hexdump is a format engine wearing six presets¶
Level: 201 · for anyone with a terminal
One line: hexdump is a small printf for bytes — its six letter flags are canned format strings you can type out yourself — and the preset it runs when you give it no flags reads your file two bytes at a time as a 16-bit number, so on any machine you are likely to own every pair comes out swapped.
The engine¶
Three ideas, and there is nothing else in the tool.
A format unit is count/size "printf format". 16/1 "%02x " means sixteen units of one byte each, each printed with %02x and a space. 8/2 "%04x " means eight units of two bytes each, and a two-byte unit is a number, which is where the trouble starts. The size is how many bytes one conversion eats; the count is how many times to do it.
The input is read in blocks, and a block is as many bytes as the longest format string asks for — 16 for every preset here. hexdump fills a block, runs the formats over it, and moves on.
Each -e string is applied to the same block, from the start. That is the part nobody guesses. Within one string the conversions run in sequence and each one consumes what it printed, so a string asking for 16 hex bytes and then 16 characters wants 32 bytes and the text column reads off the end. Give it as two -e strings and both see the same sixteen bytes — which is exactly how the canonical view is built.
The six presets, written out¶
Each of these was checked by running the flag and the -e spelling over the same file and diffing: all six are identical, so the flags really are nothing but stored strings.
| Flag | The format string it stands for | What a unit is |
|---|---|---|
| (none) | "%07.7_ax " 8/2 "%04x " "\n" |
two bytes, as a number |
-x |
"%07.7_ax " 8/2 " %04x " "\n" |
two bytes, as a number |
-o |
"%07.7_ax " 8/2 " %06o " "\n" |
two bytes, as a number |
-d |
"%07.7_ax " 8/2 " %05u " "\n" |
two bytes, as a number |
-b |
"%07.7_ax " 16/1 "%03o " "\n" |
one byte, octal |
-c |
"%07.7_ax " 16/1 "%3_c " "\n" |
one byte, as a character |
-C |
three strings, below — a table cell cannot hold them | one byte, twice |
Read the unit size column and the whole tool falls out. Four of the seven views hand you 16-bit numbers, and a 16-bit number has an end that comes first. Three hand you bytes. -C is the only one that hands you bytes and a text column, which is why it is the one to type.
-C is the odd one out, and not only because its text column contains the bar character a table cannot. It is three -e strings, which is what lets one pass over sixteen bytes print them as hex and again as text:
hexdump -e '"%08.8_Ax\n"' \
-e '"%08.8_ax " 8/1 "%02x " " " 8/1 "%02x "' \
-e '" |" 16/1 "%_p" "|\n"'
Run that and diff it against hexdump -C: identical, and section 3 of the example below does exactly that.
The _a and _A conversions are hexdump's own additions to printf: _a is the offset of the current byte and _A the offset of the last one, so %08.8_Ax on its own line is the trailing address every dump ends with. %_p is print this byte if it is printable ASCII, otherwise a dot, and %_c is print it as a character, falling back to a C escape or an octal one. Those two are the only places in the tool where anything resembling a text decision gets made, and only one of them makes it badly.
In the terminal¶
Verified output of hexdump_is_a_format_engine_sh.sh — regenerated by tools/run_examples.py, never hand-typed.
1. THE DEFAULT IS THE ONE THAT LIES
$ hexdump s.txt
0000000 6163 c366 3aa9 3120 82e2 0aac
000000c
$ hexdump -C s.txt
00000000 63 61 66 c3 a9 3a 20 31 e2 82 ac 0a |caf..: 1....|
0000000c
The same twelve bytes, twice. The file starts 63 61; the first line
says 6163. Every pair is swapped — and it is not a bug, it is the
default format doing exactly what it says.
2. WHY: THE DEFAULT READS TWO BYTES AT A TIME, AS A NUMBER
The preset behind a bare hexdump is 8 units of 2 bytes: 8/2 "%04x ".
Two bytes read as one 16-bit integer, printed in THIS CPU's byte order,
which on anything you are likely to own is little-endian: low byte
first. So 63 61 becomes the number 0x6163. -x is the same preset with
wider columns:
$ hexdump -x s.txt
0000000 6163 c366 3aa9 3120 82e2 0aac
000000c
It is the UTF-16 question — which end of a two-byte number comes first —
turning up in a tool that was only asked to show bytes.
3. THE PRESETS ARE FORMAT STRINGS, AND YOU CAN WRITE THEM OUT
-C is not built in as a view. It is three -e strings, and typing them
by hand reproduces it exactly:
$ hexdump -e '"%08.8_Ax\n"' -e '"%08.8_ax " 8/1 "%02x " " " 8/1 "%02x "' -e '" |" 16/1 "%_p" "|\n"' s.txt
00000000 63 61 66 c3 a9 3a 20 31 e2 82 ac 0a |caf..: 1....|
0000000c
diff against hexdump -C: identical, byte for byte.
4. WHY -C NEEDS THREE -e STRINGS AND NOT ONE
Within one format string the conversions run in sequence and each one
CONSUMES the bytes it printed. Ask one string for 16 hex bytes and then
16 characters and it wants 32 bytes — so the text column reads past the
end and prints nothing:
$ hexdump -e '"%08.8_ax " 8/1 "%02x " " " 8/1 "%02x " " |" 16/1 "%_p" "|\n"' s.txt
00000000 63 61 66 c3 a9 3a 20 31 e2 82 ac 0a ||
The |...| came out empty. Separate -e strings are each applied to the
SAME block of bytes, from the start — which is how one pass prints the
same sixteen bytes as hex and again as text.
5. THE LAYOUT YOU ACTUALLY WANT, IN ONE STRING
count/size "printf format" is the whole language: how many units, how
many bytes each, how to print one.
$ hexdump -e '16/1 "%02x " "\n"' s.txt
63 61 66 c3 a9 3a 20 31 e2 82 ac 0a
$ hexdump -e '8/1 "%02x " " " 8/1 "%03u " "\n"' s.txt
63 61 66 c3 a9 3a 20 31 226 130 172 010
Two columns of the same eight bytes, hex then decimal, in a layout
nothing else offers: od's shape is whatever its flags decided, and
xxd's is whatever -c and -g allow. One warning about %u — write it,
not %d. A one-byte unit under %d is SIGNED, so c3 prints as -61.
6. THE STAR: REPEATED LINES ARE HIDDEN BY DEFAULT
$ hexdump -C zeros.bin
00000000 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
*
00000040
64 bytes went in and one line came out. The * means 'and more of the
same'. -v prints every line:
$ hexdump -C -v zeros.bin
00000000 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
00000010 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
00000020 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
00000030 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
00000040
Four lines of sixteen, which is the 64 bytes. Without -v the output is
three lines — first, star, final offset — whatever the file's size, so
the length of a dump tells you nothing about the length of the file.
AND THE SQUEEZE APPLIES TO -e TOO, WHICH IS HOW IT DESTROYS DATA.
hexdump -e '1/1 "%.2x"' is the usual stand-in for xxd -p on a machine
with no xxd. On this 64-byte file, characters of hex printed:
hexdump -e (no -v) : 4
hexdump -ve : 128
xxd -p : 128
The first one printed 00 and a star. A unit here is one byte, so any
byte that repeats the one before it is squeezed away, and the output is
not hex any more — it is hex with a * in it. Always write -v when the
output is going to be parsed rather than read.
7. THE TEXT COLUMN IS ASCII, AND ONLY ASCII
$ hexdump -C s.txt
00000000 63 61 66 c3 a9 3a 20 31 e2 82 ac 0a |caf..: 1....|
0000000c
$ hexdump -C latin1.txt
00000000 63 61 66 e9 0a |caf..|
00000005
Two files, two encodings, and the same answer: a dot for every byte
above 7f. %_p asks 'is this byte a printable ASCII character' and
nothing else — no locale, no encoding, no guess. That is a limit and
a virtue: the column cannot mislead you about which encoding this is,
because it never had an opinion. Read the hex.
8. -c IS THE ONE VIEW THAT DOES HAVE AN OPINION
$ hexdump -c s.txt
0000000 c a f 303 251 : 1 342 202 254 \n
000000c
Octal escapes for the bytes it cannot draw: 303 251 is the é. This runs
under LC_ALL=C, where that is all it can do. In a UTF-8 locale BSD
hexdump prints M-C M-) here and Ubuntu's still prints 303 251 — the
page has the measurement. Same command, two machines, two answers.
9. WHAT IT CANNOT DO: COME BACK
$ xxd -p s.txt
636166c3a93a2031e282ac0a
$ xxd -p s.txt | xxd -r -p | cat
café: 1€
xxd -r reads a dump and writes the bytes. hexdump has no -r and od has
no -r; to reverse a hexdump you write the loop yourself. That single
missing flag is the reason to install xxd on a machine that lacks it.
10. SKIP AND LENGTH, WHEN THE INTERESTING BYTES ARE NOT AT THE FRONT
$ hexdump -C -s 2 -n 4 s.txt
00000002 66 c3 a9 3a |f..:|
00000006
-s skips, -n stops after that many bytes, and the offset column keeps
counting from the front of the file — so what it prints is the address
in the file, not in the window. xxd spells the same two -s and -l.
What it decides about your text¶
Almost nothing, and that is the reason to reach for it.
hexdump -C's text column is ASCII and stops there. %_p asks one question — is this byte in the range 32 to 126 — and prints a dot when the answer is no. It does not consult the locale, does not attempt a decode, and does not have a theory about what encoding the file is in. Section 7 of the run above puts a UTF-8 file and a Latin-1 file through it and gets dots from both. That is a limit, and it is also the strongest thing on this page: the column cannot mislead you, because it never claimed to be reading text. Compare od -a, which invents names for bytes it cannot draw and produces cafi for a file containing café in Latin-1 — a wrong answer shaped like a word, wrong in exactly the byte you were asking about.
hexdump -c is the exception, and it is the one BSD/GNU split in the tool:
macOS (BSD) Ubuntu (util-linux)
LC_ALL=C hexdump -c c a f 303 251 : 1 342 202 254 c a f 303 251 : 1 342 202 254
LC_ALL=…UTF-8 hexdump -c c a f M-C M-) : 1 M-b 202 M-, c a f 303 251 : 1 342 202 254
LC_ALL=C hexdump -C |caf..: 1....| |caf..: 1....|
LC_ALL=…UTF-8 hexdump -C |caf..: 1....| |caf..: 1....|
Under a UTF-8 locale BSD hexdump -c switches to M-x meta notation, which is the byte with its high bit stripped: e2 becomes M-b because 0xe2 & 0x7f is 0x62, the letter b. 82 stays octal in the same row, because 0x82 & 0x7f is 0x02 and that is not printable. So one row mixes two notations, chosen per byte, on one platform and not the other. hexdump -C is identical in all four cells — and it is the only row in that block you could paste into a bug report and expect the person reading it to see what you saw.
Everything else in the tool is byte-identical between the two implementations, in both locales, down to the trailing spaces on a short last line. That is not luck: util-linux's hexdump is a direct descendant of the BSD one, where od's two implementations grew apart and now differ in padding, in the -a names, and in whether -c decodes multi-byte characters at all.
Against the other three¶
One file, four tools, and the questions worth asking before you type one of them.
hexdump -C |
xxd |
od -An -tx1 |
hexyl |
|
|---|---|---|---|---|
On a bare ubuntu:24.04 |
not installed (bsdextrautils) |
not installed (xxd) |
already there | not installed |
| On macOS | there | there | there | brew install hexyl |
| Is the default view the file? | no — bare hexdump swaps pairs |
yes | no — octal 16-bit words | yes |
| Offsets | hex | hex | octal, unless -A x |
hex |
| Text column | ASCII, locale-immune | ASCII, locale-immune | -c / -a only, and both drift by platform and locale |
colour by byte category |
Repeats collapsed to *? |
yes, unless -v |
no — and -a collapses runs of NUL only |
yes, unless -v |
yes |
| Turn a dump back into bytes | — | xxd -r |
— | — |
| Layout under your control | -e, completely |
-c, -g, -b, -i, -e |
fixed flags only | fixed |
| Same output on macOS and Linux | yes — except -c |
yes (one source, ships with vim) | no | one implementation |
Four things in that table are worth saying in words.
1. The tool that is always there is the one with the worst default. od with no flags prints 16-bit octal words at octal offsets, so a twelve-byte file ends at offset 0000014. Two number bases, both surprising, in the output of the only dump tool a stripped-down container has. od -An -tx1 is the incantation; -Ax if you want the offsets back in hex.
2. hexdump's default is the same mistake, one base along. 63 61 prints as 6163. It is the UTF-16 byte-order question — which end of a two-byte number comes first — turning up in a tool that was only asked to show bytes. Both defaults date from a machine where the word, not the byte, was the natural unit; neither is going to change, and both are one letter away from being right (-C, -tx1).
3. The * is a silent data loss when the output is not for a person. hexdump -e '1/1 "%.2x"' is the standard stand-in for xxd -p on a box with no xxd, and section 6 above runs it over 64 zero bytes: 4 characters out instead of 128, because a unit is one byte and every byte after the first repeats it. The squeeze is not a property of the -C view, it is a property of the tool, and it applies to whatever you asked -e to print. Write -v any time the output will be parsed. xxd -p has no such mode and needs no such flag, which is the single strongest argument for installing it.
4. Only xxd goes backwards. xxd -r -p turns a hex string into the bytes it names, so a dump can be edited and put back. hexdump and od are one-way, and reversing them means writing the loop. On a machine with neither, python3 -c 'import sys;sys.stdout.buffer.write(bytes.fromhex(sys.stdin.read()))' is the whole of xxd -r -p.
Which one to reach for¶
- Looking at a file, on your own machine:
xxd, orhexylif you have installed it. Honest defaults, nothing to remember. - Pasting a dump where somebody else has to read it:
hexdump -C. It is the one view in this chapter that is byte-identical across platforms and locales, so what they see is what you saw. - On a container, a rescue shell, or anything stripped down:
od -An -tx1. It is the only one guaranteed to be there. Neverod -a. - You want a specific layout — a fixed number of columns, hex beside decimal, no offsets, no text:
hexdump -e, with-v. - You need the bytes back afterwards:
xxd -r, and nothing else.
If you are coming from Python or ABAP¶
Python. bytes.hex(' ') is the hex column and bytes.fromhex() is xxd -r -p; between them they replace most reaching for a dump tool at all, and int.from_bytes(b, 'little') is precisely what bare hexdump did to your pairs, with the byte order stated out loud instead of inherited from the CPU. If you want the full three-column view in a script, the ten-line dump() in chapter 1 is it. The habit that transfers from this page is narrower and more useful: when a tool shows you characters, ask what it does with a byte it cannot draw — repr() and %_c and od -a each answer differently, and only the hex column is the file.
ABAP (Not machine-checked — CI cannot run ABAP.) The debugger's hex view of an xstring is the middle column and nothing else — no offsets, no text guess — and that is the same trade hexdump -C makes on purpose. Two things carry over. cl_abap_codepage=>convert_to( ) gets you from a string to the bytes you can actually compare, which is the ABAP spelling of "read the hex, not the text column". And the byte-order surprise in the default view has a direct analogue in xstring handling across systems: a two-byte quantity has an end that comes first, and if nothing in your code says which, something else has already decided.
Try it¶
Then, on a file you actually have:
hexdump yourfile | head -2andhexdump -C yourfile | head -2. Find a pair of bytes and watch it swap.hexdump -C -n 4 yourfileagainstfile yourfile. The first four bytes are howfileguesses; now you can check its work.hexdump -e '1/1 "%.2x"' /dev/zero | head -c 20— count what comes out, then add-v.- Write the view you always wanted:
hexdump -ve '8/1 "%02x " " " 8/1 "%_p" "\n"'.
Practice¶
Why is my dump showing the bytes in the wrong order? Predict what bare hexdump prints for a file containing cafe\n, byte group by byte group, and compare it with xxd -p. The pairs come out swapped — say what hexdump did and why it is not a bug.
Then give two flags that show the file in file order, and state the general rule about what a dump's grouping width is claiming.
Answers
Verified output of hexdump_kata_sh.sh — regenerated by tools/run_examples.py, never hand-typed.
THE FILE, ONE BYTE AT A TIME
636166650a
hexdump WITH NO FLAGS
0000000 6163 6566 000a
0000005
Read the first group: 6163. The file starts 63 61. The two bytes came
out SWAPPED -- and so did every other pair.
WHY
The default is -x, which reads the file two bytes at a time as a
16-BIT NUMBER and prints that number. On a little-endian machine the
low byte is stored first, so the number made from 63 61 is 0x6163.
Nothing is wrong: hexdump printed the integer correctly. It is just
not showing you the file.
THE FLAGS THAT DO SHOW YOU THE FILE
hexdump -C f
00000000 63 61 66 65 0a |cafe.|
00000005
hexdump -e is the same idea written out -- the letter flags are canned
format strings, and you can type your own:
63 61 66 65 0a
THE POINT, WHICH IS BIGGER THAN ONE TOOL
A dump that groups bytes into a WIDTH has made a claim about what the
data's unit is -- and if the claim is wrong, the tool will reorder your
bytes to keep it true. hexdump's default claims 16-bit integers; xxd's
default claims nothing and never reorders.
That is the whole reason this library reaches for xxd or hexdump -C in
a bug report. A bare hexdump is a picture of your CPU's opinion, and
on a big-endian machine the same command would print the bytes in file
order -- so the output is not even reproducible across hardware.
See also¶
- Reading a hex dump — the three columns, if this page assumed one you have not met
- Inspecting a file — the same tools inside a workflow, and
od -a's invented names at length xxdis the dump you can put back — the tool with the honest default, and the only one that reversesodreads types, not bytes — the one that is always installed, and what its-tis actually asking for- The five worth installing —
hexyl, measured against what you already have - Byte order and the BOM — where the swapped pairs come from, in the format that has to care
- A character and its bytes on one line — the one-liner that turns a character into its dump