Hex is a shorthand¶

Level: 101 · for anyone starting from zero

One line: Hexadecimal is not a different kind of number. It is binary written four bits at a time, so one byte is always exactly two hex digits, and C3 A9 is a picture of sixteen switches you do not have to count.

One digit, four bits¶

Sixteen patterns fit in four bits, and hex gives each one a single character: 0–9 for the first ten, A–F for the last six. Split a byte down the middle and each half is one digit:

0100 0001      the byte for 65
 4    1        left nibble 0100 = 4, right nibble 0001 = 1
0x41           the same byte, as most tools will show it

Decimal cannot do this. 65 tells you nothing about the bits, 200 and 255 look nothing like 1100 1000 and 1111 1111, and a byte in decimal is one, two, or three digits, so a row of them has no rhythm. In hex a byte is always two digits, so a hex dump is a grid: sixteen bytes per line, every byte the same width, and the seam between bytes never falls inside a digit. That property is the entire reason the format exists.

The spellings¶

The same byte has several spellings, and each tool picks one:

You see	It means	Where
`0x41`	the number 65, written in hex	Python, Rust, C, bash arithmetic
`\x41`	the byte 65 inside a string or bytes literal	Python `b'\x41'`, Rust `b"\x41"`, `printf`
`41`	one byte, in a hex dump or an ABAP `x` field	`xxd`, `od -tx1`, `TYPE x`
`U+0041`	the Unicode code point 65, which is not a byte at all	the Unicode standard; chapter 2
`%41`	the byte 65 in a URL	web addresses

Case does not matter — 0xc3 and 0xC3 are the same byte — and the 0x prefix is for humans: int('0x41', 16) works in Python because Python is forgiving, but Rust's from_str_radix refuses it, and so does printf '%d' 0b….

In Python¶

Verified output of hex_is_a_shorthand_py.py — regenerated by tools/run_examples.py, never hand-typed.

1. ONE HEX DIGIT IS EXACTLY FOUR BITS (a 'nibble')
   bits   hex   decimal
   0000   0      0
   0001   1      1
   0010   2      2
   0011   3      3
   0100   4      4
   0101   5      5
   0110   6      6
   0111   7      7
   1000   8      8
   1001   9      9
   1010   A     10
   1011   B     11
   1100   C     12
   1101   D     13
   1110   E     14
   1111   F     15

2. SO A BYTE IS EXACTLY TWO HEX DIGITS, ALWAYS
    65 = 0100 0001 = 41   (left nibble 4, right nibble 1)
   200 = 1100 1000 = C8   (left nibble C, right nibble 8)
   255 = 1111 1111 = FF   (left nibble F, right nibble F)
     0 = 0000 0000 = 00   (left nibble 0, right nibble 0)
    15 = 0000 1111 = 0F   (left nibble 0, right nibble F)
    16 = 0001 0000 = 10   (left nibble 1, right nibble 0)
   Compare decimal: 65 / 200 / 255 have no visible relation to their bits.

3. THE SPELLINGS PYTHON GIVES YOU
   hex(65)              -> '0x41'     (0x prefix, lowercase, no padding)
   format(65, '02x')    -> '41'       (two digits, no prefix)
   format(5, '02x')     -> '05'       (padding matters: '5' would not be a byte)
   f'{65:#04x}'         -> '0x41'     (# adds the prefix, 04 counts the prefix)
   int('41', 16)        -> 65         (text -> number, base 16)
   0x41                 -> 65         (a hex literal in source)
   bytes.fromhex('41')  -> b'A'       (text -> the byte itself)
   b'A'.hex()           -> '41'       (the byte -> text)

4. THE TRAP: THE TEXT '41' IS NOT THE BYTE 0x41
   '41'.encode()        -> b'41'   which is the bytes 34 31
   Two characters, '4' and '1', are two bytes: 0x34 and 0x31.
   bytes.fromhex('41')  -> b'A'    one byte, value 0x41 = 65 = 'A'
   A hex dump SHOWS you '41'; the file CONTAINS one byte. Never confuse the picture with the thing.

5. WHY 0xFF IS THE NUMBER EVERYONE REMEMBERS
   0xFF = 255 = 11111111 = every switch on = the biggest byte
   0x100 = 256 = 100000000 = the first number that needs a second byte

Section 4 is the one to stare at. A hex dump shows you 41, and it is easy to start thinking of a file as being made of those two characters. It is not. The file contains one byte, and '41' typed into a text editor is two other bytes, 0x34 and 0x31, which are the ASCII digits four and one. bytes.fromhex('41') turns the picture back into the thing; .hex() turns the thing into a picture. Everything in a hex dump is a picture.

In the terminal¶

Verified output of hex_is_a_shorthand_sh.sh — regenerated by tools/run_examples.py, never hand-typed.

1. printf CONVERTS BETWEEN BASES

$ printf '%x\n' 65          # decimal in, hex out
41

$ printf '%02X\n' 5         # two digits, uppercase
05

$ printf '%d\n' 0x41        # hex in, decimal out
65

2. BASH ARITHMETIC DOES KNOW EVERY BASE: base#digits

$ echo $(( 16#41 ))  $(( 2#01000001 ))  $(( 8#101 ))
65 65 65

3. THE PICTURE VERSUS THE THING

$ printf '41' | xxd            # the TEXT 41: two bytes, 0x34 and 0x31
00000000: 3431                                     41

$ printf '\x41' | xxd          # the BYTE 0x41: one byte, shown as 41, read as A
00000000: 41                                       A

printf '%x' and $(( 16#41 )) convert numbers; xxd shows bytes. The last two commands are section 4 again, from the shell: the text 41 dumps as 3431, the byte \x41 dumps as 41.

In Rust¶

Verified output of hex_is_a_shorthand_rs.rs — regenerated by tools/run_examples.py, never hand-typed.

1. FOUR FORMAT SPECIFIERS, ONE VALUE
   {:x}     -> 41       lowercase hex, no padding
   {:02X}   -> 41       uppercase, two digits — the byte-shaped one
   {:#04x}  -> 0x41     prefix included, width counts the prefix
   {:08b}   -> 01000001 the same eight bits, one per digit

2. THREE LITERALS THAT ARE ALL 65
   0x41 == 0b0100_0001 == b'A'  ->  true

3. PARSING NEEDS THE BASE, AND REFUSES THE PREFIX
   u8::from_str_radix("41", 16)   -> Ok(65)
   u8::from_str_radix("0x41", 16) -> Err(ParseIntError { kind: InvalidDigit })
   (the prefix is for humans; the parser wants digits only)

4. A BYTE STRING PRINTED AS HEX, TWO DIGITS PER BYTE
   "café" -> 63 61 66 c3 a9
   five bytes for four letters; the last two are the é. Next chapter.

{:02x} is the spelling to reach for when the thing being printed is a byte: two digits, padded, so a run of them stays a grid. {:x} on a value below 16 prints one digit and quietly breaks the alignment. The Rust library's Why hexadecimal ↗ has that trap and two more, plus a kata.

Writing the literal¶

The table above is about how a byte is shown to you. This is the other direction — how you write one in source — and it is the same idea with one extra rule: the prefix names the base, and every base except decimal has one.

Verified output of writing_the_literal_py.py — regenerated by tools/run_examples.py, never hand-typed.

1. FOUR BASES, ONE NUMBER, IN SOURCE
   0b1100_0011  = 195      binary, 8 digits, one per bit
   0o303        = 195      octal, 3 digits
   195          = 195      decimal, the only base with no prefix
   0xC3         = 195      hex, 2 digits -- the byte-shaped one
   Same value four ways. The prefix is not decoration; it is the only thing
   in the line that says which base the digits are in.

2. THE UNDERSCORE IS A COMMENT YOU CAN PUT INSIDE A NUMBER
   0b1100_0011  = 195      grouped by nibble, so you can read the two hex digits off it
   0xC3_A9      = 50089    grouped by BYTE, which is the grouping that matters here
   1_000_000    = 1000000  grouped by thousand, the habit you already have
   The parser drops them, so they cost nothing and change nothing. Since 3.6.
   int('1_000')         -> 1000    -- and int() takes them too
   int('0b1010_1010', 0) -> 170    -- base 0 means 'read the prefix'

3. A LEADING ZERO IS NOT A BASE. PYTHON 3 MADE SURE OF IT
   Writing 0755 in source is a SyntaxError -- refused on purpose, not by accident.
   (The message names the fix; it is not quoted here because CPython rewords it.)
   compile('x = 0755') -> SyntaxError    confirmed on this interpreter

   And the same seven characters as a STRING, which is where it still bites:
   int('0755')          -> 755     base 10 by default; the zero is just a zero
   int('0755', 8)       -> 493     you said the base, so the zero is padding
   int('0755', 0)       -> ValueError   base 0 reads a prefix, and '0' is not one
   int('0o755', 0)      -> 493     the prefix, spelled the way Python spells it
   Three answers from one field of seven characters. If it came out of a config
   file or a form, the base is a decision somebody has to make in the open.

Section 2 is small and worth adopting today. The underscore is a separator the parser throws away, so it costs nothing and lets the literal draw its own grouping: 0b1100_0011 groups by nibble, so you can read the two hex digits straight off it, and 0xC3_A9 groups by byte, which is the grouping the octal section below is about. A number that shows its own structure is one you can check by eye.

Section 3 is the trap, and it is the picture-versus-thing confusion one level up. A leading zero is not a base — but it used to be, and in one language it still is.

Verified output of writing_the_literal_c.c — regenerated by tools/run_examples.py, never hand-typed.

1. IN C, A LEADING ZERO *IS* THE OCTAL PREFIX
   0755 = 493    <- octal. The prefix is a single '0', and it is easy to miss.
    755 = 755    <- decimal. One character apart, 262 apart in value.
   0x1ED = 493   <- the same 493 in hex, where the base is spelled out loud.
   This is the rule Python 3 removed and Rust never adopted, and it is why
   chmod(path, 0755) is correct while chmod(path, 755) compiles and is not.

2. WHICH MAKES THE ZERO THE ONLY PREFIX YOU CANNOT SEE
   0x / 0b / 0o are two characters and one of them is a letter. C's octal
   prefix is one character that is also a digit, in a language where a
   zero-padded number is a completely ordinary thing to write.

3. AND C11 HAS NO BINARY LITERAL AT ALL
   0b1100_0011 is not C. Neither is the underscore.
   C++14 added both first, and picked the APOSTROPHE as its separator --
   0b1100'0011 -- because '_' was already taken by user-defined literals.
   C23 then matched C++, nine years later. Same idea, a third spelling.

So the same seven characters, 0755, are 493 in C, 755 in Rust, and a compile error in Python 3 — three answers, each printed above or below by that language's own program, from a literal a reader would call obvious. C's octal prefix is a single 0, which is both the shortest prefix and the only one that is also a digit, in a language where writing a zero-padded number is ordinary. Python 3 removed the rule rather than keep quietly disagreeing with C about 0755, and made the old spelling a syntax error instead of a different number — which is the right way to retire a trap: fail, do not reinterpret.

The live version is int() on text you did not write. int('0755') is 755, int('0755', 8) is 493, and neither is wrong; the string simply does not say. That is the parsing side of the same question, and it has a page of its own — Hex: a number, or a picture of bytes puts Python, Rust, C and bash on one malformed input and gets four different answers, one of them silent. If those seven characters came out of a config file, a form, or a spreadsheet column, somebody has to choose the base in the open, and the choice belongs next to the code that reads the field rather than in a habit.

Verified output of writing_the_literal_rs.rs — regenerated by tools/run_examples.py, never hand-typed.

1. THE SAME FOUR BASES, THE SAME PREFIXES
   0b1100_0011 = 195      0o303 = 195      195 = 195      0xC3 = 195
   Rust took Python's prefixes unchanged, so this half transfers with no edits.

2. THE UNDERSCORE IS LOOSER HERE: ANYWHERE, ANY NUMBER OF TIMES
   0xC3_A9    = 50089   grouped by byte, the reading that matters
   0x_C3_A9_  = 50089   legal too -- leading and trailing, and Rust does not mind
   1_000_000  = 1000000

3. AND A LITERAL MAY NAME ITS OWN TYPE, WHICH PYTHON HAS NO WAY TO DO
   0xC3u8     = 195     a u8, so the width is in the literal
   0xC3_A9u16 = 50089   a u16, and the compiler checks it fits
   That suffix is the whole difference in one character: in Python 195 is an
   integer of no particular width, and in Rust you have said which byte count
   you meant before the program runs.

4. AND THE LEADING ZERO? IT IS JUST A ZERO
   0755 = 755    decimal, not octal -- Rust has no leading-zero rule at all
   One seven-character literal, and each language on this page gives it a
   different reading -- each from its own program, above and below. The
   prefix exists so that sentence can stop being true of new code.

Rust's addition is the type suffix. 0xC3u8 is a byte and 0xC3_A9u16 is two, checked before the program runs — so where Python's 195 is an integer of no particular width, Rust's literal has already said how many bytes you meant. That is the same information a hex dump gives you by counting digits, moved into the source text.

Why hex and not octal¶

Octal is the same trick at a different width: one digit for three bits, 0–7. It was the standard once, and the tooling still carries it — od is octal dump, printf '\303\251' writes two bytes, git status escapes a non-ASCII filename in octal. So why did hex win?

For no reason about hexadecimal at all. Three does not divide eight. A byte is 8 bits, so it is exactly two hex digits and never a whole number of octal digits — and once digits stop lining up with bytes, a dump stops being a grid.

Verified output of why_not_octal_py.py — regenerated by tools/run_examples.py, never hand-typed.

1. THE BASE HAS TO DIVIDE THE WORD
   one octal digit = 3 bits        one hex digit = 4 bits
   a byte is 8 bits:   8 / 4 = 2 exactly      8 / 3 = 2.67  <- does not divide
   That one fact is the whole argument. Everything below is it, seen from somewhere.

2. SO THE LEADING OCTAL DIGIT OF A BYTE IS NOT A WHOLE DIGIT
   Split 255 the way each base splits it, and look at the group sizes:
   hex        1111 1111            = 0xFF    4 + 4
              F    F
   octal      11 111 111           = 0o377   2 + 3 + 3
              3  7   7
   Three octal digits would be 9 bits. A byte has 8, so the top digit is short one bit:
   it runs 0-3 and no further. 0o377 = 255 is the biggest byte; 0o400 = 256 is past it.
   Both hex digits are full digits, 0-F. Neither is a special case you have to remember.

3. TWO BYTES: HEX SHOWS THE SEAM, OCTAL HIDES IT
   'é' in UTF-8 is 2 bytes: c3 a9
   as bits    11000011 10101001    <- the gap is the byte boundary
   as hex     c3       a9          <- two digits each, and the gap survives
   as octal   1 100 001 110 101 001 = 0o141651
                        ^^^
                        this digit (110 = 6) is part of BOTH bytes at once
   In hex no digit is ever shared, because 4 divides 8. That is the entire reason a hex dump
   can print bytes in a grid and an octal one cannot.

4. OCTAL IS NOT WRONG. IT FITS A THREE-BIT FIELD PERFECTLY
   Unix permissions are three rwx triples = 9 bits, and 9 / 3 = 3 exactly.
   octal      111 101 101          = 0o755   one digit per triple
              7   5   5
   hex        0001 1110 1101       = 0x1ED   no digit lines up
              1    E    D
   Same nine bits. `0o755` says rwx r-x r-x out loud; `0x1ED` says nothing at all.
   So the base is not a matter of taste — it is whichever one divides the field you are reading.

5. WHICH IS WHY THE ANSWER CHANGED WHEN THE CHARACTER DID
   6-bit character  (Fieldata, BCDIC, DEC SIXBIT)
      octal  6 / 3 = 2.00   exactly
      hex    6 / 4 = 1.50   NOT evenly
   8-bit byte       (System/360 onward)
      octal  8 / 3 = 2.67   NOT evenly
      hex    8 / 4 = 2.00   exactly
   Two octal digits held one 6-bit character. Two hex digits hold one 8-bit byte.
   Same property, a different width — the notation followed the character, not the fashion.

Section 3 is the one that settles it. Both bases can spell the two bytes of é; only one lets you see where the first byte ends. In c3 a9 the seam is visible because every digit belongs to exactly one byte. In 0o141651 the digit 6 is three bits taken from both bytes, so there is nowhere to put the gap — which is why no octal tool has ever had the shape of a hex dump.

Section 4 is the other half, and it is what keeps octal honest: octal is not obsolete, it is specialised. Unix permissions are three rwx triples — nine bits, and 9 ÷ 3 = 3 exactly — so 0o755 says rwx r-x r-x out loud where 0x1ED says nothing at all. Where the field is three bits wide, octal is the base that fits and hex is the awkward one.

Which makes the history a consequence rather than an accident. The machines that counted in octal had characters six bits wide — Fieldata, BCDIC, DEC SIXBIT — and 6 ÷ 3 = 2, so one character was exactly two octal digits: the same tidy property hex has today. When IBM's System/360 settled the byte at eight bits in 1964, the width changed and the base that divided it changed with it. Octal did not lose an argument. Its word size stopped being manufactured.

Verified output of why_not_octal_sh.sh — regenerated by tools/run_examples.py, never hand-typed.

1. THE TOOL NAMED AFTER OCTAL STILL DEFAULTS TO IT
   'od' is octal dump, and with no flags it prints 16-bit WORDS in octal,
   in the CPU's own byte order. Both CI runners are little-endian, so:

$ printf 'caf\303\251\n' | od | squeeze
0000000 060543 141546 005251
0000006
   The file is the six bytes 63 61 66 c3 a9 0a, and not one of those three
   numbers is a byte: 0o060543 is 0x6163, which is 'ca' read backwards.
   That default is a fossil of a machine whose unit was the word, not the byte.

2. -b IS THE FLAG THAT SHOWS BYTES IN OCTAL

$ printf 'caf\303\251\n' | od -b | squeeze
0000000 143 141 146 303 251 012
0000006
   Six bytes, three digits each: 18 octal digits, room for 54 bits, holding 48.
   The same six bytes in hex are twelve digits holding 48 bits -- exactly the file,
   which is what 'the base divides the word' buys you.

$ printf 'caf\303\251\n' | xxd -p
636166c3a90a

3. THE ESCAPE THAT SURVIVED: \NNN IS POSIX, \xHH IS BASH

$ printf '\303\251' | xxd     # octal escape: in POSIX printf, so it works in any shell
00000000: c3a9                                     ..

$ printf '\xc3\xa9' | xxd     # hex escape: a bash extension, absent from POSIX
00000000: c3a9                                     ..
   Same two bytes, the e-acute. When a script must run under /bin/sh, the octal
   escape is the portable one -- which is the last place octal is not just legacy.

od's bare default is the fossil — 16-bit words, in octal, in the CPU's byte order, which is three things a byte-oriented reader wants none of — preserved because POSIX froze the behaviour of a tool written for a machine that counted in words. It is the same swap plain hexdump still performs, one base along. Section 3 is the live reason to still know octal, though: \NNN is in POSIX printf and \xHH is not, so a script that has to run under /bin/sh writes its bytes in octal.

If you are coming from Python or ABAP¶

Python. Everything here is stdlib: hex(), int(s, 16), format(n, '02x'), bytes.fromhex(), bytes.hex(). The one thing worth adding to what you know is bytes.hex(' ') — the separator argument, since 3.8 — which is a one-call hex dump and makes section 4 of every later lesson readable. Octal is there too — 0o755, oct(), format(n, '03o') — and it is worth reaching for exactly once, on the mode argument of os.chmod.

ABAP. ABAP made hex the only spelling for bytes. A value of TYPE x or xstring is written in source as a quoted run of hex digits — DATA b TYPE x LENGTH 1 VALUE '41'. — and displayed the same way in the debugger, two digits per byte, always. There is no decimal view and no character view of an x field; if you want the character, you convert. And xstrlen( xs ) counts bytes while strlen( s ) counts characters — the two counts this whole library is about, already distinct in the language. There is no octal literal in ABAP at all, which is the section above stated as a language design: the only base ABAP offers for a byte is the one that divides it. (Not machine-checked — CI cannot run ABAP.)

Try it¶

cd 01_Bits_and_Bytes/hex_is_a_shorthand/examples
python3 hex_is_a_shorthand_py.py
bash hex_is_a_shorthand_sh.sh
rustc --edition 2024 hex_is_a_shorthand_rs.rs -o /tmp/hex && /tmp/hex
python3 why_not_octal_py.py
bash why_not_octal_sh.sh
python3 writing_the_literal_py.py
rustc --edition 2024 writing_the_literal_rs.rs -o /tmp/lit && /tmp/lit
cc -std=c11 -Wall -Wextra writing_the_literal_c.c -o /tmp/litc && /tmp/litc

Then by hand: convert 0xC3 and 0xA9 to bits (one nibble at a time), and 1110 1001 to hex. Check with format(0xC3, '08b') and hex(0b11101001).

Then the same two bytes in octal, and find the digit that belongs to both of them. Check with format(0xC3A9, 'o').

Practice¶

Four bits at a time, both directions. By hand, one nibble at a time: convert 0xC3 and 0xA9 to bits, and 1110 1001 to hex.

Then the part that shows why the shorthand works. Write C3 A9 in octal, and find the byte boundary in what you wrote. If you cannot find it, say why not.

Answers

Verified output of hex_is_a_shorthand_kata_py.py — regenerated by tools/run_examples.py, never hand-typed.

HEX TO BITS -- split the byte, convert each half
   0xc3 -> C and 3 -> 1100 and 0011 -> 11000011
   0xa9 -> A and 9 -> 1010 and 1001 -> 10101001

BITS TO HEX -- same move, the other way
   1110 1001 -> E and 9 -> 0xe9

THE TWO BYTES TOGETHER
   C3 A9 -> 1100 0011 1010 1001
   Sixteen switches, four hex digits, and no arithmetic anywhere: each
   digit is a picture of exactly four bits and never borrows from its
   neighbour. That is the whole reason hex won.

NOW THE SAME BYTES IN OCTAL, WHICH DOES BORROW
   0xC3A9 = 0o141651
   Three bits per digit against sixteen bits, so the digits do not line
   up with the bytes: the boundary between C3 and A9 falls INSIDE an
   octal digit. Read the two bytes separately and you get
   0o303 and 0o251, whose digits are nowhere in 0o141651 except by accident.
   Sixteen is a power of two that divides eight. Eight is not.