Why hexadecimal¶

Level: 101 → 201 · working knowledge

One line: Hex is not a different kind of number — it is a spelling of bits in which one digit is exactly four, so a byte is always exactly two digits and the seam between bytes never falls inside a character.

Here are four bytes, written honestly:

00001010111100000000010110111110

Find the third one. You have to count, in groups of eight, with a finger on the screen — and if you miscount by a single place, every byte after it is wrong and nothing tells you. Now the same four bytes:

0af005be

The third byte is 05. You read it the way you read the third word of a sentence, because the bytes have visible edges. Nothing about the number changed; the spelling changed, and the spelling was chosen so the edges would land where they do.

The property, stated once¶

16 is 2⁴. So one hex digit stands for exactly four bits — never three-and-a-bit, never a number of bits that depends on the value:

0x0 = 0000     0x7 = 0111     0xA = 1010     0xF = 1111

Four bits is a nibble, and a byte is two of them. That is the entire trick: two digits per byte, forever, for all 256 values, so a run of bytes becomes a run of digit-pairs you can index into. The two hex digits of a byte are literally its two nibbles — byte >> 4 is the left one and byte & 0x0F is the right one.

Everything else on this page is a consequence, including all three traps.

Why not decimal¶

Decimal has no such digit-to-bit correspondence, because 10 is not a power of two. The consequence is that the number of digits changes with the value:

byte	decimal	digits	hex	digits
7	`7`	1	`0x07`	2
42	`42`	2	`0x2A`	2
100	`100`	3	`0x64`	2
255	`255`	3	`0xFF`	2

Write four bytes in decimal and you get 10240 5190, or 102405190, or something with separators — but never a string you can cut into fixed-width pieces. Decimal is the right spelling for a quantity. Hex is the right spelling for a bit pattern, and a byte is a bit pattern that happens to also be readable as a number.

Why not octal — and where octal still wins¶

Octal is base 8, and 8 is 2³, so an octal digit is exactly three bits. That is a real digit-to-bit correspondence, so octal has the property decimal lacks. It just has it at the wrong granularity: 3 does not divide 8, so a byte is 2⅔ octal digits. 0xFF prints as 377 — three digits, of which the leading one carries only two bits. The byte boundary falls inside a character, which is the one thing the spelling was supposed to prevent.

Octal is not a historical embarrassment, though; it lost to hex only for bytes. On a machine with 12-bit or 36-bit words — the PDP-8, the PDP-10 — three-bit groups divided evenly and octal was the natural choice. And it is still the right spelling today wherever the field really is a multiple of three bits:

0o755 = 111 101 101
        rwx r-x r-x     owner / group / other

Unix file permissions are nine bits in three groups of three, so each octal digit owns exactly one group. chmod 755 is not legacy grit; it is the same argument as hex, applied to a field that happens to be 9 bits instead of 8.

Rust's four spellings¶

Rust writes all four bases as literals, and _ may be inserted anywhere in any of them purely for the eye:

0xBE          // hex
0b1011_1110   // binary
0o276         // octal
190           // decimal

All four are the same u8. The underscore is worth a habit: 0xDEAD_BEEF and 0b1011_1110 group at the boundaries that matter, and 1_000_000 does the same for a quantity.

The full set of literals, formats, and traps, run:

Verified output of why_hexadecimal.rs — regenerated by tools/run_examples.py, never hand-typed.

=== one number, four spellings ===
  binary   10111110   8 digits, unreadable, but honest about the bits
  decimal       190   3 digits, and the count changes with the value
  octal         276   3 digits, but see below
  hex            be   2 digits -- always, for every byte there is
  all four are the same u8: true

=== the whole reason: one hex digit IS four bits ===
  0x0  =  0000  =   0
  0x1  =  0001  =   1
  0x7  =  0111  =   7
  0x9  =  1001  =   9
  0xA  =  1010  =  10
  0xF  =  1111  =  15
  16 == 2^4, so a digit maps onto a fixed, whole number of bits

=== ...so a byte is two digits, and the seam falls between them ===
  0xBE      = 10111110
  high nibble = 1011 = 0xB   <- (byte >> 4)
  low  nibble = 1110 = 0xE   <- (byte & 0x0F)
  the two hex digits of a byte ARE its two nibbles

=== why not decimal: the boundary wanders ===
    7 decimal is 1 digit(s);  0x07 is always 2
   42 decimal is 2 digit(s);  0x2A is always 2
  100 decimal is 3 digit(s);  0x64 is always 2
  255 decimal is 3 digit(s);  0xFF is always 2
  10 is not a power of 2, so no decimal digit owns a fixed set of bits

=== why not octal: it fits a 3-bit group, and a byte is not one ===
  one octal digit = 3 bits, and 3 does not divide 8
  0xFF as octal   = 377  <- 3 digits for 8 bits: the top digit holds only 2
  where octal still wins: a 9-bit field, which IS three groups of 3
    0o755 = 111101101 = rwx r-x r-x   (Unix file permissions)
    111 101 101   <- owner / group / other, one digit each

=== Rust's literals -- same value, four ways, plus _ anywhere ===
  0xBE == 0b1011_1110 == 0o276 == 190 : true
  0xDEAD_BEEFu32      = 3735928559
  1_000_000           = 1000000
  b'A'                = 65 = 0x41   <- a byte literal is just a u8

=== Rust's formatting ===
  {:x}       a           lowercase, NO padding
  {:X}       A           uppercase
  {:02x}     0a          padded to one byte  <- what you almost always want
  {:#x}      0xa         with the 0x prefix
  {:#04x}    0x0a        prefix, and the width COUNTS the prefix
  {:b}       1010        binary
  {:#010b}   0b00001010  prefix + 8 bits = width 10
  {:o}       12          octal

=== TRAP 1: unpadded hex silently loses the byte boundary ===
  [0x0A, 0xB0]   naive "ab0"  padded 0ab0
  [0xAB, 0x00]   naive "ab0"  padded ab00
  two DIFFERENT arrays, one naive string: true
  padded, they stay distinct:             true
  dropping the 02 throws away the one property you chose hex for

=== TRAP 2: from_str_radix takes a RADIX, not a prefix ===
  u8::from_str_radix("ff", 16)   = Ok(255)
  u8::from_str_radix("0xff", 16) = Err("invalid digit found in string")   <- 'x' is not a hex digit
  u8::from_str_radix("100", 16)  = Err("number too large to fit in target type")   <- 0x100 does not fit a u8
  u8::from_str_radix("+f", 16)   = Ok(15)   <- a sign is accepted, even for a u8
  so {:#x} does not round-trip: printed "0xff", reads back Err("invalid digit found in string")
  strip it first:                          Ok(255)

=== TRAP 3: hex of a signed number is two's complement, not a minus sign ===
  format!("{:x}", -1i8)  = ff         (not "-1")
  format!("{:x}", -1i32) = ffffffff   (the width shows through)
  -1i8 as u8             = 255

=== bytes <-> hex, both directions ===
  bytes -> text : [10, 240, 5, 190] -> "0af005be"
  text  -> bytes: "0af005be" -> [10, 240, 5, 190]
  round-trips    : true
  and it works BECAUSE every byte spent exactly two characters

Trap 1: unpadded hex throws away the whole point¶

{:x} does not pad. A byte below 0x10 prints as one character, and the moment that happens in a sequence, the byte boundary is gone:

bytes.iter().map(|b| format!("{b:x}")).collect::<String>()   // wrong
bytes.iter().map(|b| format!("{b:02x}")).collect::<String>() // right

This is not a cosmetic bug. Two different byte arrays can produce the same string:

[0x0A, 0xB0]  ->  "a" + "b0"  =  "ab0"
[0xAB, 0x00]  ->  "ab" + "0"  =  "ab0"

The encoding stopped being reversible — and it stopped quietly, because it is correct for 240 of the 256 possible bytes. Every test you write with realistic-looking data passes. It fails on a small byte, which in real data means a leading zero, a null, a low count, a newline. The kata below counts the damage across all 65,536 two-byte inputs.

Note the shape of this trap: it is the property from the top of the page, thrown away by omitting two characters. Hex without padding is just base 16 — you kept the arithmetic and dropped the reason you were using it.

Trap 2: `from_str_radix` takes a radix, not a prefix¶

The base is the second argument, so the 0x is not merely unnecessary — it is a parse error, because x is not a hex digit:

u8::from_str_radix("ff", 16)    // Ok(255)
u8::from_str_radix("0xff", 16)  // Err(InvalidDigit)

Which produces a genuine asymmetry worth knowing: {:#x} prints a prefix that from_str_radix will not read back. Format and parse are not inverses in Rust unless you strip it yourself with strip_prefix("0x").

The other half of the same call is a feature rather than a trap: the type bounds the parse, so u8::from_str_radix("100", 16) is Err(PosOverflow) — 0x100 is 256 and does not fit. The parser refuses rather than truncating.

What it does not refuse is a sign. The grammar is an optional + or -, then digits, and the + is accepted even by an unsigned type: u8::from_str_radix("+f", 16) is Ok(15). So a two-character slice that parses is not two hex digits, and a decoder that parses pairs has to check is_ascii_hexdigit first — the one in the kata does.

Trap 3: hex of a signed integer is two's complement¶

format!("{:x}", -1i8)    // "ff"
format!("{:x}", -1i32)   // "ffffffff"

No minus sign appears, and the width of the type shows through — the same value prints as two characters or eight depending on where it was stored. This is not {:x} being unhelpful: hex is a spelling of the bits, and the bits of -1i8 genuinely are 11111111. If you wanted the human reading, you wanted decimal.

If you are coming from another language¶

Python. The literals are identical (0xBE, 0b1011_1110, 0o276, and _ separators since 3.6), and the ideas transfer whole. Three differences will bite you, all of them Rust being stricter or narrower:

int("0xbe", 16) accepts the prefix; u8::from_str_radix("0xbe", 16) rejects it.
A sign goes the other way round: int('+f', 16) and u8::from_str_radix("+f", 16) both accept it, and bytes.fromhex('+f') raises. The first two read a number and fromhex reads bytes, which is the distinction Hex: number or bytes? ↗ is built on.
bytes.hex() and bytes.fromhex() are in the standard library and pad correctly, so Trap 1 is a bug you would have to write on purpose. Rust's standard library has no [u8] -> String hex encoder at all — you write it, or you take the hex crate — so the trap is live. (f"{10:x}" is unpadded in Python too; you just rarely reach for it.)
format(-1, "x") is "-1" in Python and "ff" in Rust. Python's int has no width, so it can afford a minus sign; Rust's i8 has exactly eight bits and prints them.

ABAP. You already have the padded form as a type rather than a formatting choice: a TYPE x LENGTH 1 field is one byte and always displays as two hex digits, and XSTRING is a byte string that renders in hex by nature — so Trap 1 cannot happen to you there, because you never hand-assemble the string. What changes in Rust is exactly that: a hex fingerprint is a String, an ordinary sequence of characters with no memory of having been bytes. The discipline TYPE x gave you for free is now yours to keep, and {:02x} is where you keep it.

Practice¶

The fingerprint that collided. You want a short hex fingerprint for a ballot file, so two election observers can read it aloud and compare. Write the obvious encoder with {:x}, then find two different files it gives the same fingerprint — start with a byte below 0x10 and a byte that ends in zero.

Then fix it, and prove the fix rather than asserting it: encode every one of the 65,536 two-byte files and count how many distinct strings each version produces. Finally write the decoder, and decide what it should do with an odd number of characters, with 0x on the front, and with a character that is not a hex digit — three cases, three different answers. Then feed it +f and aéb: from_str_radix alone reads the first as a byte, and slicing two bytes at a time panics on the second.

Solution

why_hexadecimal_kata.rs in full — pasted here by tools/run_examples.py from the file CI compiles and runs.

//! Kata solution: the fingerprint that collided.
//!
//! You want a short fingerprint for a ballot file — something two election
//! observers can read aloud to each other and compare. Hex is the obvious
//! spelling. The obvious implementation is also wrong, and it is wrong in the
//! one way that matters: two different files can print the same fingerprint.
//!
//!   rustc --edition 2024 why_hexadecimal_kata.rs -o /tmp/whxk && /tmp/whxk

use std::collections::{BTreeMap, BTreeSet};

/// The version almost everyone writes first.
fn naive(bytes: &[u8]) -> String {
    bytes.iter().map(|b| format!("{b:x}")).collect()
}

/// The same thing, with the one property hex was chosen for.
fn encode(bytes: &[u8]) -> String {
    bytes.iter().map(|b| format!("{b:02x}")).collect()
}

#[derive(Debug, PartialEq)]
enum HexError {
    OddLength(usize),
    BadDigit(String),
}

/// Read a fingerprint back. Accepts an optional `0x`, because a human will type one.
fn decode(text: &str) -> Result<Vec<u8>, HexError> {
    let t = text.strip_prefix("0x").or_else(|| text.strip_prefix("0X")).unwrap_or(text);
    // Check every character before parsing any. `from_str_radix` accepts a
    // leading `+`, so the pair "+f" would read as 15 — and slicing two bytes at
    // a time panics when a pair boundary falls inside a character, as in "aéb".
    if let Some(bad) = t.chars().find(|c| !c.is_ascii_hexdigit()) {
        return Err(HexError::BadDigit(bad.to_string()));
    }
    if !t.len().is_multiple_of(2) {
        return Err(HexError::OddLength(t.len()));
    }
    Ok((0..t.len())
        .step_by(2)
        .map(|i| u8::from_str_radix(&t[i..i + 2], 16).expect("two hex digits, checked above"))
        .collect())
}

fn main() {
    println!("=== two ballot files, one fingerprint ===");
    let file_a: [u8; 3] = [0x0A, 0xB0, 0x42];
    let file_b: [u8; 3] = [0xAB, 0x00, 0x42];
    let show = |bs: &[u8]| bs.iter().map(|b| format!("0x{b:02X}")).collect::<Vec<_>>().join(", ");
    println!("  file A  [{}]  ->  naive {:?}", show(&file_a), naive(&file_a));
    println!("  file B  [{}]  ->  naive {:?}", show(&file_b), naive(&file_b));
    println!("  different files:      {}", file_a != file_b);
    println!("  same fingerprint:     {}   <- the observers agree, and they are wrong",
             naive(&file_a) == naive(&file_b));

    println!("\n=== the fix is two characters of format string ===");
    println!("  file A  ->  {:?}", encode(&file_a));
    println!("  file B  ->  {:?}", encode(&file_b));
    println!("  now distinct:         {}", encode(&file_a) != encode(&file_b));

    println!("\n=== how bad was it? every two-byte file, counted ===");
    let all: Vec<[u8; 2]> = (0..=255u8).flat_map(|x| (0..=255u8).map(move |y| [x, y])).collect();
    let naive_distinct: BTreeSet<String> = all.iter().map(|p| naive(p)).collect();
    let fixed_distinct: BTreeSet<String> = all.iter().map(|p| encode(p)).collect();
    println!("  inputs                     : {}", all.len());
    println!("  distinct naive fingerprints: {}   <- {} files lost their identity",
             naive_distinct.len(), all.len() - naive_distinct.len());
    println!("  distinct fixed fingerprints: {}   <- one each, which is the job",
             fixed_distinct.len());

    let mut buckets: BTreeMap<String, Vec<[u8; 2]>> = BTreeMap::new();
    for p in &all {
        buckets.entry(naive(p)).or_default().push(*p);
    }
    let worst = buckets.values().map(|v| v.len()).max().unwrap_or(0);
    println!("  worst single collision     : {worst} different files share one string");
    println!("  the first few collisions:");
    for (text, group) in buckets.iter().filter(|(_, g)| g.len() > 1).take(3) {
        let shown: Vec<String> = group.iter().map(|p| format!("{p:?}")).collect();
        println!("    {text:>5} <- {}", shown.join("  "));
    }

    println!("\n=== reading a fingerprint back ===");
    for input in ["0ab042", "0x0ab042", "0AB042", "0ab04", "0ab0zz", "+f", "aéb"] {
        println!("  {input:<10} -> {:?}", decode(input));
    }
    println!("  from_str_radix alone reads \"+f\" as {:?} -- it takes a sign, so check the digits first",
             u8::from_str_radix("+f", 16));
    std::panic::set_hook(Box::new(|_| {}));
    let sliced = std::panic::catch_unwind(|| &"aéb"[0..2]);
    let _ = std::panic::take_hook(); // the default hook back
    println!("  and slicing \"aéb\" two bytes at a time panics: {} -- the first pair ends inside the é",
             sliced.is_err());

    println!("\n=== the round trip, proved rather than asserted ===");
    let single_ok = (0..=255u8).all(|b| decode(&encode(&[b])) == Ok(vec![b]));
    let pairs_ok = all.iter().all(|p| decode(&encode(p)) == Ok(p.to_vec()));
    println!("  all 256 single bytes round-trip   : {single_ok}");
    println!("  all {} two-byte files round-trip: {pairs_ok}", all.len());
    let naive_single = (0..=255u8).filter(|&b| decode(&naive(&[b])) == Ok(vec![b])).count();
    println!("  ...and with the naive encoder     : {naive_single} of 256");
    let failures: Vec<u8> = (0..=255u8).filter(|&b| decode(&naive(&[b])) != Ok(vec![b])).collect();
    println!("  the {} that fail are 0x{:02X}..=0x{:02X} -- exactly the bytes one digit could spell,",
             failures.len(), failures[0], failures[failures.len() - 1]);
    println!("  which is why the bug hides: it needs a small byte to show itself at all");
}

Verified output of why_hexadecimal_kata.rs — regenerated by tools/run_examples.py, never hand-typed.

=== two ballot files, one fingerprint ===
  file A  [0x0A, 0xB0, 0x42]  ->  naive "ab042"
  file B  [0xAB, 0x00, 0x42]  ->  naive "ab042"
  different files:      true
  same fingerprint:     true   <- the observers agree, and they are wrong

=== the fix is two characters of format string ===
  file A  ->  "0ab042"
  file B  ->  "ab0042"
  now distinct:         true

=== how bad was it? every two-byte file, counted ===
  inputs                     : 65536
  distinct naive fingerprints: 61936   <- 3600 files lost their identity
  distinct fixed fingerprints: 65536   <- one each, which is the job
  worst single collision     : 2 different files share one string
  the first few collisions:
      110 <- [1, 16]  [17, 0]
      111 <- [1, 17]  [17, 1]
      112 <- [1, 18]  [17, 2]

=== reading a fingerprint back ===
  0ab042     -> Ok([10, 176, 66])
  0x0ab042   -> Ok([10, 176, 66])
  0AB042     -> Ok([10, 176, 66])
  0ab04      -> Err(OddLength(5))
  0ab0zz     -> Err(BadDigit("z"))
  +f         -> Err(BadDigit("+"))
  aéb        -> Err(BadDigit("é"))
  from_str_radix alone reads "+f" as Ok(15) -- it takes a sign, so check the digits first
  and slicing "aéb" two bytes at a time panics: true -- the first pair ends inside the é

=== the round trip, proved rather than asserted ===
  all 256 single bytes round-trip   : true
  all 65536 two-byte files round-trip: true
  ...and with the naive encoder     : 240 of 256
  the 16 that fail are 0x00..=0x0F -- exactly the bytes one digit could spell,
  which is why the bug hides: it needs a small byte to show itself at all

Po polsku¶

W polskiej szkole system szesnastkowy (hexadecimal) poznaje się przez „zamianę systemów”: dzielenie przez 16 z resztą, tabelka, wynik. To ćwiczenie uczy arytmetyki i skutecznie zasłania powód, dla którego heks w ogóle istnieje — a powód nie jest arytmetyczny, tylko typograficzny. 16 to 2⁴, więc jedna cyfra szesnastkowa odpowiada dokładnie czterem bitom, czyli półbajtowi (nibble), a bajt to zawsze i niezmiennie dwie cyfry. Dlatego 0af005be czyta się jak zdanie — trzeci bajt to 05 — a te same trzydzieści dwa bity zapisane dwójkowo trzeba liczyć palcem po ekranie. Heks nie jest inną liczbą; jest innym zapisem tych samych bitów, dobranym tak, żeby szew między bajtami nigdy nie wypadł w środku znaku.

Reszta strony wynika z tej jednej własności. Dziesiętnie nie da się tak zapisać, bo 10 nie jest potęgą dwójki, więc liczba cyfr wędruje razem z wartością: 7 to jedna cyfra, 255 trzy, a 0x07 i 0xFF po dwie, zawsze. System ósemkowy odpowiedniość ma — 8 to 2³, cyfra to trzy bity — tylko w złej ziarnistości: 3 nie dzieli 8, więc 0xFF zapisuje się jako 377, gdzie górna cyfra niesie tylko dwa bity i granica bajtu leży w środku znaku. To nie znaczy, że ósemkowy jest przeżytkiem: chmod 755 to dziewięć bitów w trzech trójkach, po jednej cyfrze na właściciela, grupę i resztę — dokładnie ten sam argument co przy heksie, tyle że dla pola o innej szerokości.

Pułapka pierwsza jest w Ruscie realna, nie teoretyczna: {:x} nie dopełnia zerami. Bajt mniejszy niż 0x10 wypisze się jako jeden znak i szew przepada — [0x0A, 0xB0] oraz [0xAB, 0x00] dają ten sam łańcuch "ab0". Kod przechodzi wszystkie testy z „realistycznymi” danymi, bo zachowuje się poprawnie dla 240 z 256 bajtów; psuje się tylko na wiodącym zerze, bajcie zerowym albo małej liczbie. Ćwiczenie na tej stronie liczy szkody: spośród 65 536 dwubajtowych plików wersja bez dopełnienia gubi tożsamość 3 600 z nich, a zawodzą dokładnie bajty 0x00..=0x0F. Piszemy {:02x} — i warto wiedzieć, dlaczego akurat w Ruscie ta pułapka żyje: standardowa biblioteka nie ma enkodera [u8] -> String, więc każdy pisze go sam albo bierze crate hex, podczas gdy w Pythonie bytes.hex() dopełnia za nas.

Dwie pozostałe pułapki też są warte zapamiętania po polsku. from_str_radix przyjmuje podstawę, a nie przedrostek, więc u8::from_str_radix("0xff", 16) kończy się błędem — x nie jest cyfrą szesnastkową — i wychodzi z tego asymetria: {:#x} wypisuje przedrostek, którego parser nie odczyta, więc formatowanie i parsowanie nie są tu operacjami odwrotnymi, dopóki sam nie zdejmiesz 0x przez strip_prefix. Za to ograniczenie typem jest zaletą: u8::from_str_radix("100", 16) zwraca błąd zamiast obciąć wartość, bo 0x100 to 256. Znaku natomiast nie odrzuca: u8::from_str_radix("+f", 16) to Ok(15), nawet dla typu bez znaku — więc dwa znaki, które się sparsowały, to jeszcze nie dwie cyfry szesnastkowe, i dekoder w ćwiczeniu sprawdza is_ascii_hexdigit, zanim cokolwiek sparsuje. I na koniec liczby ujemne: format!("{:x}", -1i8) daje ff, a -1i32 daje ffffffff — żadnego minusa, bo heks zapisuje bity, a bity liczby ujemnej to uzupełnienie do dwóch; przy okazji widać szerokość typu, w którym wartość była przechowywana. Jeśli chciałeś zapisu dla człowieka, chciałeś systemu dziesiętnego.

Szukaj po polsku: system szesnastkowy · półbajt · uzupełnienie do dwóch · rust from_str_radix radix not prefix · rust format 02x padding · rust hex encode bytes crate