`diff` compares lines, `cmp` compares bytes, neither compares text¶

Level: 201 · for anyone who has been told two files are identical

One line: Two files that print the same characters can differ on every line — one spells é as c3 a9 and the other as 65 cc 81, or one ends its lines 0d 0a — and diff will print the two identical-looking lines and call it a change, because a line to diff is the bytes between the newlines and nothing else.

What each of them compares¶

cmp walks both files a byte at a time and stops at the first pair that differ, reporting how far in it got. diff splits both files at every 0a, runs its longest-common-subsequence over the resulting byte strings, and prints the ones with no partner. That is the whole of their text model. Neither decodes, neither normalizes, neither has any idea what a character is — so both of them answer exactly one question, are these the same bytes, and print the answer in a format that looks like it is about text.

They are not wrong to do that. A byte comparison is the honest primitive, and it is the only comparison that needs no policy. The trap is on the reading end: when diff prints the same four characters on both sides of its ---, it is not malfunctioning and there is nothing wrong with your terminal. It is telling you the bytes differ, in the only vocabulary it has, about a difference the screen cannot draw.

Two ways a file acquires such a difference are ordinary rather than exotic — a colleague's editor writes CRLF endings, or a filename or a line of text comes back from macOS in NFD where yours is NFC — which is why this is the page you land on after twenty minutes of "but they're the same file".

In the terminal¶

Verified output of diff_and_cmp_sh.sh — regenerated by tools/run_examples.py, never hand-typed.

1. TWO FILES THAT PRINT THE SAME AND DIFFER ON EVERY LINE
$ cat nfc.txt nfd.txt
café
café
$ xxd -p nfc.txt
636166c3a90a
$ xxd -p nfd.txt
63616665cc810a
$ diff nfc.txt nfd.txt
1c1
< café
---
> café
   exit=1
   Six bytes against seven. Both files hold the word café: one spells the
   é as one composed character (c3 a9), the other as an e followed by a
   combining acute (65 cc 81). diff has no opinion about that. A line is
   the bytes between the newlines, those two byte strings are not equal,
   so it reports a change — and prints both sides, which look identical,
   because your terminal draws both spellings the same way.

2. cmp NAMES THE BYTE, AND CALLS IT A char
$ cmp nfc.txt nfd.txt
nfc.txt nfd.txt differ: char 4, line 1
   exit=1
   Byte 4 is c3 in one file and 65 in the other. CHARACTER 4 is é in both.
   The word in that message is POSIX's and both cmps print it; it has
   meant byte since before the distinction mattered. Read it as an offset
   into the file, never as a position in the text.
$ cmp -l nfc.txt nfd.txt 2>/dev/null | sed 's/^ *//'
4 303 145
5 251 314
6  12 201
   -l lists every differing byte: the offset, then the two values in
   OCTAL (303 is 0xc3). The leading spaces are stripped because the two
   cmps pad that column to different widths — the page has the fence. The
   third row is the interesting one: byte 6 is the newline in one file and
   201 in the other, so the files differ in LENGTH as well as in content.
   No exit status is printed under a pipe anywhere on this page: a
   pipeline's status is the LAST command's, which here would be sed's.

3. THE OTHER PAIR THAT LOOKS IDENTICAL: CRLF AGAINST LF
$ diff dos.txt unix.txt >/dev/null
   exit=1
$ diff dos.txt unix.txt | cat -vet
1,2c1,2$
< one^M$
< two^M$
---$
> one$
> two$
   Every line changed, and raw on your screen not one of them looks
   changed: the byte that differs sits at the END of the line, where
   nothing draws it. That is why the diff is piped through cat -vet here
   — ^M$ is CR LF and $ alone is LF, so the left side of the change has
   one byte the right side does not. (It is also the only way to put this
   output on a page: a raw CR does not survive being recorded as an
   answer key, for the same reason it confuses everything else.)
$ diff --strip-trailing-cr dos.txt unix.txt
   exit=0
   Both diffs have that flag. It redefines the same for one run, which is
   the right answer when a Windows checkout meets a Unix one — and the
   wrong answer if you are trying to find out why the checksum changed.

4. diff DECIDES YOUR TEXT IS BINARY AND STOPS
$ diff a16.txt b16.txt
Binary files a16.txt and b16.txt differ
   exit=1
   Two UTF-16 files, one letter apart. Every other byte of UTF-16LE
   ASCII is 00, and a NUL is how diff decides a file is not text — so it
   declines to show you the line at all.
$ diff --text a16.txt b16.txt | cat -v
2c2
< ^@t^@w^@o^@
---
> ^@t^@w^@q^@
   --text overrides the guess, and cat -v makes the NULs visible as ^@.
   The line numbering is honest and useless at the same time: diff split
   the file at the 0a bytes, so its line 2 begins with the SECOND byte of
   the previous newline. A UTF-16 line break is 0a 00, and diff can only
   see the first half of it.
$ cmp a16.txt b16.txt
a16.txt b16.txt differ: char 13, line 2
   exit=1
   And cmp answers normally. The tool with no text model at all is the one
   still working on a file whose text model diff could not guess.

5. THE TWO FLAGS THAT LOOK LIKE THE FIX, AND ARE NOT
$ diff -i upper.txt nfc.txt
1c1
< CAFÉ
---
> café
   exit=1
$ LC_ALL=<a UTF-8 locale> diff -i upper.txt nfc.txt
   exit=1
   -i folds ASCII case. É and é are two bytes each, it does not touch
   them, and asking in a UTF-8 locale changes nothing — same status on
   both platforms. The locale name is not printed because it differs per
   machine and the answer does not.
$ diff -w nbsp.txt space.txt >/dev/null
   exit=1
$ diff -w nbsp.txt space.txt | cat -v
1c1
< aM-BM- b
---
> a b
   -w ignores whitespace, and whitespace means the ASCII space and tab. A
   NO-BREAK SPACE is c2 a0, an ordinary character to diff, so the line
   that looks like it has a space in it does not have one. Read the cat -v
   rendering carefully: M-B is c2 and M- is a0 — the space you can see in
   aM-BM- b belongs to a0's own spelling, not to the file.

6. THE THREE ANSWERS, AND THE ONE A SCRIPT SHOULD READ
$ diff nfc.txt nfc.txt
   exit=0
$ diff nfc.txt nfd.txt >/dev/null
   exit=1
$ diff nfc.txt no_such_file 2>/dev/null
   exit=2
   0 same, 1 differ, 2 trouble — the contract both tools keep, and the
   reason if diff a b; then is a bug: a missing file takes the same branch
   as a difference unless you test for 2.
$ cmp -s nfc.txt nfd.txt
   exit=1
   -s prints nothing and answers with the status, which is the portable
   spelling: the statuses are identical everywhere and the messages are
   not.

7. THE LAST BYTE, WHICH ONLY ONE OF THEM WILL TELL YOU ABOUT
$ diff nl.txt nonl.txt
1c1
< one
---
> one
\ No newline at end of file
   exit=1
   One file ends with 0a and the other stops. The two lines are otherwise
   identical, and diff has a notation for exactly this. cmp reports it as
   an early EOF, on stderr, in wording that is not the same on the two
   platforms — so a script cannot read that either. The portable question
   is the status, or tail -c 1 | xxd -p.

In Python¶

Verified output of diff_and_cmp_py.py — regenerated by tools/run_examples.py, never hand-typed.

1. ONE WORD, TWO SPELLINGS, AND len() CAN SEE IT
   NFC on screen                            café
   NFD on screen                            café
   len(NFC), len(NFD)                       (4, 5)
   NFC.encode('utf-8').hex()                636166c3a9
   NFD.encode('utf-8').hex()                63616665cc81
   NFC == NFD                               False
   after NFC normalization                  True
   Four characters against five, printed identically. == on str is not a
   byte comparison — it compares code points — and it still says False,
   because these are genuinely different code points. Only naming a
   normal form makes the question answerable, and NFC is a CHOICE: it
   says composed spellings win. diff has no way to express that choice.

2. filecmp IS cmp, difflib IS diff
   filecmp.cmp(shallow=False)               False
   That is cmp -s: byte for byte, no decode, one boolean. shallow=True
   is the default and compares os.stat() — size and mtime — which is a
   different question wearing the same name.
   --- nfc.txt
   +++ nfd.txt
   @@ -1 +1 @@
   -café
   +café
   difflib prints the same unreadable change diff prints, for the same
   reason: it was handed two sequences and asked which differ. It
   compares characters rather than bytes, and on this pair that does
   not help — a decode is not a normalization.

3. THE ONE ANSWER THE SHELL TOOLS CANNOT GIVE: UNIVERSAL NEWLINES
   open(dos).read() == open(unix).read()    True
   ...with newline='' (no translation)      False
   ...as bytes                              False
   True, False, False — one pair of files, three answers, and the
   difference is which door you opened them through. Python's default
   text mode translates CRLF to \n on the way in, so the difference
   diff reports on every line of these two files is gone before your
   code sees it. That is a convenience with a cost: a program that
   reads text this way cannot tell you the file was ever a DOS file.

4. WRITE THE POLICY DOWN, THEN COMPARE
   same_text(nfc, nfd)                      True
   same_text(dos, unix)                     True
   filecmp.cmp(dos, unix, shallow=False)    False
   'CAFÉ'.casefold() == 'café'.casefold()   True
   Four lines that a shell one-liner cannot say. The last one is worth
   holding against diff -i, which folds ASCII case and leaves É alone:
   casefold() is Unicode's own case mapping, so it folds É to é — and
   ß to ss, which is why it is casefold() and not lower().
   Nothing here is more CORRECT than cmp. It is more SPECIFIC: the
   policy is four keyword arguments in one function instead of a
   default nobody chose.

The part that is not the same on both machines¶

Almost everything above is identical on the two platforms, including the parts that most look like implementation detail: the differ: char N, line M line — wording and all — the binary notice, \ No newline at end of file, and every exit status. Three things are not.

Measured 2026-09-07 — macOS 26.6.2 (Apple diff, based on FreeBSD diff; BSD cmp) and ubuntu:24.04 (GNU diffutils 3.10). Verbatim; not machine-checked, because no key can match both.

$ cmp -l nfc.txt nfd.txt        # stdout — the offset column
   macOS    "     4 303 145"    Ubuntu   "4 303 145"

$ cmp nl.txt nonl.txt           # stderr
   macOS    cmp: EOF on nonl.txt
   Ubuntu   cmp: EOF on nonl.txt after byte 3

$ diff a b                      # two files differing in one letter, with one NUL 200 kB in
   macOS    Binary files a and b differ
   Ubuntu   40001c40001
            < tail x
            ...

cmp -l pads its offset column to a fixed width on BSD and not at all on GNU — the same trap as wc -c and uniq -c, and the reason the shell example pipes it through sed 's/^ *//' before recording it. The two octal value columns are identical, so the content of that listing is safe to quote; only its shape is not.

cmp's EOF message names the byte on GNU and not on BSD. It goes to stderr, so it never reaches a pipeline that is reading cmp's output — which is a mercy, because it means a script cannot come to depend on the wording. Read the status instead.

And the binary heuristic has a different reach. Both diffs call a file binary when they meet a NUL byte, but BSD finds one anywhere in the file while GNU only looks at the first block it reads — measured here as somewhere between 4 KB and 8 KB. So one file, two verdicts: a log with a stray NUL near the end is Binary files … differ on a Mac and an ordinary line diff on Linux. The example on this page puts its NUL in byte 2, where both agree.

Four questions that all sound like "are these the same file?"¶

What you meant	How to ask it	What the answer costs
The same bytes	`cmp -s a b` — status 0 or 1, nothing printed	says nothing about the text; two spellings of one word are two files
The same lines, whatever ended them	`diff --strip-trailing-cr a b`	redefines the same for one run; a real difference in the file is now invisible
The same text, however spelled	normalize both to one form first — `unicodedata.normalize` in Python, `uconv -x nfc` from `icu4c`	NFC is a choice, not a fact, and it is not reversible: normalizing to compare is fine, normalizing in place is an edit
The same words, ignoring case	`diff -i` folds ASCII; Python's `casefold()` folds `É` and turns `ß` into `ss`	two different definitions of case, and neither is the one

Only the first row is free of policy, which is the argument for reaching for cmp first: get the byte answer, then decide which of the other three questions you were actually asking. The order matters because the byte answer never changes, and the other three all depend on a decision somebody has to write down.

That decision is exactly what version control has to make for you, which is why git ships a text/binary heuristic of its own, a core.autocrlf setting, and .gitattributes — three knobs for the two rows in the middle of that table.

If you are coming from Python or ABAP¶

Python. The three tools are all in the standard library and they are not interchangeable: filecmp.cmp(a, b, shallow=False) is cmp (and the default shallow=True compares os.stat — size and mtime — which answers a different question under the same name), difflib is diff, and == on two str compares code points, which is one step better than bytes and still says False for the two spellings of café. The one thing Python has that neither shell tool does is universal newlines: open() in text mode translates CRLF to \n on the way in, so the difference diff reports on every line of a DOS file is gone before your code sees it — newline='' is how you ask to see the file as it is.

ABAP (Not machine-checked — CI cannot run ABAP.) IF text1 = text2 on two string variables compares character by character on already-decoded data, so the CRLF question never arises inside the program — but the normalization one does, and ABAP will not normalize for you either: a string that arrived NFD from an interface is not equal to the composed one, and nothing in the comparison says so. Two ABAP-specific traps beside it. Comparing type c fields ignores trailing blanks (the shorter operand is padded), which is a definition of the same that string does not share — the same class of surprise as diff --strip-trailing-cr, decided for you by the type rather than by a flag. And comparing two xstring values is the cmp row of the table above: bytes, no policy, no decode.

Try it¶

printf 'caf\303\251\n' > a; printf 'cafe\314\201\n' > b, then diff a b, then cmp -l a b. Then wc -c a b and see which tool told you the files were even different lengths.
Take a file a Windows colleague sent you: diff it against yours, then diff --strip-trailing-cr, then cat -vet it. Only the third one shows you what is actually there.
iconv -f UTF-8 -t UTF-16 < a > a16 and the same for b, then diff a16 b16. Then convert both back to UTF-8 and diff again — that is the real fix for the binary notice, and it is a decode, not a flag.
Grep your own scripts for if diff and if ! diff. A missing file exits 2, which takes the same branch as they differ unless you tested for it.

Practice¶

Two files that print the same characters. One holds café as 63 61 66 c3 a9 0a, the other as 63 61 66 65 cc 81 0a. Predict the exit status of cmp -s, diff, diff -i and diff -w — all four are the same, and none of them is asking the question you meant.

Then: diff prints two lines that look identical to you. Explain what it is telling you. Finish with diff's three exit statuses and the if diff a b; then bug they cause.

Answers

Verified output of diff_and_cmp_kata_sh.sh — regenerated by tools/run_examples.py, never hand-typed.

TWO FILES THAT DRAW IDENTICALLY
   nfc.txt  636166c3a90a     café
   nfd.txt  63616665cc810a   café
   lengths: 6 and 7 bytes

FOUR QUESTIONS
   cmp -s              exit 1   same BYTES?           no
   diff                exit 1   same LINES as bytes?  no
   diff -i             exit 1   ASCII case folded -- irrelevant here
   diff -w             exit 1   whitespace ignored -- also irrelevant

   Every flag says different, and none of them is asking the question you
   meant. diff splits at 0a and compares the byte strings between; those
   two byte strings are not equal; there is no flag for 'same text,
   however spelled', because that needs a normalization POLICY that diff
   has no business choosing.

WHAT diff PRINTS, WHICH IS THE CONFUSING PART
   1c1
   < café
   ---
   > café
   Both sides look identical on your screen, because your terminal draws
   both spellings the same way. diff is not malfunctioning; it is
   reporting a difference the screen cannot show.

THE THREE EXIT STATUSES, AND THE BUG THEY CAUSE
   same file       exit 0
   files differ    exit 1
   missing file    exit 2
   0, 1, 2. So 'if diff a b; then ...' takes the SAME branch for 'they
   differ' and 'one of them does not exist', unless you test for 2.

THE FIX IS A DECODE, NOT A FLAG
   Normalize both sides first -- unicodedata.normalize in Python, uconv
   -x nfc from icu4c -- and then compare. Normalizing to COMPARE is fine;
   normalizing in place is an edit to somebody's data.

diff compares lines, cmp compares bytes, neither compares text¶