Skip to content

find, and filenames that are bytes

Level: 201 · for anyone who has typed a filename and been told it does not exist

One line: A filename is a bag of bytes with two forbidden values, find -name compares those bytes, and on a Mac the kernel does not — so cat NAME can open a file that find -name NAME cannot see, on one machine, in the same second.

The setup

żółw is Polish for turtle, and it can be spelled two ways that no reader can tell apart:

Spelling Code points Bytes
NFC, precomposed ż ó ł w — four c5bc c3b3 c582 77 — seven
NFD, decomposed z ◌̇ o ◌́ ł w — six 7a cc87 6f cc81 c582 77 — nine

Both draw the same word. Which one your keyboard produces depends on your keyboard, and which one is in a file you received depends on the machine it came from — a Mac's option-key accents, a paste from a web page, and a ZIP from Windows do not agree.

That is the whole problem. Everything below is a consequence.

In the terminal

Verified output of find_names_are_bytes_sh.sh — regenerated by tools/run_examples.py, never hand-typed.

1. ONE WORD, TWO SPELLINGS, DIFFERENT BYTES
   NFC  ż ó ł w        : c5bcc3b3c58277
   NFD  z+ ̇ o+ ́ ł w     : 7acc876fcc81c58277
   same on screen?     : yes on screen, NOT equal as bytes
   Seven bytes against nine. A reader sees one word; every tool on this
   page sees two different names. This is normalization, and it is the
   reason the next section behaves the way it does.

2. -name IS A BYTE COMPARISON
$ find . -name <the NFC bytes>
./żółw
$ find . -name <the NFD bytes>; echo "exit $?"
   (printed nothing)
   exit 0
   The file exists. You typed its name. find matched nothing, because it
   compared your nine bytes against the seven on disk — and it still exits
   0, because find's status answers 'did the walk succeed', never 'did
   anything match'. There is no error to notice and no status to test.

3. WHAT find ACTUALLY PRINTED, IN HEX
$ find . -name "*w" | xxd -p
2e2fc5bcc3b3c582770a
   2e2f is './', then the seven bytes of the name, then 0a for the newline
   find added. There is no encoding in that stream — a filename is a bag
   of bytes with two forbidden values, 0x00 and 0x2f ('/'), and newline is
   not one of them.

4. WHICH IS WHY COUNTING LINES IS NOT COUNTING FILES
   one file, whose name contains a newline
$ find . -type f | wc -l                      : 2
$ find . -type f -print0 | tr -dc "\0" | wc -c : 1
   Two against one. -print0 ends each name with the one byte that cannot
   occur inside a name, so the count is right and so is everything
   downstream: find … -print0 | xargs -0, and never find … | xargs.

5. THE PATTERN IS BYTES TOO, SO GLOBBING IS BYTE GLOBBING
$ find . -name "*ó*"     # o + COMBINING ACUTE, two characters
   (printed nothing)
$ find . -name "*ó*"     # U+00F3, one character
./żółw
   Both patterns are the letter o with an acute accent. One is two
   characters, the other is one. Only the spelling on disk matches, and
   your keyboard decides which one you typed — on a Mac, an option-key
   accent and a paste from a web page can differ.

6. THE HABIT
   Search by the part you are sure of. ASCII substrings are safe:
$ find . -name "*w" -o -name "*.txt"
./żółw
   And when a name must be matched exactly, do not retype it — let the
   shell hand you the bytes that are already there:
$ for f in *; do find . -name "$f"; done
./sub
./żółw

The two machines disagree, and this is where

The example above only ever creates the NFC file, because the moment you create both spellings the two operating systems stop agreeing — and the disagreement is worth seeing:

Measured 2026-09-06 — macOS 26.6 on APFS, and ubuntu:24.04. Same commands, same bytes, two filesystems. Not machine-checked: no key can match both.
$ nfc=$(printf '\305\274\303\263\305\202w')    # żółw, seven bytes
$ nfd=$(printf 'z\314\207o\314\201\305\202w')  # żółw, nine bytes
$ printf 'turtle\n' > "$nfc"
$ printf 'other\n'  > "$nfd"

# macOS, APFS                          # Ubuntu, ext4/overlayfs
$ ls | wc -l                           $ ls | wc -l
1                                      2

$ cat "$nfd"                           $ cat "$nfd"
turtle                                 other

$ [ -e "$nfd" ] && echo yes            $ [ -e "$nfd" ] && echo yes
yes                                    yes

$ find . -name "$nfd"                  $ find . -name "$nfd"
(nothing)                              ./żółw

$ grep -l turtle "$nfd"                $ grep -l turtle "$nfd"
żółw                                   (no match, exit 1)

Read the macOS column again. cat opens it. test -e says it is there. grep opens it and prints back a name you did not type. And find -name cannot see it.

Nothing there is a bug. APFS is normalization-insensitive: it stores the bytes you gave it and compares names as if both sides were normalized, so the second printf did not create a second file — it overwrote the first. Linux filesystems do neither: nine bytes is a different name from seven, so you get two files.

The dividing line is who does the comparison:

Comparison done by Behaves as So
the kernel, on open() / stat() normalization-insensitive on macOS, byte-exact on Linux cat, test -e, grep FILE all work on a Mac
the tool, on what it read from the directory always byte-exact find -name, a shell glob, and your == do not

find reads the directory, gets the seven stored bytes, and compares them to your nine. It never asks the kernel. That is why it is the odd one out — and it is also why the same script behaves differently depending on whether it opens files by name or searches for them.

One more macOS-only fact from the same measurement, worth knowing before you write a test: APFS refuses a filename that is not valid UTF-8. open(b"bad\xff\xfe", "wb") fails with Errno 92, Illegal byte sequence, where Linux creates the file happily. So "filenames are arbitrary bytes" is true on Linux and only nearly true on macOS, and code that has to handle the general case cannot be tested on a Mac.

In Python

find compares bytes and tells you by failing. Python hands you a str, which looks like it has solved the problem and has not — two strings that draw the same word are still unequal, and os.fsdecode shows you what is underneath when a name is not valid text at all.

Verified output of find_names_are_bytes_py.py — regenerated by tools/run_examples.py, never hand-typed.

1. TWO STRINGS, ONE WORD, AND == SAYS NO
   NFC                                      'żółw'  len=4
   NFD                                      'żółw'  len=6
   they look the same on screen             żółw  żółw
   NFC == NFD                               False
   as UTF-8, NFC                            c5bcc3b3c58277
   as UTF-8, NFD                            7acc876fcc81c58277
   Four characters against six, seven bytes against nine. Python is not
   wrong to say False — they are different sequences of code points. It
   is the same answer find gives, arrived at one level up.

2. THE FIX IS normalize(), AND IT HAS TO BE APPLIED TO BOTH SIDES
   normalize('NFC', NFD) == NFC             True
   normalize('NFD', NFC) == NFD             True
   casefold() alone is not enough           False
   Pick one form, put every name through it, then compare. Which form
   does not matter; that everything agrees does. This is the only
   comparison on this page that answers the question a human asked.

3. WHAT THE FILESYSTEM GAVE BACK
   we asked for                             c5bcc3b3c58277
   listdir returned                         c5bcc3b3c58277
   unchanged?                               True
   the NFD spelling is in listdir?          False
   ...after normalising both                True
   The bytes came back as they went in — both of the machines this
   library is tested on store the name they are given. What differs is
   what happens when you then ASK for the other spelling, and that is on
   the page: one of the two filesystems will hand you the same file.

4. A FILENAME IS BYTES, AND PYTHON SAYS SO WHEN PUSHED
   the bytes                                636166e92e747874
   os.fsdecode(raw)                         'caf\udce9.txt'
   round trips to the same bytes            True
   len() of that name                       8
   is it printable?                         False
   \udce9 is a lone surrogate — the surrogateescape trick, which parks an
   undecodable byte in a code point that can never come from real text, so
   the name survives a trip through str and back. It is why os.listdir()
   can hand you every file on a Linux box, including the ones nobody can
   name. print() on it raises UnicodeEncodeError; that is the cost.

5. THE RULE
   Compare filenames after normalizing BOTH sides, never by retyping a
   name you read off a screen — and when you pass names to another
   program, pass the bytes you were given rather than a spelling of them.
   In the shell that is find -print0 | xargs -0; in Python it is passing
   the listdir/os.scandir entry itself, not a string you rebuilt.

The habits

  1. Match on the ASCII part. find . -name '*.csv', find . -name '*w'. Every byte you did not type is a byte that cannot be spelled two ways.
  2. Never retype a name you read off a screen. The glyphs do not carry the spelling. Let the shell hand you the bytes: for f in *; do … "$f"; done.
  3. -print0 and xargs -0, always. A newline is legal in a filename; NUL is not. find … | wc -l counts lines, and a filename can be two of them.
  4. Normalize both sides before comparing — unicodedata.normalize('NFC', …) in Python, uconv -x nfc in the shell if you have icu4c. Which form you pick does not matter; that everything agrees does.
  5. If find cannot see a file you can cat, you have found this exact bug. Dump the name — ls | xxd — and look for cc 81, cc 87, cc a8: combining marks, the fingerprint of NFD.

If you are coming from Python or ABAP

Python: pathlib and os.listdir return str, decoded with surrogateescape, so a name that is not valid UTF-8 survives as lone surrogates and round-trips exactly — but print() on it raises. Path.glob compares the same way find -name does, character by character after decoding, so it has the identical NFC/NFD blind spot. The one thing Python gives you that the shell does not is unicodedata.normalize, which is the actual fix; os.path.normcase is not it and does nothing at all on macOS and Linux.

ABAP (Not machine-checked — CI cannot run ABAP.) On the application server the filename in OPEN DATASET is a string, converted to the platform's byte representation on the way out, so the same two spellings produce two different open() calls and you inherit exactly the behaviour of the underlying filesystem — normally Linux, so byte-exact and unforgiving. There is no normalize in the language: cl_abap_conv_* converts between code pages, which is a different operation, and normalization has to be done before the name reaches ABAP. The practical rule for interfaces is the same one as everywhere else in this library — agree on ASCII filenames at the boundary, and put the human-readable name inside the file.

Try it

  1. ls | xxd | grep -c 'cc' in your Downloads folder. Every hit is a combining mark in a filename.
  2. Create touch "$(printf 'a\nb')" and then run find . -type f | wc -l. Then do it with -print0.
  3. On a Mac: create żółw two ways, as the fence above does, and count the files. Then do the same on any Linux box you have. The number is the lesson.
  4. find . -name '*é*' in a folder you know contains one. If it comes back empty, dump the real name and compare the bytes — you have just reproduced the top of this page.

Practice

A file cat can open and find cannot see. Create a file whose name contains é, then search for it with find . -name using the composed spelling and again with the decomposed one. Predict both results and explain the difference in terms of bytes.

Then say why this is a macOS-specific trap, what the filesystem does that find does not, and give two patterns that work on either platform.

Answers

Verified output of find_kata_sh.sh — regenerated by tools/run_examples.py, never hand-typed.

ONE FILE, WHOSE NAME IS BYTES
   ls says      café.txt
   name bytes   636166c3a92e747874

SEARCH FOR IT TWO WAYS
   find . -name "<NFC>"    1 hit(s)
   find . -name "<NFD>"    0 hit(s)
   the two patterns differ: 636166c3a92e747874 vs 63616665cc812e747874

   find compares BYTES. The two patterns are different byte strings, so
   at most one of them can match -- and which one matches depends on
   what the filesystem stored, not on what you typed.

WHY THAT IS A MAC-SPECIFIC TRAP
   On Linux the kernel stores the bytes you handed it, so the NFC pattern
   finds the NFC file and that is the end of it.
   On macOS the filesystem may DECOMPOSE the name on the way in -- HFS+
   did this always, APFS normalizes on comparison -- so a file created
   with c3 a9 can be listed back as 65 cc 81. Your shell then completes
   the name correctly, cat opens it, and find -name with the composed
   spelling finds nothing. Same machine, same second, one tool comparing
   bytes and another comparing through a normalizing layer.

WHAT WORKS ON EITHER
   find . -name "caf*"     1 hit(s)   -- a pattern that avoids the letter
   Or normalize both sides before comparing, in a language that can:
       python3 -c "import unicodedata,sys;print(unicodedata.normalize('NFC',sys.argv[1]))"

THE GENERAL SHAPE
   A filename is a bag of bytes with two forbidden values, 00 and 2f. It
   is not text, it has no declared encoding, and every tool that treats
   it as text has quietly chosen one. find chose bytes -- which is the
   defensible choice, and still surprises you.

See also