What kind of file is this?¶
Level: 201 · for Python programmers
One line: Python gives you five ways to ask what a file is, they answer five different questions, and the one that reads the bytes was deleted from the standard library in 3.13 — so the only honest answer is now thirty lines you write yourself.
There is no attribute on a file that says what it is. os.path, pathlib, mimetypes and subprocess all look like they answer the same question and none of them does. Most confusion about file types in Python comes from reaching for whichever one is shortest to type and then believing its answer to a question you did not ask.
| What you call | What it reads | Opens the file? | The question it answers |
|---|---|---|---|
os.lstat / stat.S_IS* |
four bits of st_mode |
no | how is it stored? |
pathlib.Path.is_file() |
the same, after following symlinks | no | is there a regular file at the end of this path? |
mimetypes.guess_type() |
the filename | never | what is it called? |
| reading the bytes yourself | the first bytes | yes | what is in it? |
subprocess.run([path]) |
the first bytes, in the kernel | yes | can it be launched? |
Read the third column. Two of the five never look inside the file at all — which is why a .png full of Polish prose keeps its image/png label, and why is_file() is happy to call a corrupt archive a file.
The four bits¶
The filesystem's own idea of "type" is four bits wide and has seven values: regular file, directory, symlink, FIFO, socket, block device, character device. stat.S_IFMT(mode) masks them out and stat.S_IMODE(mode) gives you the permissions that make up the rest of the number. There is no eighth value, and "PNG" is not among them.
is_file() follows, and False means several things¶
Path.is_file() is stat(), so it follows symlinks — and Path.is_symlink() is lstat(), so it does not. That matters more than it sounds, because is_file() answers False for at least four different reasons: the path is a directory, the path does not exist, the path is a symlink whose target does not exist, or the call failed and pathlib swallowed the error. The run below shows a dangling symlink and a wholly absent name producing identical rows, and only is_symlink() separating them.
mimetypes never touches the disk¶
mimetypes.guess_type() is a lookup table over the extension. It answers for files that do not exist, it is wrong exactly when the filename is wrong, and its None means "no opinion about this suffix" — never "unknown kind of file" and never "I could not read it". It is the right tool for choosing a Content-Type header from a name you are about to serve, and the wrong tool for every question about a file you have in your hands.
The bytes: the standard library stopped answering¶
Python used to ship a content sniffer. imghdr identified twelve image formats from their first bytes, sndhdr did the same for audio, and both were removed in 3.13 under PEP 594 ↗ — the "dead batteries" cleanup. Nothing in the standard library replaced them.
python:3.11-slim import imghdr -> OK, with DeprecationWarning
"'imghdr' is deprecated and slated for removal in Python 3.13"
python:3.12-slim import imghdr -> OK, same warning
python:3.13-slim import imghdr -> ImportError
python:3.14-slim import imghdr -> ImportError
So on a modern Python the options are: read the first bytes and compare them yourself, as sniff() in the example does; or install something — python-magic binds the same libmagic that file(1) uses, and filetype is a pure-Python signature table. Neither is in the standard library, and CI here has no install step, which is exactly why the example hand-rolls it.
The run¶
Verified output of what_kind_of_file_is_this_py.py — regenerated by tools/run_examples.py, never hand-typed.
1. THE FILESYSTEM KNOWS SEVEN KINDS, AND NONE OF THEM IS 'PNG'
os.lstat() reads the directory entry. It never opens the file, so
it cannot be fooled by the bytes and cannot see them either.
liar.txt S_IFMT = 0o100000 -> regular file
adir S_IFMT = 0o40000 -> directory
good_link S_IFMT = 0o120000 -> symlink
The type is those four bits and nothing else. With the permissions
pinned to 0o644, st_mode splits exactly two ways:
st_mode 0o100644
S_IFMT(mode) 0o100000 the type
S_IMODE(mode) 0o644 the permissions
2. is_file() FOLLOWS SYMLINKS, AND ANSWERS False FOR SEVERAL REASONS
path is_file() is_dir() is_symlink() exists()
liar.txt True False False True
adir False True False True
good_link True False True True
dangling False False True False
absent.txt False False False False
Read the last two rows. 'dangling' is a symlink that is really
there, pointing at a name that is not; 'absent.txt' is nothing at
all. is_file() says False to both, and exists() says False to both,
because both follow the link. Only is_symlink() and os.lstat() can
tell them apart - and os.stat('dangling') raises FileNotFoundError
about a file whose directory entry you can plainly read.
3. mimetypes ASKS THE NAME AND NOTHING ELSE
It is a lookup table over the extension. It never opens the file,
so it is wrong in exactly the cases where the name is:
liar.txt guess_type -> text/plain
real.png guess_type -> image/png
empty.png guess_type -> image/png
absent.txt guess_type -> text/plain
notes.dat guess_type -> None
Note the last two. 'absent.txt' does not exist and still gets an
answer, because nothing was consulted but the string. 'notes.dat'
gets None, which means 'no opinion about .dat' - never 'unknown
kind of file', and never 'this file is unreadable'.
4. ONLY READING THE BYTES ANSWERS THE QUESTION YOU MEANT
path by inode by name by bytes
liar.txt regular file text/plain image/png
real.png regular file image/png text/plain
empty.png regular file image/png empty
Three columns, three questions, and on liar.txt three different
answers - none of them wrong. They were asked 'how is it stored',
'what is it called' and 'what is in it'. Only the third one opened
the file, and the standard library no longer ships a module that
does it: sniff() above is thirty lines you now write yourself.
5. subprocess ASKS A FIFTH QUESTION: CAN THE KERNEL LAUNCH IT?
with_shebang ran, exit 7
no_shebang refused: OSError, errno 8
not_executable refused: PermissionError, errno 13
All three are regular, non-empty files holding shell script text,
and file-by-content would call all three the same thing. errno 8 is
ENOEXEC: the kernel read offset 0, found no '#!' and no ELF header,
and no handler claimed the bytes. errno 13 is EACCES: the bytes were
never looked at, because the execute bit was not set.
Run no_shebang from a shell instead and it works, because the SHELL
catches ENOEXEC and interprets the file itself. subprocess does not.
What the run shows¶
Section 2 is the one to reread. dangling and absent.txt produce the same four booleans, and they are not the same situation: one is a symlink that is really on disk pointing at a name that is not, the other is nothing at all. os.lstat('dangling') succeeds — the directory entry is right there — while os.stat('dangling') raises FileNotFoundError about it. If your code branches on is_file(), those two paths take the same branch.
Section 3's absent.txt row is the sharpest thing on the page. guess_type('absent.txt') returns text/plain for a file that does not exist, because nothing was consulted except the string. That is not a bug; it is what the module is for. It is also how a "file type" check ends up passing on a filename a user typed.
Section 5 separates two refusals that look alike. All three files are regular, non-empty and hold shell script text — a content sniffer would call them the same thing. errno 8 (ENOEXEC) means the kernel read offset 0, found neither #! nor an ELF header, and no handler claimed the file. errno 13 (EACCES) means the bytes were never read at all, because the execute bit was missing. And no_shebang runs perfectly well from a shell, because the shell catches ENOEXEC and interprets the file itself; subprocess does not, which is why the defect only appears once you automate something that worked by hand.
One thing the example deliberately does not show, because no answer key could hold it:
normal user Path(f).is_file() -> False os.stat(f) -> PermissionError
root Path(f).is_file() -> True os.stat(f) -> OK
pathlib catches OSError and returns False, so on the first line False means "I could not tell" rather than "not a regular file". In a container you are usually root and will never see it; in production you are usually not. When the difference matters, call os.stat() and handle the exception.
If you are coming from Rust, C or ABAP¶
Rust splits the same way and is louder about it. std::fs::metadata follows symlinks and std::fs::symlink_metadata is the lstat, exactly like os.stat against os.lstat; FileType::is_file() / is_dir() / is_symlink() are the S_IS* macros. The difference worth carrying across is the error: Path::is_file() returns a bare bool and folds every failure into false, just as pathlib does — but fs::metadata() returns io::Result<Metadata>, so the idiomatic Rust spelling cannot silently hide a permission error the way the idiomatic Python spelling does. Rust has no mimetypes in std at all. See Path and PathBuf ↗.
C is where these names come from: stat(2) and lstat(2) fill a struct stat, and S_ISREG(st.st_mode) is the macro Python's stat.S_ISREG is named after. st_mode & S_IFMT is the same mask. C has no answer to "what is in it" either — file(1) is an ordinary program reading a database, not a system call.
ABAP made the opposite choice, and the contrast is the useful part: there is essentially no content sniffing anywhere in the stack. cl_gui_frontend_services=>gui_upload takes filetype = 'BIN' or 'ASC' — the caller declares the type and nothing checks the bytes against the declaration. So ABAP only ever has the mimetypes answer, and only because a developer typed it. That is why a wrong filetype corrupts data quietly: nothing downstream is in a position to notice the claim was false. (Not machine-checked — CI cannot run ABAP.)
Try it¶
- Copy any PNG you have to
notes.txt. Askmimetypes.guess_type('notes.txt'), then open it and read the first eight bytes. Two answers, both correct, different questions. mimetypes.guess_type('a_file_that_does_not_exist.png'). Explain the answer to somebody.- Make a dangling symlink with
os.symlink('nowhere', 'link'). ComparePath('link').exists(),Path('link').is_symlink(),os.lstat('link')andos.stat('link'). Only two of the four agree that it is there. - On 3.12 or earlier,
import imghdrand watch theDeprecationWarning. On 3.13 or later, watch theImportError. The module is the same age as most of the code that still calls it. - Write a two-line shell script with no
#!,chmod +xit, and run it from your shell — it works. Now run it throughsubprocess.run([path]). The difference between those two results is your shell, not the kernel.
See also¶
- Filenames are not text — one layer down: the name as bytes, and what your filesystem does to it before Python ever sees it
- Opening a file — once you have decided it is text, which encoding
open()picks when you do not say stris notbytes— whysniff()above opens in"rb"and compares againstb"..."- File type is four questions ↗ — the same layering outside Python: what
file(1)actually does, the anatomy of a magic rule, and why the desktop disagrees with it - The first two bytes ↗ — section 5 at full length: the invisible bytes that make a good script unrunnable