Descriptors 0, 1 and 2¶

Level: 201 · you have read that /dev/stdin points to /proc/self/fd/0, and your Mac says fd/0

One line: stdin, stdout and stderr are file descriptors 0, 1 and 2, and /dev/stdin, /dev/stdout, /dev/stderr and /dev/fd/N are names for them. The names agree about reading and writing but disagree about opening. On Linux, echo err > /dev/stderr in a script whose output goes to a log file opens that file again and truncates it. On a Mac the same line behaves like >&2, and >&2 is correct on both.

Measured¶

The book lists /dev/std* on Fedora and finds three symbolic links into /proc/self/fd. The same question comes first here, then what the names do when you use them:

Verified output of redir_descriptors_sh.sh on Linux — regenerated by tools/run_examples.py, never hand-typed.

$ for f in /dev/stdin /dev/stdout /dev/stderr; do echo "$f -> $(readlink "$f")"; done
/dev/stdin -> /proc/self/fd/0
/dev/stdout -> /proc/self/fd/1
/dev/stderr -> /proc/self/fd/2

$ if [ -L /dev/fd ]; then echo "/dev/fd -> $(readlink /dev/fd)"; else echo "/dev/fd is a directory"; fi
/dev/fd -> /proc/self/fd

$ if [ -d /proc/self/fd ]; then echo "/proc/self/fd exists"; else echo "there is no /proc"; fi
/proc/self/fd exists

$ { echo "to fd 1"; echo "to fd 2" > /dev/stderr; } > out.txt 2> err.txt; echo "out.txt: $(cat out.txt)"; echo "err.txt: $(cat err.txt)"
out.txt: to fd 1
err.txt: to fd 2

$ echo "read through /dev/stdin" | cat /dev/stdin
read through /dev/stdin

$ { echo "to fd 3" >&3; } 3> three.txt; cat three.txt
to fd 3

$ { echo first; echo err > /dev/stderr; echo last; } > log.txt 2>&1; cat -v log.txt
err
^@^@last

$ { echo first; echo err >> /dev/stderr; echo last; } > log.txt 2>&1; cat -v log.txt
first
last

$ { echo first; echo err >&2; echo last; } > log.txt 2>&1; cat -v log.txt
first
err
last

$ sh -c "echo first; echo err > /dev/stderr; echo last" > log.txt 2>&1; cat -v log.txt
err
^@^@last

Verified output of redir_descriptors_sh.sh on macOS — regenerated by tools/run_examples.py, never hand-typed.

$ for f in /dev/stdin /dev/stdout /dev/stderr; do echo "$f -> $(readlink "$f")"; done
/dev/stdin -> fd/0
/dev/stdout -> fd/1
/dev/stderr -> fd/2

$ if [ -L /dev/fd ]; then echo "/dev/fd -> $(readlink /dev/fd)"; else echo "/dev/fd is a directory"; fi
/dev/fd is a directory

$ if [ -d /proc/self/fd ]; then echo "/proc/self/fd exists"; else echo "there is no /proc"; fi
there is no /proc

$ { echo "to fd 1"; echo "to fd 2" > /dev/stderr; } > out.txt 2> err.txt; echo "out.txt: $(cat out.txt)"; echo "err.txt: $(cat err.txt)"
out.txt: to fd 1
err.txt: to fd 2

$ echo "read through /dev/stdin" | cat /dev/stdin
read through /dev/stdin

$ { echo "to fd 3" >&3; } 3> three.txt; cat three.txt
to fd 3

$ { echo first; echo err > /dev/stderr; echo last; } > log.txt 2>&1; cat -v log.txt
first
err
last

$ { echo first; echo err >> /dev/stderr; echo last; } > log.txt 2>&1; cat -v log.txt
first
err
last

$ { echo first; echo err >&2; echo last; } > log.txt 2>&1; cat -v log.txt
first
err
last

$ sh -c "echo first; echo err > /dev/stderr; echo last" > log.txt 2>&1; cat -v log.txt
first
err
last

Where the names point. On Linux, /dev/stdin links to /proc/self/fd/0, and /dev/fd is itself a link to /proc/self/fd, a directory the kernel fills in for whichever process looks at it. A Mac has no /proc: there /dev/stdin links to fd/0, and /dev/fd is a real directory of its own.
The names follow the descriptor. Inside { …; } 2> err.txt, writing to /dev/stderr landed in err.txt. Reading /dev/stdin read the pipe, and >&3 reached the file the block opened as descriptor 3. So far the two systems agree.
Then both descriptors point at one file. > log.txt 2>&1 is the usual way to log a script, and the block writes first, then err to /dev/stderr, then last:
On a Mac the log is complete. Opening /dev/stderr there makes a copy of descriptor 2: the same open file at the same position, and the truncation > asks for is ignored. man 4 fd on a Mac says exactly that.
On Linux the log starts with err, and first is gone. Opening /dev/stderr followed the link to log.txt and opened the file afresh, and > truncated it to nothing. err went in at offset 0. Descriptor 1 was still 6 bytes in, so last went in at offset 6, and the two bytes in between are a hole that reads back as NUL bytes: ^@^@ in cat -v.
>> does not rescue it on Linux. Appending put err at the end of the file, byte 6, which is exactly where descriptor 1 wrote last next, over it.
>&2 is right on both, because it opens nothing. It makes echo's descriptor 1 a copy of the descriptor 2 that is already open.
It is not a bash quirk. sh -c is dash on Ubuntu and bash on a Mac, and each did the same as the bash line above it. Any program that opens /dev/stderr by name will too.

On a Mac¶

Every difference is in the two fences above: the link targets, the missing /proc, and a /dev/stderr that cannot truncate anything. The Mac's answer is the forgiving one. A script tested only on a Mac can carry > /dev/stderr for years, then lose most of its log the first time it runs as script > log 2>&1 on a Linux server.

In zsh and fish¶

zsh opens /dev/stderr like any other file name, so it gets each platform's answer:

Verified output of redir_descriptors_zsh.zsh on Linux — regenerated by tools/run_examples.py, never hand-typed.

$ for f in /dev/stdin /dev/stdout /dev/stderr; do echo "$f -> $(readlink "$f")"; done
/dev/stdin -> /proc/self/fd/0
/dev/stdout -> /proc/self/fd/1
/dev/stderr -> /proc/self/fd/2

$ { echo "to fd 1"; echo "to fd 2" > /dev/stderr } > out.txt 2> err.txt; echo "out.txt: $(cat out.txt)"; echo "err.txt: $(cat err.txt)"
out.txt: to fd 1
err.txt: to fd 2

$ { echo first; echo err > /dev/stderr; echo last } > log.txt 2>&1; cat -v log.txt
err
^@^@last

$ { echo first; echo err >&2; echo last } > log.txt 2>&1; cat -v log.txt
first
err
last

$ { print first; print -u2 err; print last } > log.txt 2>&1; cat -v log.txt
first
err
last

Verified output of redir_descriptors_zsh.zsh on macOS — regenerated by tools/run_examples.py, never hand-typed.

$ for f in /dev/stdin /dev/stdout /dev/stderr; do echo "$f -> $(readlink "$f")"; done
/dev/stdin -> fd/0
/dev/stdout -> fd/1
/dev/stderr -> fd/2

$ { echo "to fd 1"; echo "to fd 2" > /dev/stderr } > out.txt 2> err.txt; echo "out.txt: $(cat out.txt)"; echo "err.txt: $(cat err.txt)"
out.txt: to fd 1
err.txt: to fd 2

$ { echo first; echo err > /dev/stderr; echo last } > log.txt 2>&1; cat -v log.txt
first
err
last

$ { echo first; echo err >&2; echo last } > log.txt 2>&1; cat -v log.txt
first
err
last

$ { print first; print -u2 err; print last } > log.txt 2>&1; cat -v log.txt
first
err
last

print -u2 is zsh's own way to write to descriptor 2. Like >&2 it opens nothing, and the log is complete on both systems.

fish adds a twist of its own, and that part is the same on both systems:

Verified output of redir_descriptors_fish.fish on Linux — regenerated by tools/run_examples.py, never hand-typed.

$ for f in /dev/stdin /dev/stdout /dev/stderr; echo "$f -> "(readlink $f); end
/dev/stdin -> /proc/self/fd/0
/dev/stdout -> /proc/self/fd/1
/dev/stderr -> /proc/self/fd/2

$ begin; echo "to fd 3" >&3; end 3> three.txt; cat three.txt
to fd 3

$ begin; echo first; echo err >&2; echo last; end > log.txt 2>&1; cat -v log.txt
first
err
last

$ begin; echo first; echo err > /dev/stderr; echo last; end > log.txt 2>&1

$ cat -v log.txt
first
last

$ cat -v fish_stderr.txt
err

$ sh -c "echo first; echo err > /dev/stderr; echo last" > log.txt 2>&1; cat -v log.txt
err
^@^@last

Verified output of redir_descriptors_fish.fish on macOS — regenerated by tools/run_examples.py, never hand-typed.

$ for f in /dev/stdin /dev/stdout /dev/stderr; echo "$f -> "(readlink $f); end
/dev/stdin -> fd/0
/dev/stdout -> fd/1
/dev/stderr -> fd/2

$ begin; echo "to fd 3" >&3; end 3> three.txt; cat three.txt
to fd 3

$ begin; echo first; echo err >&2; echo last; end > log.txt 2>&1; cat -v log.txt
first
err
last

$ begin; echo first; echo err > /dev/stderr; echo last; end > log.txt 2>&1

$ cat -v log.txt
first
last

$ cat -v fish_stderr.txt
err

$ sh -c "echo first; echo err > /dev/stderr; echo last" > log.txt 2>&1; cat -v log.txt
first
err
last

A 2> on a begin … end block catches what the block writes with >&2. But > /dev/stderr inside the block is opened by the fish process itself, and fish's own descriptor 2 was never moved. So err missed log.txt altogether and went to the child fish's own stderr, which the script had pointed at fish_stderr.txt. An external program started from fish (sh -c) does get both descriptors pointed at the log, and then the platform difference comes back. In fish, >&2 is the only spelling that means "this block's stderr".

	bash	zsh	fish
write to stderr, safely	`>&2`	`>&2`, `print -u2`	`>&2`
`> /dev/stderr` under `> log 2>&1`	Linux truncates the log; a Mac appends	the same as bash	inside a block: fish's own stderr, not the log

If you are coming from another library¶

Encodings. A pipe is not a terminal ↗ is the other question a program asks about descriptor 1, whether it is a terminal, and what changes with the answer.
Encodings. touch, : > and install are not three spellings of one command ↗ measures truncation from the other side: a descriptor opened before a file is emptied is left reading the emptied file.
Rust. Standard error, and exit status ↗ is descriptors 1 and 2 from inside a program: what belongs on each.
Python. Standard in, standard out, and pipes ↗ outlines sys.stdout, the file object Python wraps around descriptor 1.

Descriptors 0, 1 and 2¶

Measured¶

On a Mac¶

In zsh and fish¶

If you are coming from another library¶

See also¶