Skip to content

stderr does not go down the pipe

Level: 101 · you have piped a command into grep or wc -l, and its error message still landed on your screen

One line: | connects the left command's stdout and nothing else, so its error messages go straight to the terminal and the next command never sees them. 2>&1 | sends both streams down the pipe in bash, zsh and fish. bash 4 and zsh also accept |&, which the Mac's bash 3.2 rejects as a syntax error. fish spells it &| and rejects |&, and in zsh &| is not a pipe at all.

Measured

cat nosuch distros writes two lines on stdout and one error message on stderr. sed puts piped: in front of every line it reads, so a line without the prefix did not come through the pipe. The script sends its own stderr to the same place as its stdout, so the page shows both, in the order they arrived:

Verified output of stderr_pipe_sh.sh, identical on Linux and macOS — regenerated by tools/run_examples.py, never hand-typed.

$ printf "alma\ndarkfi\n" > distros

$ cat nosuch distros | sed "s/^/piped: /"
cat: nosuch: No such file or directory
piped: alma
piped: darkfi

$ cat nosuch distros 2>&1 | sed "s/^/piped: /"
piped: cat: nosuch: No such file or directory
piped: alma
piped: darkfi

$ cat nosuch distros | sed "s/^/piped: /" 2>&1
cat: nosuch: No such file or directory
piped: alma
piped: darkfi

$ cat nosuch distros 2>&1 >/dev/null | sed "s/^/piped: /"
piped: cat: nosuch: No such file or directory

$ cat nosuch distros | grep -c "No such"
cat: nosuch: No such file or directory
0

$ cat nosuch distros 2>&1 | grep -c "No such"
1
  • The error has no prefix. The pipe replaced cat's stdout and left its stderr where it was, on the terminal (here, the runner). sed read two lines, not three.
  • 2>&1 | puts the error in the pipe. The shell connects the pipe before it applies the command's own redirections, so 2>&1 points stderr at what stdout is by then: the pipe.
  • Where 2>&1 sits matters. At the end of the line it belongs to sed, not to cat, and the error is back on the terminal.
  • 2>&1 >/dev/null | sends only stderr down the pipe. stderr goes to the pipe first, then stdout goes to /dev/null. The two data lines are gone, and the error is the only line sed saw. Why the order of 2>&1 and > matters belongs to Redirection.
  • The cost of forgetting. | grep -c "No such" counted 0, with the message on the screen right above the count. Searching a program's output for its errors finds none when the program writes its errors on stderr, as cat does.

On a Mac

cat's message and every line above are the same on both machines, and 2>&1 | works in bash 3.2. The short spelling does not:

Verified output of stderr_pipe_bar_amp_sh.sh on Linux — regenerated by tools/run_examples.py, never hand-typed.

$ echo "bash ${BASH_VERSINFO[0]}"
bash 5

$ bash -c 'cat nosuch distros |& sed "s/^/piped: /"'
piped: cat: nosuch: No such file or directory
piped: alma
piped: darkfi
(exit status 0)

$ bash -c 'f() { cat nosuch distros |& sed "s/^/piped: /"; }; declare -f f'
f () 
{ 
    cat nosuch distros 2>&1 | sed "s/^/piped: /"
}
(exit status 0)

Verified output of stderr_pipe_bar_amp_sh.sh on macOS — regenerated by tools/run_examples.py, never hand-typed.

$ echo "bash ${BASH_VERSINFO[0]}"
bash 3

$ bash -c 'cat nosuch distros |& sed "s/^/piped: /"'
bash: -c: line 0: syntax error near unexpected token `&'
bash: -c: line 0: `cat nosuch distros |& sed "s/^/piped: /"'
(exit status 2)

$ bash -c 'f() { cat nosuch distros |& sed "s/^/piped: /"; }; declare -f f'
bash: -c: line 0: syntax error near unexpected token `&'
bash: -c: line 0: `f() { cat nosuch distros |& sed "s/^/piped: /"; }; declare -f f'
(exit status 2)

bash 4.0 added |&. On Linux, bash 5 ran it, and declare -f shows how bash stores it: as 2>&1 |, so it is another spelling and nothing more. On the Mac, /bin/bash 3.2 stopped at the & with a syntax error and ran nothing, not even the cat: exit status 2. So a script that uses |& and works on Linux fails on a Mac whose bash is /bin/bash, and 2>&1 | works on both. Each line runs in a child bash, so the error is reported the way you would see it at a prompt, without ending this script.

In zsh and fish

zsh accepts 2>&1 | and |&, and stores |& as 2>&1 | as well. Two of its lines do something else:

Verified output of stderr_pipe_zsh.zsh, identical on Linux and macOS — regenerated by tools/run_examples.py, never hand-typed.

$ printf "alma\ndarkfi\n" > distros

$ cat nosuch distros | sed "s/^/piped: /"
cat: nosuch: No such file or directory
piped: alma
piped: darkfi

$ cat nosuch distros |& sed "s/^/piped: /"
piped: cat: nosuch: No such file or directory
piped: alma
piped: darkfi

$ cat nosuch distros 2>&1 | sed "s/^/piped: /"
piped: cat: nosuch: No such file or directory
piped: alma
piped: darkfi

$ cat nosuch distros 2>&1 >/dev/null | sed "s/^/piped: /"
piped: cat: nosuch: No such file or directory
piped: alma
piped: darkfi

$ setopt no_multios; cat nosuch distros 2>&1 >/dev/null | sed "s/^/piped: /"; setopt multios
piped: cat: nosuch: No such file or directory

$ f() { cat nosuch distros |& sed "s/^/piped: /" }; functions f
f () {
	cat nosuch distros 2>&1 | sed "s/^/piped: /"
}

$ g() { cat nosuch distros &| sed "s/^/piped: /" }; functions g
g () {
	cat nosuch distros &|
	sed "s/^/piped: /"
}
  • 2>&1 >/dev/null | put everything in the pipe, the two data lines too, where bash sent only the error. zsh's MULTIOS option, on in zsh -f, makes a stdout that is redirected twice (here to /dev/null and to the pipe) go to both. After setopt no_multios the line does what it does in bash. tee is where MULTIOS comes back, since it is a tee built into the shell.
  • &| is not a pipe in zsh. functions g prints how zsh parsed the fish spelling: cat nosuch distros &| is one command, which zsh runs in the background and disowns, and sed is a second, separate command with nothing piped into it. The example prints the parse rather than running it, because a background job's output would race the rest of the page.

fish spells both streams &|, accepts 2>&1 |, and refuses |& with a message that names the fish spelling:

Verified output of stderr_pipe_fish.fish, identical on Linux and macOS — regenerated by tools/run_examples.py, never hand-typed.

$ printf "alma\ndarkfi\n" > distros

$ cat nosuch distros | sed "s/^/piped: /"
cat: nosuch: No such file or directory
piped: alma
piped: darkfi

$ cat nosuch distros &| sed "s/^/piped: /"
piped: cat: nosuch: No such file or directory
piped: alma
piped: darkfi

$ cat nosuch distros 2>&1 | sed "s/^/piped: /"
piped: cat: nosuch: No such file or directory
piped: alma
piped: darkfi

$ cat nosuch distros 2>&1 >/dev/null | sed "s/^/piped: /"
piped: cat: nosuch: No such file or directory

$ cat nosuch distros |& sed "s/^/piped: /"
fish: |& is not valid. In fish, use &| to pipe both stdout and stderr.
cat nosuch distros |& sed "s/^/piped: /"
                   ^^
(exit status 127)

2>&1 >/dev/null | sends only stderr, as in bash. |& is a parse error, reported before the line runs, with exit status 127, so the example runs it in a child fish.

bash 3.2 bash 5 zsh fish
both streams down the pipe 2>&1 \| 2>&1 \| or \|& 2>&1 \| or \|& 2>&1 \| or &\|
only stderr down the pipe 2>&1 >/dev/null \| 2>&1 >/dev/null \| the same, after setopt no_multios 2>&1 >/dev/null \|
the spelling that goes wrong \|&, a syntax error &\|, which backgrounds the left side \|&, a parse error

If you are coming from another library

  • Rust. Standard error, and exit status ↗ is the same split from the program's side: which stream a message belongs on, why a warning printed on stdout turns into data for the next command, and 2>&1 > f against > f 2>&1.

See also