Skip to content

chomp knows about CRLF

Level: 101 · anyone reading files written on Windows

One line: With no argument Ruby's chomp removes \r\n, \n or \r, where Perl's chomp removes only \n; but Ruby's lines and each_line still split after \n alone, so a lone \r stays inside a line unless you split on \R.

Verified output of chomp_knows_crlf_rb.rb — regenerated by tools/run_examples.py, never hand-typed.

chomp with no argument:
  "unix\n".chomp                           "unix"
  "windows\r\n".chomp                      "windows"
  "old mac\r".chomp                        "old mac"
  "no ending".chomp                        "no ending"
  "blank line after\n\n".chomp             "blank line after\n"

With an argument:
  "blank line after\n\n".chomp("")         "blank line after"
  "windows\r\n".chomp("\n")                "windows"
  "windows\r\n".chomp("\r")                "windows\r\n"

Splitting text with mixed endings:
  text.lines                               ["one\r\n", "two\rthree\n", "four"]
  text.each_line(chomp: true).to_a         ["one", "two\rthree", "four"]
  text.split(/\R/)                         ["one", "two", "three", "four"]
  text.encode(universal_newline: true)     "one\ntwo\nthree\nfour"

Reading the output

  • chomp with no argument removes one line ending, of any of the three kinds, and nothing else: "blank line after\n\n" keeps its second \n.
  • chomp("") removes every trailing newline — paragraph mode, named after Perl's $/ = "".
  • chomp("\n") still removes \r\n; chomp("\r") removes only a final \r, and "windows\r\n" does not end in one.
  • lines and each_line split after each \n. each_line(chomp: true) strips the \r\n from "one\r\n" but leaves "two\rthree" as a single line.
  • split(/\R/) treats all three endings as line breaks, and encode(universal_newline: true) rewrites them all to \n.

What to do

For a line read from a file, chomp is enough. For text whose line endings you do not know — pasted input, an old Mac file — normalise first with encode(universal_newline: true), or split with /\R/.

If you are coming from another language

Perl's chomp removes the current value of $/, which is "\n":

Verified output of chomp_knows_crlf_pl.pl — regenerated by tools/run_examples.py, never hand-typed.

  chomp on "unix\n"                 "unix"
  chomp on "windows\r\n"            "windows\r"
  chomp on "old mac\r"              "old mac\r"
  chomp on "blank line after\n\n"   "blank line after\n"
  s/\R\z// on "windows\r\n"         "windows"
  local $/ = ""; chomp              "blank line after"

So a file from Windows, read line by line, keeps a \r at the end of every line, and it turns up at the end of the last field of every split. s/\R\z// is the Perl spelling of Ruby's chomp. The Perl library's chomp leaves the CR ↗ shows where that \r ends up in a CSV row.

The Python library's What ends a line ↗ compares splitlines with Ruby's $/ and \R; the Rust library's RFC 1212 ↗ is the story of str::lines learning about \r\n; the Encodings library's CRLF vs LF ↗ has the bytes.