Skip to content

Start here

Level: 101 · read this first

One line: Perl can do everything text asks of it, but its defaults were set before Unicode and kept for compatibility — so this library runs every example and shows you the default and the fix side by side, as bytes where bytes are the point.

This library is for someone who can already write a little Perl, or a lot of sed and awk, and has been surprised by text: a one-liner that worked on one file and emptied another, a name that came out as Kraków, a CSV row that lost a field, a \w that matched in one script and not in the next.

What it assumes

That you can read Perl syntax. That "code point" and "UTF-8" are words you have met — if they are not, the encodings library ↗ builds them from the bottom up, and every page here links the page there that it depends on.

How a lesson works

  • The page — one idea, a **One line:** that states the claim, then the evidence.
  • examples/ — the program or script behind the page. A block marked Verified output was pasted in by a tool from a real run, and CI runs it again on Ubuntu and on macOS on every push.
  • demo/ — input files, for the one-liner lessons: each command on those pages can be pasted into a terminal in that folder.
  • If you are coming from another language — the page in the Python, Java, C or Rust library that teaches the same idea, so the difference is one click away.

The five to read first

If you read nothing else, read these, in this order. Between them they explain most of the text bugs Perl programs ship with:

  1. use utf8 is about the source file — why length "zażółć" is 10.
  2. "Wide character in print" — why the warning is the good case.
  3. Decode at the edges — the five things one bad byte can become.
  4. split has sharp edges — the one you will hit this week.
  5. -i edits in place — before you point a one-liner at a file you care about.

Running an example yourself

git clone https://github.com/masiarek/perl-learning-library
cd perl-learning-library/01_One_Liners/n_and_p_are_a_loop/examples
bash n_and_p_loop_sh.sh
python3 ../../../tools/run_examples.py --only n_and_p_loop_sh

The third command prints what the page's verified block shows. The last one checks that output against the recorded answer key and tells you if your machine disagrees. You need perl 5.36 or later: on a Mac, /usr/bin/perl is 5.34, so use Homebrew's.

Perl's text scorecard, honestly

A summary of the whole library, so you know what you are walking into.

Genuinely good

  • The one-liner. -n, -p, -a, -F, -l, -i and -0 make a full language as terse as sed and awk, and — unlike sed -i — the same on macOS and Linux.
  • Everything Unicode asks for ships with perl: Encode, Unicode::Normalize, Unicode::Collate with per-language tailoring, grapheme clusters as \X, case folding as fc. Several of the sibling languages need a third-party library for the last three.
  • Decoding can be strict. decode('UTF-8', $bytes, FB_CROAK) throws on the first bad byte, and says which byte.
  • B::Deparse shows you the program perl compiled from your switches, so no switch has to be taken on trust.
  • The regex engine is the one the others copied — PCRE is Perl Compatible Regular Expressions — and /a, /aa and \R give precise control over what a class means.

Genuinely bad

  • A string does not know whether it is bytes or text. Perl tracks an internal storage form, not a meaning, and nothing stops a byte string and a character string from being joined.
  • A literal is bytes unless use utf8; a filehandle has no encoding unless you add one; and printing a string with no layer writes one byte or two for the same é depending on the rest of the string.
  • Without use v5.12 or later — and in every -e one-liner — \w and uc give different answers for the same string depending on how it is stored.
  • s/// returns a count, /g changes meaning with context, a failed match in an argument list passes nothing at all, and split limits itself to the number of variables you assign to.
  • + 0 reads "42abc" as 42 and carries on.

The pattern: as in the Java text library ↗, almost every flaw is a default kept for compatibility, with a correct alternative one pragma or one flag away. Perl rarely makes the right thing hard. It makes the old thing shorter.

The chapters

Chapter What it covers
01 — One-liners The loop -n and -p build, the fields -a and -F split, what -i writes back, and what -0 makes a record
02 — Unicode text Where bytes become text and back: use utf8, "Wide character", decoding, what length counts, and fc
03 — Regex The Unicode bug, /g in list and scalar context, the counts s/// and tr/// return, and split
04 — Records and fields The \r that chomp leaves, CSV, sorting for people, and numbers that arrive as text
05 — Resources The perldoc pages behind every lesson, and the sibling libraries

Planned

Rough order, not a promise:

  • pack and unpack — fixed-width records and binary formats. The encodings library's Packing a record ↗ already uses pack "N n d>", and a page here would take it apart.
  • Here-documents and formatted output — <<~, printf, and padding text whose characters are not all one column wide.
  • Regex, further — named captures, /x, qr// composed from pieces, and backtracking that never ends.
  • Perl One-Liners, the book — Peteris Krumins' 130 programs, re-run against this library's rules: which ones change behaviour on a UTF-8 file, and what -CSDA does to them.