Skip to content

Search and replace

One line: A substitution has three parts that each vary by engine — whether the default replaces the first match or all of them, what language the replacement string is written in, and what an empty match does — and the one rule that holds everywhere is that anything beyond inserting $1 belongs in a replacement function, not a replacement string.

Level: 201 · you are about to call sub, replace, gsub, replaceAll or s///

The three questions

Question Python Perl JavaScript Ruby Java Go Rust
Default: first or all? all (re.sub) first (s///), all with /g first (replace), all with g or replaceAll first (sub), all (gsub) first (replaceFirst), all (replaceAll) all (ReplaceAllString) first (replace), all (replace_all)
Template language \1, \g<name> $1, $+{name}, \U…\E $1, $<name>, $& \1, \k<name>, \0 $1, ${name} $1, ${name} $1, ${name}
A function instead? re.sub(p, f, s) s/…/f()/e s.replace(re, f) s.gsub(re) { } m.replaceAll(f) (Java 9+) ReplaceAllStringFunc replace_all(re, f) where f: Fn(&Captures)

Python and Go default to all; the other five default to first. A one-line "fix the typo" script ported from JavaScript to Python replaces every occurrence, and one ported the other way replaces one.

Measured

Verified output of topic_search_and_replace_sh.sh — regenerated by tools/run_examples.py, never hand-typed.

construct                python      perl            node        ruby        java        go          rust        fancy
first occurrence only    f0o boo     f0o boo         f0o boo     f0o boo     f0o boo     f0o boo     f0o boo     f0o boo
$1 in the template       $2 at $1    host at me      host at me  $2 at $1    host at me  host at me  host at me  host at me
\1 in the template       host at me  \u{2} at \u{1}  \\2 at \\1  host at me  2 at 1      \\2 at \\1  \\2 at \\1  \\2 at \\1
\U$1 uppercases in Perl  err         WORD            \\Uword     \\U$1       Uword       \\Uword     \\Uword     \\Uword
empty match replaced     -ab         -ab             -ab         -ab         -ab         -ab         -ab         -ab
lookahead keeps context  NN EUR      NN EUR          NN EUR      NN EUR      NN EUR      -           -           NN EUR
\K keeps the prefix      -           price=NN        no          price=NN    -           -           -           price=NN

Reading the table

  • Row 1. The driver asks every engine for a first-only replacement, so all eight give f0o boo. In your own code, the default decides this — see the table above.
  • Rows 2–3. The $1/\1 split in its most common form, swapping two fields. Python and Ruby paste $2 at $1 in as text; Perl turns \2 into a control character; Java turns it into the digit 2. The replacement references page has fourteen rows of this.
  • Row 4, \U$1. Perl's template can change case — \U uppercases to the next \E or the end — and the result is WORD. Python refuses the escape (err); Ruby keeps the template as text; Java reads \U as an escaped U and produces Uword; JavaScript, Go and Rust keep \Uword. Case-changing in a replacement is a Perl-only template feature. Everywhere else it is a function.
  • Row 5, the empty match. x* matches the empty string at offset 0 in every engine, so replacing its first match with - produces -ab. With replace-all, this becomes -a-b-: an empty match at every position. This is the row behind "my replace inserted the text between every character".
  • Row 6, lookahead. \d+(?= EUR) replaced with NN gives NN EUR — the context was checked and left in place, because the assertion consumed nothing. Go and Rust's regex have no lookahead; the alternative there is to capture the context and put it back: (\d+)( EUR) → NN$2.
  • Row 7, \K. price=\K\d+ replaced with NN gives price=NN in Perl, Ruby and fancy-regex — the prefix was matched, then dropped from what the replacement touches. JavaScript compiled it as price=K\d+ and found nothing to replace.

When to use a function

Whenever the replacement depends on the match in any way a template cannot express: changing case outside Perl, arithmetic on a captured number, looking a captured key up in a table, or escaping. Templates are for reordering and decorating, and every engine's template language fails silently on anything else — it substitutes nothing, or pastes the template through as text. A function gets the match as a value and returns a string, and there is nothing to misspell.

The one thing to know about the function form: in JavaScript, s.replace(re, f) passes f the whole match, then each group, then the offset, then the string — positional, so a pattern with a variable number of groups makes f's signature wrong.

Keywords on this page

Replacement references · Capturing group · Lookahead · \K · *, +, ? — for the empty match.

See also