`\G` — where the last match ended¶

One line: \G succeeds only at the position where the previous match on the same subject stopped, which turns a pattern run in a loop into a tokenizer that cannot skip anything — and it exists in Perl, Ruby, Java and fancy-regex, is a letter in JavaScript, and is absent from Python, Go and Rust's regex.

Level: reference · one construct, and the loop it is for

What it means¶

Every other anchor is about the subject. \G is about the match history: it is true at the offset where the last successful match ended, or at the start when there has been none. On its own it is \A. In a loop it is the difference between "find the next token" and "find a token somewhere after here".

That distinction is the whole reason to know it. A loop of find() calls without \G skips over any text no alternative matches, silently; with \G, unmatched text is a failure at a known offset.

Measured¶

Verified output of kw_continue_anchor_sh.sh — regenerated by tools/run_examples.py, never hand-typed.

construct         python  perl  node  ruby  java  go  rust  fancy
\G at the start   -       a     no    a     a     -   -     a
\G in JavaScript  -       no    Ga    no    no    -   -     no

Perl, Ruby, Java and fancy-regex have it. JavaScript's cells are the identity-escape story again: \Ga matches the text Ga. Python, Go and Rust's regex refuse — Python offers pattern.match(subject, pos) instead, which anchors at pos the same way, and Go and Rust offer nothing (a tokenizer there is a for loop over slices, which is the honest answer anyway).

A tokenizer, in Perl¶

Verified output of kw_continue_anchor_pl.pl — regenerated by tools/run_examples.py, never hand-typed.

ID(x) OP(=) NUM(42) OP(+) ID(y7)
without \G: x = 42 y7
with \G:    x = 42 ERROR at 7

The first line is a working lexer: four token kinds and whitespace, in one while with /gc. The c matters — it stops a failed match from resetting pos(), so the next alternative is tried from the same place.

The second and third lines are the point. The subject x = 42 ? y7 contains a character no alternative accepts. Without \G the loop reports x = 42 y7 — four tokens, the ? gone without a trace, because each //g match is free to start anywhere after the last one. With \G the loop reports ERROR at 7. A lexer that swallows characters it does not understand is not a lexer.

The same loop in Java¶

Verified output of kw_continue_anchor_java.java — regenerated by tools/run_examples.py, never hand-typed.

with \G:    x = 42 ERROR at 6
without \G: x = 42 y7

Matcher.find(int) resets and searches from an offset; the check m.start() == at is what \G enforces inside the pattern, and the pattern carries it too. Java's error offset is 6 where Perl's was 7 because this pattern absorbs the whitespace inside the token (\G\s*(…)) rather than in a separate step — the reported position is where the token was expected to start.

The trap¶

Reaching for \G in an engine that has no such thing, and getting no error. JavaScript is the one here; the y (sticky) flag is its actual equivalent — re.sticky = true makes exec succeed only at lastIndex, which is \G as a flag rather than an escape.

\G — where the last match ended¶