Skip to content

Choosing an engine

One line: You rarely choose an engine — the language chooses it for you — but you always choose which features to depend on, and the useful question is which of the four families you are in, because that predicts what the engine has, what it refuses, and whether hostile input can hang it.

Level: 201 · you are starting something, or picking a crate

The four families

Family Members Has Refuses Guarantee
Perl / PCRE2 Perl, PCRE2, PHP, ABAP (PCRE, 7.55+), R (perl=TRUE), grep -P everything: recursion, conditionals, \K, verbs, atomic, possessive, \X, \p{…} nothing much none — match and depth limits instead
The middle ground Java, Ruby, JavaScript, .NET lookaround, backreferences, named groups, most of \p{…}; each missing a different corner (JavaScript: no inline flags, no \A; Ruby: (?s); Java: (?#…)) recursion (except Ruby's \g<0>), verbs, \K (except Ruby) none
Python re Python lookaround (fixed width), backreferences, conditionals, atomic and possessive since 3.11 forward references, \p{…}, (?<name>…), recursion, variable-width lookbehind none
Automata Go regexp, Rust regex, RE2, Hyperscan \p{…} in depth, named groups, leftmost-first or leftmost-longest on request backreferences, lookaround, atomic, possessive, \K — everything that needs backtracking linear in the subject, always

And a fifth that is not a language: POSIX (grep, sed, awk, C's <regex.h>), with leftmost-longest matching, no lazy quantifiers, no lookaround, and BRE/ERE spellings.

Measured

The engines chapter runs eleven constructs through five of these engines; the table is regenerated on every push:

Verified output of capability_probe_sh.sh — regenerated by tools/run_examples.py, never hand-typed.

does it COMPILE?           python  perl   node   ruby   java  
backreference              yes     yes    yes    yes    yes   
named backref, \k<w>       no      yes    yes    yes    yes   
named backref, (?P=w)      yes     yes    no     no     no    
forward reference          no      yes    yes    yes    yes   
lookahead                  yes     yes    yes    yes    yes   
lookbehind, fixed width    yes     yes    yes    yes    yes   
lookbehind, two widths     no      yes    yes    yes    yes   
lookbehind, variable width no      yes    yes    no     yes   
atomic group               yes     yes    no     yes    yes   
possessive quantifier      yes     yes    no     yes    yes   
recursion                  no      yes    no     no     no    

and does it MATCH 'aax' with the forward reference ^(?:\1x|(a))+$ ?
python  compile error
perl    yes
node    yes
ruby    yes

The "yes" cells mean compiled — and three of the constructs on this shelf are accepted by an engine while meaning something else, which is why the keyword pages ask "what did it match" as well.

The decision

  1. Does untrusted input reach the pattern? Then either the engine is an automaton (Go, Rust's regex, RE2) or the pattern has been checked for the catastrophic shape and the engine has a limit you have set (PCRE2, fancy-regex, Ruby's Regexp.timeout). Java and JavaScript have no limit; a bad pattern there holds the thread.
  2. Do you need backreferences or lookaround? Then not an automaton — or, in Rust, fancy-regex, deliberately, knowing it gives up the guarantee. Most patterns that seem to need them do not.
  3. Will the pattern live outside the code — a config file, a database column, a FIND … PCRE statement? Then it needs inline flags and (?x), which rules out JavaScript and Go as the only consumer, and it needs the portability checklist if two programs will read it.
  4. Is the text non-ASCII? Then check the family's default for \w, \d, \b and (?i) — Unicode text — and prefer an engine with \p{…}, which rules Python's re down to str methods or the third-party regex module.
  5. Is it a one-off on the command line? Then it is POSIX unless you say -P, and the command-line tools page has the three things that differ.

For ABAP specifically

ABAP is in the Perl/PCRE2 family, which is the most useful single fact about it: a pattern that works in pcre2grep, in Perl, or in a PCRE-flavour online tester is the best available prediction. A pattern that works in Python is not. The older REGEX addition is POSIX-flavoured and obsolete; moving a pattern from REGEX to PCRE deserves a test, because leftmost-longest and leftmost-first give different answers to a|ab.

Keywords on this page

The whole Keywords shelf is organised by construct with one column per engine; this page is the same information organised by engine. Backreference, lookahead, atomic group and recursion are the four whose column of - cells defines the families.

See also