(?i) is a folding table, not an instruction to ignore case¶
One line: (?i) asks the engine to apply its own case-folding table, and the tables disagree — (?i)[a-z] accepts U+212A KELVIN SIGN in Python, Perl, Ruby, Go, Rust and PCRE2, refuses it in JavaScript without u and in Java without UNICODE_CASE, and (?i)ss matches U+00DF LATIN SMALL LETTER SHARP S in Perl and Ruby and in none of the others.
Level: 301 · you use (?i) on text other people typed
Keywords on this page: (?i) · […] · \1 · (*UCP) — Topics: Unicode text · Security
Two rows in a product table look identical. One holds a lowercase k; the other holds U+212A KELVIN SIGN, the temperature symbol, which most fonts draw as a capital K and which Python's str.lower() turns into a plain k. Whether your (?i) deduplication rule collapses those two rows is not a question about your pattern. It is a question about which engine you handed the pattern to — and the answer differs between Python and Java, between JavaScript with and without one flag, and between Perl and the library Perl's syntax was copied into.
Nobody here is wrong, because (?i) has no fixed meaning. Every engine picks a case-folding table, and the tables were picked at different times for different reasons.
Three characters that expose the table¶
These three are the standard probes, chosen because their mappings are long-standing — no engine below can blame a Unicode version lag for its answer.
| Code point | Name | The fold that matters |
|---|---|---|
| U+212A | KELVIN SIGN | folds to U+006B, a plain k — so (?i)k and a temperature symbol can be the same string |
| U+017F | LATIN SMALL LETTER LONG S | folds to U+0073, a plain s — the long s of eighteenth-century printing, which arrives in quantity from OCR |
| U+00DF / U+1E9E | LATIN SMALL / CAPITAL LETTER SHARP S | simple folding relates the two; full folding maps both to ss |
Simple folding and full folding¶
Unicode defines case folding in one file, CaseFolding.txt ↗, where each line carries a status letter:
- C — common, one code point to one code point.
- S — simple, the one-to-one alternative where a full fold would expand.
- F — full, one code point to one or more: U+00DF folds to
ss, U+0130 folds toi+ U+0307. - T — Turkic, the locale-dependent exception. Everybody ignores it; see the Turkish I below.
Simple folding is C + S, and it is what a regular expression engine can implement without thinking: one character in the pattern, one character in the subject, a table lookup between them. Full folding is C + F, and it breaks that assumption — a single pattern character would have to consume two subject characters, which a character class or a state machine has no shape for.
So the prediction is that no regex engine does full folding. The prediction is wrong twice: Perl and Ruby both do, measured below. PCRE2 — Perl's own library — does not.
Meanwhile the host language's string functions frequently do full folding or full case mapping, in the same program, over the same two characters. str.casefold() in Python, String#downcase(:fold) in Ruby, str::to_uppercase in Rust and String.toUpperCase in Java all say U+00DF and ss are the same text. Their regex engines mostly say they are not. Every example below ends with that comparison, because it is the trap worth the whole page: one program, two answers, and only one of them is in the pattern.
Python: Unicode by default, ASCII on request, never in bytes¶
Verified output of case_fold_py.py — regenerated by tools/run_examples.py, never hand-typed.
does (?i) match? python 3, re.fullmatch
pattern subject str re.A bytes
k U+004B LATIN CAPITAL LETTER K yes yes yes
k U+212A KELVIN SIGN yes no no
s U+017F LATIN SMALL LETTER LONG S yes no no
U+00DF U+1E9E LATIN CAPITAL LETTER SHARP S yes no no
U+1E9E U+00DF LATIN SMALL LETTER SHARP S yes no no
ss U+00DF LATIN SMALL LETTER SHARP S no no no
[k] U+212A KELVIN SIGN yes no no
[a-z] U+212A KELVIN SIGN yes no no
i U+0131 LATIN SMALL LETTER DOTLESS I yes no no
i U+0130 LATIN CAPITAL LETTER I WITH DOT ABOVE yes no no
i U+0307 U+0130 LATIN CAPITAL LETTER I WITH DOT ABOVE no no no
does (?i) fold the BACKREFERENCE comparison too?
(?i)(a)\1 vs 'a' + U+0041 yes
(?i)(k)\1 vs 'k' + U+212A yes
one program, two answers about the same pair
str.casefold: U+00DF == 'ss' True re (?i): no
str.casefold: U+212A == 'k' True re (?i): yes
str.casefold: U+017F == 's' True re (?i): yes
str.lower(U+1E9E) == U+00DF True re (?i): yes
str.upper(U+00DF) is 2 code points True re (?i): no
normalise first, then fold -- what the advice at the end of the page does
NFKC(U+212A) = U+004B and casefolded = U+006B
NFKC(U+017F) = U+0073 and casefolded = U+0073
NFKC(U+00DF) = U+00DF and casefolded = U+0073 U+0073
NFKC(U+1E9E) = U+1E9E and casefolded = U+0073 U+0073
re.IGNORECASE on a str pattern folds the whole Unicode simple table: U+212A matches k, U+017F matches s, and the sharp-s pair folds both ways. Add re.ASCII, or hand re a bytes pattern, and every non-ASCII row turns to no — a bytes pattern is not a stricter fold, it is a different alphabet, where U+212A is three bytes that k was never going to match.
The ss row is the gap. str.casefold() says U+00DF and ss are the same string; re with re.IGNORECASE says the pattern does not match. One program printed both answers, a dozen lines apart, and nothing in the code warns you that casefold() and (?i) are different functions of the same idea.
The last block is the fix this page ends with: unicodedata.normalize("NFKC", …) turns U+212A into U+004B and U+017F into U+0073 before any pattern sees them, and casefold() finishes the job on U+00DF.
Perl: full folding, and the modifier that stops it¶
Verified output of case_fold_pl.pl — regenerated by tools/run_examples.py, never hand-typed.
does (?i) match? perl 5, anchored
pattern subject /i /ia /iaa
k U+004B LATIN CAPITAL LETTER K yes yes yes
k U+212A KELVIN SIGN yes yes no
s U+017F LATIN SMALL LETTER LONG S yes yes no
U+00DF U+1E9E LATIN CAPITAL LETTER SHARP S yes yes yes
U+1E9E U+00DF LATIN SMALL LETTER SHARP S yes yes yes
ss U+00DF LATIN SMALL LETTER SHARP S yes yes no
[k] U+212A KELVIN SIGN yes yes no
[a-z] U+212A KELVIN SIGN yes yes no
i U+0131 LATIN SMALL LETTER DOTLESS I no no no
i U+0130 LATIN CAPITAL LETTER I WITH DOT ABOVE no no no
i U+0307 U+0130 LATIN CAPITAL LETTER I WITH DOT ABOVE yes yes no
does (?i) fold the BACKREFERENCE comparison too?
(?i)(a)\1 vs 'a' + U+0041 yes
(?i)(k)\1 vs 'k' + U+212A yes
one program, two answers about the same pair
fc: U+00DF eq 'ss' true regex /i: yes
fc: U+212A eq 'k' true regex /i: yes
fc: U+017F eq 's' true regex /i: yes
lc(U+1E9E) eq U+00DF true regex /i: yes
uc(U+00DF) is 2 code points true regex /i: yes
Perl's /i is the most thorough on this page: ss matches U+00DF, SS matches it too, and i + U+0307 matches U+0130. That is full folding in the matcher, and fc — Perl's own fold-case function — agrees with the engine on every row. Perl is internally consistent in a way most of the others are not.
The two right-hand columns are the restriction knob, and the subtlety is which one works. /a changes nothing: it restricts \d, \w, \s and \b to ASCII, not the folding. You need the doubled /aa, which forbids folds that cross the ASCII boundary — so k stops matching U+212A, while U+00DF and U+1E9E keep folding to each other, both being non-ASCII. That is a precise rule, and it is the same rule PCRE2 spells (*CASELESS_RESTRICT).
JavaScript: /i and /iu are two different engines¶
Verified output of case_fold_js.js — regenerated by tools/run_examples.py, never hand-typed.
does /i match? javascript, anchored
pattern subject /i /iu
k U+004B LATIN CAPITAL LETTER K yes yes
k U+212A KELVIN SIGN no yes
s U+017F LATIN SMALL LETTER LONG S no yes
U+00DF U+1E9E LATIN CAPITAL LETTER SHARP S no yes
U+1E9E U+00DF LATIN SMALL LETTER SHARP S no yes
ss U+00DF LATIN SMALL LETTER SHARP S no no
[k] U+212A KELVIN SIGN no yes
[a-z] U+212A KELVIN SIGN no yes
i U+0131 LATIN SMALL LETTER DOTLESS I no no
i U+0130 LATIN CAPITAL LETTER I WITH DOT ABOVE no no
i U+0307 U+0130 LATIN CAPITAL LETTER I WITH DOT ABOVE no no
does /i fold the BACKREFERENCE comparison too?
(a)\1 /i vs 'a' + U+0041 yes
(k)\1 /iu vs 'k' + U+212A yes
one program, two answers about the same pair
toUpperCase(U+00DF) === 'SS' true regex /iu: no
toLowerCase(U+212A) === 'k' true regex /iu: yes
toLowerCase(U+212A) === 'k' true regex /i: no
toUpperCase(U+017F) === 'S' true regex /iu: yes
toLowerCase(U+1E9E) === U+00DF true regex /iu: yes
Without u, every non-ASCII row is no. The specification's canonicalization step uppercases the character and then throws the result away if it turned a non-ASCII character into an ASCII one — which is exactly what U+017F → S does — so the legacy /i is ASCII-only in practice for these characters. With u, canonicalization is Unicode simple case folding, and the same pattern text means something else.
/i and /iu are therefore not "the same regex, stricter parsing". A code base that adds u to its patterns for the sake of \u{…} escapes or \p{…} classes has silently widened every caseless match in it.
Ruby: full folding, like Perl¶
Verified output of case_fold_rb.rb — regenerated by tools/run_examples.py, never hand-typed.
does (?i) match? ruby, Onigmo, anchored
pattern subject utf-8 binary
k U+004B LATIN CAPITAL LETTER K yes yes
k U+212A KELVIN SIGN yes no
s U+017F LATIN SMALL LETTER LONG S yes no
U+00DF U+1E9E LATIN CAPITAL LETTER SHARP S yes no
U+1E9E U+00DF LATIN SMALL LETTER SHARP S yes no
ss U+00DF LATIN SMALL LETTER SHARP S yes no
[k] U+212A KELVIN SIGN yes no
[a-z] U+212A KELVIN SIGN yes no
i U+0131 LATIN SMALL LETTER DOTLESS I no no
i U+0130 LATIN CAPITAL LETTER I WITH DOT ABOVE no no
i U+0307 U+0130 LATIN CAPITAL LETTER I WITH DOT ABOVE yes no
does (?i) fold the BACKREFERENCE comparison too?
(?i)(a)\1 vs 'a' + U+0041 yes
(?i)(k)\1 vs 'k' + U+212A yes
one program, two answers about the same pair
casecmp?: U+00DF and 'ss' true regex //i: yes
downcase(:fold) U+00DF == 'ss' true regex //i: yes
casecmp?: U+212A and 'k' true regex //i: yes
casecmp?: U+017F and 's' true regex //i: yes
downcase U+1E9E == U+00DF true regex //i: yes
upcase U+00DF is 2 code points true regex //i: yes
Onigmo folds fully: ss matches U+00DF, and i + U+0307 matches U+0130. If you came here believing that no production regex engine implements full case folding — the belief this page started from — Ruby is the counterexample, and casecmp? agrees with it on every row.
The binary column is Ruby's version of Python's bytes: a pattern and a subject in ASCII-8BIT fold only ASCII. In Ruby the switch is the string's encoding, not a flag on the pattern, which means it is decided wherever that string was read, far from the regex.
Java: two flags, and an asymmetry nobody documents¶
Verified output of case_fold_java.java — regenerated by tools/run_examples.py, never hand-typed.
does (?i) match? java.util.regex, anchored
pattern subject (?i) (?iu)
k U+004B LATIN CAPITAL LETTER K yes yes
k U+212A KELVIN SIGN no yes
s U+017F LATIN SMALL LETTER LONG S no yes
U+00DF U+1E9E LATIN CAPITAL LETTER SHARP S no no
U+1E9E U+00DF LATIN SMALL LETTER SHARP S no yes
ss U+00DF LATIN SMALL LETTER SHARP S no no
[k] U+212A KELVIN SIGN no yes
[a-z] U+212A KELVIN SIGN no yes
i U+0131 LATIN SMALL LETTER DOTLESS I no yes
i U+0130 LATIN CAPITAL LETTER I WITH DOT ABOVE no yes
i U+0307 U+0130 LATIN CAPITAL LETTER I WITH DOT ABOVE no no
does (?i) fold the BACKREFERENCE comparison too?
(?i)(a)\1 vs 'a' + U+0041 yes
(?iu)(k)\1 vs 'k' + U+212A yes
one program, two answers about the same pair
toUpperCase(U+00DF, ROOT) is 'SS' true regex (?iu): no
equalsIgnoreCase(U+00DF, 'ss') false regex (?iu): no
equalsIgnoreCase(U+212A, 'k') true regex (?iu): yes
toLowerCase(U+1E9E, ROOT) is U+00DF true regex (?iu): no
how one character is folded onto another: lower(upper(cp))
U+00DF toUpperCase U+00DF then toLowerCase U+00DF
U+1E9E toUpperCase U+1E9E then toLowerCase U+00DF
U+0131 toUpperCase U+0049 then toLowerCase U+0069
U+0130 toUpperCase U+0130 then toLowerCase U+0069
the locale the regex engine refuses to have
toLowerCase('I', Locale.ROOT) U+0069
toLowerCase('I', Locale tr) U+0131
toUpperCase('i', Locale tr) U+0130
(?iu)i vs U+0131 yes
(?iu)I vs U+0131 yes
Pattern.CASE_INSENSITIVE alone is ASCII-only — the (?i) column is no on every non-ASCII row. Adding Pattern.UNICODE_CASE, which is what (?iu) spells inline, turns the Unicode table on. Java is the one engine here that puts the Unicode question in a separate flag from the caseless question, so Pattern.compile(p, Pattern.CASE_INSENSITIVE) — written from memory, in thousands of code bases — is an ASCII-only match that looks like a Unicode-aware one.
Then the row that has no counterpart anywhere else on this page: pattern U+00DF does not match subject U+1E9E, but pattern U+1E9E matches subject U+00DF. The lower(upper(cp)) block explains it. Java folds a character by uppercasing it and lowercasing the result, and for U+00DF both steps are the identity — as a pattern character it has no case pair at all, so it is matched literally whatever the flags say. U+1E9E goes to U+00DF, so as a pattern character it folds, and it folds onto the character that refuses to fold back. Caseless matching in Java is not symmetric, and a test that checks one direction will not discover it.
Go: simple folding, and a standard library that agrees with it¶
Verified output of case_fold_go.go — regenerated by tools/run_examples.py, never hand-typed.
does (?i) match? go regexp (RE2), anchored
pattern subject (?i)
k U+004B LATIN CAPITAL LETTER K yes
k U+212A KELVIN SIGN yes
s U+017F LATIN SMALL LETTER LONG S yes
U+00DF U+1E9E LATIN CAPITAL LETTER SHARP S yes
U+1E9E U+00DF LATIN SMALL LETTER SHARP S yes
ss U+00DF LATIN SMALL LETTER SHARP S no
[k] U+212A KELVIN SIGN yes
[a-z] U+212A KELVIN SIGN yes
i U+0131 LATIN SMALL LETTER DOTLESS I no
i U+0130 LATIN CAPITAL LETTER I WITH DOT ABOVE no
i U+0307 U+0130 LATIN CAPITAL LETTER I WITH DOT ABOVE no
does (?i) fold the BACKREFERENCE comparison too?
(?i)(a)\1 vs 'a' + U+0041 error parsing regexp: invalid escape sequence: `\1`
one program, two answers about the same pair
EqualFold: U+00DF and 'ss' false regex (?i): no
EqualFold: U+212A and 'k' true regex (?i): yes
EqualFold: U+017F and 's' true regex (?i): yes
ToLower(U+1E9E) == U+00DF true regex (?i): yes
ToUpper(U+00DF) is 2 code points false regex (?i): no
RE2 folds the full simple table, including inside [a-z], and strings.EqualFold — which walks the same folding orbit — gives the identical answer on every row. Go is the tidiest engine here: one table, used by the regexp package and the strings package alike, with no full folding anywhere in the standard library to disagree with it.
(?i)(a)\1 is not a question Go answers, because RE2 has no backreferences at all. See what a backreference costs for why that is a deliberate trade.
Rust: regex folds simply, fancy-regex forgets the backreference¶
Verified output of case_fold_rs_sh.sh — regenerated by tools/run_examples.py, never hand-typed.
does (?i) match? rust crates, anchored
pattern subject regex fancy
k U+004B LATIN CAPITAL LETTER K yes yes
k U+212A KELVIN SIGN yes yes
s U+017F LATIN SMALL LETTER LONG S yes yes
U+00DF U+1E9E LATIN CAPITAL LETTER SHARP S yes yes
U+1E9E U+00DF LATIN SMALL LETTER SHARP S yes yes
ss U+00DF LATIN SMALL LETTER SHARP S no no
[k] U+212A KELVIN SIGN yes yes
[a-z] U+212A KELVIN SIGN yes yes
i U+0131 LATIN SMALL LETTER DOTLESS I no no
i U+0130 LATIN CAPITAL LETTER I WITH DOT ABOVE no no
i U+0307 U+0130 LATIN CAPITAL LETTER I WITH DOT ABOVE no no
does (?i) fold the BACKREFERENCE comparison too?
regex (?i)(a)\1 error: backreferences are not supported
fancy-regex (?i)(a)\1 vs 'a' + U+0041 yes
fancy-regex (?i)(k)\1 vs 'k' + U+212A no
one program, two answers about the same pair
to_uppercase(U+00DF) is 'SS' true regex (?i): no
eq_ignore_ascii_case(U+00DF, 'ss') false regex (?i): no
to_lowercase(U+212A) is 'k' true regex (?i): yes
to_uppercase(U+017F) is 'S' true regex (?i): yes
to_lowercase(U+1E9E) is U+00DF true regex (?i): yes
The regex crate implements Unicode simple folding, and fancy-regex — which delegates any pattern it has no special features for straight to regex — matches it on every literal row.
The backreference is where they part company, and it is the sharpest finding on this page. (?i)(a)\1 matches aA, so fancy-regex clearly folds the backreference comparison. (?i)(k)\1 does not match k followed by U+212A. Its backreference comparison is ASCII-caseless while the rest of its matching is Unicode-caseless. Every other engine measured here folds a backreference with the same table it uses for a literal.
Rust's str methods do full case mapping — to_uppercase on U+00DF really is SS — so, like Python, a Rust program can hold both answers at once.
PCRE2 — and therefore ABAP¶
ABAP's PCRE addition is the PCRE2 library in the kernel, so PCRE2's table is ABAP's table. PCRE2 is on any machine with pcre2grep, and -q makes it answer only with an exit status: 0 a line matched, 1 none did, 2 the pattern was rejected.
Real run — PCRE2 10.48 (pcre2grep), macOS 26.6.2 x86-64, 2026-09-22. Each subject file holds one character, written as UTF-8 bytes so the transcript stays ASCII.
$ printf '\xe2\x84\xaa\n' > kelvin.txt
$ printf '\xc3\x9f\n' > sharp_s.txt
$ printf '\xe1\xba\x9e\n' > cap_sharp_s.txt
$ printf '\xc4\xb1\n' > dotless_i.txt
$ pcre2grep -q -i 'k' kelvin.txt ; echo $?
1
$ pcre2grep -q -u -i 'k' kelvin.txt ; echo $?
0
$ pcre2grep -q -u -i '(*UCP)k' kelvin.txt ; echo $?
0
$ pcre2grep -q -u -i '[a-z]' kelvin.txt ; echo $?
0
$ pcre2grep -q -u -i '(*CASELESS_RESTRICT)k' kelvin.txt ; echo $?
1
$ pcre2grep -q -u -i '\x{00DF}' cap_sharp_s.txt ; echo $?
0
$ pcre2grep -q -u -i '(*CASELESS_RESTRICT)\x{00DF}' cap_sharp_s.txt ; echo $?
0
$ pcre2grep -q -u -i 'ss' sharp_s.txt ; echo $?
1
$ pcre2grep -q -u -i 'i' dotless_i.txt ; echo $?
1
$ pcre2grep -q -u -i '(*TURKISH_CASING)I' dotless_i.txt ; echo $?
0
Four things come out of that, and the first one corrected this page's first draft:
- UTF mode is the switch, not
(*UCP). With-u, caselesskalready folds onto U+212A, and adding(*UCP)changes nothing.(*UCP)decides what\w,\dand\bmean — that is what\wand\bmatch, a different lesson — and it is the first thing reached for when a caseless match misbehaves. Here it changes nothing at all. (*CASELESS_RESTRICT)(PCRE2 10.43 and later) is Perl's/aaunder another name, and the two sharp-s lines prove it is the same rule:kstops folding onto U+212A because that fold crosses the ASCII boundary, while U+00DF and U+1E9E keep folding onto each other because neither is ASCII.- PCRE2 does simple folding only.
ssdoes not match U+00DF, so PCRE2 and Perl disagree on the one pattern people most often carry between them. (*TURKISH_CASING)exists, and it works — see the next section.
In ABAP¶
z_case_insensitive.prog.abap in full — pasted here by tools/run_examples.py from the file CI runs.
REPORT z_case_insensitive.
" Case-insensitive matching in ABAP, whose PCRE addition (7.55 and later) is the
" PCRE2 library in the kernel. Nothing in this repository can run ABAP, so this
" file is parsed by abaplint and shown on the page as source only -- there is no
" recorded output beside it, and none of the comments below is a measured claim
" about a real system.
DATA(text) = `Kelvin`.
" Two ways to ask for caseless matching, and they are not the same question.
" 1. IGNORING CASE is ABAP's own addition. It governs the WHOLE statement: there
" is no way to spell "this part caselessly, that part exactly".
FIND FIRST OCCURRENCE OF PCRE `kelvin` IN text IGNORING CASE.
ASSERT sy-subrc = 0.
" 2. (?i) is PCRE2's, so it can be scoped. (?i:...) makes one group caseless and
" leaves the rest of the pattern case-sensitive -- and a pattern that arrives
" from configuration carries its own flags with it, which IGNORING CASE in the
" calling code cannot take back.
FIND FIRST OCCURRENCE OF PCRE `(?i:kel)vin` IN text.
ASSERT sy-subrc = 0.
" Neither addition settles WHICH characters count as the same letter. That is
" the kernel's PCRE2 case-folding table, and it is the table -- not ABAP -- that
" decides whether the two patterns below find anything:
"
" \x{212A} KELVIN SIGN, which folds to a plain k in PCRE2's UTF mode
" ss which does NOT match U+00DF: PCRE2 does simple folding, not full
"
" A pattern may be written entirely in ASCII, because PCRE2's \x{...} escape
" names a code point -- worth doing in ABAP, where a literal in a transport is
" at the mercy of every editor it passes through.
FIND FIRST OCCURRENCE OF PCRE `(?i)\x{212A}` IN text.
IF sy-subrc = 0.
WRITE / `the kernel folds U+212A onto k`.
ELSE.
WRITE / `the kernel does not`.
ENDIF.
" (*CASELESS_RESTRICT) -- keep ASCII and non-ASCII apart while still folding --
" needs PCRE2 10.43 or later in the kernel, so whether it compiles is a question
" about your system, not about ABAP. Check before relying on it.
ABAP gives you two ways to ask, and only one of them is PCRE2's. IGNORING CASE is the statement's own addition: it governs the whole FIND or REPLACE, and there is no way to spell "this part caselessly, the rest exactly". (?i) is PCRE2's, so it can be scoped — (?i:…) applies to one group. The consequence worth remembering is the direction of that power: a pattern that arrives from customising or from a Z-table carries its own (?i) with it, and RESPECTING CASE in the calling code cannot take it back.
What neither addition decides is which characters are the same letter. That is the kernel's PCRE2 table, and the transcript above is the best available prediction of it. Writing \x{212A} instead of the character itself keeps an ABAP pattern in 7-bit ASCII, which is worth doing for anything that travels in a transport.
Inside a character class, and on a backreference¶
Two questions every engine answers and almost none documents prominently.
A character class folds too — including a range. Every engine that folds at all folds inside […], so (?i)[a-z] matched U+212A in Python, Perl, Ruby, Go, both Rust crates, PCRE2, JavaScript with u and Java with UNICODE_CASE. This is the one with security consequences: an allowlist written ^(?i)[a-z0-9_]+$ is not an ASCII allowlist. The measured row shows U+212A getting past it, and U+017F reaches s through the same table. Some later step — a lower(), an NFKC normalisation, a database collation — then turns them into k and s, and two identifiers that passed validation separately collide afterwards.
A backreference is compared with the same table, everywhere here except one engine: fancy-regex folds (?i)(a)\1 but not (?i)(k)\1 over U+212A. Go's regexp and Rust's regex have no backreferences to fold.
The Turkish I: the case no table can win¶
In most locales i pairs with I. In Turkish and Azerbaijani it does not: i pairs with U+0130 LATIN CAPITAL LETTER I WITH DOT ABOVE, and I pairs with U+0131 LATIN SMALL LETTER DOTLESS I. No single code-point table can serve both, which is why CaseFolding.txt marks those mappings T and why regex engines, which have no locale, leave them out.
Leaving them out is not one behaviour, though — it is three, and the measured rows show all of them:
- Python and Java
(?iu)foldionto both U+0130 and U+0131. The widest net: a patternimatches four different characters, two of which are the very characters a Turkish reader would call not ani. Python ships the extra equivalences as a table inre._casefix; Java arrives there by itslower(upper(cp))rule, which the block above shows sending both U+0131 and U+0130 to U+0069. - Perl, Ruby, Go, Rust, JavaScript and PCRE2 by default fold neither.
imatchesiandI, and the dotless U+0131 from a Turkish keyboard is simply a different letter. - PCRE2 alone offers the locale, opt-in:
(*TURKISH_CASING)(10.45 and later) makesImatch U+0131, and leaves it unmatched without the directive.
The Java block shows why no table can be right: "I".toLowerCase(Locale.ROOT) is U+0069 and "I".toLowerCase(tr) is U+0131. Same input, two correct answers, chosen by a locale the regex engine deliberately does not have. If your text is Turkish, the case decision belongs in your code, where a reviewer can see which locale you passed.
The measured summary¶
Every cell below is a line from one of the blocks above; a dash means the runs above do not answer that question for that mode.
| Engine and mode | k vs U+212A |
[a-z] vs U+212A |
U+00DF ↔ U+1E9E | ss vs U+00DF |
backref folds non-ASCII | i vs U+0131 |
|---|---|---|---|---|---|---|
Python re, str |
yes | yes | both ways | no | yes | yes |
Python re.ASCII, or bytes |
no | no | no | no | — | no |
Perl /i |
yes | yes | both ways | yes | yes | no |
Perl /iaa |
no | no | both ways | no | — | no |
| Ruby, UTF-8 | yes | yes | both ways | yes | yes | no |
Ruby, ASCII-8BIT |
no | no | no | no | — | no |
JavaScript /i |
no | no | no | no | — | no |
JavaScript /iu |
yes | yes | both ways | no | yes | no |
Java (?i) |
no | no | no | no | — | no |
Java (?iu) |
yes | yes | one way only | no | yes | yes |
Go regexp |
yes | yes | both ways | no | no backreferences | no |
Rust regex |
yes | yes | both ways | no | no backreferences | no |
Rust fancy-regex |
yes | yes | both ways | no | no | no |
PCRE2 -u, ABAP PCRE |
yes | yes | both ways | no | yes | no |
PCRE2 (*CASELESS_RESTRICT) |
no | — | both ways | — | — | — |
Versions: Python 3.14 and 3.12, Perl 5.42 and 5.38, Node 20 and 24, Ruby 4.0 and 3.4, Go 1.25, Java 25, regex 1.13 with fancy-regex 0.16, PCRE2 10.48. Each program prints the same bytes on both versions of its runtime — that was checked, because a page full of yes and no is worthless if the answer moves with a patch release.
The rule of thumb¶
For an identifier, a username, a file name, a blocklist or anything else feeding a security decision, do not let (?i) be your equality test. Normalise and fold in the host language, with a function a reviewer can name — unicodedata.normalize("NFKC", s).casefold() in Python, downcase(:fold) in Ruby, fc in Perl, toLowerCase(Locale.ROOT) in Java, to_lowercase() in Rust, strings.ToLower in Go — and then let the regex match the already-normalised form, case-sensitively.
That move buys three things the flag cannot. The folding table becomes visible in the code and versioned with it. The answer stops changing when the pattern is ported to another engine, which is the whole subject of this library. And the dedup bug at the top of the page turns into one line you can write a test for, instead of a flag whose meaning depends on which of the fifteen rows above your runtime happens to hold.
See also¶
- What
\wand\bmatch — the other half of "is this engine Unicode-aware by default", and the lesson(*UCP)really belongs to. - Unicode properties,
\p{…}— naming characters by property instead of by case. - Unicode — the chapter these three lessons belong to.
- Who supports what — the same engines, measured on eleven other constructs.
- What a backreference is — the construct whose caseless comparison split the engines above.
- UTS #18, Unicode Regular Expressions ↗ — what a "level 1" caseless match is supposed to mean, and why simple folding is the level most engines stop at.
pcre2pattern↗ — PCRE2's own account of(*UCP),(*CASELESS_RESTRICT)and(*TURKISH_CASING).