Case is not a per-character operation¶
Level: 201 · for anyone who has written .upper() on a name
One line: Uppercasing is not a table of one character per character — it can make a string longer, it can depend on what is next to a letter, and its correct answer can depend on a language your program was never told.
'ß'.upper() # 'SS' <- one character in, two out
'ΟΔΟΣ'.lower() # 'οδος' <- ends U+03C2, because that sigma is word-final
'ΟΔΟΣΑ'.lower() # 'οδοσα' <- same sigma, one letter after it, U+03C3 now
'i'.upper() # 'I' <- and in Turkish that is the wrong letter
Everybody's first model of case is a lookup table: a→A, b→B, sixty-odd rows, one character in and one character out. That model is what toupper() in C is, what a CHAR(20) column assumes, and what a for loop over characters quietly encodes. It is wrong in three separate directions, and each one breaks a different assumption.
| the assumption it breaks | the character that breaks it | |
|---|---|---|
| length | one in, one out | ß → SS |
| position | the answer is a property of the character | Σ → σ or ς |
| language | there is one right answer | i → I or İ |
None of the three is exotic. German, Greek and Turkish are not edge cases; they are three of the languages an ordinary European system holds names in.
Length: the string gets bigger¶
ß is in the cast for exactly this. It has no single uppercase letter — the uppercase of sharp s is SS, two letters — so .upper() returns a longer string than it was given.
That breaks more than a buffer. It breaks invertibility: 'ß'.upper().lower() is 'ss', not 'ß', because nothing in the output records which pair of esses used to be one letter. Case mapping is a one-way function on strings, and treating it as a round trip is a bug that only shows up on German input.
Position: the same letter, two lowercase forms¶
Greek writes sigma one way at the end of a word (ς) and another way everywhere else (σ), and has only one uppercase form for both. So lowercasing Σ is not a lookup at all — it is a question about the characters on either side.
This is the direction that makes the page's title literally true. There is no per-character table that can hold the answer, because the answer is not a property of the character.
Language: the answer depends on something the string does not carry¶
Turkish has two i's — dotted i/İ and dotless ı/I — and they case in pairs that cut across the Latin ones. In Turkish, i uppercases to İ and I lowercases to ı. In every other language using the Latin alphabet, i uppercases to I.
Python's str.upper() is locale-independent by design, so it gives the language-neutral answer and gives it always. That refusal is the lesson, and it is worth seeing rather than working around: setting the locale changes nothing.
LC_ALL=tr_TR.UTF-8 python3 -c "import locale; locale.setlocale(locale.LC_ALL, ''); print('i'.upper())"
That prints I — a correctly set Turkish locale, actively applied, and str.upper() does not look. The standard library does ship locale-sensitive text operations; they are locale.strcoll and locale.strxfrm, and they are about sorting. So the language will order your text the way your country does and refuses to case it that way.
The asymmetry is deliberate, and the argument for it is a good one: a wrong sort order is visible to whoever reads the list, while a wrong case fold silently merges two identifiers or splits one. Guessing is worse than refusing. What follows from it is that if your text is Turkish, .upper() is not the function you want, and no configuration will make it be.
In Python¶
Verified output of case_is_not_per_character_py.py — regenerated by tools/run_examples.py, never hand-typed.
1. LENGTH: ONE CHARACTER IN, TWO CHARACTERS OUT
------------------------------------------------------------------------
'ß' LATIN SMALL LETTER SHARP S
operation result chars bytes code points
the character 'ß' 1 2 00DF
.upper() 'SS' 2 2 0053 0053
.casefold() 'ss' 2 2 0073 0073
.capitalize() 'Ss' 2 2 0053 0073
.title() 'Ss' 2 2 0053 0073
So a buffer sized for the input is too small for the output, and
`for c in s: out.append(c.upper())` is not a mistake a type system
catches -- it type-checks, and it is wrong for German.
And it does not come back:
'ß'.upper().lower() -> 'ss' same as the input? False
Case mapping is not invertible. There was one sharp s and there
are now two esses, and nothing records which esses used to be one.
2. POSITION: THE SAME CHARACTER, TWO LOWERCASE FORMS
------------------------------------------------------------------------
Greek sigma is written differently at the end of a word:
U+03A3 GREEK CAPITAL LETTER SIGMA
U+03C3 GREEK SMALL LETTER SIGMA
U+03C2 GREEK SMALL LETTER FINAL SIGMA
One uppercase letter, so `.lower()` has to decide by looking around it.
input lower() final form? why
'ΟΔΟΣ' 'οδος' yes U+03C2 at the end of a word
'ΟΔΟΣ Μ' 'οδος μ' yes U+03C2 before a space: still the end
'ΟΔΟΣ.' 'οδος.' yes U+03C2 before a full stop: still the end
'ΟΔΟΣΑ' 'οδοσα' no U+03C3 a letter follows, so NOT the end
'Σ' 'σ' no U+03C3 nothing precedes it, so not a final anything
'ΑΣ' 'ας' yes U+03C2 one letter is enough to make it final
Read row 4 against row 1. The sigma is the same character in the
same word; one letter after it changes what it lowercases to. No
per-character table can express that, because the answer is not a
property of the character.
3. AND THE TWO OPERATIONS DISAGREE ON PURPOSE
------------------------------------------------------------------------
'ΟΔΟΣ' .lower() -> 'οδος' 03BF 03B4 03BF 03C2
'ΟΔΟΣ' .casefold() -> 'οδοσ' 03BF 03B4 03BF 03C3
.lower() gives the FINAL sigma; .casefold() gives the plain one.
That is not an inconsistency, it is the two jobs:
.lower() is for text a person will READ -- keep the
distinction Greek spelling makes
.casefold() is for text a program will COMPARE -- destroy every
distinction that must not separate two equal strings
The proof that they are different questions:
'οδος' == 'οδοσ' -> False
'οδος'.casefold() == 'οδοσ'.casefold() -> True
Two spellings of one Greek word. Only one of the two questions
answers `the same word` -- and it is not the one called `lower`.
And a third function answers a third way. A regex told to ignore
case compares ONE character with ONE character, so the length axis
from section 1 is invisible to it:
re.fullmatch('ß', 'SS', re.IGNORECASE) -> False
re.fullmatch('SS', 'ß', re.IGNORECASE) -> False
'ß'.casefold() == 'SS'.casefold() -> True
casefold() calls them equal and the matcher cannot, in either
direction: a comparison made one character at a time has no way
to say that one character equals two.
4. LOCALE: THE ANSWER DEPENDS ON A LANGUAGE PYTHON NEVER ASKED FOR
------------------------------------------------------------------------
Turkish has two i's, and they case in pairs that cross the
Latin ones:
char code pt name upper() lower()
'i' U+0069 LATIN SMALL LETTER I 'I' 'i'
'I' U+0049 LATIN CAPITAL LETTER I 'I' 'i'
'İ' U+0130 LATIN CAPITAL LETTER I WITH DOT ABOVE 'İ' 'i̇'
'ı' U+0131 LATIN SMALL LETTER DOTLESS I 'I' 'ı'
In Turkish, `i` uppercases to U+0130 (dotted) and `I` lowercases
to U+0131 (dotless). The table above does neither. Python's answer
is the language-neutral one, and it is wrong for Turkish text --
correctly wrong, because it was never told the text was Turkish.
That refusal is total, and it is the design. `str.upper()` reads
no environment variable, and setting one changes nothing:
LC_ALL=tr_TR.UTF-8 python3 -c "print('i'.upper())" -> 'I'
The standard library HAS locale-sensitive text operations -- they
are in `locale`, and they are about sorting:
locale.strcoll, locale.strxfrm <- collation, locale-sensitive
str.upper, str.lower <- casing, locale-INDEPENDENT
So the language will sort your text the way your country does and
refuses to case it that way. The asymmetry is deliberate: a wrong
sort order is visible, and a wrong case fold silently merges two
identifiers or splits one.
The length trap is here too, in a mapping nobody expects:
'İ'.lower() -> 'i̇' 1 char in, 2 out 0069 0307
U+0307 is a COMBINING DOT ABOVE. Lowercasing grew the string, and
the second character is a mark that renders on top of the first.
5. EVERY ANSWER ABOVE IS A BET, AND HERE IS THE SIZE OF IT
------------------------------------------------------------------------
Every mapping on this page came from `str`, which reads whichever
Unicode table this interpreter was built with. So each one is a
fact about a machine, and this library's rule is that such facts
do not become answer keys.
The usual escape is the frozen table Python ships beside the live
one -- `unicodedata.ucd_3_2_0`, sealed at Unicode 3.2 in 2002. Ask
it for a normalization and the answer cannot vary by machine:
'café' NFKC frozen 'café' live 'café'
'fi' NFKC frozen 'fi' live 'fi'
That escape is not available here, and the reason is the finding:
functions on the frozen table that concern case: 0
It carries name, category, combining class, decomposition,
normalize, bidirectional, east_asian_width and more -- and not one
case mapping among them. Case lives on `str`, and `str` has exactly
one table: the current one.
So this page records `ß`.upper() == 'SS', the two sigmas and the
four Turkish i's as keys knowing it cannot prove them stable from
inside Python. The bet is that these particular entries are load-
bearing in every text stack on earth and will not move. If one ever
does, this page's key goes red on the next run -- which is the
correct outcome, and the reason the bet is worth making out loud
rather than by saying nothing.
In Rust¶
Rust puts the whole lesson in a return type. char::to_uppercase does not return a char — it cannot, because 'ß' has no single uppercase character — so it returns an iterator, and every caller is made to deal with that at compile time.
Then the sharper half, which is the same fact one level up: char::to_lowercase and str::to_lowercase give different answers for the same text, and the char one is wrong. Not from a bug — a char is one code point with no left and no right, so the question is this sigma final is one it cannot be asked.
Verified output of case_is_not_per_character_rs.rs — regenerated by tools/run_examples.py, never hand-typed.
1. THE RETURN TYPE IS THE LESSON
------------------------------------------------------------------------
fn to_uppercase(self) -> ToUppercase // not -> char
fn to_lowercase(self) -> ToLowercase // not -> char
Both are iterators of `char`. A function returning one `char`
would be unimplementable, and the reason is a single German
letter:
char up yields code points low yields code points
'ß' 2 "SS" 0053 0053 1 "ß" 00DF
'ffi' 3 "FFI" 0046 0046 0049 1 "ffi" FB03
'İ' 1 "İ" 0130 2 "i\u{307}" 0069 0307
'A' 1 "A" 0041 1 "a" 0061
Sharp s yields two going up. The `ffi` ligature yields three,
which is the ceiling std documents. And U+0130 yields one going
up and TWO going down -- so neither direction is safe to assume,
and `.count()` is never a constant.
2. THE SAME TEXT, TWO METHODS, TWO ANSWERS
------------------------------------------------------------------------
input "ΟΔΟΣ" 039F 0394 039F 03A3
str::to_lowercase "οδος" 03BF 03B4 03BF 03C2
chars().flat_map(to_lowercase) "οδοσ" 03BF 03B4 03BF 03C3
equal? false
The two expressions look like the same operation written two
ways, and the second is the one a reviewer would call the
explicit version. It ends in U+03C3 where the correct Greek
ends in U+03C2, GREEK SMALL LETTER FINAL SIGMA.
`char::to_lowercase` is not wrong. It is asked a question that
has no answer: a `char` is one code point with no left and no
right, and whether a sigma is final is a fact about its
neighbours. std says so in its own documentation and always
returns the non-final form.
And `str::to_lowercase` only gets it right by looking:
ΟΔΟΣ -> οδος 03BF 03B4 03BF 03C2 sigma at the end of the word
ΟΔΟΣΑ -> οδοσα 03BF 03B4 03BF 03C3 03B1 same sigma, one letter added after it
3. THE PART std WILL NOT DO, AND SAYS SO
------------------------------------------------------------------------
Turkish lowercases `I` to `ı` (dotless) and uppercases `i` to
`İ` (dotted). Rust gives the language-neutral answer:
'I'.to_lowercase() -> "i"
'i'.to_uppercase() -> "I"
...and its own docs state that this holds across languages, on
purpose. There is no locale argument to pass, because `str` has
no notion of what language it is holding.
Nor is the ASCII shortcut a way out. It is a different function
with a different promise, and the promise is in the name:
text ascii_upper to_uppercase same?
"odos" "ODOS" "ODOS" true
"straße" "STRAßE" "STRASSE" false
"café" "CAFé" "CAFÉ" false
Row 2 is the one to keep. `to_ascii_uppercase` produced
"STRAßE" -- a lowercase German letter standing in the middle
of five capitals -- and did not fail, did not warn, and returned
a perfectly good String. A function that silently declines to do
its job on some of your data is only safe because it said `ascii`
in its name, which is the entire argument for the name.
4. WHAT THE STANDARD LIBRARY HANDS TO A CRATE
------------------------------------------------------------------------
std does the case axis in full and the LANGUAGE axis not at all,
and it names the crate that does: `icu_casemap`, from Unicode.
It also suggests writing the sigma rule yourself, using two
`char` predicates -- `is_cased` and `is_case_ignorable`. On the
stable compiler this file is built with, both are unstable and
the code does not compile. The advice is real; the ingredients
are nightly-only. The dated note on the page has the versions.
So the split, in one line each:
length std handles it, and the return type proves it
position std handles it on `str` and cannot on `char`
language std refuses, deliberately, and points at a crate
"ΟΔΟΣ".chars().flat_map(|c| c.to_lowercase()).collect::<String>() is the version a reviewer would call explicit, and it is the version that is wrong. Reach for str::to_lowercase unless you have a reason not to.
What the crate adds¶
std's own documentation names what it does not do, and points at the library that does:
rustc 1.98.0 (88d9e12ae 2026-08-18)
core::char::to_lowercase documents the sigma rule, always returns U+03C3,
and refers the reader to icu_casemap for
locale-dependent casing
char::is_cased unstable — feature `titlecase`, tracking #153892
char::is_case_ignorable unstable — feature `case_ignorable`, tracking #154848
char::to_titlecase unstable — feature `titlecase`, tracking #153892
Read the middle two rows together with the first. std suggests implementing the final-sigma rule yourself out of is_cased and is_case_ignorable — and on the stable compiler this library builds with, both are nightly-only, so that advice does not compile. The suggestion is sound and the ingredients are not on the shelf yet.
What ICU ↗ adds over all of this is the axis std deliberately refuses: a locale argument. icu_casemap takes a language identifier, so tr gets Turkish's dotted İ, el gets Greek's rules for accents on uppercase, and lt gets Lithuanian's retained dots. That is not a small table on top of std's — it is the whole reason the crate exists, and it is the same shape as the gap named at the foot of "Handles Unicode" is four questions.
Lower is for reading, casefold is for comparing¶
The two operations disagree about sigma on purpose, and once you see why, the choice between them stops being a matter of taste:
| what it is for | 'ΟΔΟΣ' becomes |
|
|---|---|---|
.lower() |
text a person will read — keep the distinction Greek spelling makes | οδος (final ς) |
.casefold() |
text a program will compare — destroy every distinction that must not separate two equal strings | οδοσ (plain σ) |
So 'οδος' == 'οδοσ' is False and 'οδος'.casefold() == 'οδοσ'.casefold() is True. Two spellings of one Greek word, and only one of the two functions answers the same word — the one not called lower. That is the same decision Preparing a string makes at protocol scale: fold to compare, store what they typed.
A regex asked to ignore case is a third function, and it answers a third way. It compares one character with one character, so re.fullmatch('ß', 'SS', re.IGNORECASE) is False in both directions while casefold() calls the two equal — the length axis from the top of this page, invisible to the matcher. What re.IGNORECASE does with the four Turkish i's — one class, where casefold() makes three — is the matching half of this subject, and it is measured on "Supports Unicode" is a level, not a yes.
.NET cases one character at a time — with a locale¶
Python and Rust do the length and position axes and refuse the language one. .NET is the mirror image: it has the locale argument both of them refuse, and does neither of the other two. Measured by hand on 2026-09-11 with .NET 5.0.5 on macOS 26.6.2, where .NET cases through the system's ICU (CI cannot run .NET):
length: ß |
position: ΟΔΟΣ |
language: i for Turkish |
|
|---|---|---|---|
Python str |
SS |
οδος |
I — there is no locale argument |
Rust str |
SS |
οδος |
I — std points at icu_casemap |
.NET string |
ß — de-DE included |
οδοσ — el-GR included |
İ, given new CultureInfo("tr-TR") |
The length column is the signature Rust declined to write. Rune.ToUpperInvariant takes one Rune and returns one, so the German answer cannot be expressed and ß comes back as ß; string.ToUpperInvariant applies the same one-to-one mapping a character at a time, and across every scalar value it and ToLowerInvariant changed 1,504 and 1,487 characters and the length of none. "straße" becomes "STRAßE" — the string Rust's to_ascii_uppercase produced in section 3, from a method with no ascii in its name to warn you. Position goes the same way, since a mapping shown one character cannot see where a word ends. What .NET has instead is the axis the other two refuse: every casing method takes a CultureInfo, and tr-TR gives Turkish its dotted İ and dotless ı. One oddity sits on that axis. The invariant culture — the one for text with no language — leaves İ and ı untouched in both directions, where en-US lowercases the first to i and uppercases the second to I; the runtime's native casing code special-cases the pair "to match the Windows invariant behavior" (pal_casing.c ↗).
Comparing uses a third table. StringComparison.OrdinalIgnoreCase is what Microsoft's string guidance ↗ calls "your safe default for culture-agnostic string matching", and it is not the same as comparing ToUpperInvariant results. Since .NET 5 it will not match a non-ASCII letter to an ASCII one (dotnet/runtime#32247 ↗): ſ, U+017F LATIN SMALL LETTER LONG S, is not S, and U+212A KELVIN SIGN is not k — the fold that Canonicalize, then check turns into a bypass, refused at the comparison. And it compares by uppercasing, which cannot reunite two letters that only meet in lowercase: U+1E9E ẞ and ß are different to OrdinalIgnoreCase and equal to InvariantCultureIgnoreCase. Python's casefold() calls all three pairs equal, and straße equal to STRASSE, which neither .NET comparison does.
.NET 11 gives the third table a name. ToUpperOrdinal and ToLowerOrdinal, on char, string, Rune and spans, were approved in June 2026 ↗ and ship from .NET 11 preview 7 (#130140 ↗), because OrdinalIgnoreCase "is no longer equivalent" to comparing ToUpperInvariant results; the new methods are defined to agree with the comparison and to keep the length. How far apart the two tables are depends on the machine. On this one they disagree about 83 of the 1,504 characters ToUpperInvariant changes: ſ, by the ASCII rule, and 82 letters of Vithkuqi, Garay and Beria Erfe — scripts added in Unicode 14.0, 16.0 and 17.0, all after .NET 5 shipped. ToUpperInvariant asks macOS's ICU, which has them. .NET 5's OrdinalIgnoreCase cases letters above U+FFFF with a hand-written switch over the six scripts it knew — Deseret, Osage, Old Hungarian, Warang Citi, Medefaidrin and Adlam (OrdinalCasing.Icu.cs ↗). A case table written as a switch statement has a version too.
If you are coming from Python or ABAP¶
Python. str.upper(), str.lower(), str.casefold() and str.title() all operate on the whole string, which is what lets lower() get the final sigma right — so do not decompose them into a loop over characters to "make it explicit". casefold() is the one to reach for before any comparison; upper() is not a case fold and never was. And there is no locale-aware casing anywhere in the standard library, so if you need Turkish, you need PyICU ↗ or an explicit pre-map of the four i's.
ABAP. (Not machine-checked — CI cannot run ABAP.) TRANSLATE … TO UPPER CASE is the word that transfers, and the three traps transfer with it. It operates in place on a string or a c field, so the length problem is real in a way it is not in Python: on a fixed-width c field, a ß that wants to become SS has nowhere to put the second S — use a string, and expect the length to change. It is not locale-aware, so the Turkish case is wrong there too and for the same reason. And TRANSLATE is a display transformation; for comparison, upper-casing both sides is not case folding and will not reconcile two spellings of an accented name — that needs normalization first, which is the preparing a string problem, not this one. The rule worth carrying into an ABAP system: pick one method that every comparison path calls, and never upper-case at the point of comparison.
Try it¶
- Run
.upper()over a column of real surnames from your own system and comparelen()before and after. Every row where it grew is a row a fixed-width field would have truncated. - Find the widest
CHAR(n)column in your database that holds a name, and ask what your code does whenUPPER()of a value in it is longer thann. Some databases truncate, some raise, and both are worse than the answer you assumed. - Take any Greek text you have, run
.lower()on it, then run.lower()on it one character at a time, and diff the two. If it contains a sigma at the end of a word, the two differ. - Grep your codebase for
.upper()and.lower()and sort the hits into two piles: for display and for comparison. Every hit in the second pile should becasefold(). - If your system holds Turkish data, search for a case-insensitive comparison on a value that can start with
iorI— a username, a country code, a status flag — and work out what it does to a Turkish user.
See also¶
- Preparing a string — case folding as one step of four, and what a protocol does with it
- The table has a version — why a case mapping is a fact about your interpreter, and what may be written down
- A code point is not a character — the other place a per-character loop gives the wrong answer
- UTF-16 and surrogates — the per-character loop one level further down, where a
charis half a letter - Normalization — the transformation that has to happen before a comparison, not instead of one
- "Supports Unicode" is a level, not a yes — the matching half: what a regex engine does when told to ignore case
- The hard strings — the test corpus, with the sharp s, the two sigmas and the dotless i as rows
- "Handles Unicode" is four questions — where the locale axis sits in the larger picture
- CAST.md —
ßis in the cast for this page - SpecialCasing.txt ↗ — the file the length changes and the sigma rule actually live in; it is short and readable
- Why a
charis 32 bits wide ↗ — the sibling Rust library on what acharis, and therefore what it cannot know