The string module¶
Level: 201 · for Python programmers
One line: The string module is what was left over when str grew methods — nine ASCII-only constants that the is* predicates replaced, a capwords() that is not title(), a Template whose entire value is what it cannot reach, and a Formatter you will subclass once.
A module named string in a language whose strings need no module is a historical artifact, and saying so is more useful than touring it. Python 2's string module was where the string functions lived — string.split(s), string.upper(s) — because str had no methods to hold them. Once str did, the functions were duplicates, and Python 3 dropped them. What remains is what had nowhere else to go: twelve public names, and the program below prints all twelve, because that is the entire module.
They are worth twenty minutes anyway, for four unrelated reasons. The constants are ASCII by definition, which is the only reason to reach for one — you want the ASCII answer specifically, not the Unicode one, and str.isdigit() will not give it to you. Sorted by code point they also happen to be the ASCII chart, boundaries and all. capwords() is not str.title(), and the disagreement is not a rounding error: they decide where a word begins by different rules and produce different names. Template is the $-syntax that exists because % and .format are too powerful to hand to a user — a format string that arrived from outside your program is a small program, and this page measures exactly how far it reaches. Formatter is the subclass hook almost nobody needs, with one method on it that is genuinely useful on its own.
Two things this page does not re-explain: the format specification itself belongs to the format mini-language, and string.printable.isprintable() being False is unpacked on repr is not str.
Verified output of the_string_module_py.py — regenerated by tools/run_examples.py, never hand-typed.
1. THE WHOLE DRAWER
The string module has 12 public names, and that is all of it:
9 constants ascii_letters ascii_lowercase ascii_uppercase
digits hexdigits octdigits
printable punctuation whitespace
3 callables Formatter Template capwords
Not one of them does something to a string that a str method
does not do better. That is the shape of a module which used
to hold the string functions, back when str had no methods to
hold them itself.
2. THE CONSTANTS ARE AN ASCII TABLE, WRITTEN SIDEWAYS
ascii_lowercase 26 abcdefghijklmnopqrstuvwxyz
ascii_uppercase 26 ABCDEFGHIJKLMNOPQRSTUVWXYZ
ascii_letters 52 abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ
digits 10 0123456789
hexdigits 22 0123456789abcdefABCDEF
octdigits 8 01234567
punctuation 32 !"#$%&\'()*+,-./:;<=>?@[\\]^_`{|}~
whitespace 6 \x20\t\n\r\x0b\x0c
printable 100 = digits + ascii_lowercase + ascii_uppercase
+ punctuation + whitespace
-> that concatenation IS string.printable: True
Sort those 100 characters by code point and the drawer turns
into the ASCII chart, with every boundary computed, not typed:
code points n constant the characters
------------------------------------------------------------------
09-0D 5 whitespace \t\n\x0b\x0c\r
20 1 whitespace \x20
21-2F 15 punctuation !"#$%&\'()*+,-./
30-39 10 digits 0123456789
3A-40 7 punctuation :;<=>?@
41-5A 26 ascii_uppercase ABCDEFGHIJKLMNOPQRSTUVWXYZ
5B-60 6 punctuation [\\]^_`
61-7A 26 ascii_lowercase abcdefghijklmnopqrstuvwxyz
7B-7E 4 punctuation {|}~
Four runs of punctuation, and they are exactly the gaps left
between the digit block and the two letter blocks. Those two
blocks start 0x20 apart, which is why ASCII case is one bit
and not a lookup:
ord('a') - ord('A') = 32 = 0x20
chr(ord(c) | 0x20) lowercases all 26 of them: True
3. ASCII-ONLY BY DEFINITION -- WHICH IS THE ONLY REASON TO USE THEM
A constant is a membership test against a fixed list. The
methods that replaced them ask the Unicode table instead, and
the two answers part company on the first non-ASCII character:
in the constant what Unicode says
------------------------------------------------------------------
'\u0663' ARABIC-INDIC 3 False isdigit() True
'\xa0' NO-BREAK SPACE False isspace() True
'\u2019' RIGHT QUOTE False category Pf
'\u0141' L WITH STROKE False isalpha() True
string.whitespace holds 6; str.isspace() is True for 29 code points.
So the two spellings of 'strip the whitespace' are not one:
\xa0Zoot\x1c .strip() -> 'Zoot'
\xa0Zoot\x1c .strip(string.whitespace) -> '\xa0Zoot\x1c'
And 'punctuation' is a name from before the categories were:
P* 23 punctuation !"#%&\'()*,-./:;?@[\\]_{}
S* 9 symbol $+<=>^`|~
9 of the 32 are symbols, not punctuation, by Unicode's own
accounting. The constant is not wrong about that -- it is
answering an older question: 'which ASCII characters are
neither letters nor digits nor space'.
4. capwords() IS NOT title(), AND THE DIFFERENCE IS WHERE A WORD STARTS
The entire implementation, from Lib/string.py:
(sep or ' ').join(map(str.capitalize, s.split(sep)))
input str.title() string.capwords()
------------------------------------------------------------------------------
"they're not the messiah" "They'Re Not The Messiah" "They're Not The Messiah"
'x-ray results' 'X-Ray Results' 'X-ray Results'
'e.e. cummings' 'E.E. Cummings' 'E.e. Cummings'
'3rd place' '3Rd Place' '3rd Place'
"o'brien" "O'Brien" "O'brien"
' spaced out ' ' Spaced Out ' 'Spaced Out'
title() starts a new word after every UNCASED character -- an
apostrophe, a hyphen, a digit, a dot -- which is why it
capitalises the 'r' of "they're". capwords() splits on
whitespace and nothing else, so it does not. Neither is a
title-caser: both lowercase the rest of every word, so 'IBM'
comes back 'Ibm' from either one.
capwords() also rebuilds the string instead of editing it, so
with no separator it destroys the spacing it never read:
capwords(' spaced out ', None) -> 'Spaced Out'
capwords(' spaced out ', ' ') -> ' Spaced Out '
capwords('a,b,,c', ',') -> 'A,B,,C'
With sep=None, split() collapses runs and drops the ends; with
an explicit separator it does neither. One call, two rules.
5. Template IS THE SYNTAX THAT CANNOT REACH
A template that arrived from a user, a config file or a
database is a small program once you hand it to .format().
One object, four templates, one call each -- and this module
holds a global named DATABASE_PASSWORD, set to 'hunter2':
{0.name} -> 'brian'
{0.password} -> 'swordfish'
{0.__class__.__name__} -> 'User'
{0.__init__.__globals__[DATABASE_PASSWORD]} -> 'hunter2'
A field name is a small expression language: a dot walks an
attribute, brackets index. Three hops from an object you
thought was harmless to a module global you never passed in.
The same four names through Template, which has no dot:
$name substitute -> 'brian'
$password substitute -> 'swordfish'
$name.password substitute -> 'brian.password'
$__class__ substitute -> KeyError '__class__'
'$name.password' is 'brian' followed by nine literal
characters. The placeholder grammar ends at the identifier,
so there is nothing to reach with.
6. substitute RAISES, safe_substitute NEVER DOES
template substitute() safe_substitute()
------------------------------------------------------------------
'$greeting $name' KeyError 'name' 'Hi $name'
'cost: $100' ValueError 'cost: $100'
'$$HOME' '$HOME' '$HOME'
'${bad name}' ValueError '${bad name}'
'$' ValueError '$'
'cost: $100' is the one that catches people. A price is an
invalid placeholder -- an identifier cannot begin with a digit
-- so substitute() raises on a template that was never trying
to have a placeholder in it. '$$' is the escape.
Since 3.11 you can ask before you run, which is the honest way
to accept a template from somebody else:
t = Template('$greeting, $name! ${name}s cost $$5')
t.get_identifiers() -> ['greeting', 'name']
t.is_valid() -> True
Template('$ oops').is_valid() -> False
7. AND Template'S IDENTIFIERS ARE ASCII TOO, ON PURPOSE
Template.idpattern is '(?a:[_a-z][_a-z0-9]*)'
$café identifiers ['caf'] substitute -> 'Xé'
$naïve identifiers ['na'] substitute -> 'Yïve'
$_x1 identifiers ['_x1'] substitute -> 'Z'
$Ω identifiers [] substitute -> ValueError
'$café' is the placeholder 'caf' followed by a literal 'é'.
It does not raise. It quietly substitutes a name that is a
prefix of the one you wrote.
The (?a: in that pattern is doing real work. Matching [a-z]
case-insensitively without it lets three non-ASCII characters
in, because they case-fold onto ASCII letters:
[a-z]/i (?a:[a-z])/i
U+212A KELVIN SIGN True False
U+0131 LATIN SMALL LETTER DOTLESS I True False
U+017F LATIN SMALL LETTER LONG S True False
8. Formatter: THE HOOK, AND THE ONE METHOD WORTH KNOWING
Formatter has 8 methods:
check_unused_args convert_field format format_field
get_field get_value parse vformat
Formatter().format('{:*^9.3f}', 3.14159) -> '**3.142**'
-- the same grammar, the same answer, in Python instead of C.
parse() is the part that is not a reimplementation. It is the
public parser for the format grammar, and it will tell you
what a template wants without formatting anything:
Formatter().parse('{greeting}, {0:>8.2f} and {a[1]!r}!')
literal field spec conv
'' 'greeting' '' None
', ' '0' '>8.2f' None
' and ' 'a[1]' '' 'r'
'!' None None None
And the reason to subclass it is section 5. Overriding one
method closes that hole, because get_field is exactly where
the dot and the brackets get interpreted:
{who} -> 'brian'
{who!r:>10} -> " 'brian'"
{0.password} -> ValueError: not a plain name
{0.__init__.__globals__[DATABASE_PASSWORD]} -> ValueError: not a plain name
Five lines, and the whole attribute walk is gone. That you
have to write those five lines yourself is the argument for
reaching for Template instead.
What the run shows¶
Twelve names, and none of them does anything to a string. Nine constants and three callables, of which one is a single expression and two are classes. That shape is the whole history: a module emptied out by the methods that replaced its contents, keeping only the things that were never methods in the first place.
Sorted by code point, the constants are the ASCII chart with the boundaries computed rather than typed. Nine runs and one gap: five controls at 09–0D, then nothing at all until the space at 20, then punctuation and digits and the two letter blocks interleaved to 7E. The four runs of punctuation are exactly the gaps left between digits, ascii_uppercase and ascii_lowercase — which is another way of saying string.punctuation is defined by subtraction, not by meaning. The two letter blocks starting 0x20 apart is the fact the whole | 0x20 case trick rests on; the sibling library's a character is a number ↗ is where that layout is explained rather than merely measured.
string.printable is the concatenation of the other five, and it is the ASCII range 20–7E plus five controls. Not a curated list — literally digits + ascii_lowercase + ascii_uppercase + punctuation + whitespace, which the program checks. The five extras are \t \n \x0b \x0c \r, and they are why the constant is named for the POSIX sense of printable and disagrees with str.isprintable().
ASCII-only is the point, not a defect. Every row of section 3 is a constant saying False where the modern method says True: ٣ is a digit, \xa0 is whitespace, ’ is punctuation, Ł is a letter — and none of them is in the corresponding constant. Read that as a bug and the constants look broken; read it as a promise and they become the only easy way to ask the ASCII question. Is it a letter? owns the predicates; the constants are their fossil, and the useful thing about a fossil is that it does not move.
The gap that will actually bite is strip. string.whitespace holds six characters and str.isspace() is true of twenty-nine, so s.strip() and s.strip(string.whitespace) are not the same call — the first removes a NO-BREAK SPACE and an information separator, the second leaves both in place. If you are cleaning a column of data scraped from HTML, that is the difference between a trimmed field and one that still has \xa0 on the front. The rest of that story is on strip is a set, not a prefix.
And punctuation is a name from before there were categories. Nine of its thirty-two characters — $ + < = > ^ ` | ~ — are General_Category S*, symbols rather than punctuation, by Unicode's own accounting. The constant was never wrong; it was answering an older question, which ASCII characters are neither letters nor digits nor space, and that question has no Unicode name.
capwords() and title() disagree about where a word begins, and both are wrong about names. str.title() starts a new word after every uncased character, so an apostrophe, a hyphen, a dot and a digit all start one: "they're" becomes "They'Re" and "x-ray" becomes "X-Ray". capwords() splits on whitespace and nothing else, so it gets those right and gets "o'brien" wrong in the other direction. Neither preserves interior capitals — IBM comes back Ibm from both, because both lowercase the rest of each word — so neither is a title-caser, and the one-line implementation printed in section 4 is the honest summary of what capwords is.
capwords() rebuilds the string, so it destroys the whitespace it did not read. With no separator it is ' '.join(...) over s.split(None), which collapses runs and drops both ends: " spaced out " comes back "Spaced Out". Pass an explicit separator and it does neither. One function, two behaviours, decided by whether an argument is None — worth knowing before you use it to tidy a column and quietly lose the indentation.
A format string that arrived from outside your program is a small program. Section 5 is the measurement behind the sentence: the field name in str.format is an expression language where a dot walks an attribute and brackets index, so '{0.password}' reads an attribute you never offered and '{0.__init__.__globals__[DATABASE_PASSWORD]}' walks from any object, through its own __init__, into the module that defined it, and out with a global you never passed in. Three hops. It needs no eval, no import and no cooperation from the object. This is not a hypothetical class of bug: a template held in a database column or a config file, applied to a request object, is enough. f-strings cannot do it — an f-string is compiled from a literal at parse time, so there is no way to get a runtime string into one — but that is exactly why they are no help when the template genuinely has to come from outside.
Template is the answer, and its guarantee is a grammar with nothing in it. A placeholder is $ followed by an identifier, and the grammar ends there: '$name.password' substitutes name and leaves .password as nine literal characters. There is no dot to walk and no bracket to index, so the worst a hostile template can do is name a key you did not supply. That is the whole security argument for a syntax this limited, and it is why Template is still in the standard library long after the module around it stopped being useful.
substitute raises and safe_substitute never does — including on templates that are not valid at all. '${bad name}', a lone '$' and, most usefully, 'cost: $100' all raise ValueError from substitute, because an identifier cannot begin with a digit; safe_substitute hands all three back unchanged. A price in a template is the one that catches people, and $$ is the escape. Since 3.11 there is a better move than catching the exception: get_identifiers() tells you which names a template wants and is_valid() tells you whether it is well-formed, both without running it — which is the honest way to accept a template from somebody else. Pass substitute a mapping rather than **locals(), though: Template limits what the syntax can reach, not what you hand it.
Template's identifiers are ASCII too, and that is the sharpest thing on this page. idpattern is '(?a:[_a-z][_a-z0-9]*)', so '$café' is the placeholder caf followed by a literal é. It does not raise; it substitutes a prefix of the name you wrote, which is the quietest possible failure. And the (?a: is load-bearing: matching [a-z] case-insensitively without it admits U+212A KELVIN SIGN, U+0131 DOTLESS I and U+017F LONG S, because those three case-fold onto ASCII letters. An identifier grammar that meant to be ASCII and forgot to say so would accept a Kelvin sign where you wrote a K. The sibling library measures what that costs when a security check is on the other side of it, in the check that ran too early ↗, and Python's own identifiers go the other way entirely — they accept Unicode and normalize it, which is Unicode in identifiers ↗.
Formatter is str.format reimplemented in Python so that you can override a piece of it, and parse() is the piece worth having anyway. Formatter().parse(template) is the public parser for the format grammar: it yields (literal, field_name, format_spec, conversion) and formats nothing, so it will tell you what a template requires before you supply anything — the Template.get_identifiers() of the {} world. The reason to subclass, when there is one, is section 5: get_field is where the dot and the brackets are interpreted, so overriding it in five lines removes the attribute walk entirely. That the five lines are yours to write is the argument for reaching for Template instead.
The Rust view, and C underneath both¶
Rust has no string module and no constants, because the ASCII questions are methods on the type: char::is_ascii_punctuation, is_ascii_digit, is_ascii_hexdigit, is_ascii_whitespace, and the same set again on u8. That is the design Python arrived at too — str.isdigit() is the same move — which makes the constants the fossil of the older one, and Rust simply never had the older one to fossilise.
Python 3.14.7 rustc 1.98.0
the 32 ASCII punctuation string.punctuation char::is_ascii_punctuation
identical sets -- !"#$%&'()*+,-./:;<=>?@[\]^_`{|}~
ASCII whitespace string.whitespace 6 is_ascii_whitespace 5
' \t\n\r\x0b\x0c' ' \t\n\r\x0c'
Rust's is missing U+000B VERTICAL TAB
hex digits string.hexdigits 22 is_ascii_hexdigit 22
ASCII graphic (no constant) -- is_ascii_graphic 94
The one row that disagrees is the interesting one. Rust's is_ascii_whitespace follows the WhatWG Infra definition and leaves out U+000B VERTICAL TAB; Python's string.whitespace includes it, and so does Rust's Unicode char::is_whitespace. So "ASCII whitespace" is not one set even between two standard libraries, and a port that swaps one for the other changes what gets trimmed. The char predicates are the sibling library's subject: meet the char ↗.
C's <ctype.h> is the ancestor of both, and it has a trap neither inherited. isalpha, isdigit, ispunct take an int, not a char, and the C standard requires the value to be representable as unsigned char or equal to EOF — so passing a plain char on a platform where char is signed, with any byte above 0x7F in it, is undefined behaviour rather than a false. Cast to unsigned char at every call site. The second half is that C's answers depend on LC_CTYPE: isalpha('é') is a question about the current locale, not about the character, which is the problem locale and LC_CTYPE ↗ is about. Python's constants are the fix for that half — they are frozen ASCII precisely so that no locale can move them.
If you are coming from ABAP¶
There is no string-module equivalent, and the closest idiom is the one you already reach for: a character-set constant plus CO (contains only), or the cl_abap_char_utilities attributes for the things you cannot type — newline, horizontal_tab, cr_lf. Those are the direct analogue of string.whitespace, and they carry the same warning: a hand-built set is exactly as wide as you made it, so text arriving from a web front end with a NO-BREAK SPACE in it passes an "is it blank" test written against space. What transfers cleanly is the habit of naming the set instead of typing the literal; what Python adds is a second family, the is* methods, that asks the Unicode table instead of a list — and the whole point of this page is that you must choose which one you meant.
For string templating, Template is the closest thing Python has to what you get from a text symbol or a message class: a placeholder that can only be replaced, never evaluated. If you are building output from a template stored in a customising table, that is the tool — .format() on a template from a database table is the pattern this page measures a hole in. (Not machine-checked — CI cannot run ABAP.)
Try it¶
- Rebuild
string.printablefromstrmethods alone. Loop overrange(0x80)and keep the characters a predicate calls printable; compare with the constant. You will be five characters short, and the five are the lesson — check them againstrepris notstr§4 before deciding which of the two is right. - Make
capwordscorrect for one name. Take"o'brien mcdonald x-ray"and write the title-caser you actually meant. Start by deciding, in words, where a word begins; then check your rule againsttitle()andcapwords()and see which of the three inputs each one loses. - Close the hole, then check your work. Write the
PlainNamesOnlyformatter from section 8 yourself, then try to defeat it:'{0[0]}'on a list,'{0!r}'on any object,'{}'with no name at all. Which of those should a template from a stranger be allowed to do? Then write the same rule usingTemplate.get_identifiers()and compare how long each version is.
Practice¶
Nine expressions, and the two that answer a name nobody wrote. Write down what each returns — a value, or the type of the exception — before you run anything.
string.capwords('x-ray results')'x-ray results'.title()string.capwords(' spaced out ')'\u0663' in string.digits'\xa0Zoot'.strip(string.whitespace)len(string.printable)string.Template('cost: $100').substitute()string.Template('$café').substitute(caf='X')'{0.__class__.__name__}'.format(3)
Then: lines 7 and 8 both fail on a name, and only one of them tells you. Say which, and how you would have found out about the other one without running it.
Answers
Verified output of the_string_module_kata_py.py — regenerated by tools/run_examples.py, never hand-typed.
expression answer why
-----------------------------------------------------------------------------------------------
string.capwords('x-ray results') 'X-ray Results' splits on whitespace, so 'ray' is not a word
'x-ray results'.title() 'X-Ray Results' a hyphen is uncased, so it starts a word
string.capwords(' spaced out ') 'Spaced Out' rebuilt by split()+join(): the spacing is gone
'\u0663' in string.digits False the constant is ASCII; '٣'.isdigit() is True
'\xa0Zoot'.strip(string.whitespace) '\xa0Zoot' 6 characters in the constant, 29 in str.isspace()
len(string.printable) 100 ASCII 0x20-0x7E is 95, plus five controls
string.Template('cost: $100').substitute() ValueError an identifier cannot begin with a digit
string.Template('$café').substitute(caf='X') 'Xé' the placeholder is 'caf'; the 'e-acute' is literal
'{0.__class__.__name__}'.format(3) 'int' a field name walks attributes -- that is the point
THE TWO THAT ANSWER A NAME NOBODY WROTE
Lines 7 and 8 are the same grammar seen from both sides.
Template('$café').get_identifiers() -> ['caf']
Template('cost: $100').is_valid() -> False
One asked for 'café' and got 'caf'; the other never meant
to have a placeholder and has a malformed one. Neither is
visible in the template as you read it, and both are
answerable before you run anything -- which is what those
two 3.11 methods are for.
WHY LINE 9 IS THE WHOLE ARGUMENT FOR Template
A dot in a field name walks an attribute, so a template
that came from outside your program reads whatever the
object can reach:
{0.__class__.__name__}.format(3) -> 'int'
{0.numerator}.format(3) -> '3'
{0.real}.format(3) -> '3'
Template has no dot. '$name.password' substitutes 'name'
and leaves nine literal characters, which is the entire
security argument for a syntax that can do nothing else.
THE CONSTANT-VERSUS-METHOD PAIR
Lines 4 and 5 are one question asked twice:
U+0663 in string.digits False str.isdigit() True
U+00A0 in string.whitespace False str.isspace() True
Both constants say no and both methods say yes. Neither is
wrong: a constant is a fixed ASCII list and a method is a
lookup in the Unicode table. For a validator, pick the one
whose promise you can state out loud.
See also¶
- The format mini-language — the nine-slot grammar
Formatterreimplements, and the one place the attribute walk is mentioned before this page measures it - Is it a letter? — the twelve
is*predicates that made the constants historical repris notstr— whystring.printableis not printable, by designstripis a set, not a prefix — the twenty-nine code pointsstrip()removes andstring.whitespacedoes not name- The crosswalk — which idea lives in which library
- A character is a number ↗ — why ASCII is laid out the way section 2 measures it
- Control characters ↗ — the five in
string.whitespacethat are not space - Meet the
char↗ — Rust's answer: predicates on the type, no constants at all string— Common string operations ↗ — the module docs this page is unpacking