The format mini-language¶
Level: 201 · for Python programmers
One line: f-strings, str.format, format() and __format__ are four doors onto one nine-slot grammar — and % is a fifth spelling that is a genuinely different language, kept because it is the only one that works on bytes.
Python's formatting looks like four features and is really one and a half. Everything after the colon — *^+12.3f, #010b, ,d — is a format specification, and the language does not parse it. It is handed, as a string, to the object's own __format__ method, which may mean whatever it likes by it; datetime means strftime codes, and any class of yours can mean something else. The standard spec is what the built-in types agreed on, and it has nine slots in a fixed order.
Knowing it is one grammar is most of the value, because it means the thing you learned inside an f-string works in format(), in '{}'.format(), in a __format__ you write yourself, and — with different spelling — in Rust's format!. The half that is separate is %, which predates all of it, is an operator rather than a call, and fails in its own way.
Verified output of the_format_mini_language_py.py — regenerated by tools/run_examples.py, never hand-typed.
1. FOUR DOORS, ONE GRAMMAR
the spec: '*^+12.3f'
f-string f'{value:*^+12.3f}' ***+3.142***
str.format '{:*^+12.3f}'.format(value) ***+3.142***
format() builtin format(value, '*^+12.3f') ***+3.142***
__format__ directly value.__format__('*^+12.3f') ***+3.142***
All four are the same call. f-strings and str.format both compile
down to format(), which calls the object's own __format__ with the
text after the colon -- so the spec is not parsed by the language.
It is a string handed to a type, and the type may mean anything by it:
f'{temp}' 100C <- str(), no spec
f'{temp:F}' 212.0F <- this type's OWN spec language
f'{temp:>8}' 100C <- falls back to the standard one
f'{temp!r}' Celsius(100) <- conversion happens BEFORE __format__
2. THE NINE SLOTS
[[fill]align][sign][z][#][0][width][grouping][.precision][type]
slot spec result
------------------------------------------------------
fill + align '*^12' *****42*****
align only '<8' 42 |
sign: always '+d' +42
sign: space ' d' 42|
alt form + zero pad '#010b' 0b11111111
width '8' 42|
grouping: comma ',d' 1,234,567
grouping: underscore '_d' 1_234_567
grouping in binary '_b' 1111_1111
precision on a float '.3f' 3.142
precision on a str '.3' tru
type: hex, exp, pct 'x' 'e' '%' ff 1.234000e+03 25.000000%
all of it '+08,d' +000,042
Two that surprise people: '0' before the width is not padding with
the fill character, it is a separate flag that means 'pad after the
sign' -- which is why '+08,d' puts the zeros between + and 42. And
.precision on a str is a TRUNCATION, not a rounding.
3. NUMBERING: AUTOMATIC, MANUAL, AND THE MIX THAT RAISES
automatic '{} and {}' -> spam and eggs
manual '{0} and {1}' -> spam and eggs
reordered '{1} and {0}' -> eggs and spam
reused '{0} and {0}' -> spam and spam
by name '{x} and {y}' -> spam and eggs
attribute '{0.deg}' -> 100
item '{0[1]}' -> eggs
mixed '{} and {0}' -> ValueError
Automatic numbering is a counter, so it can only count forwards.
The moment one field names its argument, the counter has nothing
sensible to do, and Python refuses rather than guessing. That is
the fact the 'you may omit the numbers' changelog note leaves out.
Note also the item lookup: '{0[1]}' takes no quotes, so the key is
always a string unless it is all digits -- there is no way to write
a lookup by the integer 1 versus the string '1'.
4. THE MINUS SIGN THAT IS NOT A NEGATIVE NUMBER
value .1f z.1f .0f z.0f
------------------------------------------------
-0.0 -0.0 0.0 -0 0
-0.04 -0.0 0.0 -0 0
-0.4 -0.4 -0.4 -0 0
0.4 0.4 0.4 0 0
-1.5 -1.5 -1.5 -2 -2
2.5 2.5 2.5 2 2
3.5 3.5 3.5 4 4
Rounding happens before the sign is chosen, so a number that is
merely small and negative can print as '-0'. In a table of results
that reads as a distinct value, and it is not one. The 'z' option
(PEP 682, Python 3.11) coerces a negative zero to a positive one
AFTER rounding, which is the only place it can be fixed.
-0.0 == 0.0 is True, so no comparison will find this for you.
The last two rows are a second surprise from the same column:
2.5 formats as 2 and 3.5 as 4. Formatting rounds half to even,
like round(), and not the way you were taught at school.
5. GROUPING, AND THE ONE THAT ASKS THE OPERATING SYSTEM
under the C locale: thousands_sep='' decimal_point='.' grouping=[]
format(1234567, ',d' ) -> 1,234,567
format(1234567, '_d' ) -> 1_234_567
format(1234567, 'n' ) -> 1234567
',' and '_' are Python's own separators and mean the same thing on
every machine. 'n' means 'ask the locale', and under C the locale
has nothing to say -- which is why this line is boring on purpose.
Set LC_NUMERIC to a locale that groups, and the same call prints
1.234.567 or 1 234 567 instead.
The documented cost is unusual enough to be worth knowing: when
the separators are non-ASCII or longer than one byte, formatting
with 'n' TEMPORARILY CHANGES LC_CTYPE for the whole process --
and the docs say outright that this affects other threads.
A formatting call that mutates global state. Use ',' unless you
are deliberately rendering for a human in a known locale.
6. format_map: THE MAPPING IS NOT COPIED
template '{name} was born in {country}'
format_map(...) Guido was born in <country>
format(**data) KeyError: 'country'
format(**m) unpacks m into a fresh plain dict, so the subclass --
and its __missing__ -- is gone before formatting starts. format_map
passes the object itself, so the hook survives. That is the whole
difference, and it is why the useful version has its own method.
7. % IS A DIFFERENT LANGUAGE, AND IT IS THE ONLY ONE THAT DOES bytes
'%s and %s' % (a, b) spam and eggs
'%(x)s' % dict(x=a) spam
'%.3f' % 3.14159 3.142
'%r' % a 'spam'
'%s' % (1, 2) TypeError
'%s %s' % ('a',) TypeError
'%s' % (1,) 1
% is one operator with one right-hand operand. A tuple on the right
is not 'the arguments' -- it is the operand, and it gets unpacked,
which is why both arity mistakes above are TypeErrors rather than
the two different errors you would get from a function call.
b'%s|%d' % (b'ab', 7) b'ab|7' <- bytes, not str
str.format and f-strings do not exist on bytes. PEP 461 put % back
on bytes in 3.5 for exactly this: binary protocols with ASCII
headers, where the alternative was building the bytes by hand.
That is the reason the oldest of the four is not going anywhere.
What the run shows¶
One spec, four spellings, identical output. f'{value:*^+12.3f}', '{:*^+12.3f}'.format(value), format(value, '*^+12.3f') and value.__format__('*^+12.3f') all produce ***+3.142***. The first two compile down to the third. So there is no "f-string formatting" to learn separately from ".format formatting" — there is one thing, reached four ways, and only the syntax around the field differs.
The spec belongs to the type, not to Python. Celsius in the program defines __format__ and gives F its own meaning, falling back to the standard spec for anything else. That is exactly how datetime works — f'{now:%Y-%m-%d}' is not a Python feature, it is datetime.__format__ reading a string Python never looked at. Two consequences: a format spec that is meaningless to one type may be meaningful to another, and a TypeError or ValueError from a spec is raised by the type, which is why the message wording varies so much.
Two slots that surprise people. The 0 before the width is not "fill with zeros" — fill is the separate first slot. It is a flag meaning pad after the sign, which is why '+08,d' on 42 gives +000,042 with the sign on the outside. And .precision on a str is a truncation, not a rounding: format('truncated', '.3') is 'tru'. That one is worth remembering because it silently shortens data in a report.
Automatic numbering is a counter, and it cannot be mixed with manual. '{} and {0}'.format(a, b) raises ValueError. The 3.1 changelog note that started this page says the numbers may be omitted; what it does not say is that omitting some of them is an error, because a counter has nothing sensible to do once a field has named its own argument. Also worth knowing about the same section: '{0[1]}' takes no quotes, so a mapping key inside a format string is always a string unless it is all digits — there is no spelling that distinguishes the key 1 from the key '1'.
A number that is not negative can print with a minus sign. Rounding happens before the sign is chosen, so -0.04 at one decimal place is -0.0 and -0.4 at zero decimal places is -0. In a column of results that reads as a distinct value and it is not one, and no comparison will catch it: -0.0 == 0.0 is True. PEP 682 ↗ added the z option in 3.11 for exactly this, and it has to act after rounding because that is the only place the problem exists. The same column carries a second surprise for free — 2.5 formats as 2 and 3.5 as 4, because formatting rounds half to even like round() does.
And one type asks the operating system. , and _ are Python's own grouping separators and behave identically everywhere. The n type means ask LC_NUMERIC, so under the C locale this program's output is deliberately boring and on a machine with a European locale the same call prints 1.234.567. The documented cost is what makes it worth a paragraph: when the locale's separators are non-ASCII or longer than one byte, formatting with n temporarily changes LC_CTYPE for the whole process, and the docs say plainly that this "affects other threads". A formatting call that mutates global state is unusual enough to be worth knowing about before you meet it. Use , unless you are deliberately rendering for a human in a locale you set.
format_map exists because format(**m) throws the mapping away. **m unpacks into a fresh plain dict, so a subclass and its __missing__ hook are gone before formatting starts; format_map passes the object itself, so Default(name='Guido') can fill in <country> instead of raising KeyError. That is the entire difference, and it is why the useful version needed its own method rather than a keyword argument.
% is one operator with one operand, and that explains both of its errors. '%s %s' % (a, b) looks like a call with three parts and is not: % takes a left string and a single right operand, and a tuple on the right is unpacked. So '%s' % (1, 2) and '%s %s' % ('a',) are both TypeError, and '%s' % (1,) — the same value written as a one-tuple — works. The one thing that keeps % alive is the last block: str.format and f-strings do not exist on bytes, and PEP 461 ↗ put % back on bytes in 3.5 for binary protocols with ASCII headers. b'%s|%d' % (b'ab', 7) is b'ab|7', and there is no other way to write it.
One security note, because section 3 shows the mechanism. A format field can walk attributes: '{0.__class__}'.format(3) gives <class 'int'>. That is a feature when the template is yours and a hazard when it is not — str.format and format_map on a template that came from a user or a database are a way to read attributes of whatever you passed in. f-strings do not have this problem, because an f-string is compiled, not interpreted at runtime. If you need user-supplied templates, string.Template is the $name syntax that exists precisely because it can do nothing else.
The Rust view¶
Rust borrowed the grammar, and the overlap is closer than it looks — the same spec string, minus the type letter, produces the same bytes.
intent Python Rust result
-----------------------------------------------------------------------------------------------------
fill, align, sign, precision f'{v:*^+12.3f}' format!("{v:*^+12.3}") ***+3.142*** both
alt form + zero pad format(255, '#010b') format!("{:#010b}", 255) 0b11111111 both
sign-aware zero pad format(42, '+08d') format!("{:+08}", 42) +0000042 both
half-to-even rounding format(2.5, '.0f') -> 2 format!("{:.0}", 2.5) -> 2 agree
negative zero on display format(-0.4, '.0f') -> -0 format!("{:.0}", -0.4) -0 both
fixing it format(-0.4, 'z.0f') -> 0 -- no z option -- Python only
digit grouping format(1234567, ',d') -- none in std -- Python only
dynamic width f'{x:>{w}}' format!("{x:>w$}") both
a name from the scope f'{name}' format!("{name}") both, since Rust 2021
a template read at runtime '{}'.format(x) does not compile Python only
wrong number of arguments IndexError at runtime does not compile Rust catches it
Two differences carry all the weight. The first is that format! is a macro over a string literal: println!(t, 1) where t is a String is a compile error — "format argument must be a string literal" — and a mismatched argument count is a compile error too. So the whole runtime-templating surface Python has around this grammar simply does not exist in Rust: no format_map, no string.Formatter, and none of the attribute-walking hazard above, because there is no way to get an attacker's template into the macro. Python pays for a genuinely useful feature — a template read from a config file — with a class of bug Rust cannot have.
The second is the z row. Both languages display -0 for a small negative number rounded to zero, so this is not a Python quirk; it is what IEEE 754 plus rounding-then-signing does. Python added a spec option for it in 3.11 and Rust has none, so in Rust you fix it before formatting or not at all.
The sibling library owns the Rust side: the format language ↗.
If you are coming from ABAP¶
The nearest thing is string templates — |Total: { lv_amount NUMBER = USER }| — and the shape of the comparison is unusual: ABAP's version is closer to n than to ,. Its formatting options are named (NUMBER, DECIMALS, WIDTH, ALIGN, PAD, CURRENCY, TIMESTAMP) rather than positional punctuation, and several of them read the user's master-data settings rather than a process locale — so the same template can render a different string for two people signed into the same system. Coming to Python, the habit to unlearn is expecting the output to be user-dependent by default: , and _ are fixed, n is the only one that asks anything, and nothing consults a user profile. Coming the other way, the habit to keep is that ABAP made you name the option, which is more readable than '+08,d' and is worth imitating in a comment. Verify the exact option set on your own system before relying on it — string templates are 7.02+ and the option list has grown. (Not machine-checked — CI cannot run ABAP.)
Try it¶
- Run the program again with
LC_NUMERICset to a locale your machine actually has (locale -awill tell you). Which line changes? Now write the one-line experiment that shows'{:n}'and'{:,}'disagreeing, and decide which one belongs in a log file and which in a report. - Give a class of yours a
__format__that understands two of its own spec letters and delegates everything else toformat(str(self), spec). Then work out what your class should do with an unknown spec — raise, or ignore it? Look at whatintdoes with'{:q}'. '{0[1]}'has no quotes. Build a dict with both the key1and the key'1', and find out which one the format string reaches. Then find the two-line workaround.- The nine slots are order-sensitive. Take
'*^+#012_.3f', scramble the slots, and see how many rearrangements Python accepts — then explain the ones it does from the grammar rather than by experiment. - Write the
bytesversion of a small binary header with%. Now try to write the same thing with an f-string, and notice how far you get before reaching for.encode()— and what that.encode()costs you if a field is not ASCII.
Practice¶
One drill per slot, then the spec read backwards. The grammar has nine slots in a fixed order:
Predict all nine — one per slot, in order:
format(42, '*^12')format(42, '+d')format(-0.4, 'z.0f')format(255, '#010b')format(42, '+08,d')format(42, '8')format(1234567, '_d')format(3.14159, '.3f')format('truncated', '.3')
Now read the grammar the other way. Write the spec that turns the left column into the right one:
| from | you want |
|---|---|
1234567 |
1_234_567 |
255 |
0b11111111 |
3.14159 |
***+3.142*** |
42 |
42 right-aligned in eight columns |
And one to explain rather than predict: '{} and {0}'.format('spam', 'eggs') raises ValueError. Automatic numbering is a counter — say what a counter has left to do once a field has named its own argument.
Answers
Verified output of the_format_mini_language_kata_py.py — regenerated by tools/run_examples.py, never hand-typed.
the grammar: [[fill]align][sign][z][#][0][width][grouping][.precision][type]
slot call result note
--------------------------------------------------------------------------------------------------------
fill + align format(42, '*^12') '*****42*****' the fill goes BEFORE the align
sign format(42, '+d') '+42' '+' always, '-' only negatives, ' ' pads
z format(-0.4, 'z.0f') '0' PEP 682: coerce a negative zero, after rounding
# alt + 0 pad format(255, '#010b') '0b11111111' the 0b counts toward the ten
0 before width format(42, '+08,d') '+000,042' NOT fill -- it means 'pad after the sign'
width format(42, '8') ' 42' a str would left-align in the same eight
grouping format(1234567, '_d') '1_234_567' ',' and '_' are Python's own, and fixed
.precision on a float format(3.14159, '.3f') '3.142' three places, rounded
.precision on a str format('truncated', '.3') 'tru' a TRUNCATION, not a rounding
THE TWO THAT ARE NOT WHAT THEY LOOK LIKE
'0' before the width is not 'fill with zeros' -- fill is the
separate FIRST slot. It is a flag meaning 'pad after the sign',
which is why the zeros land between the + and the 42:
format(42, '+08,d') '+000,042'
format(42, '08,d') '0,000,042'
format(42, '0^8,d') '00042000'
The third one puts '0' in the FILL slot and '^' in the align
slot, so it is a different spec that happens to use the same
character.
One consequence worth having met before it surprises you: with
grouping on, zero-padding rounds UP to a well-formed group,
because no group may begin with a separator. So three of these
nine widths come back one character longer than asked:
format(42, '04,d' ) 0,042 len 5 <- wider than the width
format(42, '05,d' ) 0,042 len 5
format(42, '06,d' ) 00,042 len 6
format(42, '07,d' ) 000,042 len 7
format(42, '08,d' ) 0,000,042 len 9 <- wider than the width
format(42, '09,d' ) 0,000,042 len 9
format(42, '010,d' ) 00,000,042 len 10
format(42, '011,d' ) 000,000,042 len 11
format(42, '012,d' ) 0,000,000,042 len 13 <- wider than the width
And .precision on a str truncates. It does not round, it does
not warn, and it silently shortens data in a report:
format('truncated', '.3') 'tru'
format(3.14159, '.3') '3.14'
THE SPEC READ BACKWARDS
Given the output, write the spec. One answer each:
value wanted spec got match
--------------------------------------------------------------
1234567 1_234_567| '_d' 1_234_567| True
255 0b11111111| '#010b' 0b11111111| True
3.14159 ***+3.142***| '*^+12.3f' ***+3.142***| True
42 42| '8' 42| True
(the | marks the end of the field, so the padding is visible)
THE NUMBERING THAT RAISES
'{} and {}'.format('spam', 'eggs') 'spam and eggs' automatic: a counter
'{1} and {0}'.format('spam', 'eggs') 'eggs and spam' manual: reorder freely
'{0} and {0}'.format('spam', 'eggs') 'spam and spam' manual: reuse freely
'{} and {0}'.format('spam', 'eggs') raises ValueError mixed -- and this is the one
Automatic numbering is a counter, so it can only count forwards.
The moment one field names its argument the counter has nothing
sensible to do, and Python refuses rather than guessing. Omitting
ALL the numbers is fine; omitting SOME of them is an error.
ROUNDING, AND THE MINUS SIGN THAT IS NOT A NEGATIVE NUMBER
value .1f z.1f .0f z.0f
------------------------------------------
-0.0 -0.0 0.0 -0 0
-0.04 -0.0 0.0 -0 0
-0.4 -0.4 -0.4 -0 0
0.4 0.4 0.4 0 0
-1.5 -1.5 -1.5 -2 -2
2.5 2.5 2.5 2 2
3.5 3.5 3.5 4 4
Rounding happens BEFORE the sign is chosen, so a number that is
merely small and negative prints as '-0'. In a column of results
that reads as a distinct value and it is not one -- and no
comparison will find it for you: -0.0 == 0.0 is True.
The last two rows are a second surprise from the same column:
2.5 formats as 2 and 3.5 as 4. Formatting rounds half to even,
like round(), and not the way you were taught at school.
THE FOUR DOORS ARE ONE CALL
f-string '***+3.142***'
str.format '***+3.142***'
format() builtin '***+3.142***'
__format__ direct '***+3.142***'
all four identical: True
f-strings and str.format both compile down to format(), which
hands the text after the colon to the object's own __format__.
So the spec is not parsed by the language -- it is a string
handed to a type, and the type may mean anything by it. That is
why f'{now:%Y-%m-%d}' works without Python knowing what %Y is.
See also¶
repris notstr— the!s,!rand!aconversions, and the=debug specifier, which happen before the spec is usedstris notbytes— why%onbytesneeded a PEP of its own- Sorting is not comparing — the other place a locale decides the answer
- What to write next — this page closes eight of the questions on the backlog
- The crosswalk — which idea lives in which library
- Format specification mini-language ↗ — the grammar this page is unpacking
- PEP 682 ↗ — the
zoption, and why it has to act after rounding - PEP 461 ↗ —
%onbytes, and the binary-protocol argument for it - Locale and
LC_CTYPE↗ — the six independent variables thentype reaches into