Slicing is not indexing¶
Level: 201 · for Python programmers
One line: s[n] is allowed to raise and s[:n] is not — the same brackets over two different contracts — which is why s[:n] + s[n:] == s holds for every integer there is, and why the famous s[::-1] quietly destroys any text with a combining mark in it.
Square brackets on a str are two operations wearing one syntax. s[3] passes the integer 3 to str.__getitem__. s[1:3] builds a slice object — a real value, with a repr and three attributes — and passes that. The two take different arguments, return different kinds of thing, and, the part worth knowing, make different promises about failure. An index is allowed to raise and does: ask for a position the string does not have and you get an IndexError. A slice is not allowed to raise, and does not, because every bound you hand it is first clamped to the string's actual length. s[1:100] on a five-character string is not an error; it is s[1:5] before the string is ever touched.
That clamping is not folklore, and it is not buried in C. It is a method you can call:
Start, stop and step, already in range. Every slice runs its bounds through exactly that. 100 becomes 5, -100 becomes 0, and there is nothing left for the string to refuse.
What falls out of it is the invariant from the tutorial: for any n, s[:n] + s[n:] == s. It is normally stated with the sensible values of n in mind, and it holds for the rest of them too — n past the end, n past the start, n larger than any number your machine can index with. That is a claim about every integer, and the only honest way to make it is to try a lot of them, so this page does that rather than repeating the sentence.
Verified output of slicing_is_not_indexing_py.py — regenerated by tools/run_examples.py, never hand-typed.
1. THE INVARIANT, TESTED RATHER THAN ASSERTED
s = 'Monty' len 5
For every n, cutting the string in two at n and gluing it back
together returns the original. Including n past both ends.
n s[:n] s[n:] joined == s
----------------------------------------------------
-9 '' 'Monty' 'Monty' True <- past the start
-8 '' 'Monty' 'Monty' True <- past the start
-7 '' 'Monty' 'Monty' True <- past the start
-6 '' 'Monty' 'Monty' True <- past the start
-5 '' 'Monty' 'Monty' True <- past the start
-4 'M' 'onty' 'Monty' True
-3 'Mo' 'nty' 'Monty' True
-2 'Mon' 'ty' 'Monty' True
-1 'Mont' 'y' 'Monty' True
0 '' 'Monty' 'Monty' True
1 'M' 'onty' 'Monty' True
2 'Mo' 'nty' 'Monty' True
3 'Mon' 'ty' 'Monty' True
4 'Mont' 'y' 'Monty' True
5 'Monty' '' 'Monty' True <- past the end
6 'Monty' '' 'Monty' True <- past the end
7 'Monty' '' 'Monty' True <- past the end
8 'Monty' '' 'Monty' True <- past the end
9 'Monty' '' 'Monty' True <- past the end
19 values of n, 19 times True, zero exceptions.
2. THE SWEEP: SEVEN STRINGS, AND NUMBERS NO MACHINE CAN INDEX WITH
7 strings x every n from -(len+8) to len+8
187 values of n tested, 0 failures
And n does not have to fit in a machine word:
s[:10**30] 'Monty'
s[10**30:] ''
s[-10**30:] 'Monty'
s[:10**30] + s[10**30:] 'Monty' == s: True
A slice of ten-to-the-thirty is not an error. It is 'Monty'.
3. THE SAME BRACKETS, TWO CONTRACTS
s[n] and s[n:n+1] look like the same question asked twice.
They are not. One of them is allowed to fail.
n s[n] s[n:n+1]
--------------------------------------------
-9 IndexError ''
-8 IndexError ''
-7 IndexError ''
-6 IndexError ''
-5 'M' 'M'
-4 'o' 'o'
-3 'n' 'n'
-2 't' 't'
-1 'y' '' <- the two disagree
0 'M' 'M'
1 'o' 'o'
2 'n' 'n'
3 't' 't'
4 'y' 'y'
5 IndexError ''
6 IndexError ''
7 IndexError ''
8 IndexError ''
9 IndexError ''
s[n] raised on 9 of 19 values
s[n:n+1] raised on 0 of 19, and returned '' at every n where
s[n] raised. One row goes the other way, and it is not a
third contract -- it is section 5's arithmetic again:
s[-1] 'y'
s[-1:0] '' n + 1 is 0, and 0 as a STOP means
position zero, not 'the end'. The slice asks for
everything from the last character up to the
first, which is nothing.
The bracket is one syntax over two protocols. What goes inside
decides which:
s[1] passes an int to __getitem__
s[1:3] passes a slice to __getitem__ -- a real object
s.__getitem__(slice(1, 100)) 'onty'
s.__getitem__(100) IndexError
s[10**30] IndexError
Both failures are IndexError; neither slice is.
4. WHY A SLICE CANNOT FAIL: slice.indices() IS THE CLAMPING RULE
Every slice runs its bounds through this first. It takes the
length and returns start, stop and step already in range.
written the slice object .indices(5) result
------------------------------------------------------------------------
s[:3] slice(None, 3, None) (0, 3, 1) 'Mon'
s[3:] slice(3, None, None) (3, 5, 1) 'ty'
s[1:100] slice(1, 100, None) (1, 5, 1) 'onty'
s[100:] slice(100, None, None) (5, 5, 1) ''
s[-100:] slice(-100, None, None) (0, 5, 1) 'Monty'
s[:-1] slice(None, -1, None) (0, 4, 1) 'Mont'
s[:-0] slice(None, 0, None) (0, 0, 1) ''
s[::2] slice(None, None, 2) (0, 5, 2) 'Mny'
s[::-1] slice(None, None, -1) (4, -1, -1) 'ytnoM'
Nothing in that column is out of range, because clamping happened
before the string was touched. 100 became 5. -100 became 0.
5. NEGATIVE INDICES, AND THE -0 ASYMMETRY
A negative index counts back from the end. len(s) is added to it,
once, and then the ordinary rules apply.
s[-1] -> 'y' (len + n = 4)
s[-2] -> 't' (len + n = 3)
s[-5] -> 'M' (len + n = 0)
s[-6] -> IndexError (len + n = -1)
So s[-1] is the last character. And yet:
s[:-1] 'Mont' everything but the last
s[:-0] '' empty
Not an inconsistency in slicing -- an arithmetic fact that never
reaches it. The minus is gone before the slice exists:
-0 == 0 True
repr(slice(None, -0)) slice(None, 0, None)
The slice object itself has no memory of the sign. Which means
'drop the last n characters' as s[:-n] is a bug waiting for n = 0,
and s[:len(s) - n] is the version that survives it.
6. THE THIRD SLOT
s[::2] 'Mny' every second character
s[1::2] 'ot' offset by one
s[::-1] 'ytnoM' the idiomatic reverse
s[::-2] 'ynM' backwards, every second
s[::10**30] 'M' a step no machine can count to
s[::0] ValueError the one value that DOES raise
A step of zero is the single exception to 'a slice never raises',
and it is not about the bounds -- it is a ValueError, not an
IndexError, because no clamping can make a step of zero mean
anything. Every out-of-range bound is still fine, and so is every
other step:
s[100:-100:-1] 'ytnoM'
s[::-10**30] 'y'
(A non-integer in any of the three slots is a TypeError, but that
is the argument being the wrong kind of thing, not out of range.)
7. s[::-1] REVERSES CODE POINTS, NOT CHARACTERS
The famous one-liner is correct for ASCII and wrong for a large
fraction of the world's text. Reversing a sequence is only
reversing a string when every element stands alone.
cafe + U+0301
original len 5 café 'café'
reversed len 5 ́efac '́efac'
was U+0063 U+0061 U+0066 U+0065 U+0301
now U+0301 U+0065 U+0066 U+0061 U+0063
precomposed
original len 4 café 'café'
reversed len 4 éfac 'éfac'
was U+0063 U+0061 U+0066 U+00E9
now U+00E9 U+0066 U+0061 U+0063
flag of Poland
original len 2 🇵🇱 '🇵🇱'
reversed len 2 🇱🇵 '🇱🇵'
was U+1F1F5 U+1F1F1
now U+1F1F1 U+1F1F5
family
original len 7 👨👩👧👦 '👨\u200d👩\u200d👧\u200d👦'
reversed len 7 👦👧👩👨 '👦\u200d👧\u200d👩\u200d👨'
was U+1F468 U+200D U+1F469 U+200D U+1F467 U+200D U+1F466
now U+1F466 U+200D U+1F467 U+200D U+1F469 U+200D U+1F468
Devanagari
original len 6 नमस्ते 'नमस्ते'
reversed len 6 ेत्समन 'ेत्समन'
was U+0928 U+092E U+0938 U+094D U+0924 U+0947
now U+0947 U+0924 U+094D U+0938 U+092E U+0928
Read the first pair again. 'cafe' plus a COMBINING ACUTE ACCENT
renders as cafe-with-an-accent; reversed, the accent is now the
FIRST code point in the string, with nothing before it to sit on.
A combining mark attaches to what precedes it, and after a
reversal what precedes it is whatever came next in the sentence.
U+0301 category Mn, combining class 230
Mn means 'mark, nonspacing': by definition it has no width of its
own and modifies its neighbour. Slicing does not know that.
The flag is two REGIONAL INDICATOR letters, P and L, which a font
draws as one flag. Reversed they spell L then P -- not a country
code, so it renders as two letters and Poland is gone. The family
is four people joined by three ZERO WIDTH JOINERs; the joiners
survive the reversal and the order of the people does not.
None of these raised. Every one of them is a different string that
the type system, the linter and the test suite all call a str.
8. 'THE FIRST N CHARACTERS' IS NOT WHAT s[:n] GIVES YOU
s = 'café' which renders as café -- four characters to a reader
s[:1] U+0063 renders as c
s[:2] U+0063 U+0061 renders as ca
s[:3] U+0063 U+0061 U+0066 renders as caf
s[:4] U+0063 U+0061 U+0066 U+0065 renders as cafe
s[:5] U+0063 U+0061 U+0066 U+0065 U+0301 renders as café
s[:4] is the 'first four characters' of a four-character word and
it is a different word. The accent was code point five.
flag[:1] U+1F1F5 half a flag
family[:2] U+1F468 U+200D a man and a trailing joiner
s[:n] is 'the first n code points', which is the right answer to a
question about storage and the wrong one to a question about text.
9. bytes SLICES TO bytes; bytes INDEXES TO AN int
data = b'\xc5\x81\xc3\xb3d\xc5\xba' len 7
data[0] 197 int
data[0:1] b'\xc5' bytes
Two contracts again, and this time they differ in RETURN TYPE as
well as in failure mode.
The invariant holds here too, over the same range:
data[:n] + data[n:] == data for every n: True
data[::-1] b'\xba\xc5d\xb3\xc3\x81\xc5'
data[::-1].decode() UnicodeDecodeError
Reversing bytes does not reverse text. It shreds the encoding,
and here it fails loudly -- which is the good case.
What the run shows¶
The invariant survives everything. Nineteen values of n on 'Monty', then 187 across seven strings including the empty one, then 10**30 — a number with no machine representation as an index — and there is not a single failure and not a single exception. Section 1 is the proof and section 2 is the sweep behind it. Read the -9 and 9 rows: at both extremes one half is the whole string and the other is '', which is exactly what makes the sum come out right. A slice cannot be out of range because out of range is not a state a slice can be in.
The same brackets, and only one of them may fail. Section 3 puts s[n] and s[n:n+1] side by side over that same range. They agree in the middle and part company at both ends: nine of nineteen values raise IndexError for the index and none of them do for the slice, which returns '' instead. The two lines look like the same question asked twice; one of them is allowed to fail. If you have ever written s[0] on a string that turned out to be empty, that is the whole bug — and s[:1] is the version that cannot have it.
One row goes the other way, and it is not a third contract. At n = -1, s[-1] is 'y' and s[-1:0] is ''. Nothing has changed about the promise; the arithmetic simply arrived somewhere else. n + 1 is 0, and 0 in the stop position means position zero, not "the end", so the slice asks for everything from the last character up to the first, and there is nothing there. That is the same trap as the next paragraph, one step along.
s[-1] is the last character and s[:-0] is empty, and both are correct. A negative index means "count back from the end": len(s) is added to it once, so -1 becomes 4 on a five-character string. -0 is where that breaks, and it breaks before slicing is involved — -0 == 0 is an arithmetic fact, and the minus sign is gone by the time Python builds the slice object at all. The recorded repr(slice(None, -0)) is slice(None, 0, None): the object has no memory of ever having been negative. The practical consequence is that "drop the last n characters" written as s[:-n] is correct for every n except the one that means drop nothing, where it returns the empty string instead. s[:len(s) - n] has no such edge, and is the version to write when n is computed rather than typed.
A step of zero is the single exception. s[::0] raises ValueError, and it is worth noticing what kind of failure that is not: it is not an IndexError, because it is not about the bounds. Every out-of-range bound in the world is still clamped — s[100:-100:-1] is 'ytnoM' — and a step of 10**30 is fine in both directions. Zero is the one value no clamping can give a meaning to. (A non-integer in any of the three slots is a TypeError, but that is the argument being the wrong sort of thing rather than the wrong size.)
s[::-1] reverses code points, and code points are not characters¶
This is the part that makes slicing an encodings question. s[::-1] is the idiomatic reverse, it is taught everywhere, and it is correct for ASCII and wrong for a large fraction of the world's text — because reversing a sequence is only reversing a string when every element stands on its own, and in Unicode many of them do not.
Section 7 runs it on five samples. 'cafe' followed by a COMBINING ACUTE ACCENT reverses to a string whose first code point is the accent, with nothing before it to sit on: a combining mark modifies what precedes it, and after a reversal what precedes it is whatever came next in the sentence. U+0301 is category Mn, mark, nonspacing — by definition it has no width of its own — and slicing has no idea. The precomposed spelling holds the whole é in one code point, U+00E9, so the reversal moves it about without taking it apart — which means whether this one-liner corrupts your data depends on a normalization form you almost certainly never chose. That difference is normalization's subject; here it decides whether the reverse is lossless.
The emoji cases are the same failure with better publicity. The flag is two REGIONAL INDICATOR letters, P and L, which a font draws as one flag; reversed they spell L then P, which is not a country code, so Poland becomes two letters. The family is four people joined by three ZERO WIDTH JOINERs, and the reversal keeps all seven code points and reorders the people. Devanagari नमस्ते comes back with its vowel sign and virama leading. None of these raised, none of them changed length, and every one of them is still a str that a type checker, a linter and a test suite are all perfectly happy with.
There is no fix for this in the standard library, and that is the honest answer rather than a hedge: reversing text correctly means splitting it into grapheme clusters first, which needs Unicode's segmentation algorithm, which str does not implement. A code point is not a character ↗ is the sibling library's page on the unit s[::-1] should have been using, and counting characters is where the same gap shows up as four different answers to "how long is this string".
"The first n characters" is not what s[:n] gives you¶
s[:n] is the right answer to "show me the beginning of this", and it is what anyone asking for a preview, a summary column or a truncated log line should reach for. The footnote is that it is the first n code points, which is a different quantity as soon as the text is not ASCII. Section 8 is the four-character word café written the decomposed way: s[:4] is its "first four characters" and it is cafe — a different word, with the accent left behind as code point five. flag[:1] is half a flag. family[:2] is a man and a dangling joiner.
For a preview this is usually survivable and occasionally embarrassing. For anything with a byte budget behind it — a fixed-width field, a database column, a protocol header — it is not the operation you want at all, because code points are not bytes either. Take the beginning of a string when you need the beginning of a string, and say "code points" out loud when you write the number down.
And bytes has the whole thing again, with one extra asymmetry. Slicing a bytes gives you bytes; indexing one gives you an int. data[0] is 197 and data[0:1] is b'\xc5' — the asymmetry that catches everyone once, and str is not bytes's subject rather than this page's. The slicing half is what belongs here: the invariant holds on bytes exactly as it does on str, over the same range, for the same reason — and data[::-1] shreds the encoding, which on UTF-8 usually fails loudly at the next .decode(). Failing loudly is the good case. s[::-1] on a str fails silently, which is why it is the one on this page.
The Rust view¶
Rust asks the same question with the brackets pointed at bytes, and enforces the answer: &"é"[0..1] compiles fine and panics at run time, with a message naming the char boundary it was asked to cut inside — the one string operation Rust cannot check when it compiles. The sibling library owns that page, including the checked get(0..1) that returns Option<&str> instead: slicing by byte ↗.
If you are coming from ABAP¶
ABAP's counterpart is offset/length notation — lv_text+3(5) — and the difference is in the second number: it is a length, not an end index. Python's s[3:8] and ABAP's lv_text+3(5) select the same five characters and disagree about how you say so, which matters every time you translate a loop, because Python's stop moves with the start and ABAP's length does not. There is no negative offset either — "the last five characters" is s[-5:] here and arithmetic on strlen( lv_text ) there — and no step, so s[::2] and s[::-1] have no notation at all.
The sharper difference is the contract. ABAP's offset/length is checked: reading past the end of the field raises rather than clamping, which puts it on the side of Python's s[n] and not Python's s[:n]. The habit that transfers, then, is the ABAP one — work out whether the range fits before you take it — and the habit to unlearn is expecting Python to tell you when it did not. It will not; it will hand you a short string, or an empty one, and let the wrongness surface somewhere downstream. Two more things worth carrying over: the offset form is assignable, so lv_text+3(5) = '*****' overwrites in place, and Python has no equivalent at all because a str is immutable — the nearest is s[:3] + '*****' + s[8:], a new string built from two slices. And on a Unicode system ABAP counts characters as UTF-16 code units, so the same class of bug this page describes is available one encoding along, on anything above the Basic Multilingual Plane. Verify the exact out-of-bounds behaviour and any code-page number against your own system. (Not machine-checked — CI cannot run ABAP.)
Try it¶
s[:n] + s[n:] == sis tested here for one variable. State and test the two-variable version — iss[:a] + s[a:b] + s[b:] == sfor every paira,b? Find the pairs where it is not, and say why in terms ofslice.indices.- Write the guard you would need so that
s[:-n]means "drop the lastn" for everyn ≥ 0. Then write it ass[:len(s) - n]and check that the guard is no longer needed. Which of the two would you rather read in six months? sliceis a public type. Build one, hand it around as a value, and use the same object to slice astr, abytesand alist. Then call.indices()on it with three different lengths and predict each answer before you look.- Reverse
'ångström'written both ways — precomposedU+00E5, andaplusU+030A. One reversal is lossless and one is not. Then call.upper()on both spellings and compare the results as code points rather than by eye: they render identically and they are not the same string. - Take a real string column of yours, truncate it with
s[:20], and count how many rows come back with a different number of rendered characters than you asked for. If none do, your data is ASCII and you should test it with someone else's.
Practice¶
Eight brackets over one word, and only one of them may fail. Write down what each returns — a value, or the name of the exception type — before running anything.
'Monty'[5]'Monty'[5:]'Monty'[100:200]'Monty'[:-1]'Monty'[:-0]'Monty'[-1:0]'Monty'[::0]'cafe\u0301'[::-1]
Then: two of the eight raise, and they raise different exception types. Say which two, and what makes the second failure a different kind of problem from the first. Finally, pick any of the numbers above and check that s[:n] + s[n:] is still 'Monty'.
Answers
Verified output of slicing_is_not_indexing_kata_py.py — regenerated by tools/run_examples.py, never hand-typed.
expression result why
--------------------------------------------------------------------------------------
'Monty'[5] IndexError one past the end -- an index may fail, and does
'Monty'[5:] '' the same 5, clamped to len, so there is nothing left
'Monty'[100:200] '' both bounds clamped to 5; still not an error
'Monty'[:-1] 'Mont' -1 becomes 4: everything but the last
'Monty'[:-0] '' -0 IS 0, so this is [:0] and the minus never arrived
'Monty'[-1:0] '' start 4, stop 0: from the last up to the first
'Monty'[::0] ValueError no clamping can give a step of zero a meaning
'cafe\u0301'[::-1] '\u0301efac' reversed by code point, so the accent leads
TWO FAILURES, AND THEY ARE DIFFERENT KINDS
'Monty'[5] IndexError a POSITION the string does not have
'Monty'[::0] ValueError a STEP no length can make sense of
Everything between those two rows is a slice with an out-of-range
bound, and not one of them raised. That is the page in one table:
bounds are clamped, positions are checked, and the brackets look
identical from the outside.
LINE 5 AND LINE 6 ARE THE SAME ARITHMETIC TWICE
s[-1] 'y' a negative index counts back: len + (-1) = 4
s[:-0] '' but -0 == 0, so there is nothing to count back
s[-1:0] '' start 4, stop 0 -- and 0 as a STOP is a
position, not 'the end'
repr(slice(None, -0)) slice(None, 0, None)
The slice object never sees a minus sign in either case. Both are
arithmetic that finished before slicing began.
AND THE INVARIANT, OVER EIGHT NUMBERS -- FIVE OF THEM OUT OF RANGE
n = 5 s[:n] 'Monty' s[n:] '' joined == s: True
n = 100 s[:n] 'Monty' s[n:] '' joined == s: True
n = 200 s[:n] 'Monty' s[n:] '' joined == s: True
n = -1 s[:n] 'Mont' s[n:] 'y' joined == s: True
n = 0 s[:n] '' s[n:] 'Monty' joined == s: True
n = 1 s[:n] 'M' s[n:] 'onty' joined == s: True
n = -5 s[:n] '' s[n:] 'Monty' joined == s: True
n = 10**30 s[:n] 'Monty' s[n:] '' joined == s: True
failures: 0
(-0 is not on that list, because it cannot be: written into a
list literal it is 0, exactly as it is written into a slice.)
4 of those 8 are numbers s[n] would refuse, and the
invariant does not care about any of them. It holds because
clamping happens before the cut, on both halves, and the two
clamped answers are always the two pieces of one string.
See also¶
- Comparison has a mode — a slice at an index
findreturned, landing between a letter and its combining accent stris notbytes— why indexing abytesgives anintwhile slicing one givesbytes- Counting characters — the four answers to "how long", and why
s[:n]picks one of them - Normalization — the choice that decides whether
s[::-1]is lossless stripis a set, not a prefix — the otherstroperation whose argument is not the thing it looks like- What to write next — this page closes the slicing questions; the method tour has more
- The crosswalk — Python's start/stop, ABAP's offset/length and Rust's byte range, in one row
- Slicing by byte ↗ — the same brackets in Rust, where the range is bytes and the boundary is enforced
- A code point is not a character ↗ — the grapheme cluster, which is the unit
s[::-1]should have been reversing - Sequence types in the Python docs ↗ — including the note that a slice with out-of-range bounds is not an error