Skip to content

slices and maps have iterators

Level: 201 · anyone who has written for _, k := range keys after building keys by hand

One line: The standard library hands out its sequences as iterators — slices.Values, All, Backward and Chunk, maps.Keys, Values and All, strings.SplitSeq and Lines — and takes them back with slices.Collect and maps.Collect; slices.Sorted(maps.Keys(m)) is the way to get a fixed order out of a map, and the strings and bytes iterators are documented single-use, so a second range over one finds nothing.

The program walks a slice of readings four ways, collects a hand-written iterator, chunks five orders into pairs, sorts a map's keys and values before printing anything from it, copies a map through maps.All and maps.Collect, splits a string and iterates its lines, and finally ranges twice over slices.Values, over strings.Lines, and over a line iterator built on an io.Reader. Every line it prints is in an order that cannot vary.

Verified output of slices_and_maps_have_iterators_go.go — regenerated by tools/run_examples.py, never hand-typed.

-- slices
Values: 21
Values: 19
Values: 23
All: 0 21
All: 1 19
All: 2 23
Backward: 2 23
Backward: 1 19
Backward: 0 21
Collect: [42 38 46]
Chunk: [order-1 order-2]
Chunk: [order-3 order-4]
Chunk: [order-5]
-- maps: a map iterator has no fixed order, so sort first
Sorted(Keys): [bolt nut washer]
Sorted(Values): [7 12 40]
bolt: 40
nut: 12
washer: 7
Collect(All) equal to the original: true
-- strings and bytes
SplitSeq: "bolt"
SplitSeq: "nut"
SplitSeq: ""
SplitSeq: "washer"
Lines: "first\n"
Lines: "second\n"
Lines: "third"
bytes.Lines: "first\r\n"
bytes.Lines: "second\n"
-- reusable, and single-use
slices.Values, twice: [21 19 23] [21 19 23]
strings.Lines, twice: ["first\n" "second"] []
LinesOf a reader, twice: ["first" "second"] []

Reading the output

Values, All, Backward. slices.Values ↗ is a Seq[E] over the elements; All ↗ a Seq2[int, E] of index and element, which is what for i, r := range readings gives without the function; Backward ↗ the same pairs from the last index down. All three are Go 1.23.

Collect: [42 38 46]. slices.Collect ↗ takes any Seq[E] — here a function literal written on the spot — and appends its values to a new slice. It is the bridge back from An iterator is lazy to an ordinary slice, and it is the consumer that makes the lazy work happen.

Chunk yields sub-slices of at most 2, the last shorter. slices.Chunk ↗ yields views into the original slice, not copies, with their capacity clipped to their length so that an append to a chunk cannot overwrite the next one. (The clipping is documented, not printed here.)

A map iterator has no fixed order, so the program sorts first. maps.Keys ↗: the order is not specified and not guaranteed to be the same from one call to the next — as with any range over a map. This library's rule is never to print anything from a map in map order, and slices.Sorted ↗ is the idiom: it collects an iterator into a slice and sorts it, and the iter package ↗ gives for _, key := range slices.Sorted(maps.Keys(m)) as its own example. maps.Collect(maps.All(stock)) is a copy of the map, and maps.Equal confirms it.

SplitSeq yields the empty field; Lines keeps the newline. strings.SplitSeq ↗ yields the same strings Split would return, including the "" between the two commas, without building the slice. strings.Lines ↗ yields each line with its terminating \n, and the last line without one when the string does not end in a newline; bytes.Lines ↗ splits on \n only, so a \r\n line keeps both bytes. These are Go 1.24, with SplitAfterSeq, FieldsSeq and FieldsFuncSeq beside them.

Reusable, and single-use. slices.Values ranged twice gave [21 19 23] twice: each call to the iterator walks the slice again. strings.Lines ranged twice gave ["first\n" "second"] and then []: its documentation says it "returns a single-use iterator", and its source ↗ shows why — the closure advances the s it captured, so the second call starts from an empty string. SplitSeq says the same. LinesOf over a strings.Reader is single-use for the plainer reason that the reader has been read; the iter package's Single-Use Iterators ↗ section asks doc comments to say so, and these do.

slices_and_maps_have_iterators_go.go

slices_and_maps_have_iterators_go.go in full — pasted here by tools/run_examples.py from the file CI runs.

// The standard library's iterators: slices.Values, All, Backward, Collect and
// Chunk; maps.Keys, Values, All and Collect, with slices.Sorted as the way to
// get a fixed order out of a map; strings.SplitSeq and Lines, bytes.Lines.
// Every line printed here is in an order that cannot vary.
//
//  go build slices_and_maps_have_iterators_go.go && ./slices_and_maps_have_iterators_go
package main

import (
    "bufio"
    "bytes"
    "fmt"
    "io"
    "iter"
    "maps"
    "slices"
    "strings"
)

// LinesOf reads r line by line. It is a single-use iterator: r cannot be
// rewound, so a second range over it finds nothing. strings.Lines is
// documented as single-use too, and the program checks that.
func LinesOf(r io.Reader) iter.Seq[string] {
    return func(yield func(string) bool) {
        scanner := bufio.NewScanner(r)
        for scanner.Scan() {
            if !yield(scanner.Text()) {
                return
            }
        }
    }
}

func main() {
    readings := []int{21, 19, 23}
    fmt.Println("-- slices")
    for r := range slices.Values(readings) {
        fmt.Println("Values:", r)
    }
    for i, r := range slices.All(readings) {
        fmt.Println("All:", i, r)
    }
    for i, r := range slices.Backward(readings) {
        fmt.Println("Backward:", i, r)
    }
    doubled := func(yield func(int) bool) {
        for _, r := range readings {
            if !yield(2 * r) {
                return
            }
        }
    }
    fmt.Println("Collect:", slices.Collect(doubled))
    for batch := range slices.Chunk([]string{"order-1", "order-2", "order-3", "order-4", "order-5"}, 2) {
        fmt.Println("Chunk:", batch)
    }

    fmt.Println("-- maps: a map iterator has no fixed order, so sort first")
    stock := map[string]int{"bolt": 40, "nut": 12, "washer": 7}
    fmt.Println("Sorted(Keys):", slices.Sorted(maps.Keys(stock)))
    fmt.Println("Sorted(Values):", slices.Sorted(maps.Values(stock)))
    for _, part := range slices.Sorted(maps.Keys(stock)) {
        fmt.Printf("%s: %d\n", part, stock[part])
    }
    copied := maps.Collect(maps.All(stock))
    fmt.Println("Collect(All) equal to the original:", maps.Equal(stock, copied))

    fmt.Println("-- strings and bytes")
    for field := range strings.SplitSeq("bolt,nut,,washer", ",") {
        fmt.Printf("SplitSeq: %q\n", field)
    }
    for line := range strings.Lines("first\nsecond\nthird") {
        fmt.Printf("Lines: %q\n", line)
    }
    for line := range bytes.Lines([]byte("first\r\nsecond\n")) {
        fmt.Printf("bytes.Lines: %q\n", line)
    }

    fmt.Println("-- reusable, and single-use")
    values := slices.Values(readings)
    fmt.Printf("slices.Values, twice: %v %v\n", slices.Collect(values), slices.Collect(values))
    text := "first\nsecond"
    lines := strings.Lines(text)
    fmt.Printf("strings.Lines, twice: %q %q\n", slices.Collect(lines), slices.Collect(lines))
    fromReader := LinesOf(strings.NewReader(text))
    fmt.Printf("LinesOf a reader, twice: %q %q\n", slices.Collect(fromReader), slices.Collect(fromReader))
}

What to do

  • Take and return iter.Seq and iter.Seq2 in your own APIs, and slices.Collect at the edge where a slice is needed — the iter package ↗ asks for the standard types so that everything composes.
  • Sort before printing anything from a map: slices.Sorted(maps.Keys(m)), then index the map. A test that ranges over a map and compares output is a flaky test.
  • Read the doc comment for "single-use" before ranging twice. A strings.Lines value is one pass; call strings.Lines(text) again for a second.
  • Prefer SplitSeq and Lines when the loop is all you do with the pieces, and keep Split when you need the slice — its length, an index, or a second pass.

Practice

From stock map[string]int, print the parts with fewer than 10 left in alphabetical order. Count the columns of a CSV header line with strings.SplitSeq without building a slice. Then split seven orders into batches of three with slices.Chunk and print each batch's size, first and last order.

Solution

slices_and_maps_have_iterators_kata_go.go in full — pasted here by tools/run_examples.py from the file CI runs.

// Kata: parts to reorder in alphabetical order out of a map, the columns of a
// CSV header counted without building a slice, and orders in batches of
// three -- all with the standard library's iterators.
//
//  go build slices_and_maps_have_iterators_kata_go.go && ./slices_and_maps_have_iterators_kata_go
package main

import (
    "fmt"
    "maps"
    "slices"
    "strings"
)

func main() {
    stock := map[string]int{"bolt": 40, "nut": 4, "washer": 7, "screw": 12, "rivet": 2}
    fmt.Println("-- parts with fewer than 10 left, in alphabetical order")
    for _, part := range slices.Sorted(maps.Keys(stock)) {
        if stock[part] < 10 {
            fmt.Printf("%s (%d left)\n", part, stock[part])
        }
    }

    fmt.Println("-- columns of a CSV header, counted without a slice")
    columns := 0
    for column := range strings.SplitSeq("order,customer,part,quantity", ",") {
        columns++
        fmt.Printf("column %d: %s\n", columns, column)
    }
    fmt.Println("columns:", columns)

    fmt.Println("-- orders in batches of three")
    orders := []string{"order-1", "order-2", "order-3", "order-4", "order-5", "order-6", "order-7"}
    for batch := range slices.Chunk(orders, 3) {
        fmt.Printf("batch of %d: %s .. %s\n", len(batch), batch[0], batch[len(batch)-1])
    }
}

Verified output of slices_and_maps_have_iterators_kata_go.go — regenerated by tools/run_examples.py, never hand-typed.

-- parts with fewer than 10 left, in alphabetical order
nut (4 left)
rivet (2 left)
washer (7 left)
-- columns of a CSV header, counted without a slice
column 1: order
column 2: customer
column 3: part
column 4: quantity
columns: 4
-- orders in batches of three
batch of 3: order-1 .. order-3
batch of 3: order-4 .. order-6
batch of 1: order-7 .. order-7

In other languages

  • The Rust library's iter, iter_mut, into_iter ↗ and Collect the iterator into a Vec ↗ — every Rust collection has three doors onto it and collect() is the way back; into_iter() consumes the collection, which is the ownership version of single-use. Go's slices.Values lends and never consumes; Go's single-use iterators are single-use because of what they read, not what they own.
  • Python's dict views ↗ keep insertion order, so sorted(d) is a choice rather than a necessity; and a str has no lazy splitlines, only the list-returning one — the lazy form is a file object iterated line by line.

Sources