Skip to content

Range over a function

Level: 201 · anyone who has ranged over a slice, a map or a channel and wants to range over a sequence of their own

One line: Since Go 1.23 a for … range accepts a function of type func(yield func(V) bool): the loop calls it once, the loop body becomes the yield argument, and the two sides take turns on one goroutine — iter.Seq[V] and iter.Seq2[K, V] are just names for those function types.

Readings is an iterator. It owns no loop for anyone to be inside; it calls yield once per reading and prints before and after each call. main ranges over it and prints what it gets, and the two sides' lines interleave in an order that cannot vary, because it is one goroutine calling and returning. Then main does the same thing without the range, by calling Readings with a function literal; assigns Readings to an iter.Seq[int]; ranges over an iter.Seq2[string, int] with two variables; and ranges over the third accepted shape, func(func() bool), which yields no values at all.

Verified output of range_over_a_function_go.go — regenerated by tools/run_examples.py, never hand-typed.

-- for r := range Readings
producer: yielding 21
loop: got 21
producer: yield returned true, continuing
producer: yielding 19
loop: got 19
producer: yield returned true, continuing
producer: yielding 23
loop: got 23
producer: yield returned true, continuing
producer: no more readings, returning
main: the loop ended
-- the same thing, written as a call
producer: yielding 21
loop: got 21
producer: yield returned true, continuing
producer: yielding 19
loop: got 19
producer: yield returned true, continuing
producer: yielding 23
loop: got 23
producer: yield returned true, continuing
producer: no more readings, returning
-- iter.Seq and iter.Seq2 are names for these function types
iter.Seq[int]
iter.Seq2[string,int]
loop: hall = 21
loop: garage = 19
loop: attic = 23
-- func(func() bool) is the third shape: no values at all
loop: tick
loop: tick

Reading the output

yielding 21 comes before got 21, and yield returned true comes after it. The spec ↗ says how a range over a function f proceeds: f is called with a synthesized yield function, the arguments of each yield call become the iteration values for one execution of the loop body, and after the body has run yield returns true. So the loop body ran inside the producer's call to yield — the producer is on the stack, waiting for its call to return, while the body prints. One goroutine, one stack, no scheduler: the interleaving is fixed.

no more readings, returning, then the loop ended. The loop ends when f returns. There is no close, no sentinel value and no ok flag: the producer ends the sequence by returning from its own function. That is the difference from Closing a channel ends a range, where the sender has to say "no more" with a call, and forgetting it hangs the receiver.

The hand-written call prints exactly the same eleven lines. for r := range Readings { body } runs as Readings(func(r int) bool { body; return true }). That is the mental model the Go wiki ↗ gives for the feature, and the compiler's rewrite ↗ is that call plus bookkeeping; the -gcflags=-m view in what go vet and gofmt catch shows the body under the name main-range1. The return true at the end of the literal is what falling off the end of the body, and continue, become. break becomes return false, and the next lesson, break makes yield return false, is about what the producer has to do with that.

iter.Seq[int] and iter.Seq2[string,int]. %T of the two variables. The iter package ↗ declares type Seq[V any] func(yield func(V) bool) and type Seq2[K, V any] func(yield func(K, V) bool), nothing more: a plain function such as Readings is assignable to a Seq[int] because the two have the same underlying type and only one of them is a named type (spec, Assignability ↗). Seq2 passes two values per element — an index and a value, or a key and a value — which is what a two-variable range needs; range labelled with one variable would also compile and take the first.

func(func() bool): two ticks and no value. The release notes ↗ list the three accepted signatures: func(func() bool), func(func(K) bool) and func(func(K, V) bool). The zero-value form is for "do this N times" sequences where the count is the producer's business.

range_over_a_function_go.go

range_over_a_function_go.go in full — pasted here by tools/run_examples.py from the file CI runs.

// A function of type func(yield func(int) bool) can be ranged over. The loop
// body becomes the yield argument, so the producer and the loop take turns on
// one goroutine, and the order in which the two sides print cannot vary.
//
//  go build range_over_a_function_go.go && ./range_over_a_function_go
package main

import (
    "fmt"
    "iter"
)

// Readings is an iterator: it calls yield once per reading, and stops as soon
// as yield returns false.
func Readings(yield func(int) bool) {
    for _, r := range []int{21, 19, 23} {
        fmt.Println("producer: yielding", r)
        if !yield(r) {
            fmt.Println("producer: the loop asked me to stop")
            return
        }
        fmt.Println("producer: yield returned true, continuing")
    }
    fmt.Println("producer: no more readings, returning")
}

// Labelled yields each reading with the name of its sensor: two values per
// element, which is what a two-variable range needs.
func Labelled(yield func(string, int) bool) {
    _ = yield("hall", 21) && yield("garage", 19) && yield("attic", 23)
}

func main() {
    fmt.Println("-- for r := range Readings")
    for r := range Readings {
        fmt.Println("loop: got", r)
    }
    fmt.Println("main: the loop ended")

    fmt.Println("-- the same thing, written as a call")
    Readings(func(r int) bool {
        fmt.Println("loop: got", r)
        return true
    })

    fmt.Println("-- iter.Seq and iter.Seq2 are names for these function types")
    var readings iter.Seq[int] = Readings
    var labelled iter.Seq2[string, int] = Labelled
    fmt.Printf("%T\n%T\n", readings, labelled)
    for name, r := range labelled {
        fmt.Printf("loop: %s = %d\n", name, r)
    }

    fmt.Println("-- func(func() bool) is the third shape: no values at all")
    ticks := func(yield func() bool) {
        for range 2 {
            if !yield() {
                return
            }
        }
    }
    for range ticks {
        fmt.Println("loop: tick")
    }
}

What to do

  • Write an iterator as func(yield func(V) bool) and hand it out as an iter.Seq[V] — return the named type, so callers can pass it to slices.Collect and to adapters; on a collection, name the method All, as the iter package's naming section ↗ asks.
  • Check yield's result every time, and return when it is false. The next lesson shows the panic an iterator gets for ignoring it.
  • Think of the loop body as a callback the producer calls. Code after the range runs only after the producer has returned; an iterator that does I/O does it in the middle of the loop, on the loop's goroutine.
  • Range over the function rather than over a slice it could have returned when the caller may stop early or the sequence has no natural end — An iterator is lazy.

Practice

Write Countdown(from int) iter.Seq[int] — a constructor that captures from and returns the iterator — printing from both sides so the call order shows, and Indexed(names []string) iter.Seq2[int, string] that yields each sensor name with its position. Range over both, then call Indexed by hand with a function literal that does what the loop body did, to prove they are the same call.

Solution

range_over_a_function_kata_go.go in full — pasted here by tools/run_examples.py from the file CI runs.

// Kata: an iterator made by a constructor that captures its argument, and a
// Seq2 that pairs each element with its position -- ranged over, and then
// called by hand with the same function the loop body became.
//
//  go build range_over_a_function_kata_go.go && ./range_over_a_function_kata_go
package main

import (
    "fmt"
    "iter"
)

// Countdown returns an iterator over from, from-1, ..., 1. The returned
// function is the iterator; Countdown itself only captures from.
func Countdown(from int) iter.Seq[int] {
    return func(yield func(int) bool) {
        for n := from; n >= 1; n-- {
            fmt.Println("countdown: yielding", n)
            if !yield(n) {
                return
            }
        }
        fmt.Println("countdown: done")
    }
}

// Indexed yields each sensor name with its position.
func Indexed(names []string) iter.Seq2[int, string] {
    return func(yield func(int, string) bool) {
        for i, name := range names {
            if !yield(i, name) {
                return
            }
        }
    }
}

func main() {
    for n := range Countdown(3) {
        fmt.Println("loop: got", n)
    }
    for i, name := range Indexed([]string{"hall", "garage", "attic"}) {
        fmt.Printf("loop: sensor %d is %s\n", i, name)
    }
    // The loop above and this call are the same thing.
    Indexed([]string{"hall", "garage"})(func(i int, name string) bool {
        fmt.Printf("by hand: sensor %d is %s\n", i, name)
        return true
    })
}

Verified output of range_over_a_function_kata_go.go — regenerated by tools/run_examples.py, never hand-typed.

countdown: yielding 3
loop: got 3
countdown: yielding 2
loop: got 2
countdown: yielding 1
loop: got 1
countdown: done
loop: sensor 0 is hall
loop: sensor 1 is garage
loop: sensor 2 is attic
by hand: sensor 0 is hall
by hand: sensor 1 is garage

In other languages

  • The Rust library's Implementing Iterator ↗ and Returning an iterator ↗ — a Rust iterator is a pull type: the consumer calls next() and gets an Option<Item>. Go's is push: the producer calls yield and the consumer is the function it calls. impl Iterator<Item = T> in a Rust signature is the counterpart of iter.Seq[T] here; iter.Pull needs its stop is where Go gets a next().
  • Python's generators ↗ are the closest cousin in spelling: a function with yield in it. But a Python yield suspends the function and hands the value out to whoever called next(); Go's yield is an ordinary function parameter, and the producer is the one doing the calling.
  • The Concurrency library's Coroutine ↗ — the wiki warns against reading a range over a function as a coroutine: it is calls on one stack. The coroutine arrives with iter.Pull.

Sources