Skip to content

An iterator is a method that yields

Level: 201 · anyone who wants map, select and lazy on a class of their own

One line: A class earns the whole Enumerable toolbox by writing one each that yields and including the module; return to_enum(:each) unless block_given? makes each without a block an Enumerator, and Enumerator.new { |y| … } builds one — lazily, even infinite — without a class at all.

An iterator in Ruby is not a protocol of next calls but a method that yields each element to a block: Countdown#each yields 3, 2, 1 and returns self (row 1). Add include Enumerable and the class, which defines nothing but each, answers map, select, sort, include?, min, sum and first (row 2), then each_slice, each_cons, each_with_index, reduce and to_a (row 3) — the module implements every one of them by calling each. The line return to_enum(:each) { @from } unless block_given? is what makes c.each with no block an Enumerator: an object that knows its size from that block, can be pulled with next, and chains with with_index (row 4); enum_for(__method__, xs) does the same for a plain method (row 5). An Enumerator can also be built without a class: Enumerator.new { |y| y << 1; y.yield 2 } runs its block only when consumed, so next pulls one value at a time until StopIteration, and its size is nil because nothing declared one (row 6). The block may loop forever — the consumer stops it with take, first or a slice (row 7) — and lazy chains select and map over an infinite source without materialising anything (row 8). Enumerator.produce builds the same kind of thing from a seed and a step (row 9). All of this is a mixin, so the class and its instances both report Enumerable (row 10).

Verified output of writing_an_iterator_rb.rb — regenerated by tools/run_examples.py, never hand-typed.

 1. each yields; with a block it returns self:         c.each { }.equal?(c) = true
 2. the class defines each, Enumerable does the rest:  instance_methods(false) [:each]; map [30, 20, 10], select [3, 1], sort [1, 2, 3], include?(2) true, min 1, sum 6, first(2) [3, 2]
 3. more for free:                                     each_slice(2) [[3, 2], [1]], each_cons(2) [[3, 2], [2, 1]], each_with_index [[3, 0], [2, 1], [1, 2]], reduce(:+) 6, to_a [3, 2, 1]
 4. each without a block is an Enumerator:             Enumerator, size 3, next 3, with_index(1) [[3, 1], [2, 2], [1, 3]]
 5. enum_for(__method__) in a plain method:            pair_sums([1, 2, 3]) is Enumerator, to_a [3, 5]
 6. Enumerator.new { |y| y << 1; y.yield 2 }:          to_a [1, 2, 3], next x3 [1, 2, 3], then StopIteration, size nil
 7. an infinite enumerator, consumed a piece at a time: take(8) [0, 1, 1, 2, 3, 5, 8, 13], first(5) [0, 1, 1, 2, 3], each_slice(3).first(2) [[0, 1, 1], [2, 3, 5]]
 8. lazy chains filters without materialising:         fib.lazy.select(&:even?).first(4) [0, 2, 8, 34], class Enumerator::Lazy, (1..).lazy.map { _1 * _1 }.first(3) [1, 4, 9]
 9. Enumerator.produce, a generator from a seed:       produce(1) { _1 * 2 }.take(5) [1, 2, 4, 8, 16]
10. Enumerable is a mixin, so the class says so:       Countdown.include?(Enumerable) true, c.is_a?(Enumerable) true

Reading the output

Row 1 confirms each with a block hands back the receiver, the convention that lets a call be chained. Row 2 lists [:each] as the only method Countdown defines and then seven results that came from Enumerable; row 3 adds five more, including each_with_index pairing each value with its position. Row 4 shows c.each with no block: an Enumerator whose size is 3 because the to_enum block said so, whose next is 3, and whose with_index(1) numbers from one. Row 5 does the same trick in a method with no class around it, pair_sums, returning an Enumerator that yields [3, 5]. Row 6 pulls 1, 2, 3 out of a hand-built enumerator, hits StopIteration on the fourth next, and reports size nil. Row 7 takes eight, five and two slices of three from a Fibonacci enumerator that never ends; row 8 filters the same source lazily to the first four even values and squares an endless range to [1, 4, 9], with Enumerator::Lazy as the class of the chain. Row 9 doubles from a seed of 1 five times. Row 10 is true twice.

Compared with Python

Python's iterator is a protocol rather than a method that yields: for calls iter(c), which calls __iter__, and then calls next until StopIteration. Writing __iter__ as a generator function — a def containing yield — is the idiom, and iter(c) returns a generator (row 1). There is no mixin to include: the class defines nothing but __iter__, and comprehensions, sorted, in, min, sum and itertools.islice accept any iterable (row 2), with batched, pairwise, enumerate and functools.reduce covering row 3. Countdown is an Iterable and not an Iterator — the generator that iter(c) returns is the iterator — and there is no len unless the class defines __len__ (row 4), where Ruby's to_enum took a size block. A plain def with yield is Ruby's enum_for and Enumerator.new at once (rows 5 and 6): calling it returns a generator that runs only when pulled, and running off its end raises StopIteration. Infinite generators and islice match rows 7 and 8, with one difference in the default: filter and map are already lazy iterators in Python, so there is no lazy to ask for. itertools.accumulate from a seed is Enumerator.produce (row 9), and the collections.abc classes check the protocol the way include?(Enumerable) checks the mixin (row 10). The protocol is documented under iterator types ↗, yield expressions ↗ and itertools ↗.

Verified output of writing_an_iterator_py.py — regenerated by tools/run_examples.py, never hand-typed.

 1. iter(c) calls __iter__ and gets a generator:       type generator, next(it) = 3
 2. the class defines __iter__, the builtins do the rest: own methods none; map [30, 20, 10], filter [3, 1], sorted [1, 2, 3], 2 in c True, min 1, sum 6, islice(2) [3, 2]
 3. itertools for the rest:                            batched(2) [(3, 2), (1,)], pairwise [(3, 2), (2, 1)], enumerate [(0, 3), (1, 2), (2, 1)], reduce 6, list [3, 2, 1]
 4. c is iterable, iter(c) is the iterator:            Iterable True, Iterator False / True, len(c) TypeError, enumerate(c, 1) [(1, 3), (2, 2), (3, 1)]
 5. a generator function in a plain def:               pair_sums([1, 2, 3]) is generator, list [3, 5]
 6. def three(): yield 1; yield 2; yield 3:            list [1, 2, 3], next x3 [1, 2, 3], then StopIteration, size: no len on a generator
 7. an infinite generator, consumed a piece at a time: islice(8) [0, 1, 1, 2, 3, 5, 8, 13], islice(5) [0, 1, 1, 2, 3], batched(3) first 2 [(0, 1, 1), (2, 3, 5)]
 8. lazy by default: filter and map are iterators:     islice(filter(even, fib()), 4) [0, 2, 8, 34], class filter, islice(map(sq, count(1)), 3) [1, 4, 9]
 9. accumulate as a generator from a seed:             accumulate(repeat(2), mul, initial=1) [1, 2, 4, 8, 16]
10. Iterable is a protocol, checked by an ABC:         isinstance(c, Iterable) True, isgeneratorfunction(Countdown.__iter__) True

Try it

  1. Write a Ring class over an array that yields its elements starting from an offset and wrapping around — Ring.new([1, 2, 3, 4], 1) yields 2, 3, 4, 1 — with include Enumerable, a to_enum for the block-less call, and a size block so each.size is right. Then use map, each_slice, lazy and each_with_index on it.
  2. Build a primes enumerator with Enumerator.new and an endless loop. Take the first ten with take(10), the first prime above 100 with find, and every pair of twin primes below 50 with each_cons(2) and take_while. Then write the same generator in Python and note which of the three calls needed islice.
Solution to 1

writing_an_iterator_kata_rb.rb in full — pasted here by tools/run_examples.py from the file CI runs.

# Exercise 1: Ring yields an array's elements starting at an offset and
# wrapping around; one each, Enumerable included, an Enumerator with the
# right size when no block is given.

class Ring
  include Enumerable

  def initialize(items, offset = 0)
    @items = items
    @offset = offset
  end

  def each
    return to_enum(:each) { @items.size } unless block_given?
    @items.size.times { |i| yield @items[(i + @offset) % @items.size] }
    self
  end
end

ring = Ring.new([1, 2, 3, 4], 1)
puts "to_a            #{ring.to_a}"
puts "map             #{ring.map { _1 * 10 }}"
puts "each_slice(2)   #{ring.each_slice(2).to_a}"
puts "each.size       #{ring.each.size}"
puts "each.next       #{ring.each.next}"
puts "lazy.select     #{ring.lazy.select(&:even?).first(1)}"
puts "each_with_index #{ring.each_with_index.map { |v, i| "#{i}:#{v}" }.join(' ')}"

Verified output of writing_an_iterator_kata_rb.rb — regenerated by tools/run_examples.py, never hand-typed.

to_a            [2, 3, 4, 1]
map             [20, 30, 40, 10]
each_slice(2)   [[2, 3], [4, 1]]
each.size       4
each.next       2
lazy.select     [2]
each_with_index 0:2 1:3 2:4 3:1

See also