Skip to content

Enumerable is a mixin: write each, get the rest

Level: 201 · a Python programmer who has written __iter__ and wonders where Ruby's map lives

One line: A class that defines each and says include Enumerable gains map, select, sort_by, min_by, include?, first, each_slice, lazy and fifty-odd more, because Ruby keeps the toolbox in a module that is mixed into the class, where Python keeps it in builtins and itertools and only collections.abc.Sequence plays the same trick.

Ruby's collection methods are not defined on Array and copied to Hash, Range, Struct, Set, Dir and Enumerator: they are defined once, in the module Enumerable, in terms of one method the module does not define — each. A class that writes each and includes the module gets every other method for the price of one line, and the module has no idea what it is iterating, only that each will yield the items one at a time. Shelf below is the whole recipe: an each that yields (and returns an Enumerator when called without a block, so shelf.each.next works too), plus include Enumerable. Bare is the same class without the include, to show that each on its own buys nothing.

Verified output of enumerable_is_a_mixin_rb.rb — regenerated by tools/run_examples.py, never hand-typed.

A class with only `each`, before and after `include Enumerable`
 1. Bare.new.respond_to?(:map)  (each, no include)       false
 2. Bare.new.map { } raises                              NoMethodError
 3. Shelf.ancestors.take(3)                              [Shelf, Enumerable, Object]
 4. shelf.respond_to?(:map)                              true

What one `each` bought (a handful of the toolbox)
 5. map(&:upcase)                                        ["EMMA", "DRACULA", "MIDDLEMARCH", "PERSUASION"]
 6. select { |t| t.size > 6 }                            ["Dracula", "Middlemarch", "Persuasion"]
 7. sort_by(&:size)                                      ["Emma", "Dracula", "Persuasion", "Middlemarch"]
 8. min_by(&:size)                                       "Emma"
 9. include?("Emma")                                     true
10. first, first(2)                                      ["Emma", ["Emma", "Dracula"]]
11. each_slice(3).to_a                                   [["Emma", "Dracula", "Middlemarch"], ["Persuasion"]]
12. to_a                                                 ["Emma", "Dracula", "Middlemarch", "Persuasion"]
13. each_with_index.to_a                                 [["Emma", 0], ["Dracula", 1], ["Middlemarch", 2], ["Persuasion", 3]]
14. sum("")                                              "EmmaDraculaMiddlemarchPersuasion"
15. lazy.class                                           Enumerator::Lazy
16. lazy.map(&:downcase).first(2)                        ["emma", "dracula"]
17. each.class, each.next  (no block given)              [Enumerator, "Emma"]

Core classes that include Enumerable, and two that do not
18. Array.include?(Enumerable)                           true
19. Hash.include?(Enumerable)                            true
20. Range.include?(Enumerable)                           true
21. Struct.include?(Enumerable)                          true
22. Dir.include?(Enumerable)                             true
23. Set.include?(Enumerable)                             true
24. Enumerator.include?(Enumerable)                      true
25. String.include?(Enumerable)                          false
26. Integer.include?(Enumerable)                         false

The mixin, seen from the module's side
27. Enumerable.instance_methods.include?(:each)          false
28. ...include?(:tally), ...size > 50                    [true, true]
29. Array.instance_method(:map).owner, (:inject).owner   [Array, Enumerable]
30. Comparable.instance_methods.sort  (the other mixin)  [:<, :<=, :==, :>, :>=, :between?, :clamp]
31. Shelf.include?(Comparable)  (mixins are opt-in)      false

Reading the output

Rows 1–4: Bare has an each but no map: respond_to?(:map) is false and calling it raises NoMethodError. Shelf differs by one line, and its ancestor chain shows where the methods come from — Enumerable sits between the class and Object, so a call to shelf.map misses in Shelf, hits in Enumerable, and the module's map calls shelf.each to get the items.

Rows 5–17: a handful of what arrived. map, select, sort_by, min_by, include?, first, each_slice, to_a, each_with_index, sum and lazy all work on a class that only knows how to yield four strings. Row 17 is the to_enum guard paying off: each without a block returns an Enumerator, and next pulls the first title — the external enumerators page is about that object.

Rows 18–26: the core classes that include the module. Array, Hash, Range, Struct, Dir, Set and Enumerator do; String and Integer do not. A Ruby string is not a collection of anything until you ask for each_char, each_byte or each_line, each of which returns an Enumerator that is (Not machine-checked here.).

Rows 27–31: the module from its own side. Enumerable.instance_methods does not include each (row 27) — that is the one method the includer supplies — but it does include tally, and there are more than fifty of them (row 28 prints a comparison rather than the count, which is version-sized). Row 29 is worth a look: Array overrides map with its own version (its owner is Array) but leaves inject to the module (owner Enumerable), so the same call on an Array and on a Shelf may run different code and give the same answer. Rows 30–31 show the other standard mixin, Comparable: define <=>, include it, and <, <=, ==, >, >=, between? and clamp arrive the same way — and Shelf does not have them, because a mixin is opt-in, one include at a time.

Compared with Python

Python has no module to include. An object with __iter__ is iterable, and that is the whole protocol: isinstance(shelf, Iterable) is True in row 3 without Shelf inheriting from anything, because collections.abc.Iterable recognises the method by duck typing. But nothing arrives on the object — hasattr(shelf, "map") is False in rows 1 and 4 and stays False. The toolbox is elsewhere: map, sorted, min, enumerate and sum are builtins that take an iterable, islice and batched live in itertools, and [t for t in shelf if …] is syntax. Rows 5–17 are the same handful in that form; row 14 is a real difference (sum refuses strings and the fix is "".join), and row 15 shows that Python's map is lazy already — an iterator, not a list — so there is no .lazy to call.

Rows 18–26 put the same builtins to the ABC. Every container answers True, and so does "abc": a Python str iterates its characters, where Ruby's String is not Enumerable at all. Rows 27–31 are Python's one real mixin. Iterable asks for a single method, __iter__, and gives back nothing but the isinstance check; collections.abc.Sequence asks for two, __getitem__ and __len__, and Deck — which defines exactly those — gets in, index, count and reversed for free in row 31. That is Ruby's Enumerable mechanism, applied to sequences only.

Verified output of enumerable_is_a_mixin_py.py — regenerated by tools/run_examples.py, never hand-typed.

A class with only __iter__: nothing to include, and no methods arrive
 1. hasattr(bare, "map")  (__iter__, nothing else)       False
 2. bare.map(...) raises                                 'AttributeError'
 3. Shelf.__mro__ names, isinstance(shelf, Iterable)     (['Shelf', 'object'], True)
 4. hasattr(shelf, "map")                                False

The same handful, as builtins and itertools taking the iterable
 5. list(map(str.upper, shelf))                          ['EMMA', 'DRACULA', 'MIDDLEMARCH', 'PERSUASION']
 6. [t for t in shelf if len(t) > 6]                     ['Dracula', 'Middlemarch', 'Persuasion']
 7. sorted(shelf, key=len)                               ['Emma', 'Dracula', 'Persuasion', 'Middlemarch']
 8. min(shelf, key=len)                                  'Emma'
 9. "Emma" in shelf  (in falls back to __iter__)         True
10. next(iter(shelf)), list(islice(shelf, 2))            ['Emma', ['Emma', 'Dracula']]
11. list(batched(shelf, 3))  (3.12+)                     [('Emma', 'Dracula', 'Middlemarch'), ('Persuasion',)]
12. list(shelf)                                          ['Emma', 'Dracula', 'Middlemarch', 'Persuasion']
13. list(enumerate(shelf))                               [(0, 'Emma'), (1, 'Dracula'), (2, 'Middlemarch'), (3, 'Persuasion')]
14. sum(shelf, "") raises; "".join(shelf)                ['TypeError', 'EmmaDraculaMiddlemarchPersuasion']
15. type(map(str.lower, shelf)).__name__  (lazy already) 'map'
16. list(islice(map(str.lower, shelf), 2))               ['emma', 'dracula']
17. type(iter(shelf)).__name__, next(iter(shelf))        ['generator', 'Emma']

Builtins that are Iterable, and one that is not
18. isinstance([], Iterable)                             True
19. isinstance({}, Iterable)                             True
20. isinstance(range(3), Iterable)                       True
21. isinstance(Point(1, 2), Iterable)                    True
22. isinstance(os.scandir("."), Iterable)                True
23. isinstance(set(), Iterable)                          True
24. isinstance(iter([]), Iterable)                       True
25. isinstance("abc", Iterable)                          True
26. isinstance(3, Iterable)                              False

The mixin idea, seen from the ABCs' side
27. sorted(Iterable.__abstractmethods__)                 ['__iter__']
28. hasattr(shelf, "sorted"), "sorted" in dir(builtins)  [False, True]
29. list.__mro__ names, hasattr(list, "map")             (['list', 'object'], False)
30. sorted(Sequence.__abstractmethods__)  (a real mixin) ['__getitem__', '__len__']
31. deck: "K" in, .index("Q"), .count("K"), reversed     [True, 2, 2, ['K', 'Q', 'K', 'A']]

The Python docs page for the ABCs, collections.abc ↗, has a table of which mixin methods each ABC provides; it is the Python counterpart of the Enumerable method list.

Try it

  1. Write a Fib class whose each yields the Fibonacci numbers up to a limit given to new, include Enumerable, and print to_a, select(&:even?), the differences between neighbours (each_cons(2)), include?, sum, min_by, each_slice(4), each.next and a lazy chain — nothing but each is yours to write.
  2. Make a class whose each yields two values per item (yield key, value, as Hash#each does) and print what map { |k, v| }, map { |pair| }, to_a and first give you; then change it to yield [key, value] and compare. What does each_with_index yield in each case?
Solution to 1

enumerable_is_a_mixin_kata_rb.rb in full — pasted here by tools/run_examples.py from the file CI runs.

# Kata: a Fibonacci sequence that stops at a limit, made Enumerable by one `each`.
class Fib
  include Enumerable

  def initialize(limit) = @limit = limit

  def each
    return to_enum(:each) unless block_given?
    a, b = 0, 1
    while a <= @limit
      yield a
      a, b = b, a + b
    end
    self
  end
end

def row(n, label, value) = puts(format("%2d. %-42s %s", n, label, value.inspect))

fib = Fib.new(100)
row 1, "to_a", fib.to_a
row 2, "select(&:even?)", fib.select(&:even?)
row 3, "each_cons(2).map { |a, b| b - a }", fib.each_cons(2).map { |a, b| b - a }
row 4, "include?(21), include?(22)", [fib.include?(21), fib.include?(22)]
row 5, "sum", fib.sum
row 6, "min_by { |x| (x - 50).abs }", fib.min_by { |x| (x - 50).abs }
row 7, "each_slice(4).to_a", fib.each_slice(4).to_a
row 8, "each.class, each.next", [fib.each.class, fib.each.next]
row 9, "lazy.map { |x| x * x }.first(3)", fib.lazy.map { |x| x * x }.first(3)
row 10, "each_with_index.select { |x, i| x == i }", fib.each_with_index.select { |x, i| x == i }.map(&:first)

Verified output of enumerable_is_a_mixin_kata_rb.rb — regenerated by tools/run_examples.py, never hand-typed.

 1. to_a                                       [0, 1, 1, 2, 3, 5, 8, 13, 21, 34, 55, 89]
 2. select(&:even?)                            [0, 2, 8, 34]
 3. each_cons(2).map { |a, b| b - a }          [1, 0, 1, 1, 2, 3, 5, 8, 13, 21, 34]
 4. include?(21), include?(22)                 [true, false]
 5. sum                                        232
 6. min_by { |x| (x - 50).abs }                55
 7. each_slice(4).to_a                         [[0, 1, 1, 2], [3, 5, 8, 13], [21, 34, 55, 89]]
 8. each.class, each.next                      [Enumerator, 0]
 9. lazy.map { |x| x * x }.first(3)            [0, 1, 1]
10. each_with_index.select { |x, i| x == i }   [0, 1, 5]

See also