Skip to content

A missing key is nil, and Hash.new([]) shares its default

Level: 101 · a Python programmer meeting Ruby's dict

One line: h[:zz] is nil where d["zz"] raises KeyError; fetch is the raising form; Hash.new(0) returns a default without storing it, Hash.new { |h, k| h[k] = [] } stores what its block assigns, and Hash.new([]) hands every key the same one array — Python's dict.get, Counter, defaultdict and dict.fromkeys(keys, []), in that order.

A Ruby Hash is Python's dict: keys map to values, insertion order is kept (Ruby since 1.9, Python since 3.7 (Not machine-checked here.)), and any object can be a key — Ruby hashes it with hash and compares with eql?, Python with __hash__ and __eq__. Three things differ at the surface. Keys are usually Symbols: {a: 1} has the Symbol key :a, {"a" => 1} has a String key, and {"a": 1} — the spelling a Python hand types — makes a Symbol key too. A missing key answers nil, and fetch is the method that raises (KeyError, with a message naming the key) or takes a default. And a Hash carries its own default: a value given to Hash.new is returned for every missing key but never stored, a block given to Hash.new runs for every missing key and stores whatever it assigns, and a mutable value given to Hash.new is one object shared by every miss — the trap the title names. One more thing to know before the Object Model chapter takes it further: a String key is copied and frozen on the way in, so mutating the variable you inserted with does not corrupt the hash.

Verified output of hashes_and_default_values_rb.rb — regenerated by tools/run_examples.py, never hand-typed.

 1. h = {b: 1, a: 2}; h[:c] = 3; h.keys                      [:b, :a, :c]
 2. key class of {a: 1}, {"a" => 1}, {"a": 1}                [Symbol, String, Symbol]
 3. h[:zz]  (missing key)                                    nil
 4. h.fetch(:zz)  (the raising form)                         KeyError: key not found: :zz
 5. h.fetch(:zz, 0), h.fetch(:zz) { |k| "no #{k}" }          [0, "no zz"]
 6. counts = Hash.new(0); count the chars of "hello"         {"h" => 1, "e" => 1, "l" => 2, "o" => 1}
 7. counts[:zz], counts.size  (default not stored)           [0, 4]
 8. counts.fetch(:zz)  (fetch ignores the default)           KeyError: key not found: :zz
 9. Hash.new { |h, k| h[k] = [] }; group words by size       {5 => ["apple"], 3 => ["fig"], 4 => ["kiwi", "plum"], 6 => ["banana"]}
10. groups[99]; groups.size  (the block stores)              [[], 5]
11. t = Hash.new([]); t[:a] << 1; t[:b] << 2; t, t[:zz]      [{}, [1, 2]]
12. u = {}; u[:a] ||= []; u[:a] << 1                         {a: [1]}
13. n.dig(:a, :b, 1), n.dig(:a, :zz, 1)                      [20, nil]
14. h.transform_keys(&:to_s), h.transform_values {it*10}     [{"b" => 1, "a" => 2, "c" => 3}, {b: 10, a: 20, c: 30}]
15. [[:a, 1]].to_h, h.to_a, h.to_h { |k, v| [v, k] }         [{a: 1}, [[:b, 1], [:a, 2], [:c, 3]], {1 => :b, 2 => :a, 3 => :c}]
16. h.each { |pair| } yields, then |k, v| auto-splats        [:b, 1] is an Array; k = :b, v = 1
17. h.key?(:a), h.value?(2), h.key?(:zz)                     [true, true, false]
18. key = "x"; s[key] = 1; key << "y"; s, s.keys[0].frozen?  [{"x" => 1}, true]
19. {1 => :int, 1.0 => :float}.size  (1.eql?(1.0) is false)  2
20. two equal String keys: compare_by_identity, plain        [2, 1]
21. {a: 1}.merge({b: 2}); merge({a: 2}) { |k, o, n| o + n }  [{a: 1, b: 2}, {a: 3}]
22. d = {a: 1, b: 2}; d.delete(:a), d.delete(:zz), d         [1, nil, {b: 2}]

Reading the output

Rows 1–2: the keys come back in the order they went in, and the three literal spellings give Symbol, String, Symbol.

Rows 3–5 are the missing-key contract: [] answers nil, fetch raises KeyError with key not found: :zz, and fetch with a second argument or a block answers that instead.

Rows 6–8 are Hash.new(0), the counting idiom: counts[c] += 1 reads the default 0, adds one and stores the result. Row 7 shows the default is returned, not stored — reading :zz gives 0 and the size stays 4 — and row 8 shows fetch ignores the default altogether.

Rows 9–10 are the block form, the grouping idiom: the block receives the hash and the key and stores an empty array under it, so groups[w.size] << w works on the first word of each length. Because the block assigns, merely reading groups[99] stores a key (row 10) — the opposite of row 7.

Rows 11–12 are the trap and the idiom. Hash.new([]) evaluates [] once; t[:a] << 1 appends to that one shared array and stores nothing, so t is still {} and every missing key now answers [1, 2]. The idiom that stores is u[:a] ||= [] (or the block form of row 9).

Rows 13–15: dig walks nested keys and indexes and answers nil at the first miss; transform_keys and transform_values rebuild the hash; to_h goes from pairs to a hash, to_a back, and to_h with a block does both in one step.

Rows 16–17: each yields one two-element Array per entry; a block with two parameters auto-splats it into k, v. key? asks about keys and value? about values.

Rows 18–20 are about keys as objects. The String key was duplicated and frozen on insertion, so appending to the original variable did not change the key. 1 and 1.0 are two keys, because 1.eql?(1.0) is false. compare_by_identity switches a hash from eql? to equal?, so two equal but distinct Strings become two keys.

Rows 21–22: merge takes a block to resolve a key present on both sides; delete returns the removed value, or nil for a key that was not there.

Compared with Python

The twin asks a dict, a Counter and a defaultdict the same twenty-two questions. Python has one string type and no Symbol, so row 2 has nothing to distinguish. A missing key raises KeyError (row 3); plain indexing is already Ruby's fetch, and get is Ruby's [] (rows 4–5). Counter is Hash.new(0) exactly — a read of a missing key answers 0 without storing it (rows 6–7) — but defaultdict is the block form: its read of a missing key stores the factory's value (rows 8 and 10), and get skips the factory. The shared-default trap is spelled dict.fromkeys(keys, []) (row 11): one list under every key. setdefault is ||= (row 12); there is no dig (row 13, a five-line helper); comprehensions are the transform_* methods (rows 14–15); items() yields tuples, which for k, v unpacks (row 16). Rows 18–20 are where the object models diverge: a str is immutable so there is nothing to copy, and a list key is a TypeError; 1 and 1.0 are one key, because 1 == 1.0 and they hash alike; and there is no identity mode for a dict — keying by id(obj) is the workaround (Not machine-checked here.). | merges (row 21) with the right side winning and no block to say otherwise; pop with a default is delete (row 22), and pop without one raises.

Verified output of hashes_and_default_values_py.py — regenerated by tools/run_examples.py, never hand-typed.

 1. d = {"b": 1, "a": 2}; d["c"] = 3; list(d)                ['b', 'a', 'c']
 2. key type of {'a': 1}; {'a': 1} == {"a": 1}               str, True
 3. d["zz"]  (missing key)                                   KeyError
 4. d["zz"] is already the raising form; d.get("zz")         KeyError, None
 5. d.get("zz", 0); d["zz"] if "zz" in d else "no zz"        0, 'no zz'
 6. counts = Counter("hello"); dict(counts)                  {'h': 1, 'e': 1, 'l': 2, 'o': 1}
 7. counts["zz"], len(counts)  (default not stored)          0, 4
 8. defaultdict(int): dd["zz"], len(dd); dd.get("yy")        0, 5 (stored), None
 9. defaultdict(list); group words by len                    {5: ['apple'], 3: ['fig'], 4: ['kiwi', 'plum'], 6: ['banana']}
10. groups[99]; len(groups)  (the factory stores)            [], 5
11. t = dict.fromkeys("ab", []); append via a, via b; t      {'a': [1, 2], 'b': [1, 2]}
12. u = {}; u.setdefault("a", []).append(1)                  {'a': [1]}
13. dig(n, "a", "b", 1), dig(n, "a", "zz", 1)  (helper)      20, None
14. {str(k): v ...}, {k: v * 10 ...} comprehensions          [{'b': 1, 'a': 2, 'c': 3}, {'b': 10, 'a': 20, 'c': 30}]
15. dict([("a", 1)]), list(d.items()), {v: k ...}            [{'a': 1}, [('b', 1), ('a', 2), ('c', 3)], {1: 'b', 2: 'a', 3: 'c'}]
16. for pair in d.items(): a tuple; for k, v in ...          ('b', 1) is a tuple; k = 'b', v = 1
17. "a" in d, 2 in d.values(), "zz" in d                     [True, True, False]
18. a str key is immutable (nothing to copy); a list key     TypeError
19. len({1: "int", 1.0: "float"})  (1 == 1.0, same hash)     1
20. two equal str keys: no identity mode, plain dict         -, 1
21. {"a": 1} | {"b": 2}; {"a": 1} | {"a": 2} (right wins)    [{'a': 1, 'b': 2}, {'a': 2}]
22. m.pop("a"), m.pop("zz", None), m.pop("zz")               1, None, KeyError

Try it

  1. Count the words of a sentence with Hash.new(0), then group the words by their count with Hash.new { |h, k| h[k] = [] }, and print the groups from the most frequent count down, the words within a group alphabetical. Finish by showing that reading a missing key stored one in the grouped hash and did not in the counting hash.
  2. Write invert_multi(h) that turns {a: 1, b: 1, c: 2} into {1 => [:a, :b], 2 => [:c]} twice — once with the block form of Hash.new, once with group_by and transform_values — and show what Hash#invert does to the same input.
Solution to 1

hashes_and_default_values_kata_rb.rb in full — pasted here by tools/run_examples.py from the file CI runs.

# Kata: count words with Hash.new(0), group the words by their count with
# Hash.new { |h, k| h[k] = [] }, then print the groups from the most frequent down.

text = "the cat and the dog and the bird"
counts = Hash.new(0)
text.split.each { |word| counts[word] += 1 }
puts "counts:   #{counts.inspect}"

by_count = Hash.new { |h, k| h[k] = [] }
counts.each { |word, n| by_count[n] << word }
puts "by_count: #{by_count.inspect}"

by_count.keys.sort.reverse.each do |n|
  puts "#{n} x  #{by_count[n].sort.join(', ')}"
end

by_count[99]
puts "reading by_count[99] stored a key: #{by_count.keys.inspect}"
puts "counts['zzz'] did not: #{counts['zzz']} and #{counts.keys.inspect}"

Verified output of hashes_and_default_values_kata_rb.rb — regenerated by tools/run_examples.py, never hand-typed.

counts:   {"the" => 3, "cat" => 1, "and" => 2, "dog" => 1, "bird" => 1}
by_count: {3 => ["the"], 1 => ["cat", "dog", "bird"], 2 => ["and"]}
3 x  the
2 x  and
1 x  bird, cat, dog
reading by_count[99] stored a key: [3, 1, 2, 99]
counts['zzz'] did not: 0 and ["the", "cat", "and", "dog", "bird"]

See also