Skip to content

JSON.parse gives string keys; YAML.load is safe

Level: 201 · anyone saving or exchanging data, and anyone who has typed h[:name] on a Hash that came from JSON

One line: JSON is text for everyone and loses what JSON cannot say — Symbols come back as Strings and keys come back as Strings unless you ask for symbolize_names:; YAML is text that YAML.load reads safely, refusing to build arbitrary objects unless you call unsafe_load; Marshal is bytes that keep every Ruby type and can be read only by Ruby — where Python's json behaves the same, its pickle is Marshal with no safe mode, and its standard library has tomllib but no YAML.

JSON.generate (or to_json on the object) writes the six JSON types; anything else is asked for to_json, and for a Struct or a Time that means its to_s in quotes — a String, not an error, which is the trap. JSON.parse reads back Strings, numbers, true/false/nil, Arrays and Hashes with String keys; symbolize_names: true gives Symbol keys, but a Symbol value is gone for good, because JSON has no such type. YAML is the Psych library: YAML.dump writes Symbols as :a, and since Psych 4 YAML.load is safe_load — it builds only the basic types plus Symbols, so a !ruby/object tag or a Date raises Psych::DisallowedClass until you pass permitted_classes:, and an alias raises until you pass aliases: true; YAML.unsafe_load is the old behaviour, for input you wrote yourself. Marshal.dump writes a binary String in format 4.8 that keeps Symbols, Ranges, Times, and your own classes exactly, so Marshal.load(Marshal.dump(x)) is Ruby's deep copy — and it refuses a Proc, an IO or an anonymous class, because there is no way to write them down. There is no TOML parser in the standard library.

Verified output of json_yaml_and_marshal_rb.rb — regenerated by tools/run_examples.py, never hand-typed.

 1. JSON.generate is compact, and Symbols become Strings       {"a":1,"b":[1,2.5,null,true],"c":"sym"}
 2. to_json on a Hash, Array, String, nil, Symbol, Float       {"a":1} [1,"two"] "str" null "sym" 1.5
 3. JSON.pretty_generate({a: 1, b: [1, 2]}):                   
      {
        "a": 1,
        "b": [
          1,
          2
        ]
      }
 4. JSON.parse gives String keys                               {"a" => 1, "b" => [1, 2.5, nil, true]}
 5.   symbolize_names: true gives Symbol keys                  {a: 1}
 6. a Symbol value comes back as a String                      {"a" => "sym"}
 7. non-String keys are stringified: {1 => 2, nil => 3}        {"1":2,"":3}
 8. a scalar document: parse of 3, "s", null                   [3, "s", nil]
 9. duplicate keys: the last one wins                          {"a" => 2}
10. bad input raises; its ancestors                            JSON::ParserError < JSON::JSONError < StandardError
11. NaN is not JSON: generate raises                           JSON::GeneratorError
12. non-ASCII stays UTF-8; ascii_only: true escapes it         "café" "caf\u00e9"
13. a Struct or a Time has no JSON shape: to_json is to_s, quoted "#<struct S x=1>" "2024-01-01 00:00:00 UTC"
14. YAML.dump({a: 1, "b" => [1, 2], c: nil}):                  
      ---
      :a: 1
      b:
      - 1
      - 2
      :c:
15. YAML.load: yes is true, ~ is nil, '1' stays a String       {"a" => 1, "b" => ["x"], "c" => true, "d" => nil, "e" => "1"}
16. YAML.load is safe: a !ruby/object tag raises               Psych::DisallowedClass: Tried to load unspecified class: Object
17.   YAML.unsafe_load builds it                               Object
18. Symbols are allowed; a Date is not until permitted_classes: :sym; Psych::DisallowedClass; Date
19. aliases are off by default; aliases: true allows them      Psych::AliasesNotEnabled; {"a" => 1, "b" => 1}
20. YAML round trip keeps Symbols and nesting                  {a: [1, {b: "c"}]}
21. Marshal.dump gives a binary String; dump(1) in hex; version String ASCII-8BIT; 04086906; 4.8
22. the round trip keeps Ruby types: Symbol, Range, Time       {a: [1, 2], c: 1..3, d: 2024-01-01 00:00:00 UTC}
23. load(dump(x)) is a deep copy: original, copy               {a: [1, [2]]}, {a: [1, [2, 3]]}
24. Marshal refuses a Proc and an IO                           TypeError: no _dump_data is defined for class Proc; can't dump IO
25. Marshal.load of garbage raises                             TypeError
26. TOML: no parser in Ruby's standard library                 LoadError: cannot load such file -- toml

Reading the output

Rows 1–3 are the writers: generate is compact, the Symbol key a and the Symbol value :sym both became Strings, to_json works on every JSON type, and pretty_generate indents by two. Rows 4–6 are the claim in the title: parse gives {"a" => 1, …} with String keys, symbolize_names: true gives {a: 1}, and the Symbol value that went in as :sym came back as "sym". Rows 7–9 are the corners: an Integer key becomes "1" and a nil key ""; a bare scalar is a valid document; a duplicated key keeps its last value. Rows 10–11 are the errors — JSON::ParserError is a StandardError through JSON::JSONError, and Float::NAN cannot be generated because JSON has no NaN. Row 12: by default the output is UTF-8, and ascii_only: true escapes é as é. Row 13 is the trap: a Struct and a Time serialise as their to_s, in quotes, without complaint. Rows 14–15 are YAML's shape: symbol keys are written :a:, and on the way in YAML 1.1's yes is true and ~ is nil, while a quoted '1' stays a String. Rows 16–19 are safety: the !ruby/object tag raises Psych::DisallowedClass naming the class, unsafe_load builds the Object, Symbols are allowed but a Date is not until you permit it, and an alias raises Psych::AliasesNotEnabled until you allow it. Row 20: a YAML round trip keeps Symbols, which JSON could not. Rows 21–25 are Marshal: the dump is a binary (ASCII-8BIT) String whose first two bytes 04 08 are the format version 4.8; the round trip keeps a Symbol, a Range and a Time as themselves; the copy is deep (the original's nested Array did not gain the 3); a Proc raises TypeError: no _dump_data is defined for class Proc and an IO can't dump IO; and garbage is a TypeError. Row 26: require "toml" is a LoadError.

Compared with Python

json.dumps/json.loads are rows 1–12 with small differences: the default output has spaces after , and : (row 1 — Ruby's is compact), a tuple becomes a list on the way out and comes back as a list (rows 2 and 6 — the same loss Ruby's Symbols suffer), keys are always str so there is nothing to symbolize (rows 4–5), and a None key becomes "null" where Ruby's nil key became "" (row 7). JSONDecodeError is a ValueError (row 10). Two defaults are inverted: Python writes NaN unless you pass allow_nan=False (row 11), and escapes non-ASCII unless you pass ensure_ascii=False (row 12). And where Ruby quietly stringifies a Struct or a Time, Python raises TypeError for a dataclass or a datetime and makes you say default=vars (row 13) — the stricter behaviour. There is no YAML in the standard library at all: row 14 checks sys.stdlib_module_names, and PyYAML is a package you install. pickle is Marshal — bytes in a numbered protocol (row 21; the twin pins protocol=4 because the default protocol changed between 3.12 and 3.14 — (Not machine-checked here.)), a deep copy (row 23), Python types kept (rows 20 and 22), lambdas and file objects refused (row 24), garbage an UnpicklingError (row 25) — with one difference that matters: there is no safe mode. Row 16 loads a pickle whose __reduce__ says "call print", and print ran; a pickle from an untrusted source can run anything, which is why Ruby's YAML.load became safe by default and why Marshal should be treated the same way. Python has what Ruby lacks in row 26: tomllib, a read-only TOML parser since 3.11 — the format of pyproject.toml ↗, which the Python library's page on it reads.

Verified output of json_yaml_and_marshal_py.py — regenerated by tools/run_examples.py, never hand-typed.

 1. json.dumps puts spaces after , and :; keys are already str {"a": 1, "b": [1, 2.5, null, true], "c": "sym"}
 2. dumps of a dict, list, str, None, tuple, float             {"a": 1} [1, "two"] "str" null [1, 2] 1.5
 3. json.dumps({'a': 1, 'b': [1, 2]}, indent=2):               
      {
        "a": 1,
        "b": [
          1,
          2
        ]
      }
 4. json.loads gives str keys (there is no other kind)         {'a': 1, 'b': [1, 2.5, None, True]}
 5.   no symbolize_names: a key is always a str                n/a
 6. a tuple comes back as a list                               [1, 2]
 7. non-str keys are stringified: {1: 2, None: 3}              {"1": 2, "null": 3}
 8. a scalar document: loads of 3, "s", null                   [3, 's', None]
 9. duplicate keys: the last one wins                          {'a': 2}
10. bad input raises; its ancestors                            JSONDecodeError < ValueError < Exception
11. NaN is written as NaN by default; allow_nan=False raises   NaN; ValueError
12. non-ASCII is escaped by default; ensure_ascii=False keeps it "caf\u00e9" "café"
13. a dataclass or a datetime raises; default=vars is the fix  TypeError TypeError; {"x": 1}
14. no YAML: 'yaml' in sys.stdlib_module_names; json, pickle, tomllib [False, True, True, True]
15.   (PyYAML is a third-party package)                        n/a
16. pickle.loads runs __reduce__: what loading an Evil printed 'pickle ran code\n'
17.   there is no safe load: every pickle.loads is unsafe_load n/a
18.   no permitted_classes                                     n/a
19.   no aliases option; shared references just round-trip     n/a
20. pickle round trip keeps tuples, sets and nesting           {'a': [1, {'b': 'c'}], 't': (1, 2), 's': {3}}
21. pickle.dumps gives bytes; dumps(1, protocol=4) in hex; highest bytes; 80044b012e; 5
22. the round trip keeps Python types: range, datetime         {'a': [1, 2], 'c': range(1, 4), 'd': datetime.datetime(2024, 1, 1, 0, 0, tzinfo=datetime.timezone.utc)}
23. loads(dumps(x)) is a deep copy: original, copy             {'a': [1, [2]]}, {'a': [1, [2, 3]]}
24. pickle refuses a lambda and a file object                  PicklingError < PickleError; TypeError < Exception
25. pickle.loads of garbage raises                             UnpicklingError
26. TOML: tomllib.loads (3.11+, read-only)                     {'a': 1, 's': {'b': [1, 2]}}

Try it

  1. Save a settings Hash with a Symbol value and a Time value to a JSON file and to a Marshal file in a temporary directory, load both back, and print for each key whether it survived each round trip unchanged — and what JSON gave you instead.
  2. Define class Config with a to_json(*args) that writes a Hash of its fields and a self.from_json(text) that rebuilds one, so that Config.from_json(config.to_json) == config; then try the same with YAML.dump/YAML.load and find the smallest permitted_classes: that makes it work.
Solution to 1

json_yaml_and_marshal_kata_rb.rb in full — pasted here by tools/run_examples.py from the file CI runs.

# Kata: save settings to JSON and to Marshal in a temporary directory, load
# both back, and report which values survived each round trip unchanged.
require "json"
require "tmpdir"

settings = {name: "demo", retries: 3, mode: :fast, since: Time.utc(2024, 1, 1), tags: ["a", "b"]}

Dir.mktmpdir do |dir|
  Dir.chdir(dir) do
    File.write("settings.json", JSON.generate(settings))
    File.binwrite("settings.marshal", Marshal.dump(settings))
    from_json = JSON.parse(File.read("settings.json"), symbolize_names: true)
    from_marshal = Marshal.load(File.binread("settings.marshal"))
    settings.each_key do |key|
      puts format("%-8s json: %-5s marshal: %-5s (json gave %s)", key, from_json[key] == settings[key], from_marshal[key] == settings[key], from_json[key].inspect)
    end
  end
end

Verified output of json_yaml_and_marshal_kata_rb.rb — regenerated by tools/run_examples.py, never hand-typed.

name     json: true  marshal: true  (json gave "demo")
retries  json: true  marshal: true  (json gave 3)
mode     json: false marshal: true  (json gave "fast")
since    json: false marshal: true  (json gave "2024-01-01 00:00:00 UTC")
tags     json: true  marshal: true  (json gave ["a", "b"])

See also