Skip to content

Ruby shows you its bytecode and its syntax tree

Level: 301 · someone who wants to see what 1 + 2 becomes before it runs

One line: RubyVM::InstructionSequence.compile(src).disasm lists the YARV instructions the interpreter will execute and Prism.parse(src).value the syntax tree they were compiled from — Python's compile and ast expose the same two layers, with one difference visible in the very first example: CPython folds 1 + 2 to 3 at compile time and YARV does not.

Source becomes a tree, the tree becomes an instruction sequence (an "iseq"), and the VM executes the sequence one instruction at a time. RubyVM::InstructionSequence.compile runs the first two steps on a string and disasm prints the result; this page keeps only the instruction names, because the offsets, line numbers and file paths beside them are not what it is about. Row 1 is a = 1 + 2; a * 3 in eight instructions: putobject_INT2FIX_1_ (a specialised "push 1"), putobject 2, opt_plus, setlocal, getlocal, putobject 3, opt_mult, leave. The opt_ instructions are specialised operators: opt_plus has a fast path for two Integers and falls back to a real method call otherwise, because Integer#+ can be redefined — which is also why the compiler cannot fold 1 + 2 (row 2 makes the claim explicit). Row 3 runs the sequence with eval. Row 4 looks at the serialisable form to_a — its first element names the format, the label of a compiled string is <compiled>, its kind is :top — and leaves the two version numbers that follow the magic in prose: they identify the binary format and read 4 and 0 on this Ruby, having changed with the 4.0 release. (Not machine-checked here.) Row 5 fetches the sequence behind a method with InstructionSequence.of(method(:foo)): label foo, kind :method, four instructions. Row 6 switches the specialisation off and opt_plus becomes a generic send. Row 7 is the trick behind Bootsnap: to_binary serialises a sequence and load_from_binary brings it back, ready to eval.

The tree comes from Prism, Ruby's parser since 3.4 (require "prism"). Prism.parse returns a ParseResult whose success? and errors say whether the source parsed and whose value is the ProgramNode at the root (row 8). Below it, 1 + 2 is a CallNode — a message + sent to an IntegerNode with one IntegerNode argument — because in Ruby every operator is a method call and the parser already knows it. The node knows its name, its receiver's value, its argument values and, through slice, the exact source text it came from (row 9). Row 10 parses the two-statement program from row 1: an assignment is a LocalVariableWriteNode wrapping the call, and reading a back is a LocalVariableReadNode. Row 11 is the older RubyVM::AbstractSyntaxTree API, whose node types mirror the previous parser (:SCOPE at the root); it still works, and ruby --dump=insns and --dump=parsetree print both layers from the command line.

Verified output of the_bytecode_you_can_see_rb.rb — regenerated by tools/run_examples.py, never hand-typed.

 1. compile("a = 1 + 2; a * 3"): instruction names       ["putobject_INT2FIX_1_", "putobject", "opt_plus", "setlocal_WC_0", "getlocal_WC_0", "putobject", "opt_mult", "leave"]
 2. 1 + 2 is not folded: opt_plus is in the sequence     true
 3. iseq.eval                                            9
 4. to_a[0] (format magic), label, kind                  ["YARVInstructionSequence/SimpleDataFormat", "<compiled>", :top]
 5. InstructionSequence.of(method(:foo)): label, kind, names ["foo", :method, ["getlocal_WC_0", "putobject_INT2FIX_1_", "opt_plus", "leave"]]
 6. specialized_instruction: false -> opt_plus becomes send ["putobject_INT2FIX_1_", "putobject", "send", "leave"]
 7. to_binary.class, load_from_binary(...).eval          [String, 9]
 8. Prism.parse("1 + 2"): success?, value.class, the nodes: [true, Prism::ProgramNode]
      ProgramNode
        StatementsNode
          CallNode
            IntegerNode
            ArgumentsNode
              IntegerNode
 9. the CallNode: name, receiver.value, argument values, slice [:+, 1, [2], "1 + 2"]
10. Prism.parse("a = 1 + 2; a * 3") nodes:               true
      ProgramNode
        StatementsNode
          LocalVariableWriteNode
            CallNode
              IntegerNode
              ArgumentsNode
                IntegerNode
          CallNode
            LocalVariableReadNode
            ArgumentsNode
              IntegerNode
11. RubyVM::AbstractSyntaxTree.parse("1 + 2").type (older API) :SCOPE

Reading the output

Rows 1–3: the instruction list, the presence of opt_plus in it, and the value 9. Row 4: the format's magic string, the label and the kind of a compiled string. Row 5: a method's own sequence. Row 6: the same expression without specialised instructions, send in place of opt_plus. Row 7: a String of bytecode that evaluates to 9 after a round trip. Rows 8 and 10 print each syntax tree as indented class names — Prism:: dropped — one node per line; row 9 reads the call node's fields; row 11 names the root of the older API's tree.

Compared with Python

compile(src, filename, mode) returns a code object (row 4: type code, name <module>, co_code is bytes) whose co_names lists the global names it touches and co_varnames its fast locals — empty at module level, ('x',) for foo (rows 1 and 5). Row 2 is the difference the one-line claim promised: compile("1 + 2") contains no BINARY_OP instruction because CPython's compiler folded the constant, while compile("a + 2") keeps one. It can fold because int.__add__ cannot be replaced; Ruby's open Integer class forbids the same shortcut. Row 3 evaluates a walrus version of the program to the same 9. Row 6 is Python's compile-time switch, optimize=2, which drops docstrings (and assert statements) — the counterpart of Ruby's specialized_instruction: option in spirit, not in effect. Row 7 is marshal, the format .pyc files use to store code objects, and the twin of to_binary.

ast.parse gives a Module root in exec mode and an Expression root in eval mode (rows 8 and 11); 1 + 2 is a BinOp with two Constant operands and an Add operator node — an operator, not a call, because Python's + is syntax that the VM resolves to __add__ at run time. ast.unparse turns a node back into source, as slice does for Prism (row 9), and row 10 shows Assign/Name/Store and Expr/Name/Load where Ruby has write and read nodes. Two listings this page does not print, because the two CI Pythons disagree: dis.dis output — 3.12 compiles 1 + 2 to RESUME, RETURN_CONST and stores the folded 3 in co_consts, 3.14 to RESUME, LOAD_SMALL_INT, RETURN_VALUE with the value inside the instruction and the operand 1 left in co_consts — and ast.dump, which from 3.13 omits empty fields. (Not machine-checked here.)

Verified output of the_bytecode_you_can_see_py.py — regenerated by tools/run_examples.py, never hand-typed.

 1. compile("(a := 1 + 2) * 3"): co_names, co_varnames   [('a',), ()]
 2. 1 + 2 is folded: BINARY_OP in "1 + 2" / in "a + 2"   [False, True]
 3. eval(code)                                           9
 4. type(code).__name__, co_name, type(co_code).__name__ ['code', '<module>', 'bytes']
 5. foo.__code__: co_name, co_varnames, co_argcount, BINARY_OP ['foo', ('x',), 1, True]
 6. optimize=2 drops docstrings: f.__doc__ at 0 / at 2   ['doc', None]
 7. marshal.dumps(code).__class__, eval(marshal.loads(...)) ['bytes', 9]
 8. ast.parse("1 + 2"): type, body length, the nodes:    ['Module', 1]
      Module
        Expr
          BinOp
            Constant
            Add
            Constant
 9. the BinOp: op, left.value, right.value, ast.unparse  ['Add', 1, 2, '1 + 2']
10. ast.parse("a = 1 + 2; a * 3") nodes:                 True
      Module
        Assign
          Name
            Store
          BinOp
            Constant
            Add
            Constant
        Expr
          BinOp
            Name
              Load
            Mult
            Constant
11. ast.parse("1 + 2", mode="eval") root (one ast, two modes) 'Expression'

Try it

  1. Write opcodes(src) that returns the instruction names of a source string and of every child sequence — a block or a method body is compiled as its own iseq, reachable through each_child. Print the result for x = 5, x = 5; x + 1, [1, 2].map { |x| x * 2 } and def m(a) = a * 2, remembering that a parent's disasm appends its children's listings after a blank line.
  2. Compile x = 1; x + 1 with peephole_optimization: false and with it on, and print which instruction names the optimiser removed. Then compile def m = 1, find the method body among the children and print its label and kind.
Solution to 1

the_bytecode_you_can_see_kata_rb.rb in full — pasted here by tools/run_examples.py from the file CI runs.

# Kata: opcodes(src) returns the instruction names of a source string and of
# every child instruction sequence -- a block or a method body is compiled as
# its own sequence, reachable through each_child. A parent's disasm lists its
# children after a blank line, so the listing is cut at the first blank line.

def names_of(iseq)
  iseq.disasm.lines.take_while { |line| line.strip != "" }.grep(/\A\d{4} /).map { |line| line.split[1] }
end

def opcodes(src)
  iseq = RubyVM::InstructionSequence.compile(src)
  children = []
  iseq.each_child { |child| children << [child.label, names_of(child)] }
  [names_of(iseq), children]
end

["x = 5", "x = 5; x + 1", "[1, 2].map { |x| x * 2 }", "def m(a) = a * 2"].each do |src|
  main, children = opcodes(src)
  puts "#{src.ljust(26)} -> #{main.join(' ')}"
  children.each { |label, names| puts "#{' ' * 26}    #{label}: #{names.join(' ')}" }
end

Verified output of the_bytecode_you_can_see_kata_rb.rb — regenerated by tools/run_examples.py, never hand-typed.

x = 5                      -> putobject dup setlocal_WC_0 leave
x = 5; x + 1               -> putobject setlocal_WC_0 getlocal_WC_0 putobject_INT2FIX_1_ opt_plus leave
[1, 2].map { |x| x * 2 }   -> duparray send leave
                              block in <compiled>: getlocal_WC_0 putobject opt_mult leave
def m(a) = a * 2           -> definemethod putobject leave
                              m: getlocal_WC_0 putobject opt_mult leave

See also