Skip to content

Refinements are lexical, not global

Level: 301 · a Ruby programmer who has monkey-patched String and felt uneasy about it

One line: refine String do … end inside a module builds a Refinement, and using that module activates it from that line to the end of the file — for code written there, not code called from there — so a method defined above the using line or in another file never sees it, String.method_defined? stays false, and inside the scope respond_to?, send and method all agree the method exists.

Reopening a class (classes are open) changes it for every caller in the process. A refinement is the scoped alternative: the change is packaged in a module and switched on per file (or per class body) with using. The switch is decided when the file is parsed, which is why the program below can define one method above the using line and one below it and get different answers from the two, whatever order they are called in. Ruby 4.0's refinements are also visible to reflection inside the scope — respond_to?, send, public_send and method see the refined method — while the class itself stays untouched.

Verified output of refinements_rb.rb — regenerated by tools/run_examples.py, never hand-typed.

1. a refinement is a Module holding Refinement objects
   Shout.class                                    Module
   Shout.refinements                              [#<refinement:String@Shout>]
   Shout.refinements.first.class                  Refinement
   Shout.refinements.first.target                 String
   Refinement.superclass                          Module
2. before using: String is untouched
   "hi".shout                                     NoMethodError
   "hi".respond_to?(:shout)                       false
   String.method_defined?(:shout)                 false
3. after using: the rest of this file sees it
   "hi".shout                                     HI!
   "hi".length  (an existing method, replaced)    refined length
   ["a", "b"].map(&:shout)                        ["A!", "B!"]
   after_using  (a method written below using)    HI!
4. the scope is lexical: where code is written, not when it runs
   before_using  (written above the using line)   NoMethodError
   a method in a file required after using        NoMethodError
   eval('"z".shout')  (a string eval'd here)      Z!
   using inside a method                          RuntimeError: main.using is permitted only at toplevel
5. what reflection sees inside the scope
   "hi".respond_to?(:shout)                       true
   "hi".send(:shout)                              HI!
   "hi".public_send(:shout)                       HI!
   "hi".method(:shout).owner                      #<refinement:String@Shout>
   String.instance_methods.include?(:shout)       false
   String.method_defined?(:shout)                 false
   Module.used_modules                            [Shout]
   Module.used_refinements                        [#<refinement:String@Shout>]
6. super reaches the original; a refinement sees only what was active where it was written
   "hi".upcase  (refined; calls super)            politely HI
   "hi".shout  (its upcase is still the original) HI!
   Module.used_modules                            [Politely, Shout]

Reading the output

  • 1. Shout is an ordinary Module; the refine block inside it created one Refinement object per refined class, listed by Shout.refinements, whose target is String. Refinement is a subclass of Module.
  • 2. Before using, nothing has changed: "hi".shout is a NoMethodError, respond_to? is false, and String.method_defined?(:shout) is false.
  • 3. After using Shout at the top level, the rest of the file has shout; an existing method can be replaced too (length now answers "refined length"), &:shout works, and after_using, written below the line, works.
  • 4. The scope is lexical. before_using was written above the using line, and calling it after the line still raises. A file required after the line does not see the refinement either — using reaches to the end of this file, not into the files it loads. A string eval'd in the scope does see it. using is only allowed at the top level of a file (main.using) — inside a method it raises RuntimeError: main.using is permitted only at toplevel — or at the top of a class or module body, through Module#using. (Not machine-checked here.)
  • 5. Inside the scope, respond_to? says true, send, public_send and method all reach the refined method, and method(:shout).owner is the refinement itself, #<refinement:String@Shout>. Early versions of refinements hid them from send and respond_to?; that is no longer so. (Not machine-checked here.) The class is still unchanged: String.instance_methods does not list shout and method_defined? is false. Module.used_modules and Module.used_refinements list what is active at the point of the call.
  • 6. A refined method can call super to reach the original — Politely wraps upcase. But a refinement body sees only the refinements that were active where it was written: Shout#shout calls upcase and was written before using Politely, so it still gets the original upcase and prints HI!, not POLITELY HI!. Module.used_modules now lists both, most recent first.

Compared with Python

Python has no refinements, and its two substitutes sit on either side of them. The static route is a subclass: str.shout = … is a TypeError because built-in types are closed, so class ShoutStr(str) adds the method — and loses it on the first operation, since ShoutStr("hi") + "!" is a plain str (row 1). The scoped route is unittest.mock.patch.object, and its scope is dynamic: while the with block (or decorated call) is active, every caller sees the replacement — a function written above the block, a function in another module, an eval — and the moment it ends, everything is restored (rows 3–4). Where Ruby asks where was this code written, Python asks is the patch on right now. Row 5 shows why: the patch really replaces the attribute on the class (vars(Greeter)['greet'] is loud), so reflection sees the substitute the way Ruby's send and respond_to? see a refinement, but for everyone. Row 6: the original is saved and put back, a wrapper can call the saved original much as a refinement calls super, and the built-in str cannot be patched at all (TypeError), where refine String is exactly the point. mock.patch is a testing tool; the closest production idiom is the subclass, or a wrapper function.

Verified output of refinements_py.py — regenerated by tools/run_examples.py, never hand-typed.

1. the two routes: a subclass, or a patch
   str.shout = ...  (built-in types are closed)   TypeError
   ShoutStr('hi').shout()                         HI!
   type(ShoutStr('hi') + '!').__name__            str
   type(mock.patch.object(Greeter, 'greet', loud)).__name__ _patch
2. before the patch: Greeter is untouched
   Greeter().greet()                              hello
   Greeter.greet is loud                          False
   hasattr(Greeter, 'shout')                      False
3. inside the with block: replaced
   Greeter().greet()                              HELLO
   [Greeter(), Greeter()] greeted                 ['HELLO', 'HELLO']
   inside_patch()  (a function written in the block) HELLO
   Greeter().greet()  after the block             hello
4. the scope is dynamic: whoever calls while it is active
   before_patch()  (written above the block)      HELLO
   other.call(Greeter())  (another module)        HELLO
   eval('Greeter().greet()')                      HELLO
   inside_patch()  (called from inside again)     HELLO
   inside_patch()  (called after the block)       hello
   decorated()  (the decorator form)              HELLO
   before_patch()  after decorated() returned     hello
5. what reflection sees inside the patch
   hasattr(Greeter(), 'greet')                    True
   Greeter.greet is loud                          True
   vars(Greeter)['greet'] is loud  (the class changed) True
   getattr(Greeter(), 'greet')()                  HELLO
   Greeter().greet.__func__.__name__              loud
   Greeter().greet.__func__.__name__  after       greet
6. the original is kept; a wrapper calls it; built-ins cannot be patched
   patched greet calling the saved original       politely HELLO
   Greeter.greet is original  after               True
   mock.patch.object(str, 'upper', ...)           TypeError

Try it

  1. Write a Durations module refining Integer with minutes and hours (in seconds), define a method early that calls 1.hours above the using line, then using Durations and print 2.hours + 30.minutes, [1, 2].map(&:hours), 3.respond_to?(:hours) and Integer.method_defined?(:hours); finally call early and print the exception class.
  2. Refine Array with a second method and show [1, 2, 3].second works while Array.method_defined?(:second) is false. Then reopen Array to add second globally and print Array.method_defined?(:second) again — that is the difference between this page and classes are open.
Solution to 1

refinements_kata_rb.rb in full — pasted here by tools/run_examples.py from the file CI runs.

# Exercise 1: Integer#minutes and Integer#hours as a refinement. The rest of
# the file uses them; a method written above the `using` line cannot.

module Durations
  refine Integer do
    def minutes = self * 60
    def hours = self * 3600
  end
end

def early = 1.hours # written above the using line

using Durations

puts "2.hours + 30.minutes             #{2.hours + 30.minutes}"
puts "90.minutes / 60                  #{90.minutes / 60}"
puts "[1, 2].map(&:hours)              #{[1, 2].map(&:hours).inspect}"
puts "3.respond_to?(:hours)            #{3.respond_to?(:hours)}"
puts "Integer.method_defined?(:hours)  #{Integer.method_defined?(:hours)}"
begin
  early
rescue NoMethodError => e
  puts "early  (written above using)     #{e.class}"
end

Verified output of refinements_kata_rb.rb — regenerated by tools/run_examples.py, never hand-typed.

2.hours + 30.minutes             9000
90.minutes / 60                  90
[1, 2].map(&:hours)              [3600, 7200]
3.respond_to?(:hours)            true
Integer.method_defined?(:hours)  false
early  (written above using)     NoMethodError

See also