Skip to content

Thread.current[:x] is fiber-local, not thread-local

Level: 301 · anyone who stored a request id in Thread.current[] and lost it inside an enumerator

One line: Thread.current[:x] is scoped to the fiber that set it — a new Fiber, and therefore Enumerator#next, reads nil — while thread_variable_set/thread_variable_get is the genuinely thread-wide store, and Fiber[:key] (Ruby 3.2) is a third kind that child fibers and new threads inherit, which is what Python's contextvars does.

The name is the trap. Thread.current[:x] = 1 looks like a thread-local variable, and for a program that never creates a fiber it behaves like one. But the store behind Thread#[] belongs to the current fiber: a new Fiber reads nil (row 2), and what a fiber assigns stays in that fiber (row 3). Because Enumerator#next runs its block on a private fiber — the previous page — a value set outside an enumerator is invisible inside it under next and visible under to_a (row 7). This is a real bug in real code: a request id or a database connection stashed in Thread.current[] disappears the moment a library iterates with next. Ruby's own documentation calls these fiber-local variables (Not machine-checked here.); the method that means "per thread" is thread_variable_set, whose value a fiber sees and can change (rows 4 and 5). Neither survives into a new Thread, which starts with both stores empty (row 6).

Ruby 3.2 added a third store for the case that needs inheritance: Fiber[:key], the fiber storage. A child fiber starts with a copy of its parent's storage, so it reads the parent's value (row 8) and its own assignments do not leak back (row 9); a new thread inherits it too, and so does the fiber under Enumerator#next (row 10) — which makes Fiber[] the right place for a request id. Fiber.new(storage: nil) opts a fiber out (row 11). Rows 12 and 13 show the three stores side by side — keys, thread_variables, Fiber.current.storage — and that a missing key is nil for [] and a KeyError for fetch.

Verified output of thread_locals_are_fiber_locals_rb.rb — regenerated by tools/run_examples.py, never hand-typed.

 1. Thread.current[:x] = 1; read back in the root fiber  1
 2. Thread.current[:x] inside a new Fiber                nil
 3. a fiber set [:x] = 9; the root fiber still sees      1
 4. thread_variable_get(:y) inside a new Fiber           2
 5. a fiber set :y = 3; the thread now sees              3
 6. a new Thread sees [:x] / thread_variable :y          nil / nil
 7. in an Enumerator: to_a (root fiber) / next (own fiber) [1, 3] / [nil, 3]
 8. Fiber[:req] = "abc"; inside a child Fiber            "abc"
 9. a child set Fiber[:req] = "child": child / root      "child" / "abc"
10. Fiber[:req] in a new Thread / in Enumerator#next     "abc" / "abc"
11. Fiber.new(storage: nil) starts with nothing          nil
12. keys / thread_variables / Fiber.current.storage      [:x] / [:y] / {req: "abc"}
13. a missing key: [:nope] / fetch(:nope)                nil / KeyError

Reading the output

Rows 1 to 3 are the claim in the title: 1 in the root fiber, nil in a new fiber, and still 1 after a fiber wrote 9 into its own copy. Rows 4 and 5 are the contrast: thread_variable_get(:y) is 2 inside a fiber, and after the fiber set it to 3 the thread sees 3. Row 6 prints nil / nil — a new thread has neither. Row 7 is the trap in one line: the same enumerator block reads [1, 3] when to_a runs it on the root fiber and [nil, 3] when next runs it on its own fiber; only the fiber-local :x went missing.

Rows 8 to 11 are Fiber[]. The child fiber reads "abc"; a child that assigns "child" sees its own value while the root still has "abc"; a new thread and an Enumerator#next fiber both read "abc"; a fiber created with storage: nil reads nil. Row 12 lists [:x], [:y] and {req: "abc"} — one entry per store — and row 13 ends with nil / KeyError.

Compared with Python

Python has one thread-local store, threading.local(), and no fibers, so the trap cannot happen: a generator runs on the thread that drives it, reads the thread's values, and its assignments are the thread's (rows 2 to 5 print 1, 9, 2, 3, exactly as Ruby's thread_variable_get does). Rows 4 and 5 repeat rows 2 and 3 on a second attribute because there is no second store to show. A new thread starts empty (row 6), and a generator driven by list() or by next() reads the same values either way (row 7) — the row where Ruby printed [nil, 3].

Python's counterpart of Fiber[] is contextvars. A ContextVar set in the main context is visible inside copy_context().run(...) (row 8), a value set inside the copy does not reach the caller (row 9), and asyncio copies the context into every task the way Ruby copies storage into every child fiber (Not machine-checked here.). One difference: a new threading.Thread does not inherit the context on the standard builds CI runs here — row 10 prints the default 'none' — while a Ruby thread inherits Fiber[]; an empty Context() reads the default too (row 11). Row 12 shows the two stores, vars(tl) and the context's items, and row 13 that a missing name is an AttributeError unless getattr is given a default.

Verified output of thread_locals_are_fiber_locals_py.py — regenerated by tools/run_examples.py, never hand-typed.

 1. threading.local(): tl.x = 1; read back               1
 2. tl.x inside a generator (no fibers: same thread)     1
 3. a generator set tl.x = 9; the caller now sees        9
 4. tl.y inside a generator                              2
 5. a generator set tl.y = 3; the thread now sees        3
 6. a new Thread sees tl.x / tl.y                        None / None
 7. in a generator: list() / next() twice                [9, 3] / [9, 3]
 8. ContextVar req.set("abc"); in copy_context().run     'abc'
 9. a copied context set "child": child / caller         'child' / 'abc'
10. req in a new Thread / in a generator                 'none' / 'abc'
11. contextvars.Context() (empty) .run(req.get)          'none'
12. vars(tl) / the copied context's items                {'x': 9, 'y': 3} / {'req': 'abc'}
13. a missing name: getattr(tl, 'nope', None) / tl.nope  None / AttributeError

Try it

  1. Store a request id in both Thread.current[:request_id] and Fiber[:request_id], then print both from the root fiber, from a new Fiber, from inside Enumerator#next and from a new Thread. Which store would you use for a request id that must follow the work?
  2. Write a tiny with_request_id(id) { ... } helper that sets Fiber[:request_id], runs the block, and restores the previous value in an ensure. Show that nested calls restore correctly and that a Fiber started inside the block sees the inner id.
Solution to 1

thread_locals_are_fiber_locals_kata_rb.rb in full — pasted here by tools/run_examples.py from the file CI runs.

# Kata: a request id that must follow the work. Thread.current[] does not
# cross a fiber boundary, Fiber[] does, and a new thread inherits Fiber[]
# but starts with an empty Thread.current[].
Thread.current[:request_id] = "req-42"
Fiber[:request_id] = "req-42"

def ids
  "Thread.current[:request_id]=#{Thread.current[:request_id].inspect}  Fiber[:request_id]=#{Fiber[:request_id].inspect}"
end

puts "root fiber:        #{ids}"
puts "a new Fiber:       #{Fiber.new { ids }.resume}"
puts "Enumerator#next:   #{Enumerator.new { |y| y << ids }.next}"
puts "a new Thread:      #{Thread.new { ids }.value}"

Verified output of thread_locals_are_fiber_locals_kata_rb.rb — regenerated by tools/run_examples.py, never hand-typed.

root fiber:        Thread.current[:request_id]="req-42"  Fiber[:request_id]="req-42"
a new Fiber:       Thread.current[:request_id]=nil  Fiber[:request_id]="req-42"
Enumerator#next:   Thread.current[:request_id]=nil  Fiber[:request_id]="req-42"
a new Thread:      Thread.current[:request_id]=nil  Fiber[:request_id]="req-42"

See also