Fan-out, fan-in¶
Category: Communication · Status: stub · Lessons: chapter 05, Message passing (planned)
One line: Spreading the items of one channel across several workers, then merging their results back into one channel.
How it connects¶
flowchart LR
n_channel["Channel"]
n_fan_out_fan_in["Fan-out, fan-in"]
n_fan_out_fan_in -->|uses| n_channel
classDef center stroke-width:3px
class n_fan_out_fan_in center
classDef outside stroke-dasharray: 4 3
class n_channel outside
- Is built on: Channel
- See also: Pipeline, Worker pool
In each language¶
| Rust | fan-in comes free, since cloned Senders all feed one mpsc ↗ receiver; fan-out needs that receiver shared, which the Rust Book does with Arc<Mutex<T>> ↗ |
| Go | several goroutines reading one channel (fan-out) and a merge copying many channels onto one (fan-in), both named in the Go blog's pipelines post ↗ |
| Java | ExecutorCompletionService ↗ puts each task on a queue as it completes, for take to collect |
| Python | asyncio.gather ↗ collects results in the order of its arguments, not the order they finish in |
| JavaScript | Promise.all ↗ collects fulfillment values in the order of the promises passed, regardless of completion order |
| Kotlin | the guide's fan-out ↗ (several coroutines receive from one channel) and fan-in ↗ (several coroutines send to one) |
| Erlang and Elixir | Task.async_stream ↗ runs a function over a collection with up to max_concurrency tasks and returns results in input order unless ordered: false |
Where to read more¶
- In a sibling library: Go: Fan-out, fan-in ↗
- In the books: Concurrency in Go, Katherine Cox-Buday — ch. 4, 'Concurrency Patterns in Go' → 'Fan-Out, Fan-In'
- In the books: Effective Concurrency in Go, Burak Serdar — ch. 5, 'Worker Pools and Pipelines' → 'Pipelines, fan-out, and fan-in'
- In the books: Designing Distributed Systems, Brendan Burns — ch. 7, 'Scatter/Gather'
- In the books: Effective Python, Brett Slatkin — ch. 7, 'Concurrency and Parallelism' → 'Item 57: Avoid Creating New Thread Instances for On-demand Fan-out'