Skip to content

Parallel iterators and streams

Category: Parallelism · Status: stub · Lessons: chapter 07, Parallelism

One line: A library that spreads an iterator's work over a thread pool with one method call — Rayon's par_iter, Java's parallelStream.

Also called: rayon, parallel streams, PLINQ.

How it connects

flowchart LR
  n_data_parallelism["Data parallelism"]
  n_parallel_iterators["Parallel iterators and streams"]
  n_thread_pool["Thread pool and executor"]
  n_work_stealing["Work stealing"]
  n_parallel_iterators -->|is a| n_data_parallelism
  n_parallel_iterators -->|uses| n_thread_pool
  n_parallel_iterators -->|uses| n_work_stealing
  classDef center stroke-width:3px
  class n_parallel_iterators center
  classDef outside stroke-dasharray: 4 3
  class n_data_parallelism,n_thread_pool,n_work_stealing outside

In each language
Rust Rayon's par_iter ↗ gives parallel versions of iterator methods such as map, filter and fold
C++ Standard algorithms take an execution policy ↗ rather than a parallel iterator
Java Collection.parallelStream ↗, or parallel() on a stream
Python Executor.map ↗ on a thread or process pool
C# PLINQ ↗: AsParallel() on a LINQ query
Erlang and Elixir Task.async_stream ↗ runs a function on each element in its own task, by default as many at once as there are schedulers online

Where to read more