Skip to content

Parallelism

Category: Foundations · Status: stub · Lessons: chapter 07, Parallelism

One line: Running several computations at the same instant on separate processing units so the whole finishes sooner; it needs more than one core, and concurrency does not.

Also called: parallel computing, parallel execution.

How it connects

flowchart LR
  n_asynchrony["Asynchrony"]
  n_concurrency["Concurrency"]
  n_data_parallelism["Data parallelism"]
  n_parallelism["Parallelism"]
  n_task_parallelism["Task parallelism"]
  n_asynchrony ---|vs| n_parallelism
  n_concurrency ---|vs| n_parallelism
  n_data_parallelism -->|is a| n_parallelism
  n_task_parallelism -->|is a| n_parallelism
  classDef center stroke-width:3px
  class n_parallelism center
  classDef outside stroke-dasharray: 4 3
  class n_asynchrony,n_concurrency,n_data_parallelism,n_task_parallelism outside

In each language
Rust std::thread::scope ↗ runs threads that borrow local data; parallel iterators come from the Rayon ↗ crate, not the standard library
Go Goroutines run in parallel on as many CPUs as runtime.GOMAXPROCS ↗ allows
C++ Execution policies ↗ (C++17) let standard algorithms such as std::sort and std::reduce run in parallel
Java Parallel streams ↗, and ForkJoinPool ↗ for tasks that spawn subtasks
Python Threads do not run Python code in parallel under the GIL ↗; use multiprocessing ↗ or the free-threaded build
C# Parallel programming in .NET ↗: the Task Parallel Library and PLINQ
JavaScript Each worker ↗ runs in a background thread; memory is shared between them with a SharedArrayBuffer ↗
Erlang and Elixir The VM starts one scheduler thread ↗ per logical processor by default, so processes run in parallel without any change to the code
Haskell par and pseq ↗ in the parallel package mark pure expressions that may be worth evaluating in parallel

Where to read more