GPU computing¶

Category: Parallelism · Status: stub

One line: Running thousands of small data-parallel computations on a graphics processor instead of the CPU.

Also called: GPGPU, CUDA.

How it connects¶

flowchart LR
  n_data_parallelism["Data parallelism"]
  n_gpu_computing["GPU computing"]
  n_gpu_computing -->|is a| n_data_parallelism
  classDef center stroke-width:3px
  class n_gpu_computing center
  classDef outside stroke-dasharray: 4 3
  class n_data_parallelism outside

Is a kind of: Data parallelism

In each language¶


JavaScript	Compute shaders in the WebGPU API ↗
Swift	Metal ↗ compute pipelines
Elsewhere	NVIDIA's CUDA ↗ toolkit

Where to read more¶

In the books: Pro TBB, Michael Voss, Rafael Asenjo, James Reinders — ch. 19, 'Flow Graphs on Steroids: OpenCL Nodes'
In the books: Parallel and Concurrent Programming in Haskell, Simon Marlow — ch. 6, 'GPU Programming with Accelerate'
In the books: Python Parallel Programming Cookbook, Giancarlo Zaccone — ch. 6, 'GPU Programming with Python'
In the books: Mastering C++ Multithreading, Maya Posch — ch. 10, 'Multithreading with GPGPU'
In the books: Seven Concurrency Models in Seven Weeks, Paul Butcher — ch. 7, 'Data Parallelism' → 'Day 1: GPGPU Programming'
In the books: Data Parallel C++, James Reinders, Ben Ashbaugh, James Brodman, Michael Kinsner, John Pennycook, Xinmin Tian — ch. 2, 'Where Code Executes' → 'Method#3: Using a GPU (or Other Accelerators)'
In the books: An Introduction to Parallel Programming, Peter S. Pacheco, Matthew Malensek — ch. 6, 'GPU programming with CUDA'
Reference: Wikipedia: General-purpose computing on graphics processing units ↗