Probability zero¶

Level: 201 · for anyone who has met probability

One line: For a number drawn at random from [0, 1], every set of measure zero is an event of probability zero — including events that happen on every draw — so probability zero does not mean impossible, except on a computer, which draws from a finite set.

Probability is length¶

Pick a number X uniformly at random from [0, 1]. Uniformly has a precise meaning: the chance that X lands in an interval equals the interval's length. P(0 ≤ X ≤ 1/3) = 1/3; P(1/4 ≤ X ≤ 3/4) = 1/2.

Once probability is length, everything in this chapter becomes a statement about chance.

What the program prints¶

Verified output of probability_zero.py — regenerated by tools/run_examples.py, never hand-typed.

1. THE MODEL: PROBABILITY IS LENGTH
   X is a uniform random number in [0,1]. For any interval,
   P(X lands in it) = the interval's length.
     P(0 <= X <= 1/3) = 1/3
     P(1/4 <= X <= 3/4) = 1/2
     P(9/10 <= X <= 1) = 1/10

2. ONE EXACT VALUE
   If X = 1/2, then X is inside every interval around 1/2:
     P(X = 1/2)  <=  P(X within 1/10            of 1/2) = 1/5
     P(X = 1/2)  <=  P(X within 1/1000          of 1/2) = 1/500
     P(X = 1/2)  <=  P(X within 1/1000000       of 1/2) = 1/500000
     P(X = 1/2)  <=  P(X within 1/1000000000000 of 1/2) = 1/500000000000
   P(X = 1/2) is at most every one of these numbers, so it is 0.
   Nothing about 1/2 was special: every single value has probability 0.

3. BUT X HAS TO LAND SOMEWHERE
   Every value has probability 0, and every draw produces one of them.
   So an event of probability 0 happens on every single draw:
     probability 0   does not mean   impossible
     probability 1   does not mean   certain -- it means ALMOST SURELY

   It is also why probabilities add up over a LIST of events and no
   further: [0,1] is made of its points, every point has probability 0,
   and the whole of [0,1] has probability 1.

4. MEASURE ZERO MEANS PROBABILITY ZERO
   A set inside intervals of total length eps has probability at most
   eps. A measure-zero set does that for EVERY eps, so:
     X is rational            measure 0   probability 0
     X is in the Cantor set   measure 0   probability 0
     X is irrational          measure 1   probability 1: almost surely
   Yet 1/2 is rational, 1/4 is in the Cantor set, and both are possible.

5. WHAT A COMPUTER ACTUALLY DRAWS
   Five draws from random.random(), seeded, as exact multiples of 2^-53:
     0.11911988496396309  = 1,072,936,539,072,241 / 2^53
     0.5025157552312506   = 4,526,259,536,014,527 / 2^53
     0.511822712773071    = 4,610,089,157,049,118 / 2^53
     0.8600005876492754   = 7,746,196,652,151,369 / 2^53
     0.10263685050695981  =   924,470,563,395,251 / 2^53
   Every draw is a whole number of 2^-53 steps, so random() has exactly
   2^53 = 9,007,199,254,740,992 possible values. A finite set.

   Doubles in [0, 1): 4,607,182,418,800,017,408 = 1023 x 2^52.
   random() can return 2^53 of them -- 2 in every 1023. The rest are
   never drawn, and 0.1 is one of the rest:
     0.5 is stored as 1/2                                 a multiple of 2^-53? yes
     0.1 is stored as 3602879701896397/36028797018963968  a multiple of 2^-53? no

6. THE SAME QUESTIONS, ON THE MACHINE
     P(random() == 0.5)          = 1/2^53 = 1.110e-16   not 0
     P(two draws are equal)      = 1/2^53 = 1.110e-16   not 0
     P(random() == 0.1)          = 0, and here 0 IS impossible
     P(random() is rational)     = 1, and certain: every float is a fraction
     draws until a repeat        about 119 million on average

7. TWO WORLDS, BOTH RIGHT
   the model:    X is almost surely irrational, and P(X = 0.5) = 0
   the machine:  every draw is rational, and P(random() == 0.5) = 1/2^53
   With finitely many outcomes, probability 0 means impossible again --
   the subtlety needs a continuum. And since every finite set has measure
   zero, the model gives 'X is a float' probability 0. The machine lives
   entirely inside an event of probability zero.

Every single value has probability zero¶

What is P(X = 1/2)? If X equals 1/2, it is certainly within 1/10 of 1/2, so P(X = 1/2) is at most 1/5. It is also within 1/1000 of 1/2, so P(X = 1/2) is at most 1/500 — and so on, down past any positive number (section 2). The only probability that small is 0.

Nothing about 1/2 was special: every value has probability 0. And yet X has to be some value. So on every draw, an event of probability 0 happens.

Probability 0 does not mean impossible. An event of probability 0 is one that fits inside events of arbitrarily small probability, which is the definition of measure zero, said in the language of chance.

Probability 1 does not mean certain. It means almost surely: exceptions exist, and together they have measure zero.

Adding zeros¶

Probability adds up over a list. If A₁, A₂, A₃, … are events that cannot happen together, the chance that one of them happens is P(A₁) + P(A₂) + P(A₃) + ⋯. That is why a countable set, like the rationals, has probability zero: a list of zeros adds up to zero, which is the argument of countable sets again.

It goes no further than a list. [0, 1] is made of its points, every point has probability 0, and the whole interval has probability 1. Probability, like length, only promises to add over a list — which is exactly why the definition in what measure zero means says a list of intervals.

Measure zero is probability zero¶

Section 4 puts the chapter's sets into the language of chance:

Event	Measure	Probability	Can it happen?
X is rational	0	0	yes — 1/2 is a possible draw
X is in the Cantor set	0	0	yes — 1/4 is a possible draw
X is in the fat Cantor set	1/2	1/2	yes, half the time
X is irrational	1	1	almost surely, but not certainly

The fat Cantor row is not in the program's output; its 1/2 comes from the fat Cantor set.

A random real number is almost surely irrational. That sentence is the chapter, applied to chance.

What a computer actually draws¶

A computer cannot draw from [0, 1]. It draws from a finite set, and everything above changes.

Python's random.random() returns multiples of 2⁻⁵³ — the random module documentation ↗ says so in its recipes section — so it has exactly 2⁵³ = 9,007,199,254,740,992 possible values. Section 5 shows five seeded draws, each a whole number of 2⁻⁵³ steps.

There are far more doubles below 1.0 than that: 1023 × 2⁵². Section 5 counts them by reading the bits of 1.0 as an integer, which works because for non-negative doubles the bit patterns sort in the same order as the numbers. So random() can reach only 2 in every 1023 doubles in [0, 1), and 0.1 is not one of them: the double nearest 0.1 is 3602879701896397/2⁵⁵, which is not a multiple of 2⁻⁵³.

So the same questions get different answers (section 6):

Question	The model: X uniform on [0, 1]	The machine: `random()`
P(the value is 0.5)	0, yet possible	1/2⁵³ ≈ 1.1 × 10⁻¹⁶
P(two draws are equal)	0	1/2⁵³
P(the value is 0.1)	0, yet possible	0, and impossible: the double nearest 0.1 is never returned
P(the value is rational)	0	1, and certain: every float is a fraction

The model's 0 for two equal draws is measure zero once more. The pair (X, Y) is a point in the unit square, and "equal" is the diagonal line through it, which has no area — the second room from what measure zero means. On the machine, the birthday problem takes over: expect a repeated value after about 119 million draws, √(π/2 × 2⁵³).

With finitely many outcomes, probability zero means impossible again. The subtlety of this whole lesson needs a continuum, and a computer does not have one. There is a neat way to say it in the model's own terms: the set of all floats is finite, a finite set has measure zero, so a truly uniform X is a float with probability 0. The machine lives entirely inside an event of probability zero.

Where the phrase gets used¶

"Measure zero" often turns up in technical writing as a way to say negligible: a failure that needs an exact tie, or perfectly balanced inputs, "only happens on a set of measure zero". That is fair when the inputs really are continuous. It is a mistake when they are counts. Election tallies are whole numbers, and exact ties happen. The sibling voting library uses the phrase in the careful sense, to separate failures that occupy a whole region of realistic elections from knife-edge ones that need exact ties: Reading these fairly ↗.

Run it yourself¶

From the root of your clone of this repository:

python3 02_Measure_Zero/probability_zero/examples/probability_zero.py