0.1 + 0.2 != 0.3, and 2.5.round is 3

Level: 201 · anyone who has trusted a Float with money

One line: A Ruby Float is the same 64-bit double as a Python float — 0.1 + 0.2 is 0.30000000000000004 in both — but 2.5.round is 3 where Python's round(2.5) is 2, overflow gives Infinity where Python raises, and max with a NaN raises where Python quietly picks by order.

A Float holds a binary fraction, and 0.1, 0.2 and 0.3 are not binary fractions: each literal is the nearest double, the sum of two nearest doubles is not the nearest double to 0.3, and == says so. That part is IEEE 754 and identical in every language that uses doubles. What a language adds is its rounding rule and its policy at the edges. Ruby rounds a half away from zero by default and offers half: :even, half: :down and half: :up as options; it rounds to ndigits by scaling, so 2.675.round(2) is 2.68 even though the Float is a hair below 2.675. Overflow is Float::INFINITY, not an error. NaN is unequal to itself, and max or sort over an array with a NaN in it raises ArgumentError rather than return an order-dependent answer. And to_r shows the exact value a Float holds, while rationalize recovers the short fraction you probably meant.

Verified output of floats_and_rounding_rb.rb — regenerated by tools/run_examples.py, never hand-typed.

 1. 0.1 + 0.2                          -> 0.30000000000000004    == 0.3 is false
 2. (0.1 + 0.2).round(2)               -> 0.3                    == 0.3 is true
 3. (0.1 + 0.2 - 0.3).abs              -> 5.551115123125783e-17  < Float::EPSILON is true
 4. 2.5.round, 3.5.round, -2.5.round   -> 3, 4, -3               a half rounds away from zero; the result is an Integer
 5. 2.5.round(half: :even)             -> 2                      banker's rounding on request; 3.5 gives 4, -2.5 gives -2
 6. 2.675.round(2)                     -> 2.68                   the Float is 2.67499999999999982236, but 2.675 * 100 is 267.5, and that half rounds up
 7. 0.125.round(2)                     -> 0.13                   0.125 is exact in binary: a true half, rounded away from zero
 8. 2.567.floor(2), 2.567.ceil(2)      -> 2.56, 2.57             digits arguments; 1234.567.round(-2) is 1200, an Integer
 9. Float::EPSILON                     -> 2.220446049250313e-16  MAX 1.7976931348623157e+308, MIN 2.2250738585072014e-308, DIG 15, MANT_DIG 53
10. 2.0 ** 1024                        -> Infinity               overflow is Infinity, not an error; Float::MAX * 10 is Infinity
11. nan == nan, nan.nan?               -> false, true            NaN is unequal to everything, itself included; nan <=> 1.0 is nil
12. [nan].include?(nan), [nan] == [nan] -> true, true             a container checks identity before ==, so the same NaN object is found
13. [1.0, nan].max                     -> ArgumentError: comparison of Float with 1.0 failed max and sort refuse a NaN
14. 0.1.to_r                           -> (3602879701896397/36028797018963968) the exact binary value; 0.5.to_r is (1/2)
15. 0.1.rationalize(Rational(1, 100))  -> (1/10)                 the simplest fraction within 1/100 of it
16. 1e16 + 1                           -> 1.0e+16                == 1e16 is true: the doubles are 2 apart here; 1e16 + 2 is 1.0000000000000002e+16
17. 1.0.next_float                     -> 1.0000000000000002     next_float - 1.0 == Float::EPSILON is true; 1e16.next_float is 1.0000000000000002e+16

Reading the output

Rows 1–3 are the binary fraction. 0.1 + 0.2 prints as 0.30000000000000004, the shortest decimal that reads back as that double, and it is not == 0.3; rounding to two places gives a Float that is (row 2). Row 3 measures the miss: 5.55e-17, a quarter of Float::EPSILON, which is the gap between 1.0 and the next Float — a useful yardstick only near 1.0, which is why a tolerance should scale with the numbers (the second exercise below).

Rows 4–8 are rounding. 2.5.round is 3 and -2.5.round is -3: a half goes away from zero, and the result is an Integer. half: :even gives banker's rounding, 2 for 2.5 and 4 for 3.5. Row 6 is the one to remember: 2.675 is stored as 2.67499999999999982236, yet 2.675.round(2) is 2.68, because Ruby scales first — 2.675 * 100 is exactly 267.5 in Float arithmetic — and rounds that half up. Row 7 is a true half, 0.125, which is exact in binary, rounded away from zero to 0.13. floor and ceil take a digits argument too, and a negative ndigits rounds to tens and hundreds, returning an Integer.

Rows 9–10 are the limits. Float::EPSILON, MAX, MIN, DIG (15 decimal digits survive a round trip) and MANT_DIG (53 bits of significand) are the constants of a double; 2.0 ** 1024 and Float::MAX * 10 overflow to Infinity without a word. Rows 11–13 are NaN: unequal to itself, <=> gives nil, but [nan].include?(nan) and [nan] == [nan] are true because Array compares identity before calling == — and [1.0, nan].max raises, since no order contains a NaN. Rows 14–17 show what a Float really holds: 0.1.to_r is the exact fraction 3602879701896397/36028797018963968, rationalize with a tolerance recovers (1/10), 1e16 + 1 is 1e16 because the doubles are two apart there, and next_float walks to the neighbouring double.

Compared with Python

Same bits, different rules. Rows 1–3 agree to the last digit, and Python adds math.isclose, whose default tolerance is relative. Rows 4–7 are the differences a Python programmer must unlearn in Ruby: round() is banker's rounding and nothing else — the row 5 twin reaches for decimal with ROUND_HALF_UP to get Ruby's default — and round(2.675, 2) is 2.67, because Python rounds the exact binary value rather than a scaled one, so a stored value below the half goes down. 0.125 goes to even, 0.12. floor has no digits argument, and round(1234.567, -2) stays a float. Row 10 is the loud difference: 2.0 ** 1024 raises OverflowError (multiplication by 10 does not — it returns inf), where Ruby answers Infinity. Row 13 is the quiet one: max(1.0, nan) is 1.0 and max(nan, 1.0) is nan, no error, because max keeps the first value unless a later one compares greater and a NaN never does; Ruby refuses. Fraction(0.1), limit_denominator, math.nextafter and math.ulp are the names of to_r, rationalize and next_float. The decimal module is where Python keeps rounding modes and decimal arithmetic; its Ruby counterpart, BigDecimal, is the next lesson.

Verified output of floats_and_rounding_py.py — regenerated by tools/run_examples.py, never hand-typed.

 1. 0.1 + 0.2                          -> 0.30000000000000004    == 0.3 is False
 2. round(0.1 + 0.2, 2)                -> 0.3                    == 0.3 is True
 3. abs(0.1 + 0.2 - 0.3)               -> 5.551115123125783e-17  < sys.float_info.epsilon is True; math.isclose(0.1 + 0.2, 0.3) is True
 4. round(2.5), round(3.5), round(-2.5) -> 2, 4, -2               a half rounds to even; the result is an int
 5. Decimal('2.5') quantized HALF_UP   -> 3                      away from zero on request, via decimal; 3.5 gives 4, -2.5 gives -3
 6. round(2.675, 2)                    -> 2.67                   the float is 2.67499999999999982236, and round() rounds that exact value, so down
 7. round(0.125, 2)                    -> 0.12                   0.125 is exact in binary: a true half, rounded to even
 8. math.floor(2.567 * 100) / 100      -> 2.56                   no digits argument on floor; round(1234.567, -2) is 1200.0, a float
 9. sys.float_info.epsilon             -> 2.220446049250313e-16  max 1.7976931348623157e+308, min 2.2250738585072014e-308, dig 15, mant_dig 53
10. 2.0 ** 1024                        -> OverflowError          overflow raises; sys.float_info.max * 10 is inf, no error
11. nan == nan, math.isnan(nan)        -> False, True            NaN is unequal to everything, itself included; nan < 1.0 is False
12. nan in [nan], [nan] == [nan]       -> True, True             a container checks identity before ==, so the same NaN object is found
13. max(1.0, nan), max(nan, 1.0)       -> 1.0, nan               no error: the answer depends on the order
14. Fraction(0.1)                      -> 3602879701896397/36028797018963968 the exact binary value; (0.1).hex() is 0x1.999999999999ap-4
15. Fraction(0.1).limit_denominator(100) -> 1/10                   the simplest fraction with a denominator up to 100
16. 1e16 + 1                           -> 1e+16                  == 1e16 is True: the doubles are 2 apart here; 1e16 + 2 is 1.0000000000000002e+16
17. math.nextafter(1.0, 2.0)           -> 1.0000000000000002     math.ulp(1.0) == epsilon is True; nextafter(1e16, inf) is 1.0000000000000002e+16

The Python library's Float equality and NaN ↗ is the twin of this page: why an epsilon test fails at 2000, what math.isclose scales, and the container-identity rule that row 12 shows in both languages.

Try it

1. Print a table of the halves -2.5, -1.5, -0.5, 0.5, 1.5, 2.5 under round, round(half: :even), round(half: :down), floor, ceil and truncate. Which column is symmetric about zero, and which two columns agree on every row?
2. Write close?(a, b, rel_tol: 1e-9, abs_tol: 0.0) that does what Python's math.isclose does — (a - b).abs <= [rel_tol * [a.abs, b.abs].max, abs_tol].max — and check it on 0.1 + 0.2 against 0.3, 1e16 against 1e16 + 2, and 1e-20 against 0.0 with and without an abs_tol. Then explain why (a - b).abs < Float::EPSILON gives the wrong answer for the second pair.

Solution to 1

floats_and_rounding_kata_rb.rb in full — pasted here by tools/run_examples.py from the file CI runs.

# Kata: the six roundings of a half — round, round(half: :even),
# round(half: :down), floor, ceil and truncate — on the halves from -2.5 to 2.5.

puts format("%-6s %6s %6s %6s %6s %6s %6s", "x", "round", "even", "down", "floor", "ceil", "trunc")
[-2.5, -1.5, -0.5, 0.5, 1.5, 2.5].each do |x|
  puts format("%-6s %6d %6d %6d %6d %6d %6d",
              x, x.round, x.round(half: :even), x.round(half: :down), x.floor, x.ceil, x.truncate)
end

Verified output of floats_and_rounding_kata_rb.rb — regenerated by tools/run_examples.py, never hand-typed.

x       round   even   down  floor   ceil  trunc
-2.5       -3     -2     -2     -3     -2     -2
-1.5       -2     -2     -1     -2     -1     -1
-0.5       -1      0      0     -1      0      0
0.5         1      0      0      0      1      0
1.5         2      2      1      1      2      1
2.5         3      2      2      2      3      2

See also

• 1/3r and BigDecimal("0.1") are exact where Float is not — the two types that make 0.1 + 0.2 == 0.3 true, and BigDecimal#round with a mode
• 7 / 2 is 3: integer division floors — where Infinity and NaN come from: a Float divided by zero
• 1 == 1.0 is true, but {1 => :a}[1.0] is nil — comparing across the Integer–Float line, and where 2 ** 53 + 1 fits
• inject and each_with_object — [0.1] * 10 summed with sum gives 1.0 and with inject(:+) does not
• Float equality and NaN ↗ — the Python library's twin: tolerances that scale, and NaN inside a container
• Machine numbers ↗ — the math library on which values a double can hold and why the gaps grow
• Rounding a float ↗ — the Rust library, where round is also half away from zero
• Common error messages — Twenty-nine errors you will meet in your first month of Ruby, each with the code that raises it, the…
• Time is an instant, Date is a calendar day — a Time is a point on the timeline with an offset attached
• Integers never overflow: 2 ** 200 is just an Integer — Ruby has one Integer class with no ceiling
• Numbers become text with format, to_s(16) and % — format speaks C's mini-language
• Twenty-three Ruby traps for a Python programmer — The twenty-three places where a Python reflex gives the wrong answer in Ruby
• Twenty-three Python traps for a Ruby programmer — The twenty-three places where a Ruby reflex gives the wrong answer in Python