Parsing a number from text

Level: 201 · working knowledge

One line: Turning "42" into 42 is deserialization in miniature, and C offers three functions that fail three different ways — atoi returns 0 for both zero and garbage and cannot tell you which, sscanf reports how many fields it matched but leaves the overflow undefined, and strtol tells you exactly where it stopped and sets errno on overflow, which is the only one of the three safe to point at input you did not write.

Every program that reads a configuration value, a header, a command-line argument or a line of a file does this, and it is the smallest possible version of the general problem: bytes arrive, they are supposed to mean a number, and they might not. The three C functions form a ladder from "cannot detect failure" to "detects every failure", and knowing which rung you are on is the difference between reading input and trusting it.

The program

parse_number_c.c runs ten inputs through all three functions and prints what each returns:

parse_number_c.c in full — pasted here by tools/run_examples.py from the file CI runs.

/* Turning text into a number is deserialization in miniature, and C's three
   ways of doing it fail three different ways: atoi cannot report failure at
   all, sscanf reports how many items it matched and leaves the rest, strtol
   tells you exactly where it stopped and why. */
#include <errno.h>
#include <limits.h>
#include <stdio.h>
#include <stdlib.h>

/* strtol with every check the manual page asks for. */
static const char *parse_long(const char *text, long *out)
{
    char *end;
    errno = 0;
    long v = strtol(text, &end, 10);
    if (end == text) return "no digits";
    if (*end != '\0') return "trailing characters";
    if (errno == ERANGE) return "out of range";
    *out = v;
    return NULL;
}

int main(void)
{
    /* Every input here is inside int's range, because atoi and sscanf's %d are
       undefined on a value that overflows -- so the overflow case is handled
       below, through strtol alone, which is the only one of the three that is
       allowed to see it. */
    const char *inputs[] = { "42", " 42", "42 ", "42abc", "", "abc",
                             "-7", "+7", "0x1A" };
    size_t count = sizeof inputs / sizeof inputs[0];

    printf("%-12s %-8s %-18s %s\n", "input", "atoi", "sscanf %d", "strtol, checked");
    for (size_t i = 0; i < count; i++) {
        const char *in = inputs[i];
        char shown[16];
        snprintf(shown, sizeof shown, "\"%s\"", in);

        int a = atoi(in);

        int n = 0;
        int items = sscanf(in, "%d", &n);
        char scanned[20];
        if (items == 1) snprintf(scanned, sizeof scanned, "1 item, %d", n);
        else            snprintf(scanned, sizeof scanned, "%d items", items);

        long v = 0;
        const char *why = parse_long(in, &v);
        char checked[40];
        if (why) snprintf(checked, sizeof checked, "error: %s", why);
        else     snprintf(checked, sizeof checked, "%ld", v);

        printf("%-12s %-8d %-18s %s\n", shown, a, scanned, checked);
    }

    /* The overflow case, on strtol only: atoi would be undefined here, and
       sscanf's %d with it too. strtol sets errno to ERANGE and reports it. */
    long v = 0;
    const char *why = parse_long("99999999999999999999999", &v);
    printf("\nstrtol(\"99999999999999999999999\", checked) -> error: %s\n",
           why ? why : "none");

    /* Base 0 lets the text choose: 0x for hex, a leading 0 for octal. */
    printf("strtol(\"0x1A\", base 0) = %ld, strtol(\"012\", base 0) = %ld, base 10 = %ld\n",
           strtol("0x1A", NULL, 0), strtol("012", NULL, 0), strtol("012", NULL, 10));

    /* The width on %s is the only thing between scanf and gets. */
    char name[6];
    int got = sscanf("Zbigniew", "%5s", name);
    printf("sscanf(\"Zbigniew\", \"%%5s\") -> %d item, name = \"%s\"\n", got, name);
    return 0;
}

Verified output of parse_number_c.c — regenerated by tools/run_examples.py, never hand-typed.

input        atoi     sscanf %d          strtol, checked
"42"         42       1 item, 42         42
" 42"        42       1 item, 42         42
"42 "        42       1 item, 42         error: trailing characters
"42abc"      42       1 item, 42         error: trailing characters
""           0        -1 items           error: no digits
"abc"        0        0 items            error: no digits
"-7"         -7       1 item, -7         -7
"+7"         7        1 item, 7          7
"0x1A"       0        1 item, 0          error: trailing characters

strtol("99999999999999999999999", checked) -> error: out of range
strtol("0x1A", base 0) = 26, strtol("012", base 0) = 10, base 10 = 12
sscanf("Zbigniew", "%5s") -> 1 item, name = "Zbign"

atoi cannot report failure. It returns 0 for "abc" and 0 for "0" — the same answer for "not a number" and "the number zero", with no way to tell them apart. For "42abc" it returns 42 and throws the rest away silently. And on a value too big for int its behaviour is undefined, which is why the overflow input is not passed to it at all: a program with undefined behaviour has no correct output to record. If you have ever wondered why a bad config value was silently treated as zero, atoi is often the reason.

sscanf reports how many fields matched. sscanf(in, "%d", &n) returns the number of items it converted — 1 on success, 0 when the text does not start with a number, and EOF (shown as -1) when there is nothing to read. That is genuinely more than atoi gives you, and the width modifier on %s — %5s in the last line — is the one thing standing between scanf and gets, because an unbounded %s overflows its buffer exactly like gets. But %d on a value that overflows is undefined too, so sscanf is not the function to hand untrusted numbers to.

strtol reports all four outcomes. It writes, through its second argument, a pointer to the first character it did not consume — so end == text means "no digits at all", and *end != '\0' means "trailing characters". It sets errno to ERANGE on overflow, which is the case atoi and sscanf cannot survive, and here reports out of range for the twenty-three-digit input. The four checks the parse_long helper performs are exactly the ones the manual page asks for, and skipping any of them is skipping a way the input can be wrong. Base 0 is a bonus: it lets the text choose its own base, reading 0x1A as hex and a leading 0 as octal.

The same job in Python

int() either returns the number or raises — there is no zero-on-failure — and it is more permissive than strtol in ways worth knowing. parse_number_py.py:

parse_number_py.py in full — pasted here by tools/run_examples.py from the file CI runs.

"""int() either returns the number or raises: there is no zero-on-failure.
It is also more permissive than strtol in ways worth knowing -- whitespace on
either side, underscores between digits, and no overflow at all."""
inputs = ["42", " 42", "42 ", "42abc", "", "abc", "-7", "+7", "0x1A",
          "99999999999999999999999", "1_000"]
for text in inputs:
    try:
        print(f"int({text!r:28}) = {int(text)}")
    except ValueError as e:
        print(f"int({text!r:28}) -> ValueError: {e}")

print(f"int('0x1A', 0) = {int('0x1A', 0)}, int('012', 0) -> ", end="")
try:
    print(int("012", 0))
except ValueError as e:
    print(f"ValueError: {e}")
print(f"int('0o12', 0) = {int('0o12', 0)}")

Verified output of parse_number_py.py — regenerated by tools/run_examples.py, never hand-typed.

int('42'                        ) = 42
int(' 42'                       ) = 42
int('42 '                       ) = 42
int('42abc'                     ) -> ValueError: invalid literal for int() with base 10: '42abc'
int(''                          ) -> ValueError: invalid literal for int() with base 10: ''
int('abc'                       ) -> ValueError: invalid literal for int() with base 10: 'abc'
int('-7'                        ) = -7
int('+7'                        ) = 7
int('0x1A'                      ) -> ValueError: invalid literal for int() with base 10: '0x1A'
int('99999999999999999999999'   ) = 99999999999999999999999
int('1_000'                     ) = 1000
int('0x1A', 0) = 26, int('012', 0) -> ValueError: invalid literal for int() with base 0: '012'
int('0o12', 0) = 10

The overflow input that is undefined behaviour in C's easy functions is just a number in Python, because int has no upper bound — int("99999999999999999999999") returns it exactly. Python also trims surrounding whitespace, accepts underscores between digits, and, like strtol base 0, reads a base from a 0x or 0o prefix.

And in Rust

str::parse returns a Result, and the Err says which of strtol's checks failed, in words. Nothing is trimmed and nothing is guessed. parse_number_rs.rs:

parse_number_rs.rs in full — pasted here by tools/run_examples.py from the file CI runs.

// `str::parse` returns a Result, and the Err says which of strtol's three
// checks failed. Nothing is trimmed and nothing is guessed: " 42" is an error,
// and so is a number the type cannot hold.
fn main() {
    let inputs = ["42", " 42", "42 ", "42abc", "", "abc", "-7", "+7", "0x1A",
                  "99999999999999999999999"];
    for text in inputs {
        match text.parse::<i32>() {
            Ok(v) => println!("{:28} parse::<i32>() = Ok({v})", format!("{text:?}")),
            Err(e) => println!("{:28} parse::<i32>() = Err: {e}", format!("{text:?}")),
        }
    }
    println!("\" 42 \".trim().parse::<i32>() = {:?}", " 42 ".trim().parse::<i32>());
    println!("i32::from_str_radix(\"1A\", 16) = {:?}", i32::from_str_radix("1A", 16));
    println!("\"99999999999999999999999\".parse::<u128>() = {:?}",
             "99999999999999999999999".parse::<u128>());
}

Verified output of parse_number_rs.rs — regenerated by tools/run_examples.py, never hand-typed.

"42"                         parse::<i32>() = Ok(42)
" 42"                        parse::<i32>() = Err: invalid digit found in string
"42 "                        parse::<i32>() = Err: invalid digit found in string
"42abc"                      parse::<i32>() = Err: invalid digit found in string
""                           parse::<i32>() = Err: cannot parse integer from empty string
"abc"                        parse::<i32>() = Err: invalid digit found in string
"-7"                         parse::<i32>() = Ok(-7)
"+7"                         parse::<i32>() = Ok(7)
"0x1A"                       parse::<i32>() = Err: invalid digit found in string
"99999999999999999999999"    parse::<i32>() = Err: number too large to fit in target type
" 42 ".trim().parse::<i32>() = Ok(42)
i32::from_str_radix("1A", 16) = Ok(26)
"99999999999999999999999".parse::<u128>() = Ok(99999999999999999999999)

" 42" is an Err — the leading space is a "invalid digit", because parse does not trim — and the overflow is Err: number too large to fit in target type rather than undefined behaviour, because the target type is named at the call and the check is part of it. parse::<u128>() of the same digits succeeds, because it fits there.

If you are coming from another language

ABAP. (Not machine-checked — CI cannot run ABAP.) ABAP's MOVE and arithmetic on a character field convert implicitly, and a non-numeric value raises CX_SY_CONVERSION_NO_NUMBER rather than returning a silent zero — closer to Python's raise than to C's atoi. The width and overflow questions come back at the type: a packed or integer target has a range, and the conversion is where the range is checked.

See also

• A char is a byte, not a character — the other direction bytes have to be interpreted, and the checks C leaves to you there too
• A record on the wire — parsing a binary number rather than a text one, where the failure modes are byte order and width instead of stray characters
• Parsing a string ↗ — parse and the FromStr trait in full
• Encode, decode and errors ↗ — the same "what do you do when the bytes are wrong" question, one layer down
• Numbers from text ↗ — Perl's + 0, which behaves like atoi: it reads 42abc as 42, warns, and carries on