Intel Hex¶

Level: 201 · for anyone with a hex editor open

One line: A data record's offset field is 16 bits, so an image larger than 64 KB needs the two extended-address records — one shifting by 4 bits and one by 16 — and the same data record lands at three different addresses depending on which of them last preceded it; meanwhile both hex loaders decide a file is theirs from one regular expression over its first non-blank line, which cannot tell the two formats apart.

What the loader reads¶

A record has to say what it is, how long it is, and whether it arrived already took an Intel HEX record apart field by field — the colon, the count, the offset, the type, the data, the two's-complement checksum — against Intel's 1988 specification. This page is the other half of the format: what a loader does with a file of them, which is to build a memory image. The six record types, as Ghidra's IntelHexRecord names them:

type	name	data field	what the loader does
`00`	Data	the bytes	places them at base + offset
`01`	End of File	none	stops
`02`	Extended Segment Address	a 16-bit value	base = value × 16 — the 8086's segment arithmetic, reaching 1 MB
`03`	Start Segment Address	`CS:IP`	records the entry point, 8086-style
`04`	Extended Linear Address	a 16-bit value	base = value << 16 — the upper half of a 32-bit address, reaching 4 GB
`05`	Start Linear Address	a 32-bit `EIP`	records the entry point

Base is state the loader keeps between records, and it is the whole subject.

In Python¶

Verified output of intel_hex_py.py — regenerated by tools/run_examples.py, never hand-typed.

1. THE OFFSET FIELD IS 16 BITS
------------------------------------------------------------------------
   :05FFFC00636166C3A96A
   type 0 Data   offset 0xfffc   5 bytes 63 61 66 c3 a9

   Four hex digits of offset reach 65,535 and no further. A firmware
   image for anything with more than 64 KB of address space needs
   another record to say which 64 KB, and Intel gave it two.

2. THE SAME DATA RECORD, THREE ADDRESSES
------------------------------------------------------------------------
   nothing before it                              :05010000636166C3A964   lands at 0x00000100
   02: Extended Segment Address 0x1000  (x 16)    :05010000636166C3A964   lands at 0x00010100
   04: Extended Linear Address 0x0001   (<< 16)   :05010000636166C3A964   lands at 0x00010100
   04: Extended Linear Address 0x0800   (<< 16)   :05010000636166C3A964   lands at 0x08000100

   The data record is identical in all four files: 63 61 66 c3 a9 at
   offset 0x0100. What changed is the record before it, which the
   loader keeps as state. Type 02 shifts by 4 bits, the 8086's
   segment arithmetic; type 04 shifts by 16, and reaches 4 GB.

3. A LOADER IS A LITTLE STATE MACHINE
------------------------------------------------------------------------
   :020000040800F2              4  Extended Linear Address  08 00
   :05000000636166C3A965        0  Data                     63 61 66 c3 a9
   :0400100000010203E6          0  Data                     00 01 02 03
   :020000040801F1              4  Extended Linear Address  08 01
   :02000000FFFE01              0  Data                     ff fe
   :0400000508000010DF          5  Start Linear Address     08 00 00 10
   :00000001FF                  1  End of File              

   0x08000000..0x08000004   63 61 66 c3 a9
   0x08000010..0x08000013   00 01 02 03
   0x08010000..0x08010001   ff fe
   start address   EIP 0x08000010

   Three runs of bytes at three addresses, out of seven lines of
   text, and the file itself has no notion of a total size or an
   order: a loader could read the records in any sequence as long
   as each data record follows the extended-address record it needs.

4. HOW GHIDRA DECIDES A FILE IS HEX
------------------------------------------------------------------------
   ':020000040800F2'      possible hex file  an Intel Hex record
   'S00600004844521B'     possible hex file  a Motorola S-record
   ':00000001FF  '        not a hex file     trailing spaces
   ':0000000 1FF'         not a hex file     a space inside
   ''                     not a hex file     an empty file

   One regular expression over the first non-blank line, shared by the
   Intel and Motorola loaders: a colon or an S, then hex digits to the
   end. Neither loader can tell the two formats apart at that point;
   the record mark is the only difference, and both offer every
   processor Ghidra has, because a hex file names none.

A 16-bit field and a 32-bit address¶

Section 1 shows the limit: four hex digits of offset, so a data record can say where only within 64 KB. Section 2 is the consequence. Four files hold the same data record, :05010000636166C3A964, and the bytes land at 0x100, 0x10100, 0x10100 again and 0x8000100, because what differs is the previous record — type 02 with 0x1000, multiplied by 16, or type 04 with 0x0001 or 0x0800, shifted by 16. The data record does not know its own address. A reader that takes records out of order, or drops one, places bytes in the wrong 64 KB and every checksum still passes, because a checksum is per record and the address is per file.

Two of those variants reach the same address by different arithmetic, and both are in use: 02 is what 8086 tools emit and 04 what everything since does, and a loader has to keep both, since a file may carry either. That is the same shape as MZ's paragraphs, one unit chosen in 1981 and a second added when it ran out.

The loader is a state machine¶

Section 3 runs seven records into three runs of bytes and a start address. The file has no header, no total size and no list of what it contains; what a loader knows at any line is the current base and what it has placed so far. That is why a hex file can be concatenated from parts, why a tool can emit records in any order that respects the base, and why the Motorola format, which puts a full address in every record, needs no such state at all — Motorola Hex is the comparison.

How Ghidra decides¶

Section 4 is the test, and it is one line of the source, in MotorolaHexLoader.isPossibleHexFile:

MotorolaHexLoader.java, Ghidra 12.1.3 — the whole of both hex loaders' recognition test, verbatim

return line.matches("^[S:][0-9a-fA-F]+$");

The first non-blank line, up to a hundred blank ones skipped, must be a colon or an S followed by hex digits and nothing else — so a trailing space fails it, and so does a comment line, and an S-record passes the Intel loader's test as readily as the Intel loader's own. Both loaders then offer every language Ghidra has, none preferred, because a hex file names no processor: the reader picks one from a list of hundreds, and a base address, which is Raw binary's dialog with an address column filled in.

What Ghidra checks¶

IntelHexLoader ↗ calls the Motorola loader's isPossibleHexFile and, if it passes, lists every language and compiler-spec pair as a candidate; its options are a base address, a block name and an overlay flag. Its record class knows the six types above and a maximum record length of 255. The name it shows is Intel Hex.

If you are coming from Python or ABAP¶

Python. bytes.fromhex(line[1:]) reads a whole record at once, and sum(body) & 0xFF == 0 is the checksum test; the program keeps base in one variable and a dict from address to byte, which is the simplest memory image there is. int.from_bytes(data, 'big') reads the extended-address value — Intel HEX is big-endian in its address fields although the 8086 was not, because the bytes are written left to right as text. intelhex on PyPI does all of this and is not in the standard library.

ABAP. (Not machine-checked — CI cannot run ABAP.) A hex file is text, so it arrives as a string and each record's data is xstring after a hex conversion — the armour is the conversion, the frame is the rest. The transferable lesson is the state: a record that means different things depending on what came before it is what an IDoc segment inside a parent segment is, and a parser that handles each line on its own will place every one at base 0.

Try it¶

objcopy -O ihex any.elf out.hex or an Arduino build's .hex. cut -c8-9 out.hex | sort | uniq -c: which of the six types does your toolchain emit?
Find the first 04 record and shift its value by 16. That is where the first data record lands.
Delete that 04 record and load the file in any tool. Everything is still valid and everything is at the wrong address.
Concatenate two .hex files with different 04 records and load the result.
Put a space after a record and offer the file to Ghidra. The importer's format list will not include either hex loader.