Skip to content

OMF

Level: 201 · for anyone with a hex editor open

One line: An OMF module is a stream of records, each a type byte, a 16-bit length and a checksum that makes the whole record sum to zero — unless it is zero, which means not computed and is accepted — with every name a length-prefixed string and the low bit of a record's type saying whether its fields are 16 or 32 bits wide.

What the loader reads

The Relocatable Object Module Format is Intel's 1981 object format for the 8086, later the Tool Interface Standards OMF 1.1 ↗, and the .obj format of every DOS and 16-bit Windows compiler, Borland's, Watcom's and Microsoft's. It has no header. A module is records, back to back, each:

field width what it says
type 1 80 THEADR, 96 LNAMES, 98 SEGDEF, 90 PUBDEF, A0 LEDATA, 9C FIXUPP, 8A MODEND, and some forty more; the low bit selects 32-bit fields
length 2, little-endian how many bytes follow, the checksum included
body the record's fields, with every string as a length byte and the bytes
checksum 1 the byte that makes the record sum to zero mod 256 — or zero, meaning nobody computed it

The first record is THEADR or LHEADR, naming the module, and that is what a reader recognises the format by.

In Python

Verified output of omf_py.py — regenerated by tools/run_examples.py, never hand-typed.

1. A RECORD IS A TYPE, A LENGTH, A BODY AND A CHECKSUM
------------------------------------------------------------------------
   80 08 00 06 63 61 66 65 2e 63 52

   type     0x80  THEADR
   length   08 00       = 8, little-endian, counting the checksum
   body     06 63 61 66 65 2e 63   a length byte, then the name
   checksum 0x52  sum of every byte of the record mod 256 = 0

   The same three fields Framing a format asks for -- type, length,
   check -- with Intel's checksum rule from Intel HEX: the whole record,
   type and length included, sums to zero. This is Intel's 1981 8086
   object format, and the 8086's byte order: little-endian throughout.

2. THE MODULE, RECORD BY RECORD
------------------------------------------------------------------------
   0x80 THEADR  length   8   checksum 0x52 ok    ['cafe.c']
   0x96 LNAMES  length  13   checksum 0x95 ok    ['', '_TEXT', 'CODE']
   0x98 SEGDEF  length   7   checksum 0x12 ok    48 00 01 02 03 01
   0xa0 LEDATA  length   9   checksum 0xc0 ok    01 00 00 63 61 66 c3 a9
   0x8a MODEND  length   2   checksum 0x74 ok    00

   THEADR names the module, LNAMES lists every name any later record
   will refer to by index, SEGDEF defines a segment by those indexes,
   LEDATA carries bytes for it, MODEND ends the module. No offsets:
   every record is found by walking from the one before.

3. EVERY STRING HAS A LENGTH BYTE
------------------------------------------------------------------------
   LNAMES body   00 05 5f 54 45 58 54 04 43 4f 44 45
   name 1   length 0   b''
   name 2   length 5   b'_TEXT'
   name 3   length 4   b'CODE'

   No terminator anywhere: the first name is the empty string, one
   byte long, which is what index 1 in a SEGDEF means by 'no name'.
   A name can be 255 bytes at most and may hold any byte, including
   a NUL, which a C string could not. Ghidra's OmfUtils.readString
   is exactly 'read a byte, read that many'.

4. THE LOW BIT OF THE TYPE IS THE WIDTH OF THE FIELDS
------------------------------------------------------------------------
   98 07 00 48 00 01 02 03 01 12            type 0x98 = SEGDEF     segment length 0x100 in 2 bytes
   99 09 00 48 00 00 01 00 02 03 01 0f      type 0x99 = SEGDEF32   segment length 0x10000 in 4 bytes

   0x98 and 0x99 are one record type; the low bit says whether its
   numeric fields are 16 or 32 bits, and Ghidra masks it off before
   looking the type up. The 8086 format grew 32-bit fields for the
   386 by spending one bit that was always zero.

5. A CHECKSUM OF ZERO MEANS 'NOT COMPUTED', AND IS ACCEPTED
------------------------------------------------------------------------
   as written                     checksum 0x52   sum 0x00   valid
   checksum byte set to 0         checksum 0x00   sum 0xae   valid
   one byte of the name changed   checksum 0x52   sum 0xe0   INVALID

   'Some compilers just set this to zero', says the comment in Ghidra's
   OmfRecord.validCheckSum, and so a zero passes. A check that can be
   switched off by the writer is a check the reader cannot rely on --
   and a record damaged in a way that happens to leave the byte at
   zero is, to this reader, a record nobody checked.

6. HOW GHIDRA RECOGNISES A MODULE
------------------------------------------------------------------------
   a THEADR                         type 0x80  length  8  name length 6   OMF
   an LHEADR                        type 0x82  length  5  name length 3   OMF
   a THEADR with the wrong length   type 0x80  length  9  name length 6   not OMF
   an LNAMES first                  type 0x96  length 13  name length 0   not OMF

   OmfFileHeader.checkMagicNumber: the first record must be THEADR
   or LHEADR, and its length must be the name's length plus two --
   the name byte and the checksum. There is no magic; the first
   record's arithmetic is the signature.

Three fields, and Intel's checksum again

Section 1 is a record, and it is the three fields Framing a format asks for — type, length, check — in the same company's earlier format, with the same rule for the check: add up every byte of the record and expect zero. Intel HEX has the record mark outside the sum and a text frame; OMF is binary, little-endian like the 8086, and puts the type and the length inside the sum. Section 2 walks a five-record module, and there is no table of offsets anywhere: a reader finds each record by adding the previous one's length, and the format's only structure is that order.

Every string has a length byte

Section 3: OMF has no NUL-terminated strings at all. Every name is a byte of length and that many bytes, so a name can be at most 255 long, may hold any byte including a NUL, and needs no terminator — the NE and PEF formats do the same, against ELF, Mach-O and PE, which run to a NUL. LNAMES lists every name a later record will use, and later records refer to them by index, from 1; index 0 means no name, and the first name in the list is conventionally the empty string, one byte long, so that index 1 can mean no name too.

The low bit is the width

Section 4: SEGDEF is 0x98, and 0x99 is the same record with 32-bit length fields — every type on the list with a 16-bit field has an odd twin, and Ghidra masks the bit off before looking the type up. It is the ELF class byte per record rather than per file: a 386 compiler emits odd types for segments over 64 KB and even ones otherwise, in the same module. One bit that was always zero in 1981 bought the format its 32-bit future.

A check that can be switched off

Section 5 is the finding. OmfRecord.validCheckSum accepts a checksum byte of zero without adding anything up — "Some compilers just set this to zero," the comment says — so a writer can opt out, and a record damaged in a way that leaves the byte at zero is, to this reader, a record nobody checked. Compare a DEX file, whose two hashes cannot be declined, or Intel Hex, where a zero checksum is simply a checksum that happens to be zero. Knowing what a check does not cover was the second half of adding one; knowing whether it was run is the third.

What Ghidra checks

OmfLoader ↗ calls OmfFileHeader.checkMagicNumber, which reads the first record's type, masks the low bit, requires THEADR or LHEADR, and then requires the record's length to equal the name's length byte plus two — section 6 tries four openings against that rule. There is no magic number; the first record's arithmetic is the signature. The name it shows is Relocatable Object Module Format (OMF), and a sibling Omf51Loader, not on this list, reads Intel's separate OMF-51 format for the 8051.

If you are coming from Python or ABAP

Python. bytes([len(b)]) + b writes a name and body[pos + 1:pos + 1 + body[pos]] reads one; struct.unpack_from('<H', rec, 1) is the length and sum(rec) & 0xFF == 0 the check, with or rec[-1] == 0 if you want Ghidra's leniency and a comment saying so. Walking a module is a while pos < len(data) loop that adds 3 + length each time, and a record type looked up as rectype & 0xFE.

ABAP. (Not machine-checked — CI cannot run ABAP.) A stream of typed, length-prefixed records read in order is an IDoc without its control record, or a TAB-less flat file with a record type in column one — the shape every interface has when it has no index. Two OMF rules transfer directly: a length that includes its own check byte has to be subtracted before the slice, and a checksum field that may legitimately be zero has to be tested for presence before it is tested for correctness, or the absence reads as a pass.

Try it

  1. Find any 16-bit .obj — a DOS compiler's output, or an old Windows SDK's .lib, which is OMF records wrapped in a library header — and xxd -l 3 it. The first byte is 80 or 82, and the next two are a length.
  2. Read that length, add three, and xxd -s <that> -l 3: the next record's type.
  3. Add up every byte of the first record, mod 256. Then find a record whose last byte is 00 and decide what it proves.
  4. grep -c for 99, 9D or A1 type bytes in a 386-era object and a DOS one.
  5. Write the same name as an OMF string, a NUL-terminated string and a Java writeUTF string, and compare the three byte counts.

See also