Skip to content

PEF

Level: 201 · for anyone with a hex editor open

One line: A PEF container starts with the twelve ASCII bytes Joy!peffpwpc, dates itself in seconds since 1904, and can store an initialised data section as a tiny bytecode — a 3-bit opcode and a 5-bit count in each byte, with a variable-length count when five bits are not enough — that the loader runs to produce the bytes.

What the loader reads

The Preferred Executable Format is the executable and shared-library format of classic Mac OS on the PowerPC, from 1994's System 7.1.2 until Mac OS 9 and its Carbon applications, documented in Apple's Mac OS Runtime Architectures. Everything is big-endian, as the machine was. The container header is 40 bytes:

offset field what it says
0 tag1 Joy!
4 tag2 peff
8 architecture pwpc or m68k
12 formatVersion 1
16 dateTimeStamp seconds since 1904-01-01
20 oldDefVersion, oldImpVersion, currentVersion library versions
32 sectionCount, instSectionCount 16 bits each: how many section headers follow, and how many are instantiated in memory
36 reserved

Each section header is 28 bytes — a name offset (-1 for none), a default address, three sizes, a container offset, and four bytes of kind, share kind, alignment and padding — and the kind is a small integer naming what the section is: 0 code, 1 unpacked data, 2 pattern-initialized data, 3 constant, 4 loader, 5 debug, 6 executable data, 7 exception, 8 traceback.

In Python

Verified output of pef_py.py — regenerated by tools/run_examples.py, never hand-typed.

1. TWELVE ASCII BYTES, THEN A BIG-ENDIAN HEADER
------------------------------------------------------------------------
   4a 6f 79 21 70 65 66 66 70 77 70 63 00 00 00 01   b'Joy!peffpwpc'
   tag1 b'Joy!'  tag2 b'peff'  architecture b'pwpc'   formatVersion 1   sections 2, 2 instantiated

   Joy! and peff are the two tags Ghidra's ContainerHeader compares,
   and pwpc or m68k the architecture that picks the language. Every
   number after them is big-endian, as everything on a 68000 or a
   PowerPC Mac was, and nothing in the file says so.

2. THE TIMESTAMP COUNTS FROM 1904
------------------------------------------------------------------------
   dateTimeStamp   0xc2a43140 = 3,265,540,416 seconds
   from 1904-01-01 2007-06-24 14:33:36   the Mac OS epoch, which is what the field means
   from 1970-01-01 2073-06-24 14:33:36   the Unix epoch, which is what a C program assumes
   the two epochs differ by 2,082,844,800 seconds

   An unsigned 32-bit count from 1904 runs out in 2040; a signed one
   from 1970 runs out in 2038; Windows counts 100-ns ticks from 1601.
   The number is the same kind of thing as a byte order: a choice the
   file does not record and a reader has to already know.

3. A SECTION HEADER IS 28 BYTES, AND ONE KIND IS A PROGRAM
------------------------------------------------------------------------
   section 0   kind 0 code                       address 0x10000000   total  512  unpacked  512  packed 512  at 0x64   nameOffset -1
   section 1   kind 2 pattern-initialized data   address 0x20000000   total   64  unpacked   64  packed  13  at 0x264   nameOffset -1

   nameOffset -1 means no name. Kind 2 is the one this page is about:
   totalSize bytes in memory, unpackedSize of them initialised, and
   only packedSize bytes in the file, because the initial contents
   are stored as instructions for producing them.

4. PATTERN-INITIALIZED DATA: A 3-BIT OPCODE AND A 5-BIT COUNT
------------------------------------------------------------------------
   the packed bytes   04 25 63 61 66 c3 a9 42 02 ab cd 00 31

   04                       opcode 0 Zero           count   4
   25 63 61 66 c3 a9        opcode 1 BlockCopy      count   5
   42 02 ab cd              opcode 2 RepeatedBlock  count   2   (then the repeat count, then the block)
   00 31                    opcode 0 Zero           count  49   (count in a varint after the opcode byte)

   unpacked, 64 bytes
      00  00 00 00 00 63 61 66 c3 a9 ab cd ab cd ab cd 00
      10  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
      20  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
      30  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

   Each byte's top three bits are an instruction and the low five a
   count; a count of 0 means 'read a varint next', seven bits per byte
   with the high bit as the continuation flag. Five opcodes, and a
   loader that runs them is executing the data section's bytes to
   produce the data section -- 13 bytes of file for 64 bytes of memory.

Text tags, and an epoch

Section 1: the two tags are compared as strings, Joy! and peff, and the architecture after them picks the language — twelve bytes a person reads as easily as the loader does, and a magic with the era's sense of humour. Section 2 is the timestamp, and it is the byte-order question for dates: a 32-bit count of seconds is a date only once you know the zero, and PEF's is 1904-01-01, the classic Mac epoch, where Unix counts from 1970 and Windows counts 100-nanosecond ticks from 1601. The same number is a day in 2007 or in 2073, and the file does not say which — a reader has to already know, exactly as it has to know which end comes first. An unsigned 32-bit count from 1904 runs out in 2040.

A data section that is a program

Sections 3 and 4 are the reason PEF is on this page rather than merely on the list. A section of kind 2 holds its initialised bytes as instructions for producing them: each byte's top three bits are an opcode and its low five a count, and a count of zero means the real count follows as a varint, seven bits per byte with the high bit as a continuation flag. Five opcodes — zero-fill, copy a block, repeat a block, and two interleavings — and thirteen bytes in the file become sixty-four in memory. The loader executes the data section to obtain the data section, which is the same idea as tribit's 3-bit units and DEX's ULEB128 counts, put to work as a compressor.

Ghidra's SectionHeader.unpackNextValue reads the varint, and its RepeatedBlock case reads the repeat count before the block bytes — the order the program follows, because it is the order the specification gives and the one that makes the count available before the bytes it applies to.

What Ghidra checks

PefLoader ↗ constructs a ContainerHeader, which throws unless tag1 is Joy! and tag2 is peff, then queries its language table with the architecture string; sections are loaded at 0x10000000 by default, and the loader's PackedDataOpcodes enumerates the five opcodes, with 5, 6 and 7 reserved. The name it shows is Preferred Executable Format (PEF).

If you are coming from Python or ABAP

Python. datetime(1904, 1, 1, tzinfo=timezone.utc) + timedelta(seconds=n) is the date, and the program keeps both epochs as constants so the difference, 2,082,844,800 seconds, is computed rather than quoted. The unpacker is thirty lines: b >> 5 and b & 0x1F, a varint loop, and a bytes(n) or a slice per opcode. struct.unpack('>iIIIIIBBBB', ...) is a section header, and the i for the name offset is signed because -1 means no name.

ABAP. (Not machine-checked — CI cannot run ABAP.) ABAP's dates have no epoch problem — d is a character field of eight digits — but its timestamps do: utclong counts from a fixed epoch at 100-nanosecond precision, and a Unix int8 of seconds, a Windows FILETIME and a Mac count from 1904 are three numbers that need three conversions before cl_abap_tstmp can compare them. The pattern-data lesson is the general one for any run-length field in an interface: an opcode-and-count byte is two fields sharing eight bits, and GET BIT or a division by 32 is the only way to read either.

Try it

  1. Find a PEF — a classic Mac OS application's data fork from an archive, a .shlb, or any CFM binary — and xxd -l 40 it. Joy!peff, then the architecture.
  2. Read the timestamp at offset 16, big-endian, and add it to 1904-01-01. Then to 1970-01-01, and pick the plausible one.
  3. Walk the section headers at offset 40, 28 bytes each, and find one of kind 2. Compare its packedSize with its unpackedSize.
  4. Dump the first bytes of that section and decode the first opcode by hand: b >> 5 and b & 0x1F.
  5. date -r <n> on a Unix machine with the raw timestamp, and explain the year it prints.

See also