DYLD shared cache¶
Level: 201 · for anyone with a Mac
One line: The cache's magic is a 16-byte text field, dyld_v1 and the architecture right-justified in it, so the loader compares a trimmed string rather than a number; behind it are mappings and image paths for hundreds of libraries that no longer exist as files, which is why /usr/lib/libSystem.B.dylib is a name the linker knows and ls does not.
What the loader reads¶
Since macOS 11, every system library on a Mac is prelinked into one file per architecture, the dyld shared cache, and the individual .dylib files were removed from disk. The cache is a memory image: a header, a table of mappings saying which file bytes appear at which addresses with which permissions, and a table of images, one per library, each with an address and a path. Apple's dyld_cache_format.h ↗ defines it, and the header begins:
| offset | field | width | what it says |
|---|---|---|---|
| 0 | magic |
16 | dyld_v1 followed by the architecture, right-justified, then a NUL |
| 16 | mappingOffset, mappingCount |
4 each | where the 32-byte mapping records are |
| 24 | imagesOffsetOld, imagesCountOld |
4 each | the image table in older caches; 0 in a modern one |
| 32 | dyldBaseAddress |
8 | |
| 40 | codeSignatureOffset, codeSignatureSize |
8 each | |
| … | localSymbolsOffset, localSymbolsSize, uuid, cacheType … |
some sixty more fields, added release by release |
Everything after the magic is little-endian, and every address is 64-bit. A modern cache is several files — the main one, subcaches with extensions .01, .02… and a .symbols file — and the main header carries a table of the others.
In Python¶
Verified output of dyld_shared_cache_py.py — regenerated by tools/run_examples.py, never hand-typed.
1. THE MAGIC IS A 16-BYTE TEXT FIELD
------------------------------------------------------------------------
64 79 6c 64 5f 76 31 20 20 20 20 69 33 38 36 00 b'dyld_v1 i386\x00' in the table
64 79 6c 64 5f 76 31 20 20 78 38 36 5f 36 34 00 b'dyld_v1 x86_64\x00' in the table
64 79 6c 64 5f 76 31 20 78 38 36 5f 36 34 68 00 b'dyld_v1 x86_64h\x00' in the table
64 79 6c 64 5f 76 31 20 20 61 72 6d 36 34 65 00 b'dyld_v1 arm64e\x00' in the table
64 79 6c 64 5f 76 31 61 72 6d 36 34 5f 33 32 00 b'dyld_v1arm64_32\x00' in the table
dyld_v1, then the architecture right-justified to fill 15 bytes,
then a NUL: i386 gets four spaces and arm64_32 none. Ghidra reads
sixteen bytes, decodes them, trims them, and compares the string
with twelve it knows -- a text comparison, not a number.
2. THE HEADER, THEN A MAPPING TABLE
------------------------------------------------------------------------
magic b'dyld_v1 x86_64h\x00'
mappingOffset 512 mappingCount 2 imagesOffsetOld 0 imagesCountOld 0
uuid f25a411f0e2b394f861b412f1a52f90d
address size fileOffset maxProt initProt
0x007ff800000000 650067968 0 r-x r-x
0x007ff840000000 55705600 650067968 rw- r--
Every field is little-endian and every address is 64-bit: this is
an image of memory, and the mappings say which file bytes land at
which addresses with which permissions. Nothing here is code yet.
3. AN IMAGE IS AN ADDRESS AND A PATH
------------------------------------------------------------------------
image 0 address 0x7ff800001000 pathFileOffset 640 -> '/usr/lib/libSystem.B.dylib'
image 1 address 0x7ff800011000 pathFileOffset 667 -> '/usr/lib/libc++.1.dylib'
The path is a NUL-terminated string INSIDE the cache. On a Mac
since macOS 11, /usr/lib/libSystem.B.dylib is not a file on disk
at all: the linker names it, dyld finds it here, and the only
place the name exists is this table.
4. SUBCACHES ARE NAMED BY AN EXTENSION FIELD
------------------------------------------------------------------------
subcache 0 uuid 00000000... vmOffset 0x40000000 extension '.01'
subcache 1 uuid 11111111... vmOffset 0x80000000 extension '.02'
A modern cache is one file plus subcaches named by appending the
extension -- .01, .02 -- and a .symbols file, each a cache with
its own magic; the entries here are how the main file finds them.
A magic that is a string¶
Every other signature in this chapter is compared as a number. This one is compared as text: Ghidra reads sixteen bytes, decodes them, calls trim(), and looks the result up in a list of twelve strings — dyld_v1 x86_64h, dyld_v1 arm64, dyld_v1 i386. The architecture is right-justified in the space after dyld_v1, so the number of spaces is part of the signature, and arm64_32 fills the field with none. Section 1 builds five of the twelve. It is the Java class file's 0xcafebabe test turned into a strcmp, and it has the property a text magic buys: a person reading head -c 16 knows the architecture without a table.
Measured on this Mac¶
$ ls dyld_shared_cache_x86_64h*
dyld_shared_cache_x86_64h 888,668,160 bytes .01 .02 .03 .04 .05 .06 697 MB to 1,000 MB each
dyld_shared_cache_x86_64h.atlas 1,380,114 dyld_shared_cache_x86_64h.map 1,156,310 (text)
$ xxd -l 32 dyld_shared_cache_x86_64h
00000000: 6479 6c64 5f76 3120 7838 365f 3634 6800 dyld_v1 x86_64h.
00000010: 2802 0000 0400 0000 0000 0000 0000 0000 (...............
mappingOffset 552 mappingCount 4 imagesOffsetOld 0 imagesCountOld 0
uuid f25a411f0e2b394f861b412f1a52f90d
mapping 0: address 0x7ff800000000 size 0x26bf4000 (620 MB) fileOffset 0x0 maxProt 5 initProt 5 r-x
mapping 1: address 0x7ff840000000 size 0x3520000 (53 MB) fileOffset 0x26bf4000 maxProt 3 initProt 1 rw- / r--
mapping 2: address 0x7ff843520000 size 0xde8000 (14 MB) fileOffset 0x2a114000 maxProt 3 initProt 3 rw-
mapping 3: address 0x7ff880000000 size 0x99f0000 (154 MB) fileOffset 0x2aefc000 maxProt 1 initProt 1 r--
$ ls -la /usr/lib/libSystem.B.dylib
ls: /usr/lib/libSystem.B.dylib: No such file or directory
$ grep -c dylib dyld_shared_cache_x86_64h.map
526
28 02 00 00 is 552 little-endian, the mapping table's offset; imagesOffsetOld is 0 because the modern header keeps the image table further along. Four mappings, seven files, six gigabytes, and the library the last line looks for is a string in a table. The .map beside the cache is a text listing of the same mappings, which is where the mapping lines were checked.
file-5.41 application/octet-stream Dyld shared cache version 1 x86_64h
file-5.45 application/octet-stream data
The macOS build ships a rule for the signature and the Ubuntu build, on the same 4,096 bytes, does not recognise it — the same magic database at two versions, finding 22's shape in the other direction.
What Ghidra checks¶
DyldCacheLoader ↗ calls DyldCacheUtils.isDyldCache, which reads sixteen bytes, trims, and compares with DyldArchitecture's twelve signatures; the architecture that matches names the language. The name it shows in 12.1.3 is DYLD Cache, shortened from the list's DYLD Shared Cache. A second loader in the FileFormats module, DyldCacheExtractLoader, pulls one library back out of the cache as a Mach-O, which is how a dylib that is not a file can still be imported as one.
If you are coming from Python or ABAP¶
Python. open(path, 'rb').read(0x1000) is enough for the header, and struct.unpack_from('<IIII', data, 16) for the first four counts; a mapping is '<QQQII', an image '<QQQII' too, and a path is data[off:data.index(b'\0', off)]. Do not read the whole file: the main cache alone is 888 MB here. The magic is data[:16].rstrip(b'\0').decode('ascii'), and the comparison is == on strings, as in Ghidra.
ABAP. (Not machine-checked — CI cannot run ABAP.) There is no shared cache on an application server, but the shape is a familiar one: a container whose index is the only place the members' names survive, like a transport file or an archive whose directory has been folded into the header. Read the table, never the filesystem, for the list of what is inside; and treat a 16-byte fixed-width text field as c LENGTH 16 — right-justified here, which is unusual, so condense before comparing.
Try it¶
ls /System/Volumes/Preboot/Cryptexes/OS/System/Library/dyld/on a Mac andhead -c 16each file that does not end in.mapor.atlas. Every one has the magic.xxd -s 16 -l 8the main cache and computemappingOffset, thenxxd -s <that> -l 32and read the first mapping's address.otool -L /bin/lslists dylibs by path.lsone of them. Thengrepthat path in the.mapfile.filethe first 4,096 bytes on a Mac and on Linux, and note which build recognises it.strings -n 12 dyld_shared_cache_x86_64h | grep '^/usr/lib/' | head— the image paths, as they sit in the file.
See also¶
- Mach-O — what every image in the cache was before it was prelinked
- The NUL byte — the terminator of every path in the image table
- Fixed-width byte fields — a 16-byte text field with padding, met in an interface rather than a header
- File type is four questions — why two versions of
filecan disagree about the same bytes dyld_cache_format.h↗ — Apple's header, which the field table above was checked against