What the debugger records¶

Level: 201 · for anyone who has set a breakpoint and wants to know what Ghidra adds to one

One line: A debugger reads the process a file became — the stack, the heap, the zero-filled globals the file never held bytes for — and Ghidra is not one: it drives lldb or gdb, keeps a mapping between the file you analyzed and the process the back end has stopped, and writes everything the back end reports at each stop into a trace that can be rewound after the process is gone.

The program¶

Four kinds of memory, one function that changes each of them, and a line of output the shell can check — demo/watch.c:

/* A program for a debugger to watch: four kinds of memory, one function that
   changes each of them, and a line of output the shell can check. */
#include <stdio.h>
#include <stdlib.h>

int table[4] = { 10, 20, 30, 40 };  /* .data: initialized, so its bytes are in the file */
int counter;                        /* .bss: not in the file; the loader supplies zeroes */

int bump(int *slot, int by)
{
    *slot += by;
    counter++;
    return *slot;
}

int main(void)
{
    int local = 5;                     /* the stack */
    int *heap = malloc(sizeof *heap);  /* the heap */
    if (heap == NULL) {
        return 1;
    }
    *heap = 7;
    bump(&table[1], 1);
    bump(&local, 2);
    bump(heap, 3);
    printf("table[1]=%d local=%d *heap=%d counter=%d\n", table[1], local, *heap, counter);
    free(heap);
    return 0;
}

Name	Where it lives	What the file holds for it
`table`	`.data`	its four initial values, byte for byte
`counter`	`.bss`	a size and nothing else; the loader supplies the zeroes
`local`	the stack	nothing — the stack exists only once the process does
`*heap`	the heap	nothing — `malloc` hands it out at run time

What the file holds, and what only the process has¶

Build it with debug information, run it, and ask the file what it knows about the four names. The script's kinds turns nm's type letter into a word, because the two machines disagree on the letter for the zero-filled one:

Verified output of debugger_watch_sh.sh — regenerated by tools/run_examples.py, never hand-typed.

$ cc -std=c17 -Wall -Wextra -O0 -g -fno-common -o watch watch.c
$ ./watch
table[1]=21 local=7 *heap=10 counter=3
$ kinds watch
code       bump
zero-fill  counter
code       main
data       table
$ cc -std=c17 -Wall -Wextra -O0 -o plain plain.c
$ cc -std=c17 -Wall -Wextra -O0 -o bss bss.c
$ cc -std=c17 -Wall -Wextra -O0 -o data data.c
bss:  a 400000-byte array added under 4096 bytes to the file
data: the same array, initialized, added at least 400000 bytes

bump and main are code and table is data: their bytes are in the file. counter is zero-fill: the file records that four bytes are needed at load time and stores none. The three small programs at the end measure that. A 400,000-byte array added under 4,096 bytes to the executable when it was uninitialized and at least 400,000 when it was initialized, on both compilers. local and *heap have no line at all — nothing in the file names them, and a debugger is the only way to see them.

The letter the script folds away is a real difference between the machines. GNU nm files counter under .bss and prints B; Apple's linker puts it in a section called __common and Apple's nm prints S, a section other than text, data or bss, and -fno-common — GCC's default, passed on both machines so the compilers are asked the same thing — does not move it:

Real output — nm on the clang build with -fno-common, x86-64 macOS 26

$ nm -m watch | grep -E '_counter|_table'
0000000100002010 (__DATA,__common) external _counter
0000000100002000 (__DATA,__data) external _table

Real output — GNU nm on the GCC 13.3 build with -fno-common, Docker's ubuntu:24.04

$ nm watch | grep -E ' (counter|table)$'
0000000000004024 B counter
0000000000004010 D table

A back end at work¶

Ghidra's own introduction says it plainly: Ghidra is not a debugger in itself, but rather, it relies on existing 3rd-party ("back end") debuggers. On a Mac the back end is lldb, on Linux gdb, and this is what each of them does with the program — a breakpoint on bump, and counter and *slot read at each of the three stops:

Real output — lldb-2100 from the Xcode Command Line Tools, x86-64 macOS 26; three lines naming the scratch folder cut

(lldb) breakpoint set --name bump
Breakpoint 1: where = watch`bump + 11 at watch.c:11:14, address = 0x000000010000056b
(lldb) run
Process 82133 stopped
* thread #1, queue = 'com.apple.main-thread', stop reason = breakpoint 1.1
    frame #0: 0x000000010000056b watch`bump(slot=0x0000000100002004, by=1) at watch.c:11:14
   8   	
   9   	int bump(int *slot, int by)
   10  	{
-> 11  	    *slot += by;
    	             ^
   12  	    counter++;
   13  	    return *slot;
   14  	}
Target 0: (watch) stopped.
(lldb) p counter
(int) 0
(lldb) p *slot
(int) 20
(lldb) continue
Process 82133 resuming
Process 82133 stopped
* thread #1, queue = 'com.apple.main-thread', stop reason = breakpoint 1.1
    frame #0: 0x000000010000056b watch`bump(slot=0x00007ff7bfefe7f8, by=2) at watch.c:11:14
   8   	
   9   	int bump(int *slot, int by)
   10  	{
-> 11  	    *slot += by;
    	             ^
   12  	    counter++;
   13  	    return *slot;
   14  	}
Target 0: (watch) stopped.
(lldb) p counter
(int) 1
(lldb) p *slot
(int) 5
(lldb) continue
Process 82133 resuming
Process 82133 stopped
* thread #1, queue = 'com.apple.main-thread', stop reason = breakpoint 1.1
    frame #0: 0x000000010000056b watch`bump(slot=0x00007fba57706380, by=3) at watch.c:11:14
   8   	
   9   	int bump(int *slot, int by)
   10  	{
-> 11  	    *slot += by;
    	             ^
   12  	    counter++;
   13  	    return *slot;
   14  	}
Target 0: (watch) stopped.
(lldb) p counter
(int) 2
(lldb) p *slot
(int) 7
(lldb) continue
table[1]=21 local=7 *heap=10 counter=3
Process 82133 resuming
Process 82133 exited with status = 0 (0x00000000)

Real output — GNU gdb 15.1 on the GCC 13.3 build, Docker's ubuntu:24.04 on the same Mac; the container cannot switch address-space randomization off, which is the warning

Breakpoint 1 at 0x11b8: file watch.c, line 11.
warning: Error disabling address space randomization: Operation not permitted
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".

Breakpoint 1, bump (slot=0x557fb976d014 <table+4>, by=1) at watch.c:11
11	    *slot += by;
$1 = 0
$2 = 20

Breakpoint 1, bump (slot=0x7ffd978a274c, by=2) at watch.c:11
11	    *slot += by;
$3 = 1
$4 = 5

Breakpoint 1, bump (slot=0x557fe5e522a0, by=3) at watch.c:11
11	    *slot += by;
$5 = 2
$6 = 7
table[1]=21 local=7 *heap=10 counter=3
[Inferior 1 (process 23) exited normally]

The same three stops and the same six values, in two syntaxes: (int) 0 against $1 = 0, frame #0: … bump(slot=…, by=1) against Breakpoint 1, bump (slot=…, by=1). Everything a debugger front end shows is a rewording of lines like these, and Ghidra's job starts where they end. Its lldb launcher is a shell script, local-lldb.sh ↗, and the lines that matter are these:

From Ghidra 12.1.3's local-lldb.sh, as installed by Homebrew

#@title lldb
#@env OPT_LLDB_PATH:file="lldb" "lldb command" "The path to lldb. Omit the full path to resolve using the system PATH."
#@env OPT_START_CMD:StartCmd="process launch" "Run command" "The lldb command to actually run the target."
. ../support/lldbsetuputils.sh
pypathTrace=$(ghidra-module-pypath "Debugger-rmi-trace")
pypathLldb=$(ghidra-module-pypath)
export PYTHONPATH=$pypathLldb:$pypathTrace:$PYTHONPATH
launch-lldb "$@"

It runs the same lldb with a Python connector on its path, and the connector talks back to Ghidra over a socket. The connector needs two Python packages that Apple's lldb does not have — it embeds the Command Line Tools' Python 3.9 — and Ghidra ships the wheels; installing them into that Python's user site is the whole setup on this Mac:

Real output — the Command Line Tools' Python 3.9.6, x86-64 macOS 26, 2026-09-13; the wheel folder is Ghidra/Debug/Debugger-rmi-trace/pypkg/dist inside the install

$ /Library/Developer/CommandLineTools/usr/bin/python3 -m pip install --user --no-index --find-links "$dist" protobuf psutil
Successfully installed protobuf-6.31.0 psutil-5.9.8
$ lldb -b -o 'script import google.protobuf' -o 'script import psutil' -o 'script print("DEPS", google.protobuf.__version__, psutil.__version__)'
(lldb) script import google.protobuf
(lldb) script import psutil
(lldb) script print("DEPS", google.protobuf.__version__, psutil.__version__)
DEPS 6.31.0 5.9.8

What a back end reports, and what the file never had¶

Stop a second time — bump(&local, 2), with slot pointing into main's frame — and ask the back end for the things Ghidra's windows are named after:

Real output — lldb, the second stop; the kill at the end cut

(lldb) bt
* thread #1, queue = 'com.apple.main-thread', stop reason = breakpoint 1.1
  * frame #0: 0x000000010000056b watch`bump(slot=0x00007ff7bfefe7f8, by=2) at watch.c:11:14
    frame #1: 0x00000001000005f1 watch`main at watch.c:25:5
    frame #2: 0x00007ff813c4e1c8 dyld`start + 3240
(lldb) register read rip rsp
     rip = 0x000000010000056b  watch`bump + 11 at watch.c:11:14
     rsp = 0x00007ff7bfefe7e0
(lldb) memory region $rsp
[0x00007ff7bf700000-0x00007ff7bff00000) rw-
Modified memory (dirty) page list provided, 7 entries.
Dirty pages: 0x7ff7bfef9000, 0x7ff7bfefa000, 0x7ff7bfefb000, 0x7ff7bfefc000, 0x7ff7bfefd000, 0x7ff7bfefe000, 0x7ff7bfeff000.
(lldb) memory region slot
[0x00007ff7bf700000-0x00007ff7bff00000) rw-
Modified memory (dirty) page list provided, 7 entries.
Dirty pages: 0x7ff7bfef9000, 0x7ff7bfefa000, 0x7ff7bfefb000, 0x7ff7bfefc000, 0x7ff7bfefd000, 0x7ff7bfefe000, 0x7ff7bfeff000.
(lldb) memory region &counter
[0x0000000100002000-0x0000000100003000) rw- __DATA
Modified memory (dirty) page list provided, 1 entries.
Dirty pages: 0x100002000.
(lldb) image list -b -o watch
[  0] watch 0x0000000000000000

Real output — gdb in the container, the second stop; the kill at the end cut

#0  bump (slot=0x7ffee1e286dc, by=2) at watch.c:11
#1  0x000055afdfc3724d in main () at watch.c:25
rip            0x55afdfc371b8      0x55afdfc371b8 <bump+15>
rsp            0x7ffee1e286c0      0x7ffee1e286c0
From                To                  Syms Read   Shared Object Library
0x00007f5ccf6db000  0x00007f5ccf705195  Yes         /lib64/ld-linux-x86-64.so.2
0x00007f5ccf4e4800  0x00007f5ccf66c039  Yes         /lib/x86_64-linux-gnu/libc.so.6

Three of those answers are about memory the file never had. $rsp and slot are in the same region, [0x7ff7bf700000-0x7ff7bff00000) rw- — eight megabytes of stack, of which the file said nothing — and &counter is in the __DATA page, which the file described and the loader filled with zeroes. gdb shows the same with different numbers: slot at 0x7fff… on the stack this time, 0x5622… on the heap the third time, and the first stop's 0x5621…a014 <table+4> in the data. Ghidra's introduction calls these spaces the program image does not have, and its dynamic listing shows all of them where the static listing shows only the file.

The other answer is the mapping. gdb set the breakpoint at 0x11b8 — the address in the file — and stopped at rip 0x55e5d29711b8 <bump+15>, because Linux loaded the executable at a random base. lldb stopped at 0x10000056b, the same address the file says, because lldb launches with address-space randomization off by default, and image list -o watch reports the load offset as 0x0. A front end that shows your analysis of the file beside a stopped process has to hold this offset for every module, and that is the system of mappings in Ghidra's introduction. Each window in the Debugger tool is one of those answers, kept:

Ghidra window	The back-end answer it shows	Help page at the installed tag
Dynamic Listing	the bytes at `rip`, disassembled, in the process's addresses	DebuggerListingPlugin ↗
Registers	`register read` · `info registers`	DebuggerRegistersPlugin ↗
Stack	`bt`	DebuggerStackPlugin ↗
Regions	`memory region` · `info proc mappings`	DebuggerRegionsPlugin ↗
Modules	`image list` · `info sharedlibrary`, with the load offset that makes the mapping	DebuggerModulesPlugin ↗
Threads	`thread list` · `info threads`	DebuggerThreadsPlugin ↗
Breakpoints	`breakpoint list` · `info breakpoints`, placed in both listings at once	DebuggerBreakpointsPlugin ↗
Time	the stops themselves, one snapshot each — the part no back end keeps	DebuggerTimePlugin ↗

The cursor sync between the two listings — click bump in the static listing and the dynamic one goes to 0x55e5…11b8 — is its own plugin, DynamicStaticSynchronizationPlugin ↗, and it is what puts the previous lesson's decompiler beside the process: the names and types you gave the file are shown against the memory the process actually has.

The trace¶

The last row of that table is the one that has no counterpart in either transcript. lldb printed (int) 0, then (int) 1, then (int) 2, and each line was gone the moment the next one arrived; the transcript above is the only record, and it is text. Ghidra's introduction describes the alternative: the debugger can record that target into a Ghidra Trace database … the user can rewind this recording during or after a session, and the UI will recall those observations, displaying the recorded machine state instead of the present machine state. Every stop is a snapshot; every register and every byte the back end reported at that snapshot is in the database; the Time window is a list of them; and a trace is a project file, so it can be saved, reopened after the process has exited, and — the introduction adds — committed to a Ghidra Server, though not merged.

demo/RecordBump.java is written to show that with numbers rather than a quotation. Run from the Debugger tool's Script Manager with watch open, it launches the program under lldb through Ghidra's own launcher, stops at bump three times, reads counter from the process at each stop, then rewinds to the first snapshot and reads the same four bytes from the recording while the process is still stopped at the third. It also lists the snapshots, the regions, the modules and the static mappings the launcher made, and writes it all to ~/record_bump.txt.

One fence is missing. The script above compiles against Ghidra 12.1.3's jars, but it has not been run: the Debugger tool exists only in Ghidra's GUI, and this page's tooling could not drive the GUI when the page was written. Its output belongs here as a Real output fence. To produce it: start Ghidra, import the watch built by the script above with its watch.dSYM beside it, open it in the Debugger tool, and run RecordBump.java from Window → Script Manager after copying it to ~/ghidra_scripts. The first launch shows the lldb launcher's dialog once; the run command it uses is process launch --stop-at-entry.

Until then, what the trace holds is only what Ghidra's introduction says it holds, and this page has not checked it.

If you are coming from another language¶

Rust. The same two back ends, one layer up: rust-lldb and rust-gdb ship in the toolchain's sysroot and are wrappers that load Rust's pretty-printers — the Rust library's Debugging Rust ↗ says why no page there uses one. Ghidra's launcher runs plain lldb, so it sees what the wrapper would add the printers to: a String as its raw pointer, length and capacity, which is also what the trace records.

Python. The interpreter is its own back end, and a trace is fifteen lines: sys.settrace hands a function every call the interpreter makes, and this one keeps counter at each call of bump, then reads the first snapshot back after the run — the same shape as Ghidra's recording, for one variable instead of all of memory:

Verified output of debugger_trace_py.py — regenerated by tools/run_examples.py, never hand-typed.

table[1]=21 local=7 heap=10 counter=3
snapshot 1: counter was 0 when bump was called with by=1
snapshot 2: counter was 1 when bump was called with by=2
snapshot 3: counter was 2 when bump was called with by=3
rewound to snapshot 1: counter was 0 and is now 3

ABAP. (Not machine-checked — CI cannot run ABAP.) There is no separate back end to drive: the ABAP Debugger is part of the application server and attaches to a session from a second one, so what it shows is the server's own view of the program's variables, and there is no trace to rewind — a stop is a stop, and stepping back means running again.