XML input¶

Level: 201 · for anyone who has exported a program

One line: Ghidra's XML export is a PROGRAM element with the processor, byte order and address width as attributes, a memory map whose sections point at a sidecar .bytes file by offset, and addresses as hex strings — a text container for binary facts, in which a length is a decimal string, an address a bare hex string, and a byte of the program is never written at all.

What the loader reads¶

File > Export Program with the XML format writes two files: name.xml, the program's facts, and name.xml.bytes, its memory — or, as the export dialog names it, a .bytes file beside the XML. The XML's grammar is PROGRAM.DTD ↗, and it begins:

<?xml version="1.0" standalone="yes"?>
<?program_dtd version="1"?>
<PROGRAM NAME="hello.bin" EXE_PATH="/tmp/hello.bin" EXE_FORMAT="Raw Binary" IMAGE_BASE="08000000">
    <PROCESSOR NAME="x86" ENDIAN="little" ADDRESS_MODEL="64-bit" LANGUAGE_PROVIDER="x86:LE:64:default:gcc" />
    <MEMORY_MAP>
        <MEMORY_SECTION NAME="ram" START_ADDR="08000000" LENGTH="12" PERMISSIONS="rwx">
            <MEMORY_CONTENTS FILE_NAME="hello.bin.bytes" FILE_OFFSET="0" />
        </MEMORY_SECTION>
    </MEMORY_MAP>

then SYMBOL_TABLE, CODE, DATA, COMMENTS, FUNCTIONS and the rest of the DTD's twenty children, each optional.

In Python¶

Verified output of xml_input_py.py — regenerated by tools/run_examples.py, never hand-typed.

1. THE PROGRAM ELEMENT SAYS WHAT A HEADER WOULD
------------------------------------------------------------------------
   attribute          value
   NAME               hello.bin
   EXE_PATH           /tmp/hello.bin
   EXE_FORMAT         Raw Binary
   IMAGE_BASE         08000000
   PROCESSOR/NAME     x86
   PROCESSOR/ENDIAN   little
   PROCESSOR/ADDRESS_MODEL 64-bit
   PROCESSOR/LANGUAGE_PROVIDER x86:LE:64:default:gcc

   ENDIAN and ADDRESS_MODEL are the two facts every binary header in
   this chapter encodes somewhere; here they are attribute strings,
   and the loader picks a language from them -- or from
   LANGUAGE_PROVIDER, which names it outright, if that is present.

2. THE BYTES ARE NOT IN THE XML
------------------------------------------------------------------------
   MEMORY_SECTION 'ram'   START_ADDR '08000000' -> 0x8000000   LENGTH '12' -> 12
   MEMORY_CONTENTS FILE_NAME 'hello.bin.bytes'   FILE_OFFSET 0
   the bytes, from the sidecar   63 61 66 c3 a9 00 55 48 89 e5 5d c3

   An export is two files: the .xml and a .bytes beside it, named in
   FILE_NAME. The XML never holds a byte of the program; it holds
   where in the other file the bytes are, and where in memory they
   go. Import the .xml alone and the memory map is 0xff, by design.

3. TWO NUMBERS, TWO BASES, AND THE TAG SAYS WHICH
------------------------------------------------------------------------
   START_ADDR   '08000000'   an address: hex, no prefix   as hex 134217728    as decimal 8000000
   LENGTH       '12'         a length: decimal            as hex 18           as decimal 12
   FILE_OFFSET  '0'          an offset: decimal           as hex 0            as decimal 0
   IMAGE_BASE   '08000000'   an address: hex              as hex 134217728    as decimal 8000000

   '12' is twelve as a LENGTH and eighteen as an address; '08000000'
   is 134,217,728 as an address and eight million as a length. The
   attribute name is the only thing that says which base to read a
   digit string in. Ghidra's XmlUtilities.parseInt reads decimal, or
   hex after a 0x prefix, so LENGTH="0x10" is sixteen and "10" is ten;
   an address goes through the address factory, which reads hex bare.

4. TEXT INSIDE TEXT IS ESCAPED, AND THE FILE'S ENCODING IS UTF-8 BY DEFAULT
------------------------------------------------------------------------
   in the file   <COMMENT ADDRESS="08000000" TYPE="end-of-line">the string &quot;café&quot; &amp; a NUL</COMMENT>
   parsed        'the string "café" & a NUL'
   symbol name   'café_handler'   63 61 66 c3 a9 5f 68 61 6e 64 6c 65 72

   declaration   <?xml version="1.0" standalone="yes"?>
   no encoding= : XML 1.0 says UTF-8, or UTF-16 with a byte-order mark
   UTF-8                  parses, symbol 'café_handler'
   UTF-16 with BOM        parses, symbol 'café_handler'
   Latin-1, undeclared    ParseError: the bytes are not the encoding the declaration implies

   A quote is &quot; and an ampersand &amp;, so a comment can hold
   any character the encoding can. Without encoding= the parser
   assumes UTF-8, accepts UTF-16 because the BOM says so, and refuses
   a Latin-1 e9 as malformed -- the one case a binary header would
   have let through as a number.

A header, as attributes¶

Section 1: the facts every binary header in this chapter carries somewhere — which end, how wide, where it loads, what it is — are here as strings on two elements. ENDIAN="little" is ELF's byte 5 spelled out; ADDRESS_MODEL="64-bit" is its byte 4, and Ghidra reads the number out of that string with a regular expression, (\d+)-bit. LANGUAGE_PROVIDER names the language outright when it is present, and the loader prefers it; without it the loader searches by processor name, endianness and size, and may find several.

The bytes are somewhere else¶

Section 2 is the format's shape. The XML holds no program bytes — MEMORY_CONTENTS names a file and an offset, and the loader opens that file, seeks, and reads LENGTH bytes into the section. Import the .xml without its .bytes and every section is filled with 0xff, which the source does on purpose so the map still exists. So the two files are one program, related by a filename inside one of them; the DYLD shared cache has the same arrangement with its subcaches, and a DBG with its PDB.

Two bases in one file¶

Section 3 is the page's encodings point. START_ADDR="08000000" and LENGTH="12" are both strings of digits, and they are in different bases: an address is hexadecimal with no prefix, read by Ghidra's address factory, and a length or an offset is decimal, read by XmlUtilities.parseInt — which also accepts a 0x prefix, so LENGTH="0x10" is sixteen. Nothing in the syntax distinguishes them; the attribute name does. 12 is twelve or eighteen and 08000000 is 134,217,728 or eight million, and the reader has to know which element it is standing in. Which base did you mean? is the general question; this is it asked twice on one line.

Text inside text¶

Section 4: a comment that contains a quote or an ampersand is escaped, " and &, so the file can hold any character the encoding can — and the encoding is XML's default, because the declaration names none: UTF-8, or UTF-16 if a byte-order mark says so. ElementTree parses the UTF-8 and the BOM'd UTF-16 and refuses the same text in undeclared Latin-1 as malformed, which is the difference between a text format and a binary one — a PE would have taken e9 as a number and said nothing.

What Ghidra checks¶

XmlLoader ↗ tries to parse the file with ProgramXmlMgr and takes the PROGRAM element's PROCESSOR attributes as its opinion; a file that does not parse is simply not offered. The name it shows is XML Input Format. ProgramXmlMgr ↗ writes the export, with an XmlWriter that names PROGRAM.DTD and emits the two processing instructions above.

If you are coming from Python or ABAP¶

Python. xml.etree.ElementTree.fromstring(data) on the bytes, not on a decoded string, so the parser can honour the declaration and a BOM; int(attr, 16) for an address and int(attr) for a length, and the attribute name is the only thing that tells you which — the program keeps a table. ET.fromstring on a str that begins with <?xml ... encoding="utf-16"?> raises, because a str has no bytes to be in.

ABAP. (Not machine-checked — CI cannot run ABAP.) cl_ixml and the CALL TRANSFORMATION machinery read an XML document's own declaration, and an xstring in is the right input for the same reason as Python's bytes; a string in has already been decoded by somebody. The two-bases lesson is a familiar one — a NUMC field and a hex RAW field print as the same digits — and the rule is the same: the field's type, never its picture, says what base it is in.

Try it¶

Export any program as XML and head -3 name.xml. Then ls -la name.xml.bytes and compare its size with the LENGTH attributes summed.
grep -o 'START_ADDR="[^"]*"' name.xml | head, then the same for LENGTH. Say which base each is in before you read the DTD.
Delete the .bytes file and import the XML. Every byte is ff.
Save the XML as Latin-1 with an accented comment and import it, or ET.fromstring it. Read the error.
Change ENDIAN="little" to big and import. The language list changes; the bytes do not.