Skip to content

What you see is not what runs

Level: 301 · deep dive

One line: Source code is stored as an ordered list of code points and read as a rendering, the two are allowed to differ, and a code reviewer is the only reader in the pipeline who has no access to the first one.

The shape, with a person in it

Every other page in this chapter has two programs disagreeing. This one has a program and a person:

the compiler reads code points, in order, and has no screen
the reviewer reads a rendering, which reorders — and which draws
    two different code points with the same picture

Both are reading the same file. Neither is malfunctioning. Unicode's bidirectional algorithm (UAX #9 ↗) exists because Arabic and Hebrew run right to left inside otherwise left-to-right text, and it does that job correctly — including when the text is a .rs file. The nine explicit formatting characters that steer it are invisible by design, because a mark that drew something would be in the way of the writing systems that need it.

So the attack writes itself: put an override in a comment or a string literal, and the line the reviewer reads is not the line the compiler compiles. Nicholas Boucher and Ross Anderson named it Trojan Source in 2021 and disclosed it to a dozen language teams at once, which is why so many compilers grew the same lint on the same day. It is CVE-2021-42574 ↗ for the bidi half and CVE-2021-42694 for the homoglyph half.

The two halves

Reordering. U+202E RIGHT-TO-LEFT OVERRIDE and its eight relatives change the display order of everything that follows without changing a single code point. A return inside what renders as a comment; a comment marker that renders in the middle of a line it does not begin; a string literal whose closing quote appears to arrive early. The compiler sees none of that, because the compiler has no concept of display order at all.

Homoglyphs. No control characters, no cleverness: аdmin with U+0430 CYRILLIC SMALL LETTER A and admin with ASCII a are two identifiers that are the same picture in every font. Define both, export the harmless one, call the other. The Python example does exactly this and prints which function actually ran.

The same primitive is old craft outside source code. A file named photo_ann + U+202E + gnp.exe renders in a file manager as photo_annexe.png, and the extension a user reads is a lie told by the renderer — malware campaigns have used exactly that for years. And at the level of a single letter it is the homograph domain: аpple.com spelled in Cyrillic, registered and demonstrated with a valid certificate in 2017. That end of the story is confusables and scripts; this page is the code-review end.

In Python

Verified output of trojan_source_py.py — regenerated by tools/run_examples.py, never hand-typed.

1. TWO FUNCTIONS, ONE APPEARANCE
   defined: 'is_admin'
            U+0069 U+0073 U+005F U+0061 U+0064 U+006D U+0069 U+006E
   defined: 'is_аdmin'
            U+0069 U+0073 U+005F U+0430 U+0064 U+006D U+0069 U+006E
   same name?   False
   is_admin('nobody')  -> False
   is_аdmin('nobody')  -> True
   One of those is a backdoor. Both lines above are eight characters
   long and, in every font, identical. A reviewer has nothing to see.
   Note the code points: only ONE character differs, at position four.

2. AND PYTHON SAYS NOTHING
   source contains U+202E?  True
   compile() raised?        no
   result                   2
   No error, no warning, not even under -W error. CPython's tokenizer
   has no opinion about text direction, because text direction is not
   a property of the program -- only of the screen the program is
   being read on. That gap is the whole of Trojan Source.

3. WHAT THIS PROGRAM REFUSES TO PRINT
   U+202A  LRE  LEFT-TO-RIGHT EMBEDDING
   U+202B  RLE  RIGHT-TO-LEFT EMBEDDING
   U+202C  PDF  POP DIRECTIONAL FORMATTING
   U+202D  LRO  LEFT-TO-RIGHT OVERRIDE
   U+202E  RLO  RIGHT-TO-LEFT OVERRIDE
   U+2066  LRI  LEFT-TO-RIGHT ISOLATE
   U+2067  RLI  RIGHT-TO-LEFT ISOLATE
   U+2068  FSI  FIRST STRONG ISOLATE
   U+2069  PDI  POP DIRECTIONAL ISOLATE
   Nine characters, none of them printed above as itself -- because an
   answer key holding a raw override would reorder the page that shows
   it, and this library's answer keys are read by people. Escapes only.

4. THE DETECTOR IS TEN LINES
   the two definitions:
     line 3 col 8: non-ASCII letter CYRILLIC SMALL LETTER A
   the comment:
     line 1 col 10: RIGHT-TO-LEFT OVERRIDE
   That is the whole defence, and it belongs in CI rather than in a
   reviewer's eyes: a rule a person cannot apply is not a control.

5. THE RULE
   Two readers, again -- but this time one of them is a human being.
     the compiler reads code points, in order, and has no screen
     the reviewer reads a rendering, which reorders and which draws
       two different code points with the same picture
   So: forbid the nine controls outright in source, require identifiers
   to be ASCII (or a single script), and render anything from outside
   with escapes rather than glyphs. Rust does the first by default and
   warns on the second; Python does neither, and both are lint rules
   somebody has to switch on.

Section 2 is the finding: CPython compiles a file containing U+202E and says nothing — not an error, not a warning, not under -W error. That is not a CPython defect. Text direction is a property of the screen, not of the program, and a tokenizer has no screen. Nothing in the language can fix this, which is why the fix is a lint and not a language change.

Section 3 is a small piece of discipline worth copying. The program prints the nine controls by name and code point and never as themselves, because an answer key holding a raw override would reorder the page displaying it — and this library's answer keys are read by people. The example source file follows the same rule: every control and every Cyrillic letter in it is written as an escape, so the file that demonstrates the attack cannot perform it.

Section 4 is the whole defence, in ten lines, and its last claim is the important one: this belongs in CI, not in a reviewer's eyes. A rule a person cannot apply is not a control.

In Rust

Verified output of trojan_source_rs.rs — regenerated by tools/run_examples.py, never hand-typed.

1. WHAT AN ESCAPE HOLDS
   let s = "admin\u{202E} user";
   chars 11   bytes 13
   code points  U+0061 U+0064 U+006D U+0069 U+006E U+202E U+0020 U+0075 U+0073 U+0065 U+0072
   The override is the sixth character and it draws nothing at all.
   Written as an escape it is six visible ASCII characters in the
   source, greppable and diffable. Written as itself it is nothing,
   and it reorders the line it sits in.

2. WHY THIS FILE COULD NOT HOLD THE RAW ONE
   rustc denies text-direction-codepoint-in-literal and
   text-direction-codepoint-in-comment BY DEFAULT -- not warn, deny.
   So a Trojan Source commit does not compile here, and the error
   offers the escape above as the fix. The page quotes both.
   Three more lints (confusable-idents, mixed-script-confusables,
   uncommon-codepoints) cover the homoglyph half, and those only
   WARN -- so the confusable identifier still builds.

3. THE SCANNER YOU STILL NEED
   a clean line:
     clean
   a filename from a zip:
     line 1 col 10: RLO  RIGHT-TO-LEFT OVERRIDE
   a display name:
     line 1 col 1: LRI  LEFT-TO-RIGHT ISOLATE
     line 1 col 7: PDI  POP DIRECTIONAL ISOLATE
   The second one renders as "photo_annexe.png" in a file manager
   and is an executable. rustc protected your SOURCE; it has no
   opinion about a string your program reads at run time, and that
   is where these characters actually arrive.

4. THE RULE
   Source: let the compiler deny them, and keep the deny on.
   Data:   scan on the way in, and render on the way out with
           escapes rather than glyphs -- a name, a filename or a
           commit message from outside is not source code, and it
           is not protected by anything that checks source code.

Rust is the language that took this most seriously, and the levels are worth reading off the compiler rather than from memory:

$ rustc -W help
    confusable-idents                     warn   detects visually confusable pairs between identifiers
    mixed-script-confusables              warn   detects Unicode scripts whose mixed script confusables codepoints are solely used
    uncommon-codepoints                   warn   detects uncommon Unicode codepoints in identifiers
    text-direction-codepoint-in-comment   deny   invisible directionality-changing codepoints in comment
    text-direction-codepoint-in-literal   deny   detect special Unicode codepoints that affect the visual representation of text on screen, changing the direction in which text flows

(rustc 1.98.0, macOS, 2026-09-06. Three UTS #39 lints at warn, two bidi lints at deny.)

So a Trojan Source commit does not build. The error is worth seeing in full, because of what the last two lines do:

error: unicode codepoint changing visible direction of text present in comment
 --> demo.rs:3:5
  |
3 |     // � }rehcuov si resu fi{
  |     ^^^-^^^^^^^^^^^^^^^^^^^^^^
  |     |  |
  |     |  '\u{202e}'
  |     this comment contains an invisible unicode text flow control codepoint
  |
  = note: these kind of unicode codepoints change the way text flows on applications that support them, but can cause confusion because they change the order of characters on the screen
  = note: `#[deny(text_direction_codepoint_in_comment)]` on by default
  = help: if their presence wasn't intentional, you can remove them

Two details. The � is not a copying accident: rustc replaces the override with U+FFFD in its own diagnostic, because an error message that reordered itself would be worse than useless. And on the literal version of the same lint, the help does not just say remove it — it offers the escaped spelling as a suggested rewrite, which is the correct fix when you genuinely meant to include one.

The homoglyph half only warns, so it still builds:

warning: found both `аdmin` and `admin` as identifiers, which look alike
  = note: `#[warn(confusable_idents)]` on by default

warning: the usage of Script Group `Cyrillic` in this crate consists solely of mixed script confusables
  = note: the usage includes 'а' (U+0430)
  = note: `#[warn(mixed_script_confusables)]` on by default

(Same run. Note the second one's condition — it fires because Cyrillic is used solely for confusables; a crate with real Cyrillic identifiers in it would not get this warning, which is the difference between a heuristic and a rule.)

Section 3 of the Rust example is the part that outlives the compiler's help. rustc protected your source. It has no opinion whatsoever about a filename read from a zip archive, a display name from a database, or a commit message from a stranger — and that is where these characters actually arrive.

If you are coming from Python or ABAP

Python. Nothing in the language will help you, so the two controls are external and both are cheap. Put a grep in CI for the nine controls — git grep -nP '[\x{202A}-\x{202E}\x{2066}-\x{2069}]' — and turn on your linter's non-ASCII identifier rule (ruff's RUF001/RUF002/RUF003 flag ambiguous characters in strings, docstrings and comments; PLC2401 covers non-ASCII names). Then remember the second half: print() and logging write whatever bytes you hand them to a terminal, so a display name from your database can carry an ESC or an override into your own console. ascii(s) and repr(s) escape everything non-printable and are the right way to log a value you did not write.

ABAP. ABAP source lives in the database and is read through SE38 / ADT rather than as a file, which changes the threat rather than removing it: what renders your code is the editor, and an editor that implements the bidirectional algorithm implements it for source too. There is no rustc-style lint here, so the control has to be a Code Inspector / ATC check or a review of transports, and it should look for the nine code points by number rather than by appearance. The run-time half applies unchanged and is the more likely one in practice: a customer name arriving through an interface can carry an override into a SAPscript form, a spool, an ALV column or an application-log entry, and none of those is a place you want a renderer following instructions. Escape or strip the controls when data crosses into a display, not when it is stored. (Not machine-checked — CI cannot run ABAP.)

Try it

cd 12_Adversarial/trojan_source/examples
python3 trojan_source_py.py
rustc --edition 2024 trojan_source_rs.rs -o /tmp/ts && /tmp/ts

Make rustc refuse something. The override is written as \u202e here on purpose — Python turns the escape into the real character in /tmp, so the thing that demonstrates the attack never exists in a file anybody reviews:

python3 -c 'open("/tmp/demo.rs","w").write("fn main() {\n    // \u202e )(niam\n}\n")'
rustc --edition 2024 /tmp/demo.rs -o /tmp/demo    # error: ... present in comment

Then scan something you did not write. The nine controls are one grep, and the answer is almost always zero — which is what makes a hit worth looking at:

git grep -nP '[\x{202A}-\x{202E}\x{2066}-\x{2069}]' -- '*.py' '*.rs' '*.md'

(-P needs GNU grep; on a Mac, the tools chapter has the story, or use the Python detector in the example.)

And one to reason about. The Rust example refuses to contain a raw control, and the Python example refuses to print one. Both are the same judgement: an artefact that demonstrates this attack should not be able to perform it. Where else does that judgement apply — and what does it say about a test fixture, a security blog post, or a page like this one?

Practice

Two readers, one file. A source line contains five characters of General_Category Cf. Predict what the compiler does with them and what the reviewer sees, and say why both are behaving correctly.

Then: of every reader in the pipeline — compiler, linter, tests, git, reviewer — say which one is uniquely vulnerable and why. Finish with the three-line detection, and say why it belongs in CI rather than in a reviewer's eyes.

Answers

Verified output of trojan_source_kata_py.py — regenerated by tools/run_examples.py, never hand-typed.

THE LINE, AS CODE POINTS
   length 50 characters, 60 bytes
   invisible format characters at index [24, 25, 27, 45, 47]:
       24  U+202E  RIGHT-TO-LEFT OVERRIDE
       25  U+2069  POP DIRECTIONAL ISOLATE
       27  U+2066  LEFT-TO-RIGHT ISOLATE
       45  U+2069  POP DIRECTIONAL ISOLATE
       47  U+2066  LEFT-TO-RIGHT ISOLATE

   Every one of those has General_Category Cf -- FORMAT. They have no
   width, no glyph and no effect on what the parser reads. They change
   only the ORDER a renderer draws the surrounding text in.

THE TWO READERS
   the compiler   reads the ordered list of code points, in order, and
                  ignores Cf characters entirely
   the reviewer   reads a RENDERING, produced by a bidi algorithm that
                  is doing exactly what those characters ask
   Both are correct. They are given different objects.

WHY THE REVIEWER IS THE VULNERABLE ONE
   Every other reader in the pipeline sees the bytes: the compiler, the
   linter, the test suite, git's own hashing. The code reviewer is the
   only participant whose input is a picture -- and a picture is the one
   representation the attacker gets to choose.

THE DETECTION, WHICH IS THREE LINES
   [c for c in text if unicodedata.category(c) == 'Cf']
   -> 5 hits on this line, 0 on ordinary source
   That is a whole-repo grep, it has no false-negative story to worry
   about, and it belongs in CI rather than in a reviewer's eyes. GitHub,
   rustc and gcc all added warnings for exactly this after CVE-2021-42574.

THE GENERAL LESSON
   Any time a human approves something on the strength of a rendering,
   the rendering is part of the trust boundary. Source review, a diff, a
   signed document, a URL in an address bar -- the question is always
   whether the thing the person SAW is the thing that will ACT.

See also