09_History — how text got this complicated¶
Level: 201 · for anyone starting from zero
Nobody designed the mess. Every strange thing about text is a fossil: a sensible decision, made under a real constraint, that outlived the constraint. This chapter is the four pages that explain the shape of everything in chapters 1–7 — where the rules came from, why the one that won won, why the one that lost is still in the software you use today, and what the newest constraint is already doing to text.
| # | Lesson | The question it answers | Status |
|---|---|---|---|
| 1 | From the telegraph to Unicode | Six eras, six constraints — and which scar each one left in code you write today | written |
| 2 | Why UTF-8 won | Six properties, one bill, and why a placemat design beat two committees | written |
| 3 | Why UTF-16 stayed | Why the encoding that lost is still inside Java, JavaScript, Windows and every SAP system | written |
| 4 | A token is not a character | The newest era, and its new ruler: why a model cannot count the letters, and why Polish costs more | written, 2026-09-08 |
The through-line¶
Every era solved the problem it actually had. Five bits were expensive, so text got a shift key. Eight bits were what a byte held, so 128 slots got claimed 30 different ways. Sixteen bits looked infinite in 1991, so three platforms built their strings out of them and are still stuck. And a fixed vocabulary is expensive now, so text gets chopped into whatever pieces were commonest in somebody's training corpus — a constraint being lived through rather than looked back on, and the first one in this chapter whose rule is set by counting rather than by a committee. None of it was stupid, and knowing which constraint produced which rule is the difference between memorising the traps and predicting them.
When to read it¶
Any time after chapter 2, and it works well as the thing you read when chapter 3 starts to feel like arbitrary rules. It is not on the critical path to any of the four checkpoints — but it is the page people remember, and it makes the next chapter's advice feel like conclusions rather than commandments.