Skip to content

01 — Strings carry an encoding

Python has two types for text, str and bytes. Java has one, and inside it is always UTF-16. Ruby has one type, String, and every string carries the name of an encoding: UTF-8 for a literal in a source file, whatever the locale says for a line read from a file, BINARY for a buffer of raw bytes. The characters a method sees are the bytes read through that name.

That lets text stay in the encoding it arrived in. It costs a class of error the other languages do not have — two strings whose names disagree — and a quieter one: a name that does not fit the bytes it labels.

Lesson The one thing
A string is bytes plus a label force_encoding renames; encode converts
Mixing encodings raises — unless it is all ASCII why the error waits for the first non-ASCII byte
Invalid bytes get in, and fail later which methods notice, and scrub
String.new is binary; "" is not the constructors and readers that return BINARY