Skip to content

Binary And

Level: 201 · for anyone with a hex editor open

One line: X[i] &= Operand keeps each bit the operand has and clears the rest, so And 7F is the seven-bit channel that turns café into cafC), And DF capitalises café correctly and Łódź into the wrong letter, and a mask wider than a byte lands on whichever byte the Endian toggle calls low.

What the dialog does

The manual writes Binary And as X[i] &= Operand (Hex Operations ↗). Each bit of the result is 1 only where both the value and the operand have a 1, so the operand is a mask: a 1 keeps a bit column of the file and a 0 clears it. There are no carries and no comparisons, which is why the chapter's table finds the sign makes no difference — and why the byte order still does, since a mask wider than a byte has to be laid over the value one way or the other.

In Python

Verified output of binary_and_py.py — regenerated by tools/run_examples.py, never hand-typed.

1. AND 7F IS A SEVEN-BIT CHANNEL
------------------------------------------------------------------------
   Clear the top bit of every byte, the way a mail relay built for
   seven-bit ASCII once did:

     before   63 61 66 c3 a9   'café'
     after    63 61 66 43 29   'cafC)'

   C3 A9 lost the bit that marked them as part of a UTF-8 character and
   became C and ). Nothing in the result says a bit was ever there.

2. AND DF UPPERCASES ASCII, AND SOMETIMES MORE
------------------------------------------------------------------------
   A lowercase ASCII letter is its capital plus 0x20, and DF is every
   bit except that one. The same mask, over UTF-8:

     word   And DF                 reads as
     café   43 41 46 c3 89         'CAFÉ'
     Łódź   c5 81 c3 93 44 c5 9a   'ŁÓDŚ'
     €      c2 82 8c               not UTF-8

   The non-ASCII letters it changed, by name:

     U+00E9 LATIN SMALL LETTER E WITH ACUTE -> U+00C9 LATIN CAPITAL LETTER E WITH ACUTE
     U+00F3 LATIN SMALL LETTER O WITH ACUTE -> U+00D3 LATIN CAPITAL LETTER O WITH ACUTE
     U+017A LATIN SMALL LETTER Z WITH ACUTE -> U+015A LATIN CAPITAL LETTER S WITH ACUTE

   é and É are 0x20 apart, and so are ó and Ó, and UTF-8 keeps that
   bit in the continuation byte -- so clearing it capitalised them. ź
   and its capital are not 0x20 apart, and clearing the same bit made a
   different letter. € lost a bit its lead byte needed.

3. A WIDER MASK LANDS WHERE THE ENDIAN TOGGLE PUTS IT
------------------------------------------------------------------------
   And 0x00FF as Unsigned Short keeps the low byte of the value. Which
   byte of the file that is depends on the toggle:

     little  34 12 -> 34 00
     big     34 12 -> 00 12

4. AND FF CHANGES NOTHING, AND AND 00 IS ASSIGN 0
------------------------------------------------------------------------
     And FF leaves all 256 bytes alone                    True
     And 00 writes 00 over all 256                        True
     Signed and Unsigned write the same, operands 0-127   True

   Every other operand is in between: one bit column kept or cleared
   per bit of the mask, and no carries, so the sign never enters into it.

And 7F is a seven-bit channel

Section 1 clears the top bit of every byte, which is what a transport built for seven-bit ASCII did to anything that passed through it. café came out as cafC): C3 lost its top bit and became C, A9 became ), and nothing in the result says a bit was ever there. UTF-7, and the seven-bit transport is what was built to survive this. Set Maximum 7F is the other way to force bytes below 80, and it writes different bytes for 127 of the 256 values.

And DF, and the letters it gets wrong

A lowercase ASCII letter is its capital plus 0x20, and DF is every bit except that one, so And DF uppercases ASCII. Section 2 applies it to UTF-8 and prints the names of the non-ASCII letters that changed:

  • café becomes CAFÉ, correctly. é and É are U+00E9 and U+00C9, 0x20 apart, and UTF-8 keeps that bit in the continuation byte, so A9 becomes 89 and the letter becomes its capital. ó and Ó work the same way.
  • Łódź becomes ŁÓDŚ. ź is U+017A and its capital is U+0179 — one apart, not 0x20 — so clearing the same bit of its continuation byte makes U+015A, LATIN CAPITAL LETTER S WITH ACUTE: a real letter, valid UTF-8, and the wrong word.
  • € stops being UTF-8. Its lead byte E2 lost a bit it needed to announce a three-byte character.

The mask did the same thing to every byte. Whether the result is right depends on where each alphabet happens to put its capitals, which is exactly what case is not a per-character operation says about case in general.

A wider mask lands where the toggle puts it

Section 3 applies And 0x00FF as an Unsigned Short to 34 12. The mask keeps the low byte of the value both times, and the low byte of the value is the first byte of the file under Little Endian and the second under Big: 34 00 against 00 12. The mask is a number, and a number laid over bytes needs an order.

What the manual does not say

  • Whether Binary And is offered for Float and Double, where C has no &.
  • What becomes of a last value too short for the type.

If you are coming from Python or ABAP

Python. bytes(b & 0x7F for b in data) is section 1, and for a mask over the whole range, (int.from_bytes(data, 'big') & mask).to_bytes(len(data), 'big'). For case, reach for the text methods: 'Łódź'.upper() knows the capital of ź is Ź, where no mask can, and bytes.upper() changes only ASCII letters and leaves every other byte alone.

ABAP. (Not machine-checked — CI cannot run ABAP.) BIT-AND is one of four bit operators, and its operands must be byte-like, x or xstring ↗. So an ABAP mask is bytes, not a number — '00FF' is the two bytes in the order you typed them — and section 3's question does not arise: there is no Endian toggle because there is no integer to lay out.

Try it

  1. Select a UTF-8 paragraph with accents in a copy of a file, And 7F as Unsigned Byte, and read it as ASCII. Count how many characters each accented letter became.
  2. And DF over a French or Polish sentence, then compare it with the same sentence properly uppercased. Find the letters the mask got wrong, and look up how far apart each one is from its capital.
  3. On two copies of a file, And 0xFF00 as Unsigned Short under Little Endian, and And 0x00FF under Big Endian. Before comparing them, explain why they should be identical.
  4. In a file of fixed-size records, clear one flag byte in every record with And 0x00, Treat Data As Unsigned Byte, and Skip Bytes set to the rest of the record.

See also