Character encoding – Unicode, ASCII and UTF-8
Computer ScienceData & BinaryAges 15–16
Loading…
Sign in to playType a line of text to see that every character has a Unicode code point, which UTF-8 stores as 1 to 4 bytes. Tap a character to see how the bits of its code point are split across the bytes, and why é, € and emoji need 2, 3 and 4 bytes. The Decode workspace turns a sequence of hex or binary bytes back into text and marks invalid bytes; the Mojibake workspace shows text breaking apart when UTF-8 bytes are read with an old one-byte encoding.
Lesson: Text data: ASCII, Unicode, UTF-8 encoding, characters versus bytes
What it shows
Computers store text as numbers. Unicode gives every character a code point, written U+ followed by hexadecimal digits, while ASCII covers only 128 characters with 7 bits. UTF-8 stores each code point in 1 to 4 bytes: the first byte begins with 0, 110, 1110 or 11110 to announce the length, and every following byte begins with 10. ASCII text is therefore unchanged in UTF-8, but accented letters, symbols and emoji need more bytes than characters. Reading UTF-8 bytes with a one-byte encoding produces garbled text called mojibake.
How to use
Choose a Sample or type in the Text box, then tap a character card to see its bits split into bytes and compare the character and byte counts. In Decode bytes, press Take from text or type hex bytes, then press Add bad bytes to see errors. In Mojibake, switch between Windows-1252 and Latin-1 and compare the output.
Parameters you can change
- Workspace Encode text into bytes, Decode bytes into text, Garbled text (mojibake)
- Sample text Characters of 1, 2, 3, 4 bytes, ASCII only, Accented Latin letters, Symbols and currency, Emoji, Several writing systems
- Show bytes in binary
- Wrong encoding used for reading Windows-1252, Latin-1 (ISO-8859-1)
Questions to explore
- Why does é need 2 bytes in UTF-8 while e needs only 1 byte?
- How many bytes does a line of five emoji take in UTF-8?
- Why does the word café sometimes appear as café on a web page?
