Unicode
also: UTF-8, UTF-16, character encoding
Unicode is an international standard for encoding and representing text characters from all world languages and symbols. It assigns a unique number (codepoint) to each character, enabling consistent text handling across different systems and languages.
Unicode solves the problem of representing text in multiple languages and scripts using a single, universal standard. Instead of relying on language-specific encodings like ASCII or ISO-8859-1, Unicode defines over 140,000 characters with assigned numerical values (codepoints), ranging from U+0000 to U+10FFFF.
UTF-8 and UTF-16 are common encoding schemes that convert Unicode codepoints into bytes for storage and transmission. UTF-8 is particularly popular in Linux because it's backward-compatible with ASCII and uses variable-length encoding, making it efficient for English text while still supporting all Unicode characters.
Example: The character 'é' has Unicode codepoint U+00E9. In a UTF-8 encoded file, it's stored as the bytes c3 a9, while in ISO-8859-1 it would be a single byte. Modern Linux systems default to UTF-8 locale settings (checked with locale) to properly handle multilingual text in files, terminals, and applications.