DataTerminology.comNEW
A Unicode encoding using one or two 16-bit code units, with optional byte order mark to indicate endianness.
DESCRIPTION
UTF-16 encodes the entire Unicode repertoire using 16-bit code units, employing surrogate pairs for code points above U+FFFF. It can be serialized as big-endian or little-endian and often uses a byte order mark (BOM) to signal endianness. UTF-16 is common in Windows APIs and some programming languages’ internal string representations. For web interchange, UTF-8 is generally preferred due to compatibility and efficiency for ASCII-dominant text. Proper handling of surrogates and endianness is essential to avoid data corruption.
ALTERNATIVE DEFINITIONS
Wikipedia
UTF-16 is a character encoding capable of encoding all 1,112,064 valid code points of Unicode.
Stresses full coverage of the Unicode code space.
IETF RFC 2781
UTF-16 is a Unicode encoding form that uses a sequence of one or two 16-bit code units to represent Unicode scalar values.
Formal standard wording highlighting 16-bit code units and variable length.
RELATED TERMS
Unicode Code Point Endianness BOM Surrogate Pair Unicode Transformation Format Character Encoding ASCII
COMPETING TERMS
KNOWLEDGE AREAS
Data Integration & Interoperability Data Architecture Data Storage & Operations Data Management
TARGET AUDIENCE
Data Engineer Software Developer Data Architect Cloud Architect Data Steward
EXTERNAL SOURCES
- Wikipedia - UTF-16
- IETF - UTF-16 standard specification
- Unicode Consortium - Unicode FAQ on UTFs and BOM
TOP QUESTIONS
When should UTF-16 be used?
Use UTF-16 when APIs or platforms require it, or when text commonly contains many non-ASCII characters and internal 16-bit representation simplifies processing.
Category: Opinion
How does UTF-16 differ from UTF-8?
UTF-16 uses 16-bit units and surrogate pairs; UTF-8 uses variable-length bytes. UTF-8 is dominant for web and interoperability.
Category: Misunderstanding
What is a surrogate pair in UTF-16?
It’s two 16-bit code units representing a single code point above U+FFFF, enabling full Unicode coverage.
Category: Example
Why does endianness matter for UTF-16?
Code units are 16-bit, so byte order must be known. A BOM or external metadata resolves endianness.
Category: Requirements
Is a BOM required for UTF-16?
Not strictly, but it’s commonly used to signal endianness in serialized text.
Category: Requirements
What are common UTF-16 pitfalls?
Dropping surrogate halves, misinterpreting endianness, or assuming fixed-length code units cause data corruption.
Category: Mistakes
Is UTF-16 efficient for ASCII-heavy text?
No. UTF-8 is typically more compact for ASCII-dominant content.
Category: Opinion
How do databases handle UTF-16?
Many store as UTF-8 internally but support UTF-16 IO. Check collation and encoding settings consistently.
Category: Management
How to convert between UTF-16 and UTF-8 safely?
Use standard libraries that validate surrogate pairs and handle BOMs, avoiding manual byte manipulation.
Category: HowTo