Character encodings

[Rick Jelliffe]

 From: Chris Maden <crism at oreilly.com>



>[Chris von See]

>A UCS-2 file starts with the byte order mark (BOM), \xFEFF.  The
>ordering of the two bytes shows whether the document is big-endian or
>little-endian.
>
>From 4.3.3:
>
>   Entities encoded in UTF-16 must begin with the Byte Order Mark
>   described by ISO/IEC 10646 Annex E and Unicode Appendix B (the ZERO
>   WIDTH NO-BREAK SPACE character, #xFEFF). This is an encoding
>   signature, not part of either the markup or the character data of the
>   XML document. XML processors must be able to use this character to
>   differentiate between UTF-8 and UTF-16 encoded documents.
>
>What's unsaid is that UCS-2 and UTF-16 are the same.

After the encoding signature (i.e., 8 bit|16 bit|32 bit and endianness)
has been detected
(e.g., from the BOM) and the code family (e.g., ASCII code-superset or
EBCDIC code-
superset) then the XML encoding header can be read. This can contain
usual header mechanism; the encoding attribute allows any IANA-registered
character
encoding name; you can explicitly specify
    "UTF-7",
    "UTF-8",
    "UTF-16"  (don't use "UNICODE-1-1" ),
    "ISO-10646-UCS-2", or
    "ISO-10646-UCS-4".

Actually, I cannot find UTF-16 in the IANA list at
    ftp://ftp.isi.edu/in-notes/iana/assignments/character-sets
I don't know which crack it has slipped down.

But don't get caught up too much in the XML header. For web service, the
MIME headers are the authority. The XML headers are there to make
sure that the MIME headers can get their information from somewhere,
and for systems that are using non-HTTP file access. (In otherwords,
XML is a format developed for use with HTTP but it does not require
HTTP.)

Rick Jelliffe



xml-dev: A list for W3C XML Developers. To post, mailto:xml-dev at ic.ac.uk
Archived as: http://www.lists.ic.ac.uk/hypermail/xml-dev/
To (un)subscribe, mailto:majordomo at ic.ac.uk the following message;
(un)subscribe xml-dev
To subscribe to the digests, mailto:majordomo at ic.ac.uk the following message;
subscribe xml-dev-digest
List coordinator, Henry Rzepa (mailto:rzepa at ic.ac.uk)