Character encodings

[Chris Maden]

[Chris von See]
> I am relatively new to XML and am trying to develop a program that
> can generate XML in various encodings.  In section 4.3.3, the XML
> spec implies that support of ISO 10646 UCS-2 encoding (i.e. Unicode)
> is valid, but in the section on autodetection of encodings (Appendix
> F) there's no mention of how to detect UCS-2 encoding.  I would
> *assume* that UCS-2 would start with \x00 \x3c\ x00 \x3f ("<?") - is
> that right?  If so, then is the spec wrong in not including this in
> Appendix F as valid?  Is it reasonable to expect that many people
> will use UCS-2 because of its similarity to Unicode?

A UCS-2 file starts with the byte order mark (BOM), \xFEFF.  The
ordering of the two bytes shows whether the document is big-endian or
little-endian.

>From 4.3.3:

   Entities encoded in UTF-16 must begin with the Byte Order Mark
   described by ISO/IEC 10646 Annex E and Unicode Appendix B (the ZERO 
   WIDTH NO-BREAK SPACE character, #xFEFF). This is an encoding
   signature, not part of either the markup or the character data of the
   XML document. XML processors must be able to use this character to
   differentiate between UTF-8 and UTF-16 encoded documents. 

What's unsaid is that UCS-2 and UTF-16 are the same.

-Chris
-- 
<!NOTATION SGML.Geek PUBLIC "-//Anonymous//NOTATION SGML Geek//EN">
<!ENTITY crism PUBLIC "-//O'Reilly//NONSGML Christopher R. Maden//EN"
"<URL>http://www.oreilly.com/people/staff/crism/ <TEL>+1.617.499.7487
<USMAIL>90 Sherman Street, Cambridge, MA 02140 USA" NDATA SGML.Geek>

xml-dev: A list for W3C XML Developers. To post, mailto:xml-dev at ic.ac.uk
Archived as: http://www.lists.ic.ac.uk/hypermail/xml-dev/
To (un)subscribe, mailto:majordomo at ic.ac.uk the following message;
(un)subscribe xml-dev
To subscribe to the digests, mailto:majordomo at ic.ac.uk the following message;
subscribe xml-dev-digest
List coordinator, Henry Rzepa (mailto:rzepa at ic.ac.uk)