Free Tool for Efficient XML Data Compression

[Eric Bohlman]

On Sat, 18 Dec 1999, Elliotte Rusty Harold wrote:

> While XMill sounds interesting, I really have to take issue with this 
> statement.  I've seen no evidence that XML based, uncompressed file 
> formats are larger than the corresponding binary file formats. This 
> is a common fear about XML but I have not seen it borne out in my 
> tests. For instance, my 700K, very verbose baseball statistics 
> example is more than two megabytes in both FileMaker 3 and Microsoft 
> Excel.

Slightly playing devil's advocate here, I rather strongly suspect that
what you're seeing is the size advantage of a variable-length-field data
format over a fixed-length-field data format; your database files probably
have 1.3+ megabytes of blank padding.  What does the comparison look like
if you export the files to CSV?

While I do believe there's a lot of FUD running around regarding the size
of XML files, and that shouldn't be the primary selling point of something
like XMill, it does appear that the XMill developers are on to something:
XML allows specifying the structure of data, and XMill appears to be able
to use that structural information to get better compression than would be
available by treating the data as an unstructured blob of text.  It's very
often the case that the average "entropy" of the contents of each "field"
of a set of "records" is much lower than the "entropy" of the set as a
whole, and thus the sum of the gains from compressing each "column"
individually can be greater than the gain from compressing the entire set
as if it were homogeneous.



xml-dev: A list for W3C XML Developers. To post, mailto:xml-dev at ic.ac.uk
Archived as: http://www.lists.ic.ac.uk/hypermail/xml-dev/ and on CD-ROM/ISBN 981-02-3594-1
To unsubscribe, mailto:majordomo at ic.ac.uk the following message;
unsubscribe xml-dev
To subscribe to the digests, mailto:majordomo at ic.ac.uk the following message;
subscribe xml-dev-digest
List coordinator, Henry Rzepa (mailto:rzepa at ic.ac.uk)