XML tools and big documents (was: Re: Is there a size limitation on XML file given to MSXSL as input?)

[David Megginson]

Ingo Macherius writes:

 > My afterall impression is that most available tools do well with
 > toy examples, but any input being in the MB range easily blasts
 > them. At least that's true for what came from MS so far.

I don't think that that's true in general.  Most of the Java-based XML
parsers I've tried seem to be able to handle Jon Bosak's XML Old
Testament (nearly 4MB) just fine, if somewhat slowly -- I used ot.xml
for routine testing and profiling while developing AElfred, and
AElfred barely kicked up a sweat.

The problem comes if the parser tries to build a tree rather than
simply reporting an event stream.  Depending on the implementation,
document trees tend to be very large.  With a naive tree
implementation, a 10MB document might use 100MB or more of virtual
memory for the document tree -- that'll bring most current desktop
systems to a screeching halt.


All the best,


David

-- 
David Megginson                 david at megginson.com
           http://www.megginson.com/

xml-dev: A list for W3C XML Developers. To post, mailto:xml-dev at ic.ac.uk
Archived as: http://www.lists.ic.ac.uk/hypermail/xml-dev/
To (un)subscribe, mailto:majordomo at ic.ac.uk the following message;
(un)subscribe xml-dev
To subscribe to the digests, mailto:majordomo at ic.ac.uk the following message;
subscribe xml-dev-digest
List coordinator, Henry Rzepa (mailto:rzepa at ic.ac.uk)