XML parsing memory overhead concerns

[Stephen R. Savitzky]

Paul Miller <stele at fxtech.com> writes:

> As I've written before I've been working on a callback-based streaming
> XML parser that is sort of DOM-like, specifically for reading
> application data from XML files where you know what the object hierarchy
> is.

This is probably similar to what I'm using: the parser's API is basically 
a tree traverser:  it has methods like:

  toNextSibling
  toFirstChild
  toParent
  ... and a bunch of methods to examine the current node.

Though it's DOM-like, you never actually have to build the whole tree.

It does have the feature (some might call it a bug) that you're processing
large chunks of the document before you know that it doesn't validate.
Since I deal with human-written documents I consider it a feature, since it
permits graceful error recovery.

> What do people think? Spare the memory and provide a simpler (and
> slightly less capable) solution or store the entire thing in memory and
> use the nice stuff in expat and give more features?

I believe there's a low-level module in expat that can handle the lexical
part of the parsing but still be called in pull mode.  Or you could use
"full" expat if you have threads (i.e. run expat as a coroutine).

But like you, I'd really like to see a traversal-oriented (pull) parser API
to go along with the current event-oriented (push) ones.  There are times
when it's just the right thing to do.

-- 
Stephen R. Savitzky  <steve at rsv.ricoh.com>  <http://rsv.ricoh.com/~steve/>
Platform for Information Applications:      <http://RiSource.org/PIA/>
Chief Software Scientist, Ricoh Silicon Valley, Inc. Calif. Research Center
 voice: 650.496.5710  front desk: 650.496.5700  fax: 650.854.8740 
  home: <steve at theStarport.org> URL: http://theStarport.org/people/steve/

xml-dev: A list for W3C XML Developers. To post, mailto:xml-dev at ic.ac.uk
Archived as: http://www.lists.ic.ac.uk/hypermail/xml-dev/ and on CD-ROM/ISBN 981-02-3594-1
To unsubscribe, mailto:majordomo at ic.ac.uk the following message;
unsubscribe xml-dev
To subscribe to the digests, mailto:majordomo at ic.ac.uk the following message;
subscribe xml-dev-digest
List coordinator, Henry Rzepa (mailto:rzepa at ic.ac.uk)