XML parsing memory overhead concerns

[Stephen R. Savitzky]

"Clark C. Evans" <clark.evans at manhattanproject.com> writes:

> On 16 Dec 1999, Stephen R. Savitzky wrote:
> > This is probably similar to what I'm using: the parser's API is basically 
> > a tree traverser:  it has methods like:
> > 
> >   toNextSibling
> >   toFirstChild
> >   toParent
> >   ... and a bunch of methods to examine the current node.
> > 
> > Though it's DOM-like, you never actually have to build the whole tree.
> 
> So, some random access storage is needed with this technique?

Yes -- you need a stack to match start tags and end tags, and you need to
build a parse tree for any element that needs to be processed as a whole.
This parser is part of a system that does only local tree transformations,
so any tokens that don't have to be transformed can just be sent on to the
output, which has the interface of a tree _constructor_ (or SAX application,
if you want to look at it that way).

> Why not have expat build the sub tree you are interested,
> put it in memory.  You can then "pull" from this?

You can, of course, use SAX or some other "push" parser on the input and
either copy events (tokens) or build subtrees as appropriate.  Doesn't work
as well if you have to process an entire document, but could do it in
streaming mode.  Works in the more usual case of a handful of active tags
embedded in a mostly-passive document, because most of the subtrees that
require processing in that case are small.

> I'd like to hear *much* more about this.

Our system is open source, at http://RiSource.org/PIA/

-- 
Stephen R. Savitzky  <steve at rsv.ricoh.com>  <http://rsv.ricoh.com/~steve/>
Platform for Information Applications:      <http://RiSource.org/PIA/>
Chief Software Scientist, Ricoh Silicon Valley, Inc. Calif. Research Center
 voice: 650.496.5710  front desk: 650.496.5700  fax: 650.854.8740 
  home: <steve at theStarport.org> URL: http://theStarport.org/people/steve/

xml-dev: A list for W3C XML Developers. To post, mailto:xml-dev at ic.ac.uk
Archived as: http://www.lists.ic.ac.uk/hypermail/xml-dev/ and on CD-ROM/ISBN 981-02-3594-1
To unsubscribe, mailto:majordomo at ic.ac.uk the following message;
unsubscribe xml-dev
To subscribe to the digests, mailto:majordomo at ic.ac.uk the following message;
subscribe xml-dev-digest
List coordinator, Henry Rzepa (mailto:rzepa at ic.ac.uk)