ANN: XML and Databases article

[Ingo Macherius]

Ken North <ken_north at compuserve.com> wrote at 9 Sep 99, 13:17:

> Ingo Macherius wrote
> > The other problem is you can store XML, but
> > have to explicitely model relations (parent, sibling, child, ... and
> > all other axis you need) by foreign keys.

> This reads like you always decomposed or mapped XML documents to store them,
> as opposed to storing some XML documents as a single column. The choice of
> how to store documents in a database will vary on a case-by-case basis,
> depending on application needs.

First, this is not a plea for help on a customers problem. We are a 
research institute, so all problems are homemade and imginary. They 
may, however, be applicable in the real world.

And yes, we need a fully decomposed XML view. On top of the database 
an XQL processor is running, which is implememented in terms of the 
W3C-DOM methods. I consider XQL (and XPath and all the other 
variations of XML path-expression QLs) basically as declarative 
notation for DOM traversals.

If there was a fixed schema and XML was an exchange format only I 
agree with you. The best thing to do is to define fixed mappings, 
maybe parse some values from the XML into tables, and store the whole 
XML as a BLOB or VARCHAR. But this can not be done with arbitrary, 
well-formed XML when you want to query them.

The main advantage of storing XML along the "natural" DOM structure 
is the fact you don't need mappings anywhere. Defining a mapping from 
XML to a DB schema (e.g. in Oracle8i) always requires administrator 
interaction, skill, and time. With a DOM-DB solution you just drop in 
the XML and you are done. Excelon comes very close to this.

Of course it depends on what you wanna do with the data. For mostly 
fixed applications it is reasonable to define a fixed mapping once. 
But in my case we need to deal with wrapper outputs, so there is no 
such thing as a fixed schema, there are hundreds :)
 
> > The number of "metatables" with structural information quickly outnumbers
> the useful data.
> 
> That's an accepted behavior in the OLAP world -- even more so now that disk
> and memory are inexpensive.

What hurts is not disk space, but the number of foreign keys and 
joins to be performed while evaluating a query. The system gets slow, 
and a simple "checkout" (in XQL the "/" query) may take very long. My 
knowledge of OLAP is limited, but can this technique(s) model

- arbitrary access of structural node-neighbours (sibling, parent, 
child, namespace, ...)
- transparent storage of query-processor internal metadata, e.g. as 
used for structural indices, along with each node

with reasonable access times ? With the PDOM, we can pump through 3.3 
MB of XML (or about 160.000 DOM objects) per second without any 
caching. The number may be 10 times better if the cache is doing 
fine, but this depends on access patterns.

	++im

--
Ingo Macherius//Dolivostrasse 15//D-64293 Darmstadt//+49-6151-869-882
GMD-IPSI German National Research Center for Information Technology
mailto:macherius at gmd.de http://www.darmstadt.gmd.de/~inim/
Information!=Knowledge!=Wisdom!=Truth!=Beauty!=Love!=Music==BEST (Zappa)

xml-dev: A list for W3C XML Developers. To post, mailto:xml-dev at ic.ac.uk
Archived as: http://www.lists.ic.ac.uk/hypermail/xml-dev/ and on CD-ROM/ISBN 981-02-3594-1
To (un)subscribe, mailto:majordomo at ic.ac.uk the following message;
(un)subscribe xml-dev
To subscribe to the digests, mailto:majordomo at ic.ac.uk the following message;
subscribe xml-dev-digest
List coordinator, Henry Rzepa (mailto:rzepa at ic.ac.uk)