What is a good database for very large collections? (was ...)

[Simon St.Laurent]

(I'm responding in part to reduce the length of our crazy subject header)

At 04:18 AM 2/2/99 +1100, Rick Jelliffe wrote:
>Can I try to shift it back to a vital question asked earlier, but not
>answered?
>
>What is a good database for XML?
>
>The criteria are:
>    * over 20, 000, 000 document fragments, each less than 256
>characters, each with some flat metadata, able to be incrementally
>reloaded onto the live system
>    * about simultaneous 30 users accessing about 10 fragments a minute
>each, grouped together (along with other dynamic data) and transformed,
>with a high need for immediate response
>    * constant data-mining tools using various adhoc AI and linguitic
>retrieval software augmenting the metadata in the background.

Wow!  That's quite a set of criteria, and looks almost nothing at all like
my criteria, which are more like:

* over 20,000 document fragments, ranging in length from 1 to 100,000
characters, all with some metadata, which will remain on the system in
mostly stable form.
* about 5 simultaneous authors, up to maybe a thousand people reading the
information.
* indexing and searching moving around in the background.

Given these wildly different criteria (and I'm sure others out there have
different ideas as well), the concept of a database for XML seems pretty
weird.  Maybe we should focus on tools for getting information into and out
of a repository, and let vendors create different back ends created to
match our widely differing needs.  That way we can still share tools, and
read each other's material, but aren't locked into a particular vendor
whose approach won't work for everyone.


Simon St.Laurent
XML: A Primer / Building XML Applications (March)
Sharing Bandwidth / Cookies
http://www.simonstl.com

xml-dev: A list for W3C XML Developers. To post, mailto:xml-dev at ic.ac.uk
Archived as: http://www.lists.ic.ac.uk/hypermail/xml-dev/
To (un)subscribe, mailto:majordomo at ic.ac.uk the following message;
(un)subscribe xml-dev
To subscribe to the digests, mailto:majordomo at ic.ac.uk the following message;
subscribe xml-dev-digest
List coordinator, Henry Rzepa (mailto:rzepa at ic.ac.uk)