Frontier as a scalable XML repository (was Re: Is XML dead already or what?)

[David Megginson]

Dave Winer writes:

[to me, on scalable XML repositories]

> We already have this. I hate it when people say things like "No one
> has a good, scalable solution for this now." We do. Are you supposed
> to be an expert on this stuff or do you just like to hear yourself
> talk?

[and to Paul, on size]

> 50MB databases are very common. It scales up to 500MB, but few of
> our customers have databases that large. I wouldn't promote, at this
> time, our odb being used for a multi-gigabyte application in a
> single file, but if it can be separated into multiple files, I don't
> see a problem.

First, I don't claim to be an expert in RDBMS or in document
repositories, and I do like to hear myself talk, so I'll take Dave's
first comment in a friendly spirit.

I said that there is not a good, scalable XML repository available
right now.  There are at least three attempts built on top of OODB's,
including Frontier -- I won't name the other two, but they have
consistently failed to scale up for my customers when they've tried
them, so they're probably suitable only up to the workgroup level
right now (i.e. 20-50 authors working on the same text).  The problem,
as every one who has tried has reported to me, is that the underlying
OODB technology does not scale up to higher levels, though perhaps
it's only a matter of time until the technology catches up.

I have not heard from anyone who has tested Frontier yet, but until
now I have tended to think of it as a web-content management tool
rather than a general purpose document repository -- I'll take a close
look at the demo release when I have a chance.

In the mean time, perhaps Dave can help us by answering a few
questions (since even once I've read the Frontier docs, most of the
rest of the list will not have).  Remember that I was suggesting that
there was not yet a scalable XML repository, not that there were no
repositories that could manage XML.


1. Frontier as a general Document Repository
--------------------------------------------

In my opinion, there are six minimum criteria that a product must meet
to call itself a document repository.  Note that these must be
available out of the box when I install the software -- if the
software merely gives me the ability to write the stuff myself, it
doesn't count (I can do all of this with Perl and RCS if I want to
write it myself).

Which of the following do I have immediately when I install Frontier
on my computer?

1. version control (perferably by storing deltas rather than complete
   objects)

2. access control (check-in/check-out, object locking, etc.)

3. hierarchical object management (virtual folders)

4. virtual compound documents

5. where-used and what-uses tracking

6. efficient searching and querying

In addition to these, workflow is highly desirable though not
absolutely required.  Others may want to add criteria that I missed.


2. Frontier as an XML document repository
-----------------------------------------

In the original message, we were discussing a repository that could
manage XML at the element level.  Here are the what I consider the
three additional requirements for Frontier to be considered an XML
repository -- again, these must be available out of the box to count.

(I don't know if these are necessarily good things, but they're what
SGML repositories do.)

Which of the following do I have immediately when I install Frontier
on my computer?

1. manages documents at the element level -- elements can be reused in 
   multiple documents, and branches of an XML document can be checked
   out and locked independently

2. allows efficient context-sensitive searching and querying, at least
   to the extent necessary to support the XPointer queries; a lot of
   indexing will be necessary here

3. provides full Unicode support, both for storage and for collating

In addition to these, a DOM interface is highly desirable though not
absolutely required (and I take it as a given that non-XML objects can
still be managed).  Others may want to add criteria that I missed.


3. Frontier as a scalable XML document repository
-------------------------------------------------

It's very hard to write a scalable document repository, whether or not
XML is involved.  Here are what I consider the minimum requirements
for scalability.

Which of the following do I have immediately when I install Frontier
on my computer?

1. no arbitrary limits on object or database size

2. passes the ACID test (the database is always in a consistent state
   and transactions can be rolled back, even with other users updating 
   the repository concurrently)

3. a single repository may be distributed across multiple servers

4. the repository can incrementally back itself up on another system and
   ensure that the backup is also in a consistent state, all without
   going offline

5. the underlying database can deal with *at least* several hundred
   concurrent requests without noticable delay

6. the query component provides query optimisation for the sake of
   efficiency, and complex queries do not cause delays for other users

The big RDBMS vendors have dealt with these problems by throwing
billions of dollars at them, and they still struggle sometimes.
Others may want to add criteria that I missed.

Thanks again to Dave for pointing out that Frontier can be thought of
as a document repository as well as a web-content management tool.  I
will look forward to learning more about Frontier from his reply.


All the best,


David

-- 
David Megginson                 david at megginson.com
           http://www.megginson.com/

xml-dev: A list for W3C XML Developers. To post, mailto:xml-dev at ic.ac.uk
Archived as: http://www.lists.ic.ac.uk/hypermail/xml-dev/
To (un)subscribe, mailto:majordomo at ic.ac.uk the following message;
(un)subscribe xml-dev
To subscribe to the digests, mailto:majordomo at ic.ac.uk the following message;
subscribe xml-dev-digest
List coordinator, Henry Rzepa (mailto:rzepa at ic.ac.uk)