A processing instruction for robots

[tbray at textuality.com]

From: Walter Underwood <wunder at infoseek.com>
> Adding the robots info to every DTD in the world requires
> unanimous agreement. Adding a PI requires non-interference
> with other PIs, a vastly simpler task. Waiting for XML
> to support mixin vocabularies and for those to be widely
> used, could take a few years.

Walter is right on both counts, but I'm having trouble getting comfortable
with his PI idea.  Not violently against it, but two things make me
uncomfortable.  First of all, PIs basically suck.  Having said that, if you
gotta use them, this is the kind of thing to use them for.

But my big problem is with the idea that individual resources ought to embed
robot-steering information. It just feels like the wrong level of
granularity.  Either this ought to be done externally in something like
robots.txt but smarter, at the webmaster/administrator level, or, with a
namespaced vocabulary at the individual element level.  Note that the
external file and the embedded element-level stuff could have the same
namespaced vocabulary.  The PI has the characteristic that it *has* to be in
the document and can modify *only* the whole document.  Also I question the
ability of authors to do the right thing with this kind of a macro-level
control.  Also I question the ability of robot authors to do the right thing
at the individual document level.

In any case, there really should be a namespace with a bunch of predeclared
attributes for this purpose; then for those who want to do fancy things,
they can do so in a clean way at the individual element level.  For those
who *don't* want to wire robot stuff into their document structure, but *do*
want individual resource-level control and *don't* want to do it in a
centralized way, I guess the PI is a tolerable kludge; but it doesn't seem
like much more than that.

Anyhow, is there enough XML on the web to make this interesting?  Serious
question, I don't know the answer. -T.


xml-dev: A list for W3C XML Developers. To post, mailto:xml-dev at ic.ac.uk
Archived as: http://www.lists.ic.ac.uk/hypermail/xml-dev/ and on CD-ROM/ISBN 981-02-3594-1
To unsubscribe, mailto:majordomo at ic.ac.uk the following message;
unsubscribe xml-dev
To subscribe to the digests, mailto:majordomo at ic.ac.uk the following message;
subscribe xml-dev-digest
List coordinator, Henry Rzepa (mailto:rzepa at ic.ac.uk)