Web server logs in XML?

[Dave Winer]

Michael:

Here's what I would do with those XML logs, I'd read them into a database
and throw away the tagged text. Even then, the log info in the database
probably won't be as compact as a straight ASCII text file, but I'll be
able to write scripts that get me more info from them. 

There's also a certain amount of hunting for the killer app going on. I
don't think anyone really knows what it's going to be or if for sure
there's going to be one. For example, a lot of people think it's going to
be a search engine, or that XML is going to replace HTML, but I see no
evidence of that, and neither of these make sense to me.

Anyway, the main reason to store log files in XML is so that XML-aware apps
can read them and process them. Maybe there's no gold in this here hill,
but why not give it a try and see? It's at most two days worth of work. Not
a big deal.

And those big hard disks are coming down in price, thankfully!

Dave


At 11:21 PM 9/30/98 -0700, Michael Ax wrote:
>Hello.
>
>Please pardon my chiming in... I try, but I'm not quite up to speed with
>everything that's going on here and I have the nagging suspicion that I am
>missing something big. 
>
>Could someone please explain to me the difference why one would want to,
>oh, lets be conservative, quadrupple the amount of data to be processed by
>logging XML-enties rather than using XML to describe the structure of the
>data?
>
>It makes a lot of sense to me to think about each log-entry as an XML
>entity, but I have a hard time fathoming that anyone would want to go to
>the extreme which I understand is being discussed here.  
>
>I have close to 10gb of web-server logs from the last 2 yrs and while this
>is theoretically appealing I, perhaps short-sightedly, have read everything
>that goes on here with an eye toward using XML as a, well, markup language
>used for universal exchange, not as the be-all & end all of native
>data-storage. 
>
>But perhaps I've ignorantly assumed that there would be something like a
>log/data file and an XML description of the data which I could use as
>inputs to filters that would have the ability to let me seek through and
>utilize such data and which might perhaps have the smarts to
>translate/copy/move contents from one meta-data-structure to another. ..
>sort-of, kind-of like an object-oriented file-system might.
>
>e.g. what's the benefit of this crazy seeming thread-subject, and why don't
>I get it? 
>
>So, can somebody please elaborate or point me to more info about how XML
>and Meta-Data are supposed to co-exist these days or to whatever happened
>to the great database debate? Did someone kill the DBAs? Who's keeping score?
>
>I have no question that XML will grow to rule the world, but I'm trembling
>at the thought that somebody might take all of the science discussed here a
>bit to literally and force me to get another 5 or 6 10gb drives and make me
>keep my logs in raw xml. 
>
>Many thanks,
>
>Best Regards,
>Michael Ax
>
>p.s. the log file business if doubly scary in light of my experiences that
>3NF typically nets a 16:1 compression. Am I misreading something? Is XML
>trying to describe a storage or a representation/exchange format? are there
>proposals for middle-layers/ translation filters? os-plug-ins?
>
>
>At 10:11 PM 9/30/98 , Don Park wrote:
>>You are right about the first draft of the XLF spec being not much of a spec
>>but a list of ideas and promises (you were always good at cutting through
>>the bull#$!@).  If you want an example, you will have to make one up
>>yourself because there isn't one except for the fragments found in various
>>XLF documents, proposals, and the archived messages.
>..
>
>xml-dev: A list for W3C XML Developers. To post, mailto:xml-dev at ic.ac.uk
>Archived as: http://www.lists.ic.ac.uk/hypermail/xml-dev/
>To (un)subscribe, mailto:majordomo at ic.ac.uk the following message;
>(un)subscribe xml-dev
>To subscribe to the digests, mailto:majordomo at ic.ac.uk the following message;
>subscribe xml-dev-digest
>List coordinator, Henry Rzepa (mailto:rzepa at ic.ac.uk)
>
>

--------------------------------------
http://www.userland.com/directory.html


xml-dev: A list for W3C XML Developers. To post, mailto:xml-dev at ic.ac.uk
Archived as: http://www.lists.ic.ac.uk/hypermail/xml-dev/
To (un)subscribe, mailto:majordomo at ic.ac.uk the following message;
(un)subscribe xml-dev
To subscribe to the digests, mailto:majordomo at ic.ac.uk the following message;
subscribe xml-dev-digest
List coordinator, Henry Rzepa (mailto:rzepa at ic.ac.uk)