SAX: ignorable whitespace question

[Eric Prud'hommeaux]

On Sun, 2 Aug 1998, Peter Murray-Rust wrote:

>                    XML (unlike HTML) does not normalise character content
> and all characters that are not markup are passed to the application.
> Ignorable whitespace is a device that SAX provides to help the application
> decide what action it may be able to take. If you are writing a SAX-based
> application you will need to understand this concept.

Oops, I was reading the XML spec without casting away a couple of HTML
assuptions. Thank you.

I'm writing a parser (in perl) now and wasn't sure I should take the
actions of Microstar's SAX driver as gospel. Admittedly, I've been
lazy about testing other implementations, but I was hoping for an
authoritative answer rather than an empirical study.

I've also crawled around Microstar's and Megginson's web spaces
looking for the SAX spec or at least a whitepaper. Do you have a
pointer?

> At 01:08 02/08/98 -0400, Eric Prud'hommeaux wrote:
> >Should "abcd#20$20efgh" be reported as:
> 
> Assuming this is in element content (e.g. <FOO>abcd  efgh</FOO>
>  and means "abcd  efgh", then it is passed without change, using
> characters(). Ignorable whitespace occurs between elements and not inside
> character strings. 

In that regard, it would seem that text is handled differently from
system identifiers and attribute values.

How about leading and trailing whitespace, or tags with just
whitespace? For example, is "<tag>some  text\r\n\t</tag>" reported
completely as characters and not split into characters("some  text")
and ignorable("\r\n\t")? Is the whitespace in "<t1>\n  <t2/>\n</t1>"
ignorable? I also assume from the XML spec that SAX is acting in the
role of XML processor and must translate \r's so it would really be
characters(" some  text\n\t").

Current (apparently overzealous) algorythm:

#####
# reportText - break up ignorable whitespace from characters

sub reportText {
    my $self = shift;
    my $characters = shift;

    # fold \r\n and \r into \n - XML:2.11
    $characters =~ s/\r\n/\n/g;
    $characters =~ tr/\r/\n/g;

    if ($characters =~ m/\A(\s+)/) {		# leading ignorables
	$self->ignorableWhitespace($&);
	$characters = $';
    }
    while ($characters =~ m/(\s{2,})/) {	# nested ignorables
	$self->characters($`) if ($` ne '');
	$self->ignorableWhitespace($&) if ($& ne '');
	$characters = $';
    }
    if ($characters =~ m/(\s+)\Z/) {		# trailing ignorables
	$self->characters($`) if ($` ne '');
	$self->ignorableWhitespace($&) if ($& ne '');
    }
}

Algorythm Lite:

sub reportText {
    my $self = shift;
    my $characters = shift;

    # fold \r\n and \r into \n - XML:2.11
    $characters =~ s/\r\n/\n/g;
    $characters =~ tr/\r/\n/g;

    if ($characters =~ m/\S/) {		# any non-white space?
	$self->characters($characters);
    } else {
	$self->ignorableWhitespace($&);
    }
}

-eric


xml-dev: A list for W3C XML Developers. To post, mailto:xml-dev at ic.ac.uk
Archived as: http://www.lists.ic.ac.uk/hypermail/xml-dev/
To (un)subscribe, mailto:majordomo at ic.ac.uk the following message;
(un)subscribe xml-dev
To subscribe to the digests, mailto:majordomo at ic.ac.uk the following message;
subscribe xml-dev-digest
List coordinator, Henry Rzepa (mailto:rzepa at ic.ac.uk)