XML::Twig question and problem

BenHopkins has asked for the wisdom of the Perl Monks concerning the following question:

Hey monks! I grovel in the gravel and genuinely genuflect gratefulness.

I'm using XML::Twig (3.30) and want to replace all entities in the data with ascii equivalents, i.e. — with --.

I tried using a _default_ root handler like this:

my $t = XML::Twig->new(
    twig_roots => {'nitf/body/body.head/hedline/hl1' => \&fix_hl1,
                   'nitf/body/body.head/hedline/hl2' => \&fix_hl2,
                    _default_ => \&fixup,
                  },
        twig_print_outside_roots => 1,
        keep_encoding => 1,
);
[download]

And the subroutine fixup looks like this:

sub fixup {
    my ($tree, $elem) = @_;

    my $tag = $elem->tag;
    plog("in fixup:  tag = $tag");

    if ($fixup_tag{$tag}) {
        my $t = $elem->text;
        conv_chars(\$t);
        $elem->set_text($t);
    }

    $elem->print;
}
[download]

The fixup_tag hash has p, byttl, person, hl1, etc., otherwise I was changing higher level tags, like nitf and that removed all the internal tags (not a good thing).

Question: is there some way to apply a filter to just the data and leave the tagging and attributes alone.

I have a feeling that a good answer to the above will obviate what comes next:

Problem: after running this, the output is invalid. The input document has <nitf><head><title></title> followed by a bunch of <meta> tags. The output has (after the DOCTYPE) a <title></title>, the <meta> tags, then the <head>, with <title> and <meta> duplicated, then comes a copy of the whole mess, this time inside the <nitf></nitf> tags!

Obviously, I'm doing something wrong.

Anybody have an idea of what it is?

Comment on XML::Twig question and problem Select or Download Code

Replies are listed 'Best First'.
Re: XML::Twig question and problem by mirod (Canon) on Jun 28, 2007 at 05:04 UTC
Question: is there some way to apply a filter to just the data and leave the tagging and attributes alone. Look at the `output_text_filter` option, it filters only text and attribute values. For your other problem, you can't use `_default` with `print_roots`, that just messes up everything. The `output_text_filter` thing will take care of it. And if you still have a problem with the beginning of the document being duplicated, get the development version of the module, I fixed a bug with it that last week.	[reply]
Re^2: XML::Twig question and problem by BenHopkins (Sexton) on Jun 28, 2007 at 06:26 UTC
I saw that, but couldn't figure out how to use it. If I say `... output_text_filter => \%fixup,` [download] and `sub fixup { my $t = $@; conv_chars(\$t); # subroutine that operates on a string return($t); }` [download] ...doesn't work. When I say`my $t = shift;`, $t contains tag names. I'm missing something here.	[reply] [d/l] [select]
Re^3: XML::Twig question and problem by mirod (Canon) on Jun 28, 2007 at 07:48 UTC
OK, so here the problem is that the filter opions do not operate when using the `twig_print_outside_roots`. At least they are not applied to the data outside the roots. Which makes sense as you are supposed to only care about what's inside the roots, that's the whole point of the "roots mode". Below is a way of solving your problem. I must say I would not use it though. Instead I would do a pre or post processing of the data, using regexps or maybe Text::Unidecode, to replace the characters you don't want. I assume you want to replace those characters everywhere in the document, so most likely you don't need to use an XML aware tool. #!/usr/bin/perl use strict; use warnings; use XML::Twig; my $t = XML::Twig->new( twig_handlers => { hl1 => \&fix_hl1, hl2 => \&fix_hl2, _all_ => \&fixup, }, keep_encoding => 1, keep_spaces => 1, ) ->parse( \*DATA); sub fixup { my( $t, $elt)= @_; foreach my $pcdata ($elt->children( '#PCDATA')) { $pcdata->set_text( conv_chars( $pcdata->text)); } $elt->flush unless( $_->in( 'hl1') \|\| $_->in( 'hl2')); } sub conv_chars { my $t= shift; $t=~ s{—}{--}g; return $t; } sub fix_hl1 { $_->set_tag( 'new_hl1')->prefix( 'fixed hl1:'); } sub fix_hl2 { $_->set_tag( 'new_hl2')->prefix( 'fixed hl2:'); } __DATA__ <!DOCTYPE nitf [ <!ENTITY foo "bar"> ]> <nitf><s><hl1>hl1 with a &foo; and an other &foo; and — and &#82 +12;</hl1> <p>text, no foo</p> <p>text, no foo but —</p> <p>text, with &foo;</p> <hl2>hl2 with a &foo; and an other &foo; and — and &#82 +12;</hl2> <hl2>hl2 no foo </hl2> <hl2>hl2 with a bold <b>&foo;</b>, <b>—</b> and an othe +r &foo;</hl2> </s> </nitf> [download]	[reply] [d/l]
Re^4: XML::Twig question and problem by BenHopkins (Sexton) on Jun 28, 2007 at 08:19 UTC


more useful options
	PerlMonks