in reply to Re^4: Wierd behaviour with HTML::Entities::decode_entities()
in thread Wierd behaviour with HTML::Entities::decode_entities()

The source for decode_entities_old, which I thought was just a pure-Perl version of decode_entities, looks roughly like this (I've stripped out some context dependence):

sub decode_entities_old { my $array = [ @_ ]; my $c; for (@$array) { s/(&\#(\d+);?)/$2 < 256 ? chr($2) : $1/eg; s/(&\#[xX]([0-9a-fA-F]+);?)/$c = hex($2); $c < 256 ? chr($c) : $1/ +eg; s/(&(\w+);?)/$entity2char{$2} || $1/eg; } return @$array; }
With this code, and the under-populated hash my %entity2char = ( amp => '&', quot => '"' ), we have
say decode_entities_old '&#x26;amp;quot;'; => &quot; say decode_entities_old '&amp;#x26;quot;'; => &#x26;quot;
We would get (essentially) the opposite behaviour if we switched the second and third substitutions; that's what I mean by “This is very particular to the ordering”.

Of course, your example shows that decode_entities (which I guess is implemented in XS—I couldn't find it in the source) doesn't exhibit this buggy behaviour, so I guess that there's a reason that the sub I quoted has _old postpended. :-)

UPDATE: I just noticed that I'd mangled my intended bug-inducer, &&#x71;uot;, above. I wonder if decode_entities (not decode_entities_old) handles it correctly?

Replies are listed 'Best First'.
Re^6: Wierd behaviour with HTML::Entities::decode_entities()
by ikegami (Patriarch) on Dec 14, 2009 at 18:05 UTC

    If HTML::Entities were to decode &#x26;amp;quot; to &quot;, it would be buggy. I did understand you correctly. Does what I said make more sense now?

    I wonder if decode_entities (not decode_entities_old) handles it correctly?

    I don't know if that's legal in SGML/HTML — unescaped ampersand — but yes.

    $ perl -MHTML::Entities -le'print decode_entities "&&#x71;uot;"' &quot;
      You can test with this -
      <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w"> <html xmlns="" xml:lang="sv" lang="sv"> <head> <meta http-equiv="Content-Type" content="text/html;charset=iso-885 +9-1"/> </head> <body> Mic i vĀr replokal &amp;quot;The Dungeon&amp;quot; </body> </html>
      Now it very clear what the problem is...

        If you get

        Mic i vĀr replokal &amp;quot;The Dungeon&amp;quot;

        and you're expecting

        Mic i vĀr replokal "The Dungeon"

        the problem is on their end. As you can see by loading that page in a browser,

        Mic i vĀr replokal &amp;quot;The Dungeon&amp;quot;


        Mic i vĀr replokal &quot;The Dungeon&quot;