http://qs321.pair.com?node_id=812747


in reply to Re^4: Wierd behaviour with HTML::Entities::decode_entities()
in thread Wierd behaviour with HTML::Entities::decode_entities()

The source for decode_entities_old, which I thought was just a pure-Perl version of decode_entities, looks roughly like this (I've stripped out some context dependence): 

sub decode_entities_old
{
    my $array = [ @_ ];
    my $c;
    for (@$array) {
    s/(&\#(\d+);?)/$2 < 256 ? chr($2) : $1/eg;
    s/(&\#[xX]([0-9a-fA-F]+);?)/$c = hex($2); $c < 256 ? chr($c) : $1/
+eg;
    s/(&(\w+);?)/$entity2char{$2} || $1/eg;
    }
    return @$array;
}

[download]
With this code, and the under-populated hash my %entity2char = ( amp => '&', quot => '"' ), we have 
say decode_entities_old '&#x26;amp;quot;'; => &quot;
say decode_entities_old '&amp;#x26;quot;'; => &#x26;quot;

[download]
We would get (essentially) the opposite behaviour if we switched the second and third substitutions; that's what I mean by “This is very particular to the ordering”.

Of course, your example shows that decode_entities (which I guess is implemented in XS—I couldn't find it in the source) doesn't exhibit this buggy behaviour, so I guess that there's a reason that the sub I quoted has _old postpended. :-)

UPDATE: I just noticed that I'd mangled my intended bug-inducer, &&#x71;uot;, above. I wonder if decode_entities (not decode_entities_old) handles it correctly?

Replies are listed 'Best First'.
Re^6: Wierd behaviour with HTML::Entities::decode_entities()
by ikegami (Patriarch) on Dec 14, 2009 at 18:05 UTC

    If HTML::Entities were to decode &#x26;amp;quot; to &quot;, it would be buggy. I did understand you correctly. Does what I said make more sense now?

    I wonder if decode_entities (not decode_entities_old) handles it correctly?

    I don't know if that's legal in SGML/HTML — unescaped ampersand — but yes. 

    $ perl -MHTML::Entities -le'print decode_entities "&&#x71;uot;"'
&quot;

    [download]
[reply]
[d/l]
[select]
      You can test with this - 
      
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w
+3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="sv" lang="sv">
<head>
    <meta http-equiv="Content-Type" content="text/html;charset=iso-885
+9-1"/>
</head>
<body>

Mic i vĀr replokal &amp;quot;The Dungeon&amp;quot;

</body>
</html>

      [download]
      Now it very clear what the problem is...
[reply]
[d/l]

        If you get

        Mic i vĀr replokal &amp;quot;The Dungeon&amp;quot;

        and you're expecting

        Mic i vĀr replokal "The Dungeon"

        the problem is on their end. As you can see by loading that page in a browser,

        Mic i vĀr replokal &amp;quot;The Dungeon&amp;quot;

        means

        Mic i vĀr replokal &quot;The Dungeon&quot;

[reply]
[d/l]
[select]

In Section Seekers of Perl Wisdom