http://qs321.pair.com?node_id=1199892


in reply to Problem upgrading XML::Fast from 0.11 to 0.17

I have discovered adding utf8decode => 1 to the xml2hash makes it work now but I don't really understand why. I am doing anything wrong here? What might have changed in XML::Fast to cause this to happen?

The changelog isn't really verbose, but I suspect it might be #71532 which was marked as fixed recently. There is also #71533 which is still open, and which indicates that the module might have issues with Unicode data in general. I haven't investigated deeper than that*, but if you have verified with a hex dump that your input file is indeed ISO-8859-1, where "ø" should be represented by 0xF8, and you have verified with Devel::Peek that the Perl variable contains the correct data² ("ø" is Unicode U+00F8), then the error message "[�ndby IF]", where � is the Unicode character "Replacement Character" (U+FFFD), indicates that there is indeed some strange Unicode stuff going on in the module, since you are opening the file with the correct encoding².

* Update before posting: I see ablanke has investigated the code.

² Update: Actually, ikegami is right that the XML parser should be doing the decoding.

Replies are listed 'Best First'.
Re^2: Problem upgrading XML::Fast from 0.11 to 0.17
by mje (Curate) on Sep 22, 2017 at 10:40 UTC