Re^2: XML::Twig - Parsing XML file with incorrect encoding in declaration


No such thing as a small change
	PerlMonks

Re^2: XML::Twig - Parsing XML file with incorrect encoding in declaration

by ateague (Monk)

on Sep 01, 2017 at 23:47 UTC ( [id://1198557]=note: print w/replies, xml )

Need Help??

in reply to Re: XML::Twig - Parsing XML file with incorrect encoding in declaration
in thread [SOLVED] XML::Twig - Parsing XML file with incorrect encoding in declaration

Can you try slurping in the XML content, and using perl to zap the encoding, before feeding to XML::Twig ?

That's what I figured I would need to do (reading in 128 MiB chunks in this case; 73 GiB is a bit much to slurp in all at once ;) ). I just wanted to see if there was an "official" method before I broke out the full brute-force nuclear option.

(Of course it probably goes without saying that the ideal solution is to fix this fluster-cluck of an XML travesty...)

EDIT:

I just saw the SO link in your update after I posted my reply. That particular example will not work in this because the data is not UTF-16 even though the encoding says otherwise.

Comment on Re^2: XML::Twig - Parsing XML file with incorrect encoding in declaration

Replies are listed 'Best First'.
Re^3: XML::Twig - Parsing XML file with incorrect encoding in declaration by NetWallah (Canon) on Sep 02, 2017 at 01:07 UTC
If it is too big to slurp .. you could "pipe-filter" it through. I haven't tried this, but you should be able to setup a FIFO , and pass a handle to LibXML to read. Then read your XML file, a record at a time, filter out the encoding garbage, and feed it to the FIFO. Probably need 2 threads to make this work in a single program. All power corrupts, but we need electricity.	[reply]
Re^4: XML::Twig - Parsing XML file with incorrect encoding in declaration by ateague (Monk) on Sep 18, 2017 at 13:42 UTC
I haven't tried this, but you should be able to setup a FIFO Would be slightly more difficult to do on Windows, but it is an option to experiment with. Sounds like a good opportunity to experiment and dabble with threads...	[reply]

In Section Seekers of Perl Wisdom

Domain Nodelet^?

www.com | www.net | www.org

Node Status^?

node history
Node Type: note [id://1198557]
help

Chatterbox^?

How do I use this? • Last hour • Other CB clients

Other Users^?

Others admiring the Monastery: (1)

chatterbot

As of 2024-04-25 00:49 GMT

Sections^?

Information^?

Find Nodes^?

Leftovers^?

Today I Learned

Voting Booth^?

No recent polls found