http://qs321.pair.com?node_id=995682


in reply to Re: Chicanery Needed to Handle Unicode Text on Microsoft Windows
in thread Chicanery Needed to Handle Unicode Text on Microsoft Windows

This thread is refreshing to read!!! As a Windows user that is somewhat new to Perl, I spent the past few hours trying to figure out why one of my supplied 193 xml files would keep outputting as a bunch of Chinese (?) characters. Jim described exactly what I kept trying.

I finished my script. Everything else works - it does all my replaces beautifully. I have maybe spent 8 hours total on my script and it will save me about 3 days of work.

But, for now, I have to go to that specific XML file, open it in Notepad, and save it as 'ANSI' instead of 'Unicode' before my script will work right.

I have tried adding the use ' $string' supplied in this thread, but I get this error:

Unknown PerlIO layer 'raw:perlio:encoding(UTF-16LE):crlf:utf8'

I really would like to create re-usable code out of my script, but I have yet to find the answer.

  • Comment on Re^2: Chicanery Needed to Handle Unicode Text on Microsoft Windows
  • Download Code

Replies are listed 'Best First'.
Re^3: Chicanery Needed to Handle Unicode Text on Microsoft Windows
by Anonymous Monk on Sep 26, 2012 at 18:38 UTC

    I have tried adding the use ' $string' supplied in this thread, but I get this error:

    Which perl version do you have?

Re^3: Chicanery Needed to Handle Unicode Text on Microsoft Windows
by Anonymous Monk on Sep 26, 2012 at 18:44 UTC

    open it in Notepad, and save it as 'ANSI' instead of 'Unicode' before my script will work right.

    You probably shouldn't do that :) save as UTF-8 instead

    iconv -f UTF-16 -t UTF-8 < in > out

    piconv -f UTF-16LE -t UTF-8 < in > out

Re^3: Chicanery Needed to Handle Unicode Text on Microsoft Windows
by Anonymous Monk on Sep 26, 2012 at 18:52 UTC