Beefy Boxes and Bandwidth Generously Provided by pair Networks
P is for Practical
 
PerlMonks  

Encoding/decoding for korean strings

by siva_pm (Novice)
on Jun 15, 2016 at 17:51 UTC ( #1165763=perlquestion: print w/replies, xml ) Need Help??

siva_pm has asked for the wisdom of the Perl Monks concerning the following question:

I have some perl code which is getting korean input from UI and storing it in database (Sybase). The input is sent as a request to a handler where response is JSON and JSONs are being encoded and decoded in utf8. However it is unable to print this on UI complaining about the following. Wed Jun 15 16:08:06 2016] info 29922 Mojo::Reactor::EV: Write failed: Wide character in syswrite at /usr/lib/perl5/5.10.1/i386-linux/IO/Handle.pm line 465.

Only if I encode the data it prints on UI. Any help to print the data on UI in korean?

Replies are listed 'Best First'.
Re: Encoding/decoding for korean strings
by Your Mother (Archbishop) on Jun 15, 2016 at 18:30 UTC

    Sounds like you either need to give the EV stuff a UTF-8 layered file handle (which may not be an option; see UTF-8 caveats in syswrite) or you need to encode_utf8() the characters you're giving to it back to bytes for it to write. I poked around the sources for Mojo::Reactor::EV and IO::Handle and did not see a clear place to suggest more. If you can give a minimal example that exhibits the problem, I guarantee someone here will be able to help. The mojo IRC would probably be a place to get immediate help too. Please post your solution here if you arrive at it elsewhere. :P

      I tried to use the encode_utf8() and it doesn't complaint in that . However, this corrupts the characters on the UI and they're not korean anymore and when I try to decode_utf8(encode_utf8($mystring)) it complaints that it can't write wide character again.

        Every single layer has to agree upon what it's looking at from the input (JSON is always UTF-8) to the parsing to the writing to the DB to the fetching from the DB to the UI. So if the characters are being corrupted, it might be because your DB is actually expecting EUC-KR or something. In that case you might try encode("EUC-KR", $utf8_string); check your DB schema and see what it expects. But the problem might be in the UI uptake or output or your DB might be incorrectly configured/specified.

        Update: didn't see your reply before I replied. I might look at this later but it's a lot to digest. :(

        Below are the methods that deal with the encoding/decoding So I'm calling handleRequest($fun,$parameters) from my cgi file which calls a Handler.pm which initially does an eval call to handleRequest which firstly does eval call looks like this:

        sub handle_req_cgi{ eval { $log->info("pjb.cgi - Started function $function , user: $user_ +name"); $msg = UI::PJB::handleRequest($function, $parameters); 1; } //Response $tx->res->code($res_code); $tx->res->headers->content_type('application/json'); $tx->res->body(encode('UTF-8',$msg)); #THIS IS THE LINE which corrupts the msg but if I print just the msg w +ithout encode then #wide character warning is thrown by Mojo $tx->rendered; }
        handleRequest() # looks like this: { $jsonDecodedParameters = $J->decode($parameters); my $result = executeFunction($fun, $jsonDecodedParameters); if(ref $result eq ref {} && defined $result->{$HTTP_REQUEST_REDIRECT_U +RL_KEY}) { $jsonEncodedResult = $J->encode($result); returns this $jsonEncodedResult }

        then the control inside Handler.pm proceeds to write the response of

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: perlquestion [id://1165763]
Approved by stevieb
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others imbibing at the Monastery: (6)
As of 2021-04-20 08:37 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found

    Notices?