Beefy Boxes and Bandwidth Generously Provided by pair Networks
Perl-Sensitive Sunglasses
 
PerlMonks  

Re^2: Encoding/decoding for korean strings

by siva_pm (Novice)
on Jun 15, 2016 at 19:11 UTC ( [id://1165780]=note: print w/replies, xml ) Need Help??


in reply to Re: Encoding/decoding for korean strings
in thread Encoding/decoding for korean strings

I tried to use the encode_utf8() and it doesn't complaint in that . However, this corrupts the characters on the UI and they're not korean anymore and when I try to decode_utf8(encode_utf8($mystring)) it complaints that it can't write wide character again.

Replies are listed 'Best First'.
Re^3: Encoding/decoding for korean strings
by Your Mother (Archbishop) on Jun 15, 2016 at 19:27 UTC

    Every single layer has to agree upon what it's looking at from the input (JSON is always UTF-8) to the parsing to the writing to the DB to the fetching from the DB to the UI. So if the characters are being corrupted, it might be because your DB is actually expecting EUC-KR or something. In that case you might try encode("EUC-KR", $utf8_string); check your DB schema and see what it expects. But the problem might be in the UI uptake or output or your DB might be incorrectly configured/specified.

    Update: didn't see your reply before I replied. I might look at this later but it's a lot to digest. :(

      So I have my db accepting these korean characters and can see them in korean in the tables too. I cannot use EUC-KR as I might have to accept all kinds of languages as input (one byte/two byte)
Re^3: Encoding/decoding for korean strings
by siva_pm (Novice) on Jun 15, 2016 at 19:19 UTC

    Below are the methods that deal with the encoding/decoding So I'm calling handleRequest($fun,$parameters) from my cgi file which calls a Handler.pm which initially does an eval call to handleRequest which firstly does eval call looks like this:

    sub handle_req_cgi{ eval { $log->info("pjb.cgi - Started function $function , user: $user_ +name"); $msg = UI::PJB::handleRequest($function, $parameters); 1; } //Response $tx->res->code($res_code); $tx->res->headers->content_type('application/json'); $tx->res->body(encode('UTF-8',$msg)); #THIS IS THE LINE which corrupts the msg but if I print just the msg w +ithout encode then #wide character warning is thrown by Mojo $tx->rendered; }
    handleRequest() # looks like this: { $jsonDecodedParameters = $J->decode($parameters); my $result = executeFunction($fun, $jsonDecodedParameters); if(ref $result eq ref {} && defined $result->{$HTTP_REQUEST_REDIRECT_U +RL_KEY}) { $jsonEncodedResult = $J->encode($result); returns this $jsonEncodedResult }

    then the control inside Handler.pm proceeds to write the response of

      Maybe there's something I'm not understanding, but there's nothing in the first snippet that calls the "handle_pjb_cgi" function in the second snippet, and so I have no idea what the "$tx" thing is. Assuming "handle_pjb_cgi" is being passed the sort of object it should be getting, the description in the OP makes it sound like whatever file handle is being used to "print" the the "message" has not been set to use a utf8 layer.

      I gather from your other reply below that you've confirmed there is valid utf8 data stored in the database, so the problem would seem to be limited to the output process, and probably involves the discipline setting on the output file handle. But I'm just guessing here.

        the first snippet which has the sub handle_pjb_cgi has eval (if you look into it, which calls handleRequest()) tx is a transaction obj (not needed for now)

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: note [id://1165780]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others admiring the Monastery: (5)
As of 2024-03-28 21:05 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found