http://qs321.pair.com?node_id=11122078


in reply to Re^4: How to Encode/Decode double encoded string.
in thread How to Encode/Decode double encoded string.

If your database actually contains ����, then it will no longer be able to provide the invalid UTF-8 string which was converted to this sequence of replacement characters. So either the legacy version of the application didn't store correct data in the first place, or something went wrong when converting the server encoding to UTF-8. In the latter case, you might be able to grab an old database backup and restart from there.

We don't have access to your database nor to your application, so it is up to you to find out which part of software did the bogus UTF-8-decoding (bogus because it failed to check for errors). In Perl, you can (and should) trap this type of error by catching errors like this:

Use Encode; eval { decode('UTF-8',$string,Encode::FB_CROAK) }; if ($@) { # Well, that $string wasn't a valid UTF-8-string. # Go now and die in what way seems best to you. -- Denethor }