Beefy Boxes and Bandwidth Generously Provided by pair Networks
P is for Practical
 
PerlMonks  

UTF-8 vs. ISO-8859-1

by BorgCopyeditor (Friar)
on Jul 11, 2002 at 00:41 UTC ( [id://180916]=note: print w/replies, xml ) Need Help??


in reply to questions about character encoding

Maybe this will help. Those accented characters you're using are probably iso-8859-1, more or less an extension of ASCII to 8 bits. So, ι, for example, is 234. UTF-8 is another encoding altogether that uses multiple bytes to encode character sets from many different writing systems (as you probably know).

Here's where the problem is (and what the error message was telling you). Under UTF-8, 7-bit ASCII characters retain their old mappings. But in order to encode multi-byte characters, UTF-8 uses the high bits. In order to encode a character that is two bytes long, the first byte's high bit is set to one, the second to zero. And so forth for characters that take up more bytes. If you take a string of Latin-A characters (I think they're called), their high bits won't match the carefully thought-out system of UTF-8 encoding, and make a mess of it. The string ικλ, for example, has three high bits set to one in a row...

...which is what "malformed UTF-8 character" means.

Update: that should have read "Latin 1," not "Latin A." Also, here's where I learned the little I know about Unicode.

BCE
--Your punctuation skills are insufficient!

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: note [id://180916]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others admiring the Monastery: (3)
As of 2024-04-24 00:03 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found