Beefy Boxes and Bandwidth Generously Provided by pair Networks
Just another Perl shrine
 
PerlMonks  

Re: length() miscounting UTF8 characters?

by amon (Scribe)
on Apr 27, 2014 at 22:12 UTC ( [id://1084052]=note: print w/replies, xml ) Need Help??


in reply to length() miscounting UTF8 characters?

I still haven't figured out how use open is supposed to work. Hypothesis: it doesn't actually apply the IO layer to your handle. We can test that by querying the IO layers.

use open IO => ":utf8"; while (<>) { my @layers = PerlIO::get_layers(\*ARGV); say "(@layers)"; }

If it just outputs something like (unix perlio), it obviously didn't apply the layer.

The explicit way should work, I guess:

use strict; use warnings; use autodie; binmode STDOUT, ":utf8"; for my $file (@ARGV) { my $fh; if ($file eq "-") { $fh = \*STDIN; binmode $fh, ":utf8"; } else { open $fh, "<:utf8", $file; } while (<$fh>) { s/[[:ascii:]]//g; print length, " ", $_, "\n"; } }

Replies are listed 'Best First'.
Re^2: length() miscounting UTF8 characters?
by AppleFritter (Vicar) on Apr 27, 2014 at 22:35 UTC

    Hey, thanks a lot, that second script actually works! That's the practical problem I was facing all solved, then.

    I'm still curious why mine wasn't working. Your first script indeed just outputs "(unix perlio)", but I lack the knowledge to dig any deeper there.

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: note [id://1084052]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others avoiding work at the Monastery: (5)
As of 2024-04-24 07:22 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found