utf8 in perl

theravadamonk has asked for the wisdom of the Perl Monks concerning the following question:

Hi, Perl Monks,

I have a maillog file which contains Subject and From addresses with utf8.

Here's a bit of the maillog. Pls pay attention to Subject: and From:

mail_id: 3gTkbdw2YlQK, b: pru-s06Xc, Hits: 8.111, size: 40657, Subject
+: "Heritage The Latest & Most Luxurious Banquet Hall In Anuradhapura 
+(raw: =?utf-8?Q?Heritage=20The=20Latest=20&=20Most=20Luxurious=20Banq
+uet=20Hall=20In=20Anuradhapura?=)", From: Heritage_Hotel_Anuradhapura
+_<ads@adzinair.net>_(raw:_=?utf-8?Q?Heritage=20Hotel=20Anuradhapura?=
+_<ads@adzinair.net>), X-Mailer: MailChimp_Mailer_-_**CID8fa1f72d0e77e
+89b5794**
[download]

I want to get it displayed in proper way. I still can't it. It STILL gives raw: =?utf-8?Q? etc..

I have added below lines to my perl code.


#!/usr/bin/perl
use CGI ':standard';
use strict;
use warnings;
use CGI::Carp 'fatalsToBrowser'; # use only for testing

print "Content-Type: text/html; charset=utf-8\n\n";


open FILE, '<:encoding(utf8)', '/var/log/mail.log' or die $!;

my @file = reverse <FILE>;
[download]

Anyway, I am reading the file in Revers Order. where hv I gone wrong?

I expect your answers. I hv been trying since yesterday evening. But, I still haven't succeeded.

I googled a lot. I came across some. I hope this url is useful.

https://en.wikibooks.org/wiki/Perl_Programming/Unicode_UTF-8

2018-07-11 Athanasius added code tags to maillog quote and linkified the Wikibooks link

Comment on utf8 in perl
Select or Download Code

Replies are listed 'Best First'.
Re: utf8 in perl by haj (Vicar) on Jul 04, 2018 at 05:36 UTC
As of now, e-mail headers are restricted to 7-bit ASCII characters and can't be UTF-8-encoded. These header fields are encoded according to RFC 2047: Message Header Extensions for Non-ASCII Text. In Perl, you can use the core module Encode::MIME::Header to convert these headers to UTF-8 characters.	[reply]
Re^2: utf8 in perl by theravadamonk (Scribe) on Jul 05, 2018 at 10:40 UTC
Hi, Thanks for directing me to good source. Anyway, I wrote a simple perl code. here' my code. #!/usr/bin/perl use CGI ':standard'; use strict; use warnings; use CGI::Carp 'fatalsToBrowser'; # use only for testing use Encode qw(encode decode); no warnings 'utf8'; print "Content-Type: text/html; charset=utf-8\n\n"; my $subject = "Room Rush \303\242\302\200\302\223 Enjoy 25% off on you +r stay. (raw: Room Rush =?utf-8?b?4oCT?= Enjoy 25% off on your stay.) +"; $subject =~ s/[^[:ascii:]]+//g; # get rid of non-ASCII characters my $subject_decoded = decode("MIME-Header", $subject); #my $subject_decoded = decode("MIME-B", $subject); #my $subject_decoded = decode("MIME-Q", $subject); print "\n"; print "<br/>"; print "subject: $subject \n\n"; print "<br/>"; print "subject_decoded: $subject_decoded \n\n"; [download] here's what I get via web browser. subject: Room Rush Enjoy 25% off on your stay. (raw: Room Rush =?utf-8?b?4oCT?= Enjoy 25% off on your stay.) subject_decoded: Room Rush Enjoy 25% off on your stay. (raw: Room Rush – Enjoy 25% off on your stay.) But, the word " raw: " still appears? Is this code good? How to enhance it? Anyway, I spent many hours to write the code since I still learn perl	[reply] [d/l]
Re^3: utf8 in perl by hippo (Bishop) on Jul 05, 2018 at 14:00 UTC
I'm not quite sure how you are getting that output. Here's an SSCCE which you might be able to tailor to your requirements. I've removed all the CGI so you can just run this from the command line. `#!/usr/bin/env perl use strict; use warnings; use Encode qw(encode decode); my $subject = "Room Rush \303\242\302\200\302\223 Enjoy 25% off on you +r stay. (raw: Room Rush =?utf-8?b?4oCT?= Enjoy 25% off on your stay.) +"; my $decoded = decode ("MIME-Header", $subject); print encode ("UTF-8", $decoded) . "\n";` [download] Running this gives: Room Rush รขยย Enjoy 25% off on your stay. (raw: Room Rush – Enjoy 25% off on your stay.) which demonstrates that we have successfully decoded the raw part of the header (and encoded it to UTF-8 for output). HTH.	[reply] [d/l]
Re^4: utf8 in perl by theravadamonk (Scribe) on Jul 05, 2018 at 16:57 UTC
Re^5: utf8 in perl by hippo (Bishop) on Jul 05, 2018 at 17:15 UTC
Some notes below your chosen depth have not been shown here
Re^3: utf8 in perl by Corion (Patriarch) on Jul 05, 2018 at 10:44 UTC
Maybe I misunderstand you, but `raw:` is in the original input string too.	[reply] [d/l]
Technical note (Re: utf8 in perl) by Dallaylaen (Chaplain) on Jul 09, 2018 at 07:41 UTC
Could you please wrap the e-mail header example into `<code>...</code>` so that looooong lines are wrapped? As of current, it breaks formatting on the front page. Thank you.	[reply]
Re: utf8 in perl by Anonymous Monk on Jul 04, 2018 at 13:01 UTC
Bear in mind also that whatever you are using to display the data must also be aware that it is being given UTF data. Some software is prepared to handle it; some turns it into an ASCII display-string; some display garbage. PerlMonks correctly displays symbols such as ฉ Copyright because the browser knows to expect UTF.	[reply]