Beefy Boxes and Bandwidth Generously Provided by pair Networks
Problems? Is your data what you think it is?
 
PerlMonks  

Re^5: Speeds vs functionality

by farang (Chaplain)
on Jul 31, 2014 at 23:50 UTC ( #1095825=note: print w/replies, xml ) Need Help??


in reply to Re^4: Speeds vs functionality
in thread Speeds vs functionality

Text::CSV_PP is able to parse that text, at least in UTF-8.

use v5.12;
use warnings;
use utf8::all;
use Text::CSV_PP;

my $csv = Text::CSV_PP->new ( 
    { binary      => 1 , 
      quote_char  => '🎥' ,
      escape_char => '🎥' ,
      sep_char    => '🎬'  } )
  or die "Cannot use CSV_PP: "
   .Text::CSV_PP->error_diag ();

my @rows;
my $fh = *DATA;
while ( my $row = $csv->getline( $fh ) ) {
         push @rows, $row;
}
$csv->eof or $csv->error_diag();
for ( @rows ) {
    printf("%-25s%s\n", $_->[0], $_->[4]);
}
__DATA__
🎥Film🎥🎬🎥Year🎥🎬🎥Awards🎥🎬🎥Nominations🎥🎬🎥Director🎥
🎥12 Years a Slave🎥🎬2013🎬3🎬9🎬🎥🎥🎥 Steve McQueen🎥
🎥Argo🎥🎬2012🎬3🎬7🎬🎥🎥🎥 Ben Affleck🎥
🎥The Artist🎥🎬2012🎬5🎬10🎬🎥🎥🎥 Michel Hazanavicius🎥
🎥The King's Speech🎥🎬2010🎬4🎬12🎬🎥🎥🎥 Tom Hooper🎥
🎥The Hurt Locker🎥🎬2009🎬6🎬9🎬🎥🎥🎥 Kathryn Bigelow🎥
🎥Slumdog Millionaire🎥🎬2008🎬8🎬10🎬🎥🎥🎥 Danny Boyle🎥
🎥No Country for Old Men🎥🎬2007🎬4🎬8🎬🎥🎥🎥 Joel Coen 🎥🎥 Ethan Coen🎥
🎥The Departed🎥🎬2006🎬4🎬5🎬🎥🎥🎥 Martin Scorsese🎥

Output:

Film                     Director
12 Years a Slave         🎥 Steve McQueen
Argo                     🎥 Ben Affleck
The Artist               🎥 Michel Hazanavicius
The King's Speech        🎥 Tom Hooper
The Hurt Locker          🎥 Kathryn Bigelow
Slumdog Millionaire      🎥 Danny Boyle
No Country for Old Men   🎥 Joel Coen 🎥 Ethan Coen
The Departed             🎥 Martin Scorsese

Replies are listed 'Best First'.
Re^6: Speeds vs functionality
by Jim (Curate) on Aug 01, 2014 at 01:38 UTC

    Booyah! farang FTW!

    Here's my test with a very lightly refactored version of the same script:

    use v5.14;
    use strict;
    use warnings;
    use utf8;
    
    use Text::CSV_PP;
    
    binmode STDOUT, ':encoding(UTF-8)';
    
    my $csv = Text::CSV_PP->new({
        sep_char    => '🎬',
        quote_char  => '🎥',
        escape_char => '🎥',
        binary      => 1,
    });
    
    my @rows;
    
    my $fh = *DATA;
    
    while (my $row = $csv->getline($fh)) {
        push @rows, $row;
    }
    
    $csv->eof() or $csv->error_diag();
    
    for my $row (@rows) {
        $row->[4] =~ s/\n\s*/, /g;
    
        printf "%-24s %s\n", $row->[0], $row->[4];
    }
    
    exit 0;
    
    __DATA__
    🎥Film🎥🎬🎥Year🎥🎬🎥Awards🎥🎬🎥Nominations🎥🎬🎥Director🎥
    🎥12 Years a Slave🎥🎬2013🎬3🎬9🎬🎥🎥🎥 Steve McQueen🎥
    🎥Argo🎥🎬2012🎬3🎬7🎬🎥🎥🎥 Ben Affleck🎥
    🎥The Artist🎥🎬2012🎬5🎬10🎬🎥🎥🎥 Michel Hazanavicius🎥
    🎥The King's Speech🎥🎬2010🎬4🎬12🎬🎥🎥🎥 Tom Hooper🎥
    🎥The Hurt Locker🎥🎬2009🎬6🎬9🎬🎥🎥🎥 Kathryn Bigelow🎥
    🎥Slumdog Millionaire🎥🎬2008🎬8🎬10🎬🎥🎥🎥 Danny Boyle🎥
    🎥No Country for Old Men🎥🎬2007🎬4🎬8🎬🎥🎥🎥 Joel Coen
    🎥🎥 Ethan Coen🎥
    🎥The Departed🎥🎬2006🎬4🎬5🎬🎥🎥🎥 Martin Scorsese🎥
    

    This correctly produces:

    Film                     Director
    12 Years a Slave         🎥 Steve McQueen
    Argo                     🎥 Ben Affleck
    The Artist               🎥 Michel Hazanavicius
    The King's Speech        🎥 Tom Hooper
    The Hurt Locker          🎥 Kathryn Bigelow
    Slumdog Millionaire      🎥 Danny Boyle
    No Country for Old Men   🎥 Joel Coen, 🎥 Ethan Coen
    The Departed             🎥 Martin Scorsese
    

    Notice that this version handles the literal newline (\n, CR-LF) in the Coen brothers record, which I change to ',' in the output.

    Thank you, farang. I stand corrected:  there is a Unicode-capable CSV parser/generator Perl module on CPAN. And I think you just solved a very long-lived problem for me.

Re^6: Speeds vs functionality
by Anonymous Monk on Aug 01, 2014 at 02:32 UTC
    shoot :) I had used the mutators to set seperator and those values and I couldn't get it to work :) thanks farang
Re^6: Speeds vs functionality
by Jim (Curate) on Aug 01, 2014 at 18:23 UTC

    It works on a UTF-16 CSV file.

    use v5.14;
    use strict;
    use warnings;
    use utf8;
    
    use autodie qw( open close );
    use Text::CSV_PP;
    
    @ARGV == 1 or die "Usage: perl $0 <CSV file>\n";
    
    my $file = shift;
    
    open my $fh, '<:raw:perlio:encoding(UTF-16):crlf', $file;
    
    my $csv = Text::CSV_PP->new({
        sep_char    => '🎬',
        quote_char  => '🎥',
        escape_char => '🎥',
        binary      => 1,
    });
    
    my @rows;
    
    while (my $row = $csv->getline($fh)) {
        push @rows, $row;
    }
    
    $csv->eof() or $csv->error_diag();
    
    close $fh;
    
    binmode STDOUT, ':raw:perlio::encoding(UTF-16LE):crlf';
    
    for my $row (@rows) {
        $row->[4] =~ s/\n\s*/, /g;
    
        printf "%-24s %s\n", $row->[0], $row->[4];
    }
    
    exit 0;
    

    See these nodes for an explanation of the UTF-16 PerlIO nonsense required on Microsoft Windows.

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://1095825]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others making s'mores by the fire in the courtyard of the Monastery: (6)
As of 2020-10-29 10:35 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?
    My favourite web site is:












    Results (269 votes). Check out past polls.

    Notices?