Beefy Boxes and Bandwidth Generously Provided by pair Networks
laziness, impatience, and hubris

line ending problem Text::CSV alternative Text::ParseWords?

by GertMT (Hermit)
on Oct 06, 2009 at 11:11 UTC ( #799438=perlquestion: print w/replies, xml ) Need Help??

GertMT has asked for the wisdom of the Perl Monks concerning the following question:

Dear monks,
it seems that the script I've been using has a problem with the line endings when used on a Mac. I've tried
eol => '\012' eol => '\015' eol => '\l' eol => '\r'
But neither of these did work. Did I forget an alternative?
#!/usr/bin/perl -w use strict; use diagnostics; use Text::CSV; my $qfn_in = 'data.csv'; open( my $fh_in, '<', $qfn_in ) or die("Can't open file \"$qfn_in\": $!\n"); my $fh_out = \*STDOUT; my $csv_in = Text::CSV_XS->new( { sep_char => ',', binary => 1, eol => '\015' } ) +; my $csv_out = Text::CSV_XS->new( { sep_char => "\t", eol => $/ } ); while ( my $row = $csv_in->getline($fh_in) ) { $csv_out->print( $fh_out, [ @$row[ 2, 3, 0, 4 ], ] ); }

As an alternative I looked at using *Text::ParseWords* (see below) but just couldn't get more then data of just the first line produced. Maybe someone could help me with that.
Thanks in advance,
#!/usr/bin/perl -w use strict; use diagnostics; use Text::ParseWords; my $mydata = "data.csv"; open( FILE, "<$mydata" ) or die("Couldn't open CSV file $mydata:$!\\n"); my $line; my @fields; while ( $line = <FILE> ) { @fields = &quotewords( ',', 0, $line ) or ( warn "a problem on line $.:$_" ); # Set variable values based on the array values. my $id = $fields[0]; my $brand = $fields[1]; my $dbt = $fields[2]; my $cdt = $fields[3]; my $color = $fields[4]; my $number = $fields[5]; } exit;

Replies are listed 'Best First'.
Re: line ending problem Text::CSV alternative Text::ParseWords?
by ELISHEVA (Prior) on Oct 06, 2009 at 11:28 UTC

    What happened with your first code sample? Did it also stop on the first line?

    I see you are only using strict, not warnings. Diagnostics does not turn on warnings automatically. When you add use warnings, what do you see? Do you get any warnings? What diagnostics print out with them?

    If you really have a problem with line endings it should be obvious in your loops. In the first bit of code, what happens if you include the line local $"='|'; print STDERR "<@$row>\n" at the top of the while loop in the first example? If you include print STDERR "<$line>\n"., what happens?

    In the two print statements, you'll note that I sent output to STDERR. This is to make the debugging statements easier to find when you are ready to comment them out or delete them. I also surrounded the variable I wanted to print with "<...>" and changed the separator between array elements in a string to a pipe ('|'). This is done so that the presence of whitespace is more obvious.

    Best, beth

      Thanks for your reply,
      thought that the -w would trigger warnings as well. Anyhow I did put use warnings; in my code but no warnings.
      guess I'm losing my way a bit.. should it look like this?
      my $row; local $"='|'; print STDERR "<@$row>\n"; while ( my $row = $csv_in->getline($fh_in) ) { $csv_out->print( $fh_out, [ @$row[ 2, 3, 0, 4 ], ] ); } print STDERR "<$row>\n";
      I get
      Can't use an undefined value as an ARRAY reference at but I'm not sure I put the code in the correct way. Trying to figure this out...

        Like this:

        while ( my $row = $csv_in->getline($fh_in) ) { # print out each line you visit during the while loop local $"='|'; print STDERR "<@$row>\n"; $csv_out->print( $fh_out, [ @$row[ 2, 3, 0, 4 ], ] ); }

        Your code was getting Can't use an undefined value as an ARRAY reference at because you were trying to dereference $row as an array reference before you ever used it as an array reference. Also "my" in a while statement scopes the variable to inside the while loop. my $row outside the while loop and while (my $row...) are actually to different variables with two different locations in memory. Using the variable as an array reference inside the while loop doesn't have any effect on how Perl sees the variable of the same name found outside of the while loop.

        Best, beth

Re: line ending problem Text::CSV alternative Text::ParseWords?
by Tux (Canon) on Oct 06, 2009 at 12:17 UTC

    When parsing CSV with Text::CSV_XS or Text::CSV don't specify the eol attribute at all, and getline () most likely will do-the-right-thing.

    open my $fh, "<", $file or die "$file: $!"; # Don't use auto_diag on older Text::CSV_XS my $csv_in = Text::CSV_XS->new ({ binary => 1, auto_diag => 1 }); my $csv_out = Text::CSV_XS->new ({ binary => 1, sep_char => "\t", eol + => $/ }); while (my $row = $csv->getline ($fh)) { $csv_out->print (*STDOUT, [ @{$row}[2, 3, 0, 4] ]); } $csv->eof () or $csv->error_diag (); # Not needed with auto_diag close $fh;

    update: filled in the correct output syntax for a array-ref-slice

    Important update: I was wrong. A single \r is not a valid line ending in the default configuration according to the CSV specs. In that case you'd have to specify that with eol => "\r". I will see if I can make it legal if the system $/ already is a "\r".

    Enjoy, Have FUN! H.Merijn

      that gives me:

      # CSV_XS ERROR: 2023 - EIQ - QUO character not allowed

      and nothing further.

      If I leave out the auto_diag = 1 part I get nothing back on the screen.

        From the man page:

        2023 "EIQ - QUO character not allowed" Sequences like ""foo "bar" baz",quux" and "2023,",2008-04-05, +"Foo, Bar",\n" will cause this error.

        Can I see the actual data?

        update: FWIW those errors might go away if you parse with

        Text::CSV_XS->new ({ binary => 1, allow_loose_quotes => 1 });


        Text::CSV_XS->new ({ binary => 1, escape_char => undef });

        Enjoy, Have FUN! H.Merijn

Log In?

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: perlquestion [id://799438]
Approved by ELISHEVA
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others taking refuge in the Monastery: (3)
As of 2021-10-28 19:50 GMT
Find Nodes?
    Voting Booth?
    My first memorable Perl project was:

    Results (96 votes). Check out past polls.