|
GertMT has asked for the wisdom of the Perl Monks concerning the following question:
Dear monks,
it seems that the script I've been using has a problem with the line endings when used on a Mac. I've tried
eol => '\012'
eol => '\015'
eol => '\l'
eol => '\r'
But neither of these did work. Did I forget an alternative?
#!/usr/bin/perl -w
use strict;
use diagnostics;
use Text::CSV;
my $qfn_in = 'data.csv';
open( my $fh_in, '<', $qfn_in )
or die("Can't open file \"$qfn_in\": $!\n");
my $fh_out = \*STDOUT;
my $csv_in =
Text::CSV_XS->new( { sep_char => ',', binary => 1, eol => '\015' } )
+;
my $csv_out = Text::CSV_XS->new( { sep_char => "\t", eol => $/ } );
while ( my $row = $csv_in->getline($fh_in) ) {
$csv_out->print( $fh_out, [ @$row[ 2, 3, 0, 4 ], ] );
}
As an alternative I looked at using *Text::ParseWords* (see below) but just couldn't get more then data of just the first line produced. Maybe someone could help me with that.
Thanks in advance, Gert
#!/usr/bin/perl -w
use strict;
use diagnostics;
use Text::ParseWords;
my $mydata = "data.csv";
open( FILE, "<$mydata" )
or die("Couldn't open CSV file $mydata:$!\\n");
my $line;
my @fields;
while ( $line = <FILE> ) {
@fields = "ewords( ',', 0, $line )
or ( warn "a problem on line $.:$_" );
# Set variable values based on the array values.
my $id = $fields[0];
my $brand = $fields[1];
my $dbt = $fields[2];
my $cdt = $fields[3];
my $color = $fields[4];
my $number = $fields[5];
}
exit;
|
Re: line ending problem Text::CSV alternative Text::ParseWords?
by ELISHEVA (Prior) on Oct 06, 2009 at 11:28 UTC
|
What happened with your first code sample? Did it also stop on the first line?
I see you are only using strict, not warnings. Diagnostics does not turn on warnings automatically. When you add use warnings, what do you see? Do you get any warnings? What diagnostics print out with them?
If you really have a problem with line endings it should be obvious in your loops. In the first bit of code, what happens if you include the line local $"='|'; print STDERR "<@$row>\n" at the top of the while loop in the first example? If you include print STDERR "<$line>\n"., what happens?
In the two print statements, you'll note that I sent output to STDERR. This is to make the debugging statements easier to find when you are ready to comment them out or delete them. I also surrounded the variable I wanted to print with "<...>" and changed the separator between array elements in a string to a pipe ('|'). This is done so that the presence of whitespace is more obvious.
Best, beth
| [reply]
[d/l]
[select] |
|
|
Thanks for your reply,
thought that the -w would trigger warnings as well. Anyhow I did put use warnings; in my code but no warnings.
guess I'm losing my way a bit.. should it look like this?
my $row;
local $"='|'; print STDERR "<@$row>\n";
while ( my $row = $csv_in->getline($fh_in) ) {
$csv_out->print( $fh_out, [ @$row[ 2, 3, 0, 4 ], ] );
}
print STDERR "<$row>\n";
I get
Can't use an undefined value as an ARRAY reference at but I'm not sure I put the code in the correct way. Trying to figure this out... | [reply]
[d/l]
[select] |
|
|
while ( my $row = $csv_in->getline($fh_in) ) {
# print out each line you visit during the while loop
local $"='|'; print STDERR "<@$row>\n";
$csv_out->print( $fh_out, [ @$row[ 2, 3, 0, 4 ], ] );
}
Your code was getting Can't use an undefined value as an ARRAY reference at because you were trying to dereference $row as an array reference before you ever used it as an array reference. Also "my" in a while statement scopes the variable to inside the while loop. my $row outside the while loop and while (my $row...) are actually to different variables with two different locations in memory. Using the variable as an array reference inside the while loop doesn't have any effect on how Perl sees the variable of the same name found outside of the while loop.
Best, beth | [reply]
[d/l]
[select] |
|
|
|
|
|
Re: line ending problem Text::CSV alternative Text::ParseWords?
by Tux (Canon) on Oct 06, 2009 at 12:17 UTC
|
When parsing CSV with Text::CSV_XS or Text::CSV don't specify the eol attribute at all, and getline () most likely will do-the-right-thing.
open my $fh, "<", $file or die "$file: $!";
# Don't use auto_diag on older Text::CSV_XS
my $csv_in = Text::CSV_XS->new ({ binary => 1, auto_diag => 1 });
my $csv_out = Text::CSV_XS->new ({ binary => 1, sep_char => "\t", eol
+ => $/ });
while (my $row = $csv->getline ($fh)) {
$csv_out->print (*STDOUT, [ @{$row}[2, 3, 0, 4] ]);
}
$csv->eof () or $csv->error_diag (); # Not needed with auto_diag
close $fh;
update: filled in the correct output syntax for a array-ref-slice
Important update: I was wrong. A single \r is not a valid line ending in the default configuration according to the CSV specs. In that case you'd have to specify that with eol => "\r". I will see if I can make it legal if the system $/ already is a "\r".
Enjoy, Have FUN! H.Merijn
| [reply]
[d/l]
[select] |
|
|
| [reply]
[d/l]
[select] |
|
|
2023 "EIQ - QUO character not allowed"
Sequences like ""foo "bar" baz",quux" and "2023,",2008-04-05,
+"Foo,
Bar",\n" will cause this error.
Can I see the actual data?
update: FWIW those errors might go away if you parse with
Text::CSV_XS->new ({ binary => 1, allow_loose_quotes => 1 });
or
Text::CSV_XS->new ({ binary => 1, escape_char => undef });
Enjoy, Have FUN! H.Merijn
| [reply]
[d/l]
[select] |
|
|
|
|
|
|
