Beefy Boxes and Bandwidth Generously Provided by pair Networks
P is for Practical
 
PerlMonks  

Re: Re: regular expression (search and destroy)

by demerphq (Chancellor)
on Nov 13, 2003 at 15:21 UTC ( [id://306811]=note: print w/replies, xml ) Need Help??


in reply to Re: regular expression (search and destroy)
in thread regular expression (search and destroy)

This should work with Excel generated CSV files.

Nope. That code doesn't do Excel CSV files at all. Excel CSV files are composed of records terminated by an unquoted newline, which are composed of fields that are either quoted with double quotes, with embedded quotes being doubled as the only form of escaping, or unquoted sections composed of non comma (actually seperator) and non quote data.

This means that Excel CSV files cannot be correctly parsed by a simple "read a line and seperate out the fields" approach. You need to know about the state of the previous line to determine if the current line is in fact part of a multiline record, or if it is not. The following code, as far as I can tell will handle CSV files correctly. However I personally would use a module, or if I really couldnt install a module (as anybody experienced knows, this is almost never really true for pure perl code) I would steal tillys code from Text::xSV and cut and paste it. But anyway, this was a fun exercise while I waited for something else to complete.

use strict; use warnings; use Data::Dumper; my $text=""; while (<DATA>) { $text.=$_; my @new; pos($text)=0; my $last=0; while ($text=~m{ \G (?: "((?:[^"]+|"")*)" # quoted section | ([^",]+) # unquoted | ((?=\s*,)) ) (?:\s*,\s*|\s*\z) }gmx) { push (@new, $+); $last=$+[0]; $new[-1]=~s/""/"/g; } if ($last==length($text)) { push (@new, "") if $text=~/,$/; print "Rec:".Data::Dumper->new([\@new])->Terse(1) ->Useqq(1)->Indent(0)->Dump()."\n"; $text=""; } } print "Error: $text\n" if $text; __DATA__ 1,the,simple,"case " "with","quoted" , "strings that contain spaces" "with","quoted" , "comma, internally" "with","quoted" , "comma, internally, with ""null"" data",,, """"""""

Incidentally it outputs:

Rec:[1,"the","simple","case\n"] Rec:["with","quoted","strings that contain spaces"] Rec:["with","quoted","comma,\ninternally"] Rec:["with","quoted","comma, internally, with \"null\" data","","",""] Rec:["\"\"\""]

---
demerphq

    First they ignore you, then they laugh at you, then they fight you, then you win.
    -- Gandhi


Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: note [id://306811]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others goofing around in the Monastery: (2)
As of 2024-04-25 19:08 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found