Beefy Boxes and Bandwidth Generously Provided by pair Networks
Just another Perl shrine
 
PerlMonks  

Re: CSV manipulation with perl

by Tux (Canon)
on Mar 09, 2009 at 14:14 UTC ( [id://749292]=note: print w/replies, xml ) Need Help??


in reply to CSV manipulation with perl

Don't! Use Text::CSV_XS instead (or Text::CSV if you do not have a C-compiler)

use DBI; use Text::CSV_XS; my $dbh = DBI->connect (...); my $sth = $dbh->prepare ("insert into table values (?, ?, ?, ...)"); my $csv = Text::CSV_XS->new ({ binary => 1 }); open my $dta, "<", "data.csv" or die "data: $!\n"; while (my $row = $csv->getline ($dta)) { $sth->execute (@$row); } $csv->eof or $csv->error_diag; $sth->finish; $dbh->commit;

Enjoy, Have FUN! H.Merijn

Replies are listed 'Best First'.
Re^2: CSV manipulation with perl
by Bloodnok (Vicar) on Mar 09, 2009 at 14:17 UTC
    ... or even Text::xSV.

    A user level that continues to overstate my experience :-))
Re^2: CSV manipulation with perl
by CountZero (Bishop) on Mar 09, 2009 at 19:41 UTC
    Doesn't Text::CSV choose either Text::CSV_XS or Text::CSV_PP depending on what is available on your system? I definitely remember you saying so at YAPC::EU 2008 in Copenhagen.

    BTW, are the slides of that presentation somewhere available? I use Text::CSV quite a lot and every ounce of speed I can wring out of it helps.

    CountZero

    A program should be light and agile, its subroutines connected like a string of pearls. The spirit and intent of the program should be retained throughout. There should be neither too little or too much, neither needless loops nor useless variables, neither lack of structure nor overwhelming rigidity." - The Tao of Programming, 4.1 - Geoffrey James

      Correct, but if you only install Text::CSV, it doesn't automatically install Text::CSV_XS, so you don't get the speed, so I keep promotong the latter

      Talk is here. Opera browser might be required, as it depends on <link rel="next" type="text/html" href="..." /> in the <head> section, which Opera supports with FastForward. To skip directly to the first slide, click here.


      Enjoy, Have FUN! H.Merijn
Re^2: CSV manipulation with perl
by joec_ (Scribe) on Mar 09, 2009 at 14:27 UTC
    Does this code take 123, 124 and 125 from the csv create a string "123|124|125" and then insert into the database? And then do the same for text, text1, text2 etc etc

    Doesnt appear to... if it does, please could you explain how? At the moment, i know it might not be the best "way" of doing things but that is what i got....thanks.

    -----

    Eschew obfuscation, espouse eludication!

      No, this was just an example of how to mix CSV and DBI

      Your problem could be translated to something like

      my $csv = Text::CSV_XS->new ({ binary => 1 }); open my $dta, "<", "data.csv" or die "data: $!\n"; my @rows; while (my $row = $csv->getline ($dta)) { push @rows, $row; } $csv->eof or $csv->error_diag; close $dta; # pivot my @data = map { my $col = $_; join "|", map { $_->[$col] } @rows } 0. +.$#{$rows[0]};

      Which will convert

      123,foo,2,FOO 234,bar,2,BAR 345,zap,3,ZAP 456,bok,5,BOK

      into

      ( "123|234|345|456", "foo|bar|zap|bok", "2|2|3|5", "FOO|BAR|ZAP|BOK")

      Assuming all records have equal length


      Enjoy, Have FUN! H.Merijn

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: note [id://749292]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others cooling their heels in the Monastery: (6)
As of 2024-03-28 21:23 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found