DBI: pipe data to "load data local infile"?by richardwfrancis (Beadle)
|on Jan 18, 2010 at 09:59 UTC||Need Help??|
richardwfrancis has asked for the wisdom of the Perl Monks concerning the following question:
In a nutshell, I want to get large amounts of data in to my MySQL database from a text file. I'm using DBI to work with the database.
"Load data local infile" is obviously the way to go but my problem is that between reading in the text file and loading the data, I have to do a number of processing steps on the data.
I'd rather not have to write the processed data back to the filesystem and then use "Load data local infile" as my files can be quite large (~300MB). I would however like to use the speed of "Load data local infile" as opposed to insert statements.
Can anyone think of a way to do this?
Many thanks in advance,Rich