in reply to DBI: speed up insertions in generating database

Hah. This is simple - use a hash for storing prepared statements. Untested pseudocode:
while (my $line = FILE) { analyze $line to extract ($table, $value) pairs foreach ($table, $value) pair { my $sql_statement = "INSERT INTO $table VALUES(?)"; my $sth = store_or_retrieve($sql_statement); $sth->execute($value); } } my $sth_hashref; sub store_or_retrieve { my $sql_statement = shift; if (exists $sth_hashref->($sql_statement)) { return $sth_hashref->{sql_statement; } else { my $sth = $dbh->prepare($sql_statement); $sth_hashref->{$sql_statement} = $sth; return $sth; } }
However, I would suggest you probably have more work to do regarding your database structure - do you really only have one row in each table?