|Pathologically Eclectic Rubbish Lister|
Big database queriesby ezekiel (Pilgrim)
|on Aug 30, 2002 at 00:14 UTC||Need Help??|
ezekiel has asked for the wisdom of the Perl Monks concerning the following question:
I have a really simple script that queries a database then loops over each row like this:
There is nothing fancy here except the query brings back 60 million rows! As a result, when I try to run the script it brings the database and machine running the database to its knees during the execution of the SQL statement.
Short of database level things or memory upgrades (that are largely outside my control) does anyone have any suggestions for handling these big queries in a more effective manner?
One thought I had was to break the query up into smaller chunks as such:
but that seems ugly to me. Surely there must be a more elegant solution?