When I have done this type of task, I've broken it up into several discrete portions and stitched them together using a shell script. Right now, I have a daily load from a sub-optimally normalized Oracle database into a MySQL database (for generating reports). I use the following process:
- Oracle's SQL*Plus connections kicked off from command-line to generate tab-delimited data files
- A cleanup step to make sure all backslashes are backslashes, there's a \N for empty datafields, and the like. This a shell script that calls Perl in commandline.
- MySQL's client is kicked off in batch mode. I have to do some massaging of the data, as well, so I need to use MySQL.
While all of those things could've been done in Perl, it's a lot quicker to do it in the native client. And, the reason I wrote the second step as a shell script around commandlines is that it's easier to do
foreach f in *.xsv
perl -pi -e 's/asdf/jkl;/g' $f
than it is for me to open the file, loop through all the lines, and write the file back out. Why make it harder for yourself?
We are the carpenters and bricklayers of the Information Age.
Then there are Damian modules.... *sigh* ... that's not about being less-lazy -- that's about being on some really good drugs -- you know, there is no spoon. - flyingmoose
I shouldn't have to say this, but any code, unless otherwise stated, is untested
Are you posting in the right place? Check out Where do I post X? to know for sure.
Posts may use any of the Perl Monks Approved HTML tags. Currently these include the following:
<code> <a> <b> <big>
<blockquote> <br /> <dd>
<dl> <dt> <em> <font>
<h1> <h2> <h3> <h4>
<h5> <h6> <hr /> <i>
<li> <nbsp> <ol> <p>
<small> <strike> <strong>
<sub> <sup> <table>
<td> <th> <tr> <tt>
Snippets of code should be wrapped in
<code> tags not
<pre> tags. In fact, <pre>
tags should generally be avoided. If they must
be used, extreme care should be
taken to ensure that their contents do not
have long lines (<70 chars), in order to prevent
horizontal scrolling (and possible janitor
Want more info? How to link
or How to display code and escape characters
are good places to start.