hacker,
This seems extremely straight forward.
Step 1:
To fetch the first page, I would probably use WWW::Mechanize provided that you comply with the robots.txt of the site.
Step 2:
To parse the directory listing page, I would use HTML::Parser, HTML::TokeParser::Simple, or HTML::TableContentParser - depending on what is appropriate.
Step 3:
The directories, if in the YYYYMMDD format as you describe, will sort correctly without the use of any module. Unless the listing was more than a few dozen entries - I wouldn't even bother with a modified water mark algorithm (I would just sort). If you do end up needing to use a date module, DateTime is the way to go.
Step 4:
Again, I would use WWW::Mechanize to go into the desired directory and get a contents listing. It should be easy to use the links() method to retrieve a list of files and then a simple regex to filter that list for files containing the date in question.
Step 6:
This is that hard part. I do not know of any modules on CPAN that allow you to resume downloads so you may have to implement this feature yourself. You could try WWW::Curl which is a wrapper module. In any case, this shouldn't be too difficult.
-
Are you posting in the right place? Check out Where do I post X? to know for sure.
-
Posts may use any of the Perl Monks Approved HTML tags. Currently these include the following:
<code> <a> <b> <big>
<blockquote> <br /> <dd>
<dl> <dt> <em> <font>
<h1> <h2> <h3> <h4>
<h5> <h6> <hr /> <i>
<li> <nbsp> <ol> <p>
<small> <strike> <strong>
<sub> <sup> <table>
<td> <th> <tr> <tt>
<u> <ul>
-
Snippets of code should be wrapped in
<code> tags not
<pre> tags. In fact, <pre>
tags should generally be avoided. If they must
be used, extreme care should be
taken to ensure that their contents do not
have long lines (<70 chars), in order to prevent
horizontal scrolling (and possible janitor
intervention).
-
Want more info? How to link
or How to display code and escape characters
are good places to start.
|