Welcome to the Monastery | |
PerlMonks |
Re^7: collect data from web pages and insert into mysqlby wfsp (Abbot) |
on Aug 02, 2010 at 14:23 UTC ( [id://852478]=note: print w/replies, xml ) | Need Help?? |
When you go to the first page the top right side says, "PAGE 2 >". When you click on that you're on page 2. Then the top right hand side says, "PAGE 3 >".
On page three there is nothing (there isn't a next page).
What that sub (get_next_page) does is to check if there is a link to a next page. If there is it returns the page number and that is the page that is processed next. If there isn't a page number it returns undef and that exits you out of the loop. With hindsight I should have called the sub get_next_page_number because that is what it is doing (it's not loading the page). The sub (get_sids) returns a list of all the sids. I reckon it would be simplest to do that and then decided which ones you want. grep might help with that. A tab delimited record sounds as thought it would do fine. By the way, there are, in this case, three calls to the website. So you have to give it a moment to finish. Let us know how you get on.
In Section
Seekers of Perl Wisdom
|
|