Beefy Boxes and Bandwidth Generously Provided by pair Networks
good chemistry is complicated,
and a little bit messy -LW
 
PerlMonks  

Re: How can I do this better?

by mwp (Hermit)
on Jan 19, 2001 at 07:10 UTC ( [id://52921]=note: print w/replies, xml ) Need Help??


in reply to How can I do this better?

For the lazy in you, allow me to introduce LWP::Simple:
use strict; use LWP::Simple qw(getstore head is_success); my @allowed = ('text/html', 'image/gif'); my $content = (head($url))[0]; die "Invalid content type! Please check URL.\n" unless grep($content =~ /\Q$_\E/, @allowed); FETCH: for(1..5) { if(is_success getstore($url, $output)) { printf "$output %d bytes OK\n", -s $output; last FETCH; } else { sleep 5; } } continue { die "Unable to fetch and store: $!\n$url => $output\n"; }
Not bad, eh? Let it do the work for you. :)

Update: Fixed a small bug, thanks chipmunk!

Update: spaz, try this on for size (LWP::Simple perldoc, approx line 148):

use LWP::Simple qw($ua); $ua->timeout(40); # set timeout, default 180

Replies are listed 'Best First'.
Re: Re: How can I do this better?
by spaz (Pilgrim) on Jan 19, 2001 at 22:53 UTC
    That's all well and good, but the reason I did it myself was so that I could set the timeout value. And I didn't see anyway to do that in man LWP::Simple.

    Any more help?

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: note [id://52921]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others browsing the Monastery: (8)
As of 2024-04-19 07:24 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found