Beefy Boxes and Bandwidth Generously Provided by pair Networks
There's more than one way to do things
 
PerlMonks  

Re: fork subroutine

by jweed (Chaplain)
on Apr 12, 2004 at 20:39 UTC ( [id://344516]=note: print w/replies, xml ) Need Help??


in reply to fork subroutine

I would write it like this. Basically, the change that I made is to have it so that the parent continues to fork off children to do its bidding, while the child downloads then exits. You were very close:
#!/usr/bin/perl -w use LWP::Simple; use strict; my @urls = qw!http://www.cnn.com/ http://www.bbc.com/ http://www.news.com/!; for my $url (@urls) { get_em($url); } sub get_em { my $url = shift; unless (my $pid = fork) { die "Couldn't fork on $url" unless defined $pid; die "Malformed url $url" unless $url =~ m!http://www.(.*?)/!; my $response = getstore($url, "$1.txt"); die "$url get failed" if is_error($response); exit; } }
Update: Moved the malformed url check
Update: See tilly's reply



Code is (almost) always untested.
http://www.justicepoetic.net/

Replies are listed 'Best First'.
Re: Re: fork subroutine
by tilly (Archbishop) on Apr 13, 2004 at 01:43 UTC
    For future reference, if you code up an example robot, please show how to fetch robots.txt and use http://WWW::RobotRules.

    It is unlikely that the person asking will know that this is important, and therefore it is our responsibility to pass along best practices.

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: note [id://344516]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others admiring the Monastery: (5)
As of 2024-04-25 10:42 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found