Beefy Boxes and Bandwidth Generously Provided by pair Networks
The stupid question is the question not asked
 
PerlMonks  

Re: finding urls in a string

by Vautrin (Hermit)
on Apr 07, 2004 at 17:11 UTC ( [id://343342]=note: print w/replies, xml ) Need Help??


in reply to finding urls in a string

Others have mentioned how to find URLs, but you may want to double check whether or not the URLs are actually valid, using a module like LWP::Simple or LWP::UserAgent For instance, for all urls @urls you find, you might do something like:

# assuming you've already populated @urls # and done: use LWP::UserAgent; use strict; use warnings; # try this: my @old_urls = @urls; @urls = (); my $user_agent = LWP::UserAgent->new; while (@old_urls) { my $url = shift (@old_urls); my $response = $user_agent->get($url); if ($response->is_success) { push @urls, $url; # or, if you want to get more detailed: # push @urls, { # url => $url, # type => $response->content_type, # }; } }

Want to support the EFF and FSF by buying cool stuff? Click here.

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: note [id://343342]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others surveying the Monastery: (5)
As of 2024-04-23 08:28 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found