my code error-ing out trying to grab html

RayRay459 has asked for the wisdom of the Perl Monks concerning the following question:

Fellow monks, i am in need of your help. I am trying to write a script that will look though a list of urls and open them, grab the html and stuff it into an array and look for a string and print the string. I usually start small with the scripts and add in more to make sure the core of it is working. I have this code so far:

#!/usr/bin/perl

use HTTP::Request;
use HTTP::Headers;
use LWP::UserAgent;

my($req, $ua, $responsecode, $res, $row, $url);


    $url = "http://listings.test.com/aw/listings/list/category12";
    $request = new HTTP::Request('GET',$url);
    $ua->timeout(10);
    $response = $ua->request($request);
    my $responsecode = $response->code();
     next if $responsecode != 200;
                    
        #} 
        @ARRAY_OF_LINES = (split "\n", $ua->request($request)->as_stri
+ng);
         $request->as_string;
         foreach $row (@ARRAY_OF_LINES) {
        chomp($row);
        }
        if ($row =~ /Updated/){
            print $1;            
}else{
                            next;
                            }
[download]

I get this error when i run this:

Can't call method "request" on an undefined value at url.pl line 7.
[download]

Any advice that you have for me would be greatly appreciated.
Thanks in advance.
Ray

Comment on my code error-ing out trying to grab html Select or Download Code

Replies are listed 'Best First'.
Re: my code error-ing out trying to grab html by mdillon (Priest) on Mar 12, 2002 at 08:18 UTC
try adding the following after the variable declarations: `$ua = LWP::UserAgent->new;`	[reply] [d/l]
Re: Re: my code error-ing out trying to grab html by RayRay459 (Pilgrim) on Mar 12, 2002 at 09:00 UTC
thank you for your help. that fixed the error. Ray	[reply]
Re: my code error-ing out trying to grab html by RayRay459 (Pilgrim) on Mar 12, 2002 at 08:59 UTC
per the advice given to me, i added $ua = LWP::UserAgent->new; after my variable declaration. The fixed the previous error (thanks so much)However, now my problem lies within my regex. Here's the updated code: #!/usr/bin/perl use HTTP::Request; use HTTP::Headers; use LWP::UserAgent; my(@ListingsUrls, $req, $ua, $responsecode, $res, $row, $url); $ua = LWP::UserAgent->new; $url = "http://listings.test.com/aw/listings/list/category12"; $request = new HTTP::Request('GET',$url); $ua->timeout(10); $response = $ua->request($request); my $responsecode = $response->code(); next if $responsecode != 200; @ARRAY_OF_LINES = (split "\n", $ua->request($request)->as_stri +ng); $request->as_string; foreach $row (@ARRAY_OF_LINES) { chomp($row); print $row . "\n"; } if ($row =~ (/Updated\s:\s\w+\s-\s\d{1,2}:\d{1,2}\d{1,2}\s +* PST/)){ print $1; }else{ print "html didn't contain Updated"; } [download] I added a print statement to $row and i can see what i am looking for(this is an excerpt from my console when the line print $row is executed): `<tr> <td> <p></p><br> <center> <font face="Arial, Helvetica" size="-1"> <b>Updated: Mar-11 23:05:50 PST</b>` [download] Any advice would be greatly appreciated. thnx, Ray	[reply] [d/l] [select]
Re: Re: my code error-ing out trying to grab html by davis (Vicar) on Mar 12, 2002 at 09:33 UTC
Hi, You were correct - the regex isn't matching. Here's a fixed-up and /x modified version: `#!/usr/bin/perl -w use strict; my $row = "Updated: Mar-11 23:05:50 PST"; if($row =~ /Updated\s: #The String "Updated", followed b +y zero or more spaces, then a colon \s\w+\s-\s\d{1,2}\s* #This matches " Mar-11 " \d{1,2}:\d{1,2}:\d{1,2} #The time \sPST/x) { #The timezone - do you really nee +d to be this specific? print "Match\n"; } else { print "No Match\n"; }` [download] On the other hand: should you really be using a regex for this at all? If it's a date, then Date::Calc could do very well for you (check out parse_date). I'd also recommend thinking about whether you really need to have " PST" in your regex at all - do all the "Updated" strings contain that timezone information? hope this helps davis Is this going out live? No, Homer, very few cartoons are broadcast live - it's a terrible strain on the animator's wrist Update:* On re-reading your code, it appears that you're using parentheses to try and capture the whole string - these need to go inside the regex delimeters. And I won't even mention that you should be "use"ing "strict" and "warnings" ;-)	[reply] [d/l]
Re: Re: Re: my code error-ing out trying to grab html by zengargoyle (Deacon) on Mar 12, 2002 at 09:54 UTC
Break out the bulky, slow Date::Manip. `... use Date::Manip; if (/>Updated: (.*?)</ && $date = ParseDate($1)) { &do_something_with($date); } else { warn "no parsum date!\n"; &fail_gracefully; }` [download] Date::Manip does have the advantage of parsing almost anything that can be a date.	[reply] [d/l]


good chemistry is complicated, and a little bit messy -LW
	PerlMonks