http://qs321.pair.com?node_id=984906

SilverSpring8 has asked for the wisdom of the Perl Monks concerning the following question:

O wise monks:

Noob (me) wants to extract a sports schedule from a website. Here's my code:

#!/usr/bin/perl -w use strict; use LWP::Simple; use HTML::TokeParser; my $url = 'http://www.georgiastatesports.com/SportSelect.dbml?DB_OEM_I +D=12700&SPID=5671&SPSID=53628'; my $raw = get($url); my $stream = HTML::TokeParser->new(\$raw) or die "$url $!"; while(my $token = $stream->get_token() ) { my $ttype = shift @{ $token }; if ($ttype eq "S") { my($tag, $attr, $attrseq, $rawtxt) = @{ $token }; next until ($tag eq "table"); my $class = $attr->{'class'}; if ($class eq "ScheduleTable") { my $sched = $stream->get_trimmed_text("/$tag"); print "$sched\n"; } # class eq "ScheduleTable" } # $ttype eq "S" } # while

What I hoped to return:

Date Opponent Location Time (ET) Results Overall CAA Media Thu, Aug 30 South Carolina State Georgia Dome 7:30 PM Sat, Sep 08 Tennessee at Knoxville, Tenn. 4:00 PM PPV Sat, Sep 15 UTSA Georgia Dome 6:00 PM Sat, Sep 22 Richmond * Georgia Dome 3:30 PM Sat, Sep 29 William & Mary * at Williamsburg, Va. 7:00 PM CSS-TV Sat, Oct 06 New Hampshire * Georgia Dome 3:30 PM Sat, Oct 13 Rhode Island * at Kingston, R.I. 1:00 PM Sat, Oct 20 Villanova (Homecoming) * Georgia Dome 3:30 PM Sat, Oct 27 James Madison * at Harrisonburg, Va. 3:30 PM Sat, Nov 03 Old Dominion * Georgia Dome 3:30 PM Sat, Nov 10 Maine * at Orono, Maine 2:00 PM

What I got:

Use of uninitialized value $class in string eq at ./gsu2.pl line 18. Use of uninitialized value $class in string eq at ./gsu2.pl line 18. Use of uninitialized value $class in string eq at ./gsu2.pl line 18. Use of uninitialized value $class in string eq at ./gsu2.pl line 18. Use of uninitialized value $class in string eq at ./gsu2.pl line 18. Date Opponent Location Time (ET) Results Overall CAA Media ? Thu, Aug +30 ?South Carolina State Georgia Dome ? 7:30 PM ? Sat, Sep 08 ?Tennes +see at Knoxville, Tenn. ? 4:00 PM PPV ? Sat, Sep 15 ?UTSA Georgia Dom +e ? 6:00 PM ? Sat, Sep 22 ?Richmond * Georgia Dome ? 3:30 PM ? Sat, S +ep 29 ?William & Mary * at Williamsburg, Va. ? 7:00 PM CSS-TV ? Sat, +Oct 06 ?New Hampshire * Georgia Dome ? 3:30 PM ? Sat, Oct 13 ?Rhode I +sland * at Kingston, R.I. ? 1:00 PM ? Sat, Oct 20 ?Villanova (Homecom +ing) * Georgia Dome ? 3:30 PM ? Sat, Oct 27 ?James Madison * at Harri +sonburg, Va. ? 3:30 PM ? Sat, Nov 03 ?Old Dominion * Georgia Dome ? 3 +:30 PM ? Sat, Nov 10 ?Maine * at Orono, Maine ? 2:00 PM Use of uninitialized value $class in string eq at ./gsu2.pl line 18. Use of uninitialized value $class in string eq at ./gsu2.pl line 18. Use of uninitialized value $class in string eq at ./gsu2.pl line 18. Use of uninitialized value $class in string eq at ./gsu2.pl line 18.

A hexdump where the newlines should be shows a character sequence of \0x20\0xA0\0x20. I tried

local $/ = \0xA0;

between get_trimmed_text() and print, without any apparent effect.

How would I get newlines before each date and handle the "uninitialized" warnings? I await your wisdom.

Replies are listed 'Best First'.
Re: TokeParser and newlines
by suaveant (Parson) on Aug 02, 2012 at 00:27 UTC
    Well, this isn't exactly right but it'll move you along, don't have time to continue. Probably finding a more comprehensive parser as previously suggested would help
    #!/usr/bin/perl -w use strict; use LWP::Simple; use HTML::TokeParser; my $url = 'http://www.georgiastatesports.com/SportSelect.dbml?DB_OEM_I +D=12700&SPID=5671&SPSID=53628'; my $raw = get($url); my $stream = HTML::TokeParser->new(\$raw) or die "$url $!"; while(my $token = $stream->get_token() ) { my $ttype = shift @{ $token }; if ($ttype eq "S") { my($tag, $attr, $attrseq, $rawtxt) = @{ $token }; next until ($tag eq "table"); my $class = $attr->{'class'}; if ($class eq "ScheduleTable") { while($stream->get_tag('tr')) {# my $sched = $stream->get_trimmed_text('/tr'); print "$sched\n"; } } # class eq "ScheduleTable" } # $ttype eq "S" } # while

                    - Ant
                    - Some of my best work - (1 2 3)

Re: TokeParser and newlines
by Anonymous Monk on Aug 01, 2012 at 22:33 UTC

      Thanks for the advice and the links. I'll check it out!