With some prodding from shmem, I was able to make progress, culminating in the code post above for "Update 2".
However, I still can't match the text I want, becuase the page fetched with WWW::Mech differs from the text I get when I open the url in firefox and do view source.
At any rate, to simply reiterate the test case for my problem (basically the same as Update 2 above, but shorter and simpler):
my $price = '249,9';
print "price: $price\n";
my $url = 'http://www.baur.de/is-bin/INTERSHOP.enfinity/WFS/Baur-BaurD
my $mech = WWW::Mechanize->new();
$mech->agent('Firefox/1.0 (Windows; U; Win98; en-US; Localization; rv:
+1.4) Gecko/20030624 Netscape/7.1 (ax)');
$mech->get( $url );
my $html = $mech->content;
print "html from $url doesn't match $price\n" unless $html =~ /$price/
print "but paste into browser and view source, and it does\n";
print "final url after firefox redirect (but not www::mech redirect) i
+s something like " . 'http://www.baur.de/is-bin/INTERSHOP.enfinity/WF
+lse' . "\n";
# uncomment this to print html, which is totally different from what y
+ou get from firefox, show source.
# print "html: $html";
Can anyone prod me further in the right direction?
PS: I'm still wondering if I would get better results with Win32::IE::Mechanize or WWW::Selenium or Mozilla::Mechanize.
UPDATE: better user agent string after perrin comment.
Are you posting in the right place? Check out Where do I post X? to know for sure.
Posts may use any of the Perl Monks Approved HTML tags. Currently these include the following:
<code> <a> <b> <big>
<blockquote> <br /> <dd>
<dl> <dt> <em> <font>
<h1> <h2> <h3> <h4>
<h5> <h6> <hr /> <i>
<li> <nbsp> <ol> <p>
<small> <strike> <strong>
<sub> <sup> <table>
<td> <th> <tr> <tt>
Snippets of code should be wrapped in
<code> tags not
<pre> tags. In fact, <pre>
tags should generally be avoided. If they must
be used, extreme care should be
taken to ensure that their contents do not
have long lines (<70 chars), in order to prevent
horizontal scrolling (and possible janitor
Want more info? How to link
or How to display code and escape characters
are good places to start.