web mechanize issue with javascript...

perlknight has asked for the wisdom of the Perl Monks concerning the following question:

All, I am trying to automate upload of pics to "http://www1.snapfish.com/welcomenphu". But I am having issue with javascript. First cut at automating the login with web mechanize:

use WWW::Mechanize;
#use HTML::TokenParser;

my $agent = WWW::Mechanize->new;
$agent->get("http://www1.snapfish.com/welcomenphu");
$agent->form("form1");
$agent->field("emailaddress","me@mykingdom");
$agent->field("password","123");
$agent->click();
print "$agent->{content}";
[download]

The next page, if I login successfully will have my photo album dir. However, this is not the case; I dump out the output in html format and used firefox to view it. It prompts me that the account already exists and asked if I want to create another account or click the login button. When I click on the login button, it brings me to the page where my album dir is located. I am thinking it's a javascript issue. Any idea? Thanks.

Comment on web mechanize issue with javascript... Download Code

Replies are listed 'Best First'.
Re: web mechanize issue with javascript... by dragonchild (Archbishop) on May 01, 2006 at 03:43 UTC
WWW::Mechanize doesn't do Javascript. Now, the IE Mechanize might because it's using OLE to actually run an IE instance. But, the WWW::Mechanize doesn't do JS. Of course, patches are always welcome. My criteria for good software: Does it work? Can someone else come in, make a change, and be reasonably certain no bugs were introduced?	[reply]
Re^2: web mechanize issue with javascript... by bart (Canon) on May 01, 2006 at 08:06 UTC
Yes, Win32::IE::Mechanize does do Javascript. It behaves a little oddly: code like `document.write(...);` includes both the original Javascript, as the generated result, in the "source". So it can't be a true compatible replacement for WWW::Mechanize... Once you take that behaviour into account, it works out very well. An obvious limitation is that it only works on Windows, and you need an install of MSIE. But, given the former, the latter comes as a given. ;-)	[reply] [d/l]
Re: web mechanize issue with javascript... by perrin (Chancellor) on May 01, 2006 at 05:21 UTC
It sounds like you're jumping to conclusions. What reason do you have for thinking it's javascript? If you turn off javascript in Firefox, can you access the page you're trying to get to? More likely, you're just hitting the wrong form or missing a field. You can see the exact data sent by Firefox with the LiveHTTPHeaders extension.	[reply]
Re: web mechanize issue with javascript... by crashtest (Curate) on May 01, 2006 at 05:00 UTC
A quick look at http://www1.snapfish.com/welcomenphu shows that the login information is posted via https. Do you have OpenSSL installed, or have Perl set up for HTTPS some other way? I suspect the SSL is your problem, not JavaScript in this case (but I didn't look too closely). Hope this helps.	[reply]
Re: web mechanize issue with javascript... by Anonymous Monk on May 01, 2006 at 03:52 UTC
Learn to search: Mechanize and javascript (yes, another) WWW::Mechanize and javascript Knocking Down Pesky JavaScript Popups from WWW::Mechanize Clicking a checkbox using www::mechanize Re: WWW::Mechanize for nonexistent fields?	[reply]
Re: web mechanize issue with javascript... by teabag (Pilgrim) on May 03, 2006 at 09:20 UTC
Hey perlknight, I had a similar problem recently. I had to deal with javascript on a page and I wanted to use WWW::Mechanize. Checking the monastery gave me some help, but didn't give me a specific answer, just pointers towards a long and winding road. Browsing though CPAN I stumbled upon Win32::IEAutomation, web application automation using Internet Explorer. Now I don't know if your code should run on windows or an other platform. Since Win32::IEAutomation uses Win32::OLE to create automation object for IE, it's only usefull on windows. But I had a script whipped up in a couple of minutes. from the docs: It drives internet explorer using its DOM properties and methods. The module allows user to interact with web page controls like links, buttons, radios, checkbox, text fields etc. It also supports frames, popup window and interaction with javascript dialogs like security alert, confirmation, warning dialog etc. There's no ppm for it at activestate, but if you download it, untar it and just put the folder in your Win32 folder in your perl directory it will function perfectly. Good luck, teabag Blessed is the end user who expects nothing, for he/she will not be disappointed.	[reply]


Welcome to the Monastery
	PerlMonks