sailortailorson has asked for the wisdom of the Perl Monks concerning the following question:
O Denizens of the Cloister of the Fruit of the Oyster,
Using LWP::UserAgent, when I look at the content of a page that has lots of Ajaxy stuff, I see the Javascript, but not the results of the Javascript.
In some cases, I need to see the final page, after all the Javascript has altered the page. Is there a way to get this from the U.A.?
For the present, I have switched over to using IE as driven by Win32::OLE, but if it is possible to do in LWP, or some other more Perlish way, I would rather do that.
Also (in a somewhat related question) I have searched CPAN for 'Greasemonkey' related scripts, but find nothing. Is anyone doing anything that simulates Greasemonkey for LWP scripts?
Just another Nabob of Nacre
Re: LWP and Javascript
by Joost (Canon) on Mar 29, 2007 at 22:28 UTC
|
Do you just want to look at the new stuff DOM nodes created by the scripts or do you want to control the whole process with perl. If you just want to look (and mess about by hand) check out the firebug firefox extension.
If you want perl control, check out Mozilla::Mechanize::GUITester and/or Mozilla::Mechanize or WWW::Selenium. update2: Win32::IE::Mechanize
update: just to be clear: LWP does not support javascript at all, and as far as I know no-one is planning on adding javascript support. All options I linked drive conventional GUI browsers since that's a lot easier to build and support than creating your own compliant and effectively useful DOM/BOM with javascript support. I should know - I gave it a try once :-)
| [reply] |
|
Joost,
Thanks. This all looks like the kind of things I need to add to a general toolbox for making webpages jump through hoops, mostly for automating testing, and computer aided manual testing. The specific task I wanted to solve is counting sponsored links (ads) on a web page. I cannot see them in LWP, only the Ajax that goes and gets them.
For some reason, I have not even tried the mechanize modules. I guess I thought I was getting more control from LWP. I just heard about Selenium last week for the first time, so I will take a look at that too.
In regard to your note about LWP, that confirms the sense I had about it. So I know now not to bark up that tree anymore.
Thanks
| [reply] |
|
| [reply] |
|
|