Trying to automate POST with LWP

andarian has asked for the wisdom of the Perl Monks concerning the following question:

Monks hello,
I've been trying to use the LWP module to automate POST. I've managed to get a response object and it looks like the POST has successed in writing the text I told it to into the proper fields. The problem is that it looks that all needs to be done now is to press the "Submit" button, which is a javascript submit function and i don't know how to interface with it. Looking at the HTML source I found that button and it looks like this

<tr style="padding-top:8px;padding-bottom:3px;">

                                        <td colspan="3" style="padding
+-right:155px;">

                                        <a href="javascript:submitFina
+l();">

                                        <img src="images/buttons/b_sea
+rch.gif" border=0></td>

                                        </a>

                                    </tr>
[download]

Looking at the start of HTML i found that function, which checks a few conditions and if those are OK it will return a new page, the part that returns a new page looks like this

document.forms.frmData.action="PhoneNumResults.asp"

document.forms.frmData.submit();
[download]

The purpose of my program is to recieve that "PhoneNumResults.asp" and procces it.
Please help

20051020 Janitored by Corion: Added formatting

Comment on Trying to automate POST with LWP Select or Download Code

Replies are listed 'Best First'.
Re: Trying to automate POST with LWP by swares (Monk) on Oct 20, 2005 at 11:31 UTC
It sounds like you may know how to use LWP but do not know what is being passed between the client and server due to the javascript that is used. If that is the case you could use a proxy such as burp to intercept the requests and responses when you access the site with a browser to figure out what is really required and received when the site is accessed. Then you can craft your perl code with that information.	[reply]
Re: Trying to automate POST with LWP by Trimbach (Curate) on Oct 20, 2005 at 12:51 UTC
Try posting the code you're using right now, and let us look at it. POST'ing from LWP is not hard, but it does take a little getting used to if you're not familiar with it. Gary Blackburn Trained Killer	[reply]
Re: Trying to automate POST with LWP by planetscape (Chancellor) on Oct 21, 2005 at 06:52 UTC
WWW::Mechanize is much easier to use than LWP. Some tools that can assist you in creating scripts for WWW::Mechanize (which has its own cookbook) include HTTP::Recorder and Ethereal (the latter being a Network Protocol Analyzer which lets you see exactly what your browser is communicating to a site and vice-versa.) castaway suggested Ethereal to me a few weeks ago when I was stumped on a web-automation problem. An article that will help you get started with HTTP::Recorder may be found here. If you have any further questions about HTTP::Recorder, its author, leira is one very helpful PerlMonk. :-) HTH, planetscape	[reply]


Clear questions and runnable code get the best and fastest answer
	PerlMonks