Beefy Boxes and Bandwidth Generously Provided by pair Networks
Syntactic Confectionery Delight
 
PerlMonks  

Submitting form data to Javascript form

by gebelo (Initiate)
on Mar 25, 2005 at 18:31 UTC ( [id://442388]=perlquestion: print w/replies, xml ) Need Help??

gebelo has asked for the wisdom of the Perl Monks concerning the following question:

I'm attempting my first complicated scraping project and have hit a wall. To get at the data I want to scrape, I need to submit several forms first. I'm using www:mech and got through the first form easy:
$browser->get( $starting_url ); $browser->form_name("GS"); $browser->submit();
This takes me to a new page with another form -- but this one has a Javascript "onClick" routine, and when I try various permutations of the following, it doesn't accept my input: This is the form code:
<form name="gs" METHOD="post" action="" onSubmit="cntyzip()" > <table width=70% align="center"> <tr> <td align=center colspan=2> You must either choose a <b>County < +font color="#FF0000"> &nbsp;or&nbsp;</font></b> enter a <b>Zip Code +</b></td></tr> <tr><td>&nbsp;</td></tr> <tr> <td colspan="2"> <table width = 76% align="center"> <tr> <td width="16%" align=left><font size=2>County:</font></td +> <td width="30%" align=left><font size=2> <select name="county"> <option value="">Choose a county</option> <option value="01ATLANTIC">ATLANTIC</option>
And these are two of the methods I've attempted to get the thing to go:
$browser->form_name( "gs" ); $browser->set_fields( county => '01ATLANTIC' ); $browser->submit(); # $browser->form_name("gs"); # $browser->field( "county", "01ATLANTIC" ); # $browser->cntyzip();
I've tried this in both regular mech and the Win32::ie version, but no luck... Any advice?

Replies are listed 'Best First'.
Re: Submitting form data to Javascript form
by Kanji (Parson) on Mar 25, 2005 at 19:52 UTC

    Find out what cntyzip() actually does and rewrite your HTML as appropriate before form submitting.

    $browser->get( $starting_url ); ## Reverse engineer cntryzip() here. my $html = $brwoser->content; $html =~ s/" onSubmit="cntyzip()"/cntyzip.cgi"/isg; ## Insert de-JS'd version here. $browser->update_html($html); $browser->form_name("GS"); $browser->submit();
    Update: s/mech/browser/; # oops

        --k.


      This is the solution I came up with, with the help of a friend.. it basically re-writes the form HTML and bypasses the Java script

      In this case, the original search was by county and $line = the value I pulled in from a list...

      my $form = $browser->form_name( 'formname' ); $form->action( 'https://destinationurl' ); $form->value( county => $line ); $browser->request( $form->click( 'buttonname' ) );

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: perlquestion [id://442388]
Approved by b10m
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others chilling in the Monastery: (4)
As of 2024-03-28 23:18 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found