Beefy Boxes and Bandwidth Generously Provided by pair Networks
No such thing as a small change

Re: How to extract links from a webpage and store them in a mysql database

by g0n (Priest)
on Dec 05, 2006 at 12:53 UTC ( [id://587856]=note: print w/replies, xml ) Need Help??

in reply to How to extract links from a webpage and store them in a mysql database

Hi syedahmed.uos,

Running your code and passing it a single URL, I get a single row in the table, with the base in column 'webpage', and a space separated set of fully qualified links in the 'links' column.

You say that you want to have each link in a separate row of the table. To do that, you'll need to iterate over the list of links. Something like this:

for my $link (@a) { $dbh->do("INSERT INTO htmllinks VALUES ('$base',' $link')"); }

Just making that change won't work because you've set 'webpage' to be the primary key, so you'll get an error because the value for 'webpage' is the same for every row. If you take the 'primary key' out of your table create, and put the loop in as above, you'll get something like this:

+---------------------------------------------------------+ | webpage | htmllinks | +---------------------------------------------------------+ | | | | | | +---------------------------------------------------------+

Is that what you're looking for?


"If there is such a phenomenon as absolute evil, it consists in treating another human being as a thing."
John Brunner, "The Shockwave Rider".

Replies are listed 'Best First'.
A reply falls below the community's threshold of quality. You may see it by logging in.

Log In?

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: note [id://587856]
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others browsing the Monastery: (2)
As of 2024-04-25 02:34 GMT
Find Nodes?
    Voting Booth?

    No recent polls found