Beefy Boxes and Bandwidth Generously Provided by pair Networks
Problems? Is your data what you think it is?
 
PerlMonks  

Peculiar Behaviour with DBI and MySQL

by Anonymous Monk
on Jul 22, 2004 at 13:55 UTC ( [id://376570]=perlquestion: print w/replies, xml ) Need Help??

Anonymous Monk has asked for the wisdom of the Perl Monks concerning the following question:

Hello Monks, Long time fan, first time poster. I have a Perl script that collects a webpage and picks out particular pieces of code in order to draw up an array of links. These links are then inserted into a MySQL database. Next, a 'foreach' loop takes each URL from the array and analyses that page creating a hash where the hash key is the column name in a MySQL table and the hash value is the entry for that column. Finally, another SQL statement, via DBI, updates the database with all of the hash values. I had this working fine before but now, after moving from my dev. server to a real one, find that the script stops updating the database after ten of these updates (11 updates in total including the initial one). I appreciate that this could be a MySQL problem but the setup for the two servers appears to be identical so I suspect it may be the Perl. Any help greatly appreciated. Thanks Chris

Replies are listed 'Best First'.
Re: Peculiar Behaviour with DBI and MySQL
by jlongino (Parson) on Jul 22, 2004 at 14:27 UTC
    You might want to check the execution time/cpu limits for both MySQL and the operating system on the production server. I don't delve deeply into MySQL and don't know what OS you're running, but this would be a good place to start. Even if it's not the problem, you should be aware of what these values are.
Re: Peculiar Behaviour with DBI and MySQL
by ChrisJ-UK (Novice) on Jul 22, 2004 at 13:59 UTC

    Sorry, I screwed up the formatting.
    This may be a little easier to read:

    Hello Monks,
    Long time fan, first time poster.

    I have a Perl script that collects a webpage and picks out particular pieces of code in order to draw up an array of links. These links are then inserted into a MySQL database.

    Next, a 'foreach' loop takes each URL from the array and analyses that page creating a hash where the hash key is the column name in a MySQL table and the hash value is the entry for that column.

    Finally, another SQL statement, via DBI, updates the database with all of the hash values. I had this working fine before but now, after moving from my dev. server to a real one, find that the script stops updating the database after ten of these updates (11 updates in total including the initial one).

    I appreciate that this could be a MySQL problem but the setup for the two servers appears to be identical so I suspect it may be the Perl.

    Any help greatly appreciated.

    Thanks Chris

      If you could post your code, we would help. Could you provide the code that interacts with MySQL? Specifically the code preceding the INSERT.

        Thanks both of you for the quick replies.

        The code for updating the database is shown below, jlongino: thanks for the pointer, I'm not much of an admin and it all looks to be set up the same. I'll keep on it though.

        %hash: $ColName => $Value
        The subroutine below is called with $page_url to tell it where to apply the update. The hash is global.

        sub update_database { my($page_url)=@_; my(@categorys); my($category); my($dbfield); @categorys = keys(%hash); foreach $category (@categorys) { # Key is prefixed to match database $dbfield="Prefix".$category; # Establish database connection with MySQL my $dsn = 'DBI:mysql:*database*:localhost'; my $db_user_name = '*username*'; my $db_password = '*password*'; my $dbh = DBI->connect($dsn, $db_user_name, $db_password); my $sth = $dbh->prepare(qq{ update table set $dbfield = '$hash{$category}' where pageu +rl = '$page_url' }); $sth->execute(); $sth->finish(); } }
Re: Peculiar Behaviour with DBI and MySQL
by ChrisJ-UK (Novice) on Jul 23, 2004 at 11:10 UTC

    Thanks very much for the help you all offered.

    I have since realised that it is something else causing the problems, not the database.

    I've moved on to a different query now.

    Thanks again

    Chris

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: perlquestion [id://376570]
Approved by ysth
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others exploiting the Monastery: (2)
As of 2024-04-16 20:32 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found