comment on

Hai i have a huge table containing around 34748806 rows out of which there are duplicates as well so i need to get each and every value and check whether the value is existing in other table. But when i run the fetchrow_array value it takes too much of time(more than 18 hrs). So can any one suggest me with more fast method on how to do it. here is the snippet of the query which i have run

$data{RBS_query} = qq~select UserName from table where UserName <> '' 
+and UserName not like 'deleted%' order by UserName~;
#print "Query--> ".$data{RBS_query}."\n";
$data{cur} = $data{dbh}->prepare($data{RBS_query});
$data{cur}->execute or die($data{dbh}->errstr);
$data{num} = $data{cur}->rows;
print "RBS Rows--> ".$data{num}."\n";
while(my @rows = $data{cur}->fetchrow_array()){
     #print "Username --> ".$rows[0]."\n";
     if(grep {$_ eq $rows[0]} @RBS_User_Names){
     #  print "Value exist in array\n";
     }else{
        push(@RBS_User_Names,$rows[0]);
        #print "value doesnot exist in array\n";
     }
}
[download]

i have tried to use the distinct also in mysql query but still it was of no use. the program was running but no result was displaying even after long time. Any help would be grately appreciated. Thank you Regards Chaitanya

In reply to Fetch Mysql huge database by ym_chaitu

Are you posting in the right place? Check out Where do I post X? to know for sure.
Posts may use any of the Perl Monks Approved HTML tags. Currently these include the following:
<code> <a> <b> <big> <blockquote> <br /> <dd> <dl> <dt> <em> <font> <h1> <h2> <h3> <h4> <h5> <h6> <hr /> <i> <li> <nbsp> <ol> <p> <small> <strike> <strong> <sub> <sup> <table> <td> <th> <tr> <tt> <u> <ul>
Snippets of code should be wrapped in <code> tags not <pre> tags. In fact, <pre> tags should generally be avoided. If they must be used, extreme care should be taken to ensure that their contents do not have long lines (<70 chars), in order to prevent horizontal scrolling (and possible janitor intervention).
Want more info? How to link or How to display code and escape characters are good places to start.


Just another Perl shrine
	PerlMonks