Re^3: Counting fields from database

Sorry, but your code is not runnable, and doesn't seem to be representative of the problem you're having. Please see How do I post a question effectively?, I know what I mean. Why don't you?, and Short, Self-Contained, Correct Example.

This is a guess of what you want that I've pieced together from your various posts.

use warnings;
use strict;
use Data::Dump;
use DBI;

my $titlesperpage = 10;

# set up a dummy database
my $dbh = DBI->connect("dbi:SQLite::memory:", undef, undef,
    { RaiseError=>1 });
$dbh->do(<<'END');
CREATE TABLE messages (
    msgID INTEGER,
    threadID INTEGER
)
END
my $sth_i = $dbh->prepare(
    'INSERT INTO messages (msgID, threadID) VALUES (?,?)');
$sth_i->execute($_,$_>>4) for 0..127;

for my $currentpage (1..5) { # simulate requests for different pages
    
    my $startcount = ($currentpage - 1) * $titlesperpage + 1;
    my $stopcount = $currentpage * $titlesperpage;
    dd $startcount, $stopcount;
    
    my $sth_s = $dbh->prepare('SELECT msgID, threadID FROM messages');
    $sth_s->execute;
    my %threadIDs;
    my $rowcnt = 1;
    while ( my $row = $sth_s->fetchrow_hashref ) {
        $threadIDs{ $row->{threadID} }++;
        if ( $rowcnt >= $startcount && $rowcnt <= $stopcount ) {
            dd $row->{msgID};
        }
    } continue { $rowcnt++ }
    dd \%threadIDs;
    
}
[download]

But this loops through all results for every request. You can have the database do the pagination, even though you have to hit the database twice it still doesn't require you to loop through all records, only those for the current page.

for my $currentpage (1..5) { # simulate requests for different pages
    
    my $startcount = ($currentpage - 1) * $titlesperpage + 1;
    
    dd $dbh->selectall_arrayref(
        'SELECT threadID, COUNT(*) FROM messages GROUP BY threadID');
    
    my $sth_s = $dbh->prepare('SELECT msgID, threadID FROM messages '
        .'ORDER BY msgID LIMIT ?,?');
    $sth_s->execute($startcount, $titlesperpage);
    while ( my $row = $sth_s->fetchrow_hashref ) {
        dd $row->{msgID};
    }
    
}
[download]

Comment on Re^3: Counting fields from database Select or Download Code

Replies are listed 'Best First'.

Re^4: Counting fields from database - (Updated)
by htmanning (Friar) on Oct 03, 2020 at 21:34 UTC

This script was written years ago in Perl 4 and I'm trying to go through and fix things but I need to just hire someone to do it. I think I will just increase the titlesperpage to 50 and call it a day. Nobody should get that many messages.

UPDATE: I may have made some progress. If I use the GROUP BY threadID in the SQL search, I can get rid of this if statement:

if ($threadIDlist !~ /\b\Q$threadID\E\b/) {
}
[download]

I end up with the following. Please forgive me if this is not "runable" as your previous note suggests. The connections to the database are wrapped in subroutines. I've been told this is a bad way of doing it but that's what is in here.

    
    
    $currentpage = param('currentpage');
    
    &Conn_to_DB;
    $SQL =  "Select * from messages where (username='$username' or ton
+ame='$username') and message!='' GROUP BY threadID order by dateadded
+ desc, ID desc";
    &DoSQL;
    $recordcount = $sth->rows;


    if ($recordcount <= 0) {
      print "Error message goes here.";
    } else {
    
    
# Separately counting the total number of threads is no longer needed 
+(apparently).    
#   $SQL2 = "Select * from messages where (username='$username' or ton
+ame='$username') and message!='' order by ID desc";
#     &DoSQL2;
#          %threads;
#          while ($pointer2 = $sth2->fetchrow_hashref){
#             $threads{$pointer2->{'threadID'}}++; 
#            }
#            $threadcount_all = scalar keys %threads; 

           
    
        $titlesperpage = 10 if ($titlesperpage eq "");
        $currentpage = 1 if ($currentpage eq "" || $currentpage < 1);
        $startcount = ($currentpage - 1) * $titlesperpage + 1;
        $stopcount = $currentpage * $titlesperpage;
        $current_count = 0;
        $threadcount=0;
        $threadIDlist = "";
        
        while (($pointer2 = $sth2->fetchrow_hashref) && ($current_coun
+t <= $stopcount)){
            $current_count++;
  
            if ($current_count >= $startcount && $current_count <= $st
+opcount) {

            $name = $pointer2->{'name'};
            $message = $pointer2->{'message'};
            $date = $pointer2->{'date'};
        
        
            #this is no longer needed    
            #if ($threadIDlist !~ /\b\Q$threadID\E\b/) {
            #$threadIDlist=$threadID . "," . $threadIDlist;
            #$threadcount++;    
        
        
        
        print qq~ Content goes here~;
    
    
    
            #} end if ($threadIDlist !~ /\b\Q$threadID\E\b/) {
        
          } #end if ($current_count >= $startcount && $current_count <
+= $stopcount) {
      
    } #end while (($pointer2 = $sth2->fetchrow_hashref) && ($current_c
+ount <= $stopcount)){
           
           
           
sub Conn_to_DB{
  use DBI;
  $DSN = "DBI:mysql:database_name:db.domain.com";
  $sqluser = "user";
  $sqlpass = "pw";
  $dbh = DBI->connect($DSN,$sqluser,$sqlpass)
    || die "Cannot connect: $DBI::errstr\n" unless $dbh;
  return;
} 

           
sub DoSQL{
  eval {
    $sth = $dbh->prepare($SQL);
  }; # end of eval

  # check for errors
  if($@){
    $dbh->disconnect;
    print "Content-type: text/html\n\n";
    print "An ERROR occurred! $@\n";
    exit;
  } else {
    $sth->execute;
  } # end of if/else
  return ($sth);
}


sub DoSQL2{
  eval {
    $sth2 = $dbh->prepare($SQL2);
  }; # end of eval

  # check for errors
  if($@){
    $dbh->disconnect;
    print "Content-type: text/html\n\n";
    print "An ERROR occurred! $@\n";
    exit;
  } else {
    $sth2->execute;
  } # end of if/else
  return ($sth2);
}
[download]

The above seems to work. It returns only the number of threads as titlesperpage asks for. Please let me know if I'm wrong.

Really appreciate the help.

[reply]
[d/l]
[select]


P is for Practical
	PerlMonks