http://qs321.pair.com?node_id=1224818

astroboy has asked for the wisdom of the Perl Monks concerning the following question:

I'm trying to query data from a Hive table on a MapR platform. My query is: 

use Modern::Perl;
use DBI;
use Data::Dumper;

my $dbh = DBI->connect('dbi:ODBC:DSN=myDSN', undef, undef, {RaiseError
+ => 1});
$dbh->{LongReadLen} = 1000000; 
$dbh->{LongTruncOk} = 1;


my $sth = $dbh->prepare(q{select * from table});
$sth->execute;

while (my $row = $sth->fetchall_arrayref) {
    say Dumper $row;
}

[download]

What I get is the following line repeated hundreds of times to STDERR

DBD::ODBC::st fetchall_arrayref warning: no select statement currently
+ executing
 at conn.pl line 17.

[download]
Then I get the one row in the table, followed by a never-ending stream of the following to STDOUT 
$VAR1 = [];

$VAR1 = [];

$VAR1 = [];

$VAR1 = [];

$VAR1 = [];

$VAR1 = [];

$VAR1 = [];

$VAR1 = [];

$VAR1 = [];

$VAR1 = [];

$VAR1 = [];

$VAR1 = [];

$VAR1 = [];

$VAR1 = [];

$VAR1 = [];

$VAR1 = [];

$VAR1 = [];

$VAR1 = [];

$VAR1 = [];

[download]
And the following to STDERR 
DBD::ODBC::st fetchall_arrayref warning: no select statement currently
+ executing
 at conn.pl line 17.
DBD::ODBC::st fetchall_arrayref warning: no select statement currently
+ executing
 at conn.pl line 17.
DBD::ODBC::st fetchall_arrayref warning: no select statement currently
+ executing
 at conn.pl line 17.
DBD::ODBC::st fetchall_arrayref warning: no select statement currently
+ executing
 at conn.pl line 17.
DBD::ODBC::st fetchall_arrayref warning: no select statement currently
+ executing
 at conn.pl line 17.
DBD::ODBC::st fetchall_arrayref warning: no select statement currently
+ executing
 at conn.pl line 17.
DBD::ODBC::st fetchall_arrayref warning: no select statement currently
+ executing
 at conn.pl line 17.
DBD::ODBC::st fetchall_arrayref warning: no select statement currently
+ executing
 ... repeat forever ....

[download]
I have to Control-C to stop this. Any explanation and way forward is appreciated

Replies are listed 'Best First'.
Re: Querying Hive tables using DBD::ODBC
by poj (Abbot) on Oct 29, 2018 at 07:34 UTC
    ->fetchall_arrayref fetches all records so you don't need to loop (unless you don't have enough memory to fetch and return all the rows in one go.)

    my $rows = $sth->fetchall_arrayref;
say Dumper $rows;

    [download]

    Try fetching each row using ->fetchrow_arrayref.

    #while (my $row = $sth->fetchall_arrayref) {
while (my $row = $sth->fetchrow_arrayref) {
   say Dumper $row;
}
    poj
[reply]
[d/l]
[select]
      Duh! You wouldn't think I'd been using DBI since the late 90s! Thanks for the fresh pair of eyes - I couldn't see the wood for the trees
[reply]
Re: Querying Hive tables using DBD::ODBC
by afoken (Chancellor) on Oct 29, 2018 at 07:27 UTC 
    use Modern::Perl;
use DBI;
use Data::Dumper;

my $dbh = DBI->connect('dbi:ODBC:DSN=myDSN', undef, undef, {RaiseError
+ => 1});
$dbh->{LongReadLen} = 1000000; 
$dbh->{LongTruncOk} = 1;


my $sth = $dbh->prepare(q{select * from table});
$sth->execute;

while (my $row = $sth->fetchall_arrayref) {
    say Dumper $row;
}

    [download]

    What I get is the following line repeated hundreds of times to STDERR

    DBD::ODBC::st fetchall_arrayref warning: no select statement currently
+ executing
 at conn.pl line 17.

    [download]

    The code you are showing us is not the code you are running. You posted only 15 lines, so there is no line 17.

    Show the code that is really running. If it is to long (more than about 30 lines), copy it, shorten it to the relevant parts, run it. Post the shortened code and the error messages and warnings it generates.

    Alexander

    --
    Today I will gladly share my knowledge and experience, for there are no sweeter words than "I told you so". ;-)
[reply]
[d/l]
[select]

      Yip, I had the code in one window and the output in another, I remved three lines to make the code more succinct but forgot to run the output again

[reply]

Back to Seekers of Perl Wisdom