fetchrow_array loop in perl 5.10

Anonymous Monk has asked for the wisdom of the Perl Monks concerning the following question:

Replies are listed 'Best First'.
Re: fetchrow_array loop in perl 5.10 by Corion (Patriarch) on Sep 18, 2014 at 08:08 UTC
Depending on (the version of) your database driver, it might be that your database driver does not like more than one statement handle being active at the same time. In your case, that would be `$searchSQL` during the loop and in that loop the `$insertSQL`. A workaround could be to first fetch all items into an array and then execute the insert for each element of that array. In my experience, at least SQLite had this behaviour of not really wanting more than one active statement handle.	[reply] [d/l] [select]
Re^2: fetchrow_array loop in perl 5.10 by choroba (Cardinal) on Sep 18, 2014 at 10:39 UTC
first fetch all items into an array and then execute the insert for each element of that array Another option might be to keep two database handles, and run the search in one, and the insert in the second one. لսႽ† ᥲᥒ⚪⟊Ⴙᘓᖇ Ꮅᘓᖇ⎱ Ⴙᥲ𝇋ƙᘓᖇ	[reply]
Re^3: fetchrow_array loop in perl 5.10 by Anonymous Monk on Sep 18, 2014 at 13:43 UTC
yep, we already found a workaround for it, but we wanted to find a centralized fix for it, this is an existing code (thousand lines of codes are coded this way) and was working in perl 5.8	[reply]
Re^2: fetchrow_array loop in perl 5.10 by bulrush (Scribe) on Sep 18, 2014 at 19:33 UTC
Depending on (the version of) your database driver, it might be that your database driver does not like more than one statement handle being active at the same time. Yep, that's what happened to me. I had to declare $dbh2 to make my stuff work. $dbh cannot handle both statements. I use DBD::Pg for Postgresql. Perl 5.8.8 on Redhat Linux RHEL 5.5.56 (64-bit)	[reply]
Re^3: fetchrow_array loop in perl 5.10 by erix (Prior) on Sep 18, 2014 at 19:54 UTC
See perhaps, DBI, fork, and clone. I can't tell if applicable to those problems (yours and that of the OP), but it's interesting anyway :)	[reply]
Re: fetchrow_array loop in perl 5.10 by McA (Priest) on Sep 18, 2014 at 07:20 UTC
Hi, I can't say much about that issue. The only thing I can see from the code is the line `$insertSQL->finish;` [download] which makes me wonder. Why do you have this statement in the loop body? As soon as you have more than one record to fetch your insertSQL statement handle isn't valid anymore. At least I would expect to have this statement at the end of the script outside of the while loop when you've executed all insert statements. Probably a point to start from. Regards McA	[reply] [d/l]
Re^2: fetchrow_array loop in perl 5.10 by Anonymous Monk on Sep 18, 2014 at 07:34 UTC
thanks for the reply,I tried commenting it out, issue is still there (didn't loop thru all the records).	[reply]
Re: fetchrow_array loop in perl 5.10 by mje (Curate) on Sep 18, 2014 at 08:27 UTC
You definitely do not want to call finish on that insert statement. However, I see you say commenting it out does not help. What DBD are you using and what are the versions of it and DBI?	[reply]
Re^2: fetchrow_array loop in perl 5.10 by Anonymous Monk on Sep 18, 2014 at 09:02 UTC
from DBD::DB2 :1.78 DBI :1.611 to: DBD::DB2 :1.85 DBI :1.631	[reply]
Re^3: fetchrow_array loop in perl 5.10 by mje (Curate) on Sep 18, 2014 at 10:07 UTC
Could you cut the code down to the smallest that fails then run with DBI_TRACE env var set to 15 like this: `DBI_TRACE=15=x.log export DBI_TRACE run code show us the last few hundred lines of x.log` [download]	[reply] [d/l]
Re^4: fetchrow_array loop in perl 5.10 by Anonymous Monk on Sep 18, 2014 at 13:26 UTC
Re^5: fetchrow_array loop in perl 5.10 by mje (Curate) on Sep 18, 2014 at 15:08 UTC
Some notes below your chosen depth have not been shown here
Re: fetchrow_array loop in perl 5.10 by sundialsvc4 (Abbot) on Sep 18, 2014 at 12:21 UTC
Did you re-install the packages when you upgraded from 5.8 to 5.10, and are you certain that your PERL5LIB is pointing to the right location? In this case, I trust the implementation of DBI implicitly, and your code looks okay, so I suspect that there was a problem introduced by the upgrade process that you used. Some of this stuff uses “XS” (C-language code, libraries, etc.) very extensively, and upgrades can upset its delicate clockwork. A clean-reinstall of the package will, for example, recompile those components. Also, you could well have two packages out there (in two different Perl-version-specific directories), and be pointing to the wrong one (which might have the same name and version-code).	[reply]
Re: fetchrow_array loop in perl 5.10 by andal (Hermit) on Sep 18, 2014 at 14:12 UTC
Potentially, this is result of DB upgrade. For example SQLite databases have changed the way how they manipulate locks on files. Depending on which table is read and which table is updated you may get different locking problems. One can play with BEGIN TRANSACTION or autocommit settings to avoid dead-locking. Anyway, most likely you should provide information about DB that is being used and SQL statements that are executed.	[reply]
A reply falls below the community's threshold of quality. You may see it by logging in.
Re: fetchrow_array loop in perl 5.10 by TomDLux (Vicar) on Sep 19, 2014 at 00:41 UTC
You expect people to determine from deep understanding of the code why your program doesn't work. I view programming the other way around ..... it doesn't work? What is the return value of execute()? What does DBI::errstr() return? If you run the program in the debugger, what happens in the first ( and only successful ) execute()? Is there a second execute() attempted but it fails? Or does the loop terminate after the first execute? As Occam said: Entia non sunt multiplicanda praeter necessitatem.	[reply]
Re^2: fetchrow_array loop in perl 5.10 by Anonymous Monk on Sep 19, 2014 at 08:50 UTC
errstr has no value for every after line, the issue here is that the loop (fetchrow_array) ends after 2 rows (out of 4) always. again the code perfectly works before the perl upgrade.	[reply]
Re^3: fetchrow_array loop in perl 5.10 by mr_mischief (Monsignor) on Sep 19, 2014 at 13:44 UTC
If there are no errors but the loop ends unexpectedly, have you checked to see that the loop conditional is what you expect?	[reply]
Re: fetchrow_array loop in perl 5.10 by Limbic~Region (Chancellor) on Nov 17, 2016 at 19:02 UTC
Anonymous Monk, I had the same problem and was able to further isolate it - it only breaks for me if one of the columns you are selecting is a LOB. I'm not sure if that helps or not. Cheers - L~R	[reply]


Just another Perl shrine
	PerlMonks