Beefy Boxes and Bandwidth Generously Provided by pair Networks
Perl: the Markov chain saw
 
PerlMonks  

Re^5: RFC: Placeholder creation for SQL statements

by Eily (Monsignor)
on Mar 10, 2018 at 00:27 UTC ( [id://1210596]=note: print w/replies, xml ) Need Help??


in reply to Re^4: RFC: Placeholder creation for SQL statements
in thread RFC: Placeholder creation for SQL statements

I read your post 3 times and I couldn't find a question about package vars. neither "our" nor "package"
Using the fact that I forgot to ask a question to not answer it, that's low :P.

Package vars are actually easier to handle than lexicals, that's why it's not included in the first demo.
OK, package vars not a problem then. Honestly my main concern was with the explicit use of closures, I didn't think this through so I didn't imagine what this could look like with the (&) prototype. I think it isn't as bad with the prototype version. At least I agree that there's no issue with package vars now that you've answered the question I didn't ask.

I wrote this:

my $username = 'LanX'; my $req1 = $db->xprepare(ph { "SELECT from users WHERE nick = '$userna +me'" }); $username = 'Eily'; my $req2 = $db->xprepare(ph { "SELECT from users WHERE nick = '$userna +me'" }); $req1->xexecute(); $req2->xexecute();
because in your example the variables where interpolated directly in the string fed to prepare. So unless that's with some version of perl with an extra layer of magic, the prepare is completly useless, which means probably not understood. So someone who wrote code with all the values in the prepare call might have written code like this:
my $username = 'LanX'; my $req1 = $db->prepare("SELECT from users WHERE nick = '$username'"); $username = 'Eily'; my $req2 = $db->prepare("SELECT from users WHERE nick = '$username'"); $req1->execute(); $req2->execute();
In which case you will get two different results. Someone who thinks your version is pretty much the same but is "optimized somehow" might fall into that trap. And that's counter intuitive anyway, people expect that when you use a variable in a statement, changing the variable later doesn't change the result of the previous statement. So by "as intended" I mostly meant by "as intended by a newbie". Notice that I called xprepare twice though, not once before changing the values like in your proposal. While it may be understood that calling xprepare once and changing the values will execute a different request each time, you could expect that calling xprepare again overwrites the biding, so the previously prepared queries are left unchanged. Maybe calling xprepare twice without calling xexecute should be forbidden (die), because according to your interpretation (twice the same result), the second call to xprepare is useless. If someone writes useless code on purpose, they probably don't understand what it does, in this case, they probably interpreted it like me (two different results).

prepare statements are meant to speed up repeated execution
They do speed up execution most of the time, but they can also do that across sessions, if the optimizer sees the exact same request prepared again, it can reuse the execution plan from the previous call and skip the evaluation phase. But the main advantage of bind variables is security, because they are protected against injection. So even if you had to call xprepare again each time before a xexecute, replacing the interpolated variables by bound variables would still be a huge benefit, that's why I didn't realize you were concentrating on the "one prepare, several executes" case.

BTW, since xprepare is supposed to be called instead of prepare, calling it several times in a row could change the values sent to execute without calling prepare again if the query string is the same. That way your colleagues could even write (with xdo to mean xprepare + xexecute):

$first_node = 1210590; $first = $dbh->xdo(ph { "SELECT * WHERE node_id = $first_node" }); # +Calls prepare and execute $second_node = 1210586; $second = $dbh->xdo(ph { "SELECT * WHERE node_id = $second_node" }); # + Same query, skips prepare, calls execute $third_node = 1210578; $third = $dbh->xdo(ph { "SELECT * WHERE node_id = $third_node " }); # + Prepare still not called
Although the DB manager might already do that for you (ignore a prepare that it already processed). And since I'm putting myself in the place of someone who doesn't understand everything or even doesn't want to understand (because the starting point is people who are reluctant to rewrite the code with placeholders), I didn't do the smart thing and select all three at once.

> my %vars = (rank => 'Pope', xp => 4); well I could also allow that bound variables are overriden if xexecute gets arguments. (not sure if that's what you mean)
No I didn't think about that, but that's a good idea, I guess you could imagine something like that:
my %default = (rank => "Pope", xp => 4); my $req = $dbh->xprepare(ph { "SELECT * FROM users WHERE rank = $_{ran +k} AND xp > $_{xp} AND username != $_{username}" } %default); $req->execute(rank => "Prior", username => "Eily");
Where xp uses the value bound during xprepare, rank is overwritten, and username is required because it doesn't have a default. At least that way if you want to replace some, or all values you don't have something that looks like you change a value after it was used. And when you reuse several times the same query in a row, some of the values might be left unchanged each time. And for this version to work you don't need to look into variables a closure closed over (ie accessing lexicals outside of their scope), prototypes are enough, and maybe tie if you want to die when a required parameter is missing (ie, when trying to read a value for a key that doesn't exist in %_)

FYI, probably more than half of that conversation happened in my head, because I realized things as I typed, and corrected myself and rewrote a lot. That's why my posts may lack questions I intended to ask, and be confusing. And I meant "you" as in "someone" BTW :D

Replies are listed 'Best First'.
Re^6: RFC: Placeholder creation for SQL statements
by Eily (Monsignor) on Mar 10, 2018 at 00:37 UTC

    That post is long enough, so I won't edit it to add new things :).

    I just realized, among the advantages of the ph BLOCK [HASH] version (syntax that looks like the other command BLOCK PARAMS, not having to parse the opcode, access pads or lexicals out of their scope), it also means you don't need that rule with the leading _, just use any variable that's not %_:

    my %default = (rank => "Bishop"); $dbh->xprepare(ph { "SELECT * FROM $table WHERE rank = $_{rank} AND le +ngth(username) = 4" } %default)->execute(rank => "Prior");

    I'll give it a try tomorrow, but I think that should be fairly easy to implement.

      > That post is long enough,

      yes, (tomorrow maybe =)

      > but I think that should be fairly easy to implement.

      yes, with Tie::Hash I suppose.

      goodnight! :)

      Cheers Rolf
      (addicted to the Perl Programming Language and ☆☆☆☆ :)
      Wikisyntax for the Monastery

Re^6: RFC: Placeholder creation for SQL statements
by LanX (Saint) on Mar 10, 2018 at 17:40 UTC
    Please let me answer in chunks, without answering all questions at once, even the imaginated once. :P

    > In which case you will get two different results.

    I get your point, code can't compensate bad programing.

    I'd say I'll offer two alternative versions of prepare (dynamic references vs static values) and emphasize on a do which does the execution after a prepare_cached right away.

    >  Maybe calling xprepare twice without calling xexecute should be forbidden (die),

    Nah, but I can throw a warning if two dynamic prepares bind the same refs.

    >  but they can also do that across sessions, if the optimizer sees the exact same request prepared again,

    Does it mean that prepare_cached from DBI is not necessary anymore?

    In any case I think that "deinterpolated" templates with placeholders are easier to cache than queries with hardcoded $names.

    Again, my emphasis ATM is on the "deinterpolater" and not the SQL api.

    I could think of many use cases, like generating / refactoring to printf or various HTML templates.

    Cheers Rolf
    (addicted to the Perl Programming Language and ☆☆☆☆ :)
    Wikisyntax for the Monastery

      Does it mean that prepare_cached from DBI is not necessary anymore?
      It depends on the DB manager that DBI calls.

      I did try my own version (yes I have a funny notion of what "tomorrow" means :). It relies on a tied hash that keeps track of the accessed keys rather than parsing the optree. And while there are two closures in my examples, if the output of ph is fed directly to xprepare or xdo, it is invisible (and if the user tries to access the output of ph, you get a subref, so it's easier to expect the closure behaviour).

      use strict; use warnings; use feature "say"; # Hash where every value is "?", but that remembers the order in which + the keys where accessed package PlaceHolderHash { use Tie::Hash; our @ISA = qw(Tie::StdHash); my %use_order; sub FETCH { my $self = shift; push @{ $use_order{$self} }, shift; "?"; } sub get_required { $use_order{$_[0]}; } } # A dummy DBI that prints "PREPARE $string" instead of actually callin +g prepare package MyDBTemplate { use Data::Dump qw( pp ); sub new { bless {}; } sub prepare { say qq<PREPARE "$_[1]">; } sub xprepare { my ($self, $params) = @_; $self->prepare($params->{Prepare}); my %query = %{$params}{qw(Default Required)}; # Yeah, hash-slice! bless \%query, 'MyQuery'; } sub xdo { my $self = shift; $self->xprepare(@_)->xexecute(); } } # A dummy query object that prints "EXECUTE $string" instead of actual +ly calling execute package MyQuery { sub execute { my $self = shift; local $" = ", "; say "EXECUTE (@_)"; } sub xexecute { my $self = shift; my %values = (%{$self->{Default}}, @_); my @required = @{ $self->{Required} }; exists $values{$_} or die "Param $_ required but not defined" for +@required; $self->execute(@values{@required}); } } # called as ph BLOCK HASH, # returns a hash with # - Prepare: the string to send with prepare() # - Default: the default value for named placeholders # - Required: the list of required values, and the order they were use +d in sub ph(&%) { tie my %prepare, 'PlaceHolderHash'; local *_ = \%prepare; my $sub = shift; { Prepare => $sub->(), Default => { @_ }, Required => (tied %prepare)->get_required() }; } my $dbh = new MyDBTemplate; my %default = (a => "A", b => "X"); my $table = "table"; say "First query"; # Table isn't part of %_, so do not use a place older my $query = $dbh->xprepare(ph { "SELECT * FROM $table WHERE _a = $_{a} + OR _b1 = $_{b} AND _c = $_{c} AND _b2 = $_{b}" } %default); $table = "I can change it now, it doesn't matter"; eval { $query->xexecute() } or print "$@"; $query->xexecute(b => "B", c => "C"); $query->xexecute(b => "B2", c => "C2"); say "\nSecond query"; my %values = (rank => 'Prior', xp => 42); $table = "users"; $dbh->xdo(ph { "SELECT * FROM $table WHERE rank = $_{rank} AND xp > $_ +{xp}" } %values);
      First query PREPARE "SELECT * FROM table WHERE _a = ? OR _b1 = ? AND _c = ? AND _b +2 = ?" Param c required but not defined at pm_1210536.pl line 71. EXECUTE (A, B, C, B) EXECUTE (A, B2, C2, B2) Second query PREPARE "SELECT * FROM users WHERE rank = ? AND xp > ?" EXECUTE (Prior, 42)

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: note [id://1210596]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others rifling through the Monastery: (4)
As of 2024-04-20 00:29 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found