Beefy Boxes and Bandwidth Generously Provided by pair Networks
Just another Perl shrine
 
PerlMonks  

Dynamic SQL

by Anonymous Monk
on Apr 06, 2015 at 15:28 UTC ( [id://1122563]=perlquestion: print w/replies, xml ) Need Help??

Anonymous Monk has asked for the wisdom of the Perl Monks concerning the following question:

I have no option but to use dynamic sql i.e construct queries on fly because the final query depends on conditions,for example:
if ($x>10) { $query="(select Name from Customers where CustId='$custid')" } else { $query="(select Name from Sales where SalesId='$salesid')" }

what I am worried about is 1.SQL injection, since I can't use placeholders and 2.performance, since the query won't be cached

What should I do about both of those points?

Replies are listed 'Best First'.
Re: Dynamic SQL
by bitingduck (Chaplain) on Apr 06, 2015 at 15:34 UTC

    You should use placeholders.

    $query1="(select Name from Customers where CustId=?)"; $query2="(select Name from Sales where SalesId=?)"; $sth1=$dbh->prepare($query1); $sth2=$dbh->prepare($query2); if ($x>10) { $sth1->execute($custid); } else { $sth2->execute($salesid); }
    (minor tweaks to fix typos..)
      If you have a complex query you can write this way for the easier readability:
      $query1 = 'select Name from Customers where CustId = :custid'; $query2 = 'select Name from Sales where SalesId = :salesid'; $sth1 = $dbh->prepare($query1); $sth2 = $dbh->prepare($query2); if ($x > 10) { $sth1->bind_param('custid', $custid); $sth1->execute(); while ($row = $sth1->fetchrow_hashref) { ... } } else { $sth2->bind_param('salesid', $salesid); $sth2->execute(); while ($row = $sth2->fetchrow_hashref) { ... } }
      'prepare' can be replaced with 'prepare_cached' in order to avoid multiple preparation. Excerpt from DBI man page:
      Like "prepare" except that the statement handle returned will be store +d in a hash associated with the $dbh. If another call is made to "pre +pare_cached" with the same $statement and %attr parameter values, the +n the corresponding cached $sth will be returned without contacting t +he database server.
      Update: I am not sure that this kind of named binding work with MySQL.
      because the query parameters varry too :
      if ($x>10) { $query="(select Name from Customers where CustId='$custid +')" } else { $query="(select Name from Sales where SalesId='$salesid' + and CustId='$custid')" }
      so when preparing and executing the query with the first query:
      $sth->execute(?)
      with the second:
      $sth->execute(?,?)
      so I can't have a single execute covering both occasions

        Did you notice that my version has two separate statement handles that are being executed? You can do the same thing with yours. Prepared statements with placeholders will likely be faster than constructing queries on the fly and executing them, and certainly much safer.

        How many possible queries can you need to construct that you can't afford to use prepared statements and placeholders?

        my ( $query, @args ); if ( $x > 10 ) { $query = 'select Name from Customers where CustID = ?'; @args = ( $custid ); } else { $query = 'select Name from Sales where SalesId = ? and CustID = ?' +; @args = ( $salesid, $custid ); } my $sth = $dbh->prepare( $query ); $sth->execute( @args );
Re: Dynamic SQL
by TGI (Parson) on Apr 07, 2015 at 02:25 UTC

    MidLifeXis was so kind as to recommend my module DBIx::PreQL. It's aim is to simplify the horrendous task of maintaining dynamically generated SQL.

    Here's one way to handle the your particular case.

    my $preql = <<'PREQL' * SELECT * Name * FROM CUST Customer SALE Sales * WHERE CUST CustID = ?customer_id? SALES SalesID = ?sales_id? PREQL my ($query, @params) = DBIx::PreQL->build_query( query => $preql, data => { sales_id => $salesid, customer_id => $custid, }, wanted => $x > 10 ? 'CUST' : 'SALES' ); my $got = $db->selectall_hashref( $query, @params );

    Why do all this stuff?

    • You get the placeholders you wanted for your dynamic query.
    • This is the least sucky way I've found to build dynamic SQL. The code I maintain has tons of dynamic SQL and I spent years experimenting before I hit upon this approach.
    • You will get a good error message if you put yourself in a condition where you are expecting a customer id but one isn't available.

    In fact, at $WORK, we have found it worthwhile to convert static queries to PreQL. It has helped with debugging and maintenance. Because all the placeholders are labeled, they are easier to understand. PreQL also provides useful error messages that identify missing, but expected placeholder values.


    TGI says moo

Re: Dynamic SQL
by Your Mother (Archbishop) on Apr 06, 2015 at 16:30 UTC

    Your code seems very badly organized if you can really mix into a single line of execution like that. You might look at refactoring the design. That said, caching the cached statements yourself is trivial with Perl–

    my %query_map; # Actually declared in higher (more outer) scope. if ( $x > 10 ) { $query = "SELECT Name FROM Customers WHERE CustId = ?"; } else { $query = "SELECT Name FROM Sales WHERE SalesId = ?"; } $query_map{$query} ||= $dbh->prepare($query); $query_map{$query}->execute(@args_that_will_match_query);
Re: Dynamic SQL
by MidLifeXis (Monsignor) on Apr 06, 2015 at 17:05 UTC

    See also DBIx::PreQL for a tool to help manage query clauses like this.

    --MidLifeXis

Re: Dynamic SQL
by mr_mischief (Monsignor) on Apr 06, 2015 at 16:21 UTC
    if ( $x > 10 ) { $query = '(select Name from Customers where CustId=?)' $cust_sth = $dbh->prepare( $query ); $cust_sth->execute( $custid ); } else { $query = '(select Name from Sales where SalesId=? and SalesTerrito +ry=?)' $sales_sth = $dbh->prepare( $query ); $sales_sth->execute( $salesid, $territory ); }
Re: Dynamic SQL
by Anonymous Monk on Apr 06, 2015 at 15:33 UTC
    since I can't use placeholders ... since the query won't be cached

    Why not? The example you showed would allow both these things.

    I am worried about ... performance

    Instead of worrying, implement and measure. Or at least be more specific, e.g. how many of these queries per hour?

      because the query parameters varry too :
      if ($x>10) { $query="(select Name from Customers where CustId='$custid')" } else { $query="(select Name from Sales where SalesId='$salesid' and CustId='$custid')" }
      so when preparing and executing the query with the first query:
      $sth->execute(?)
      with the second:
      $sth->execute(?,?)
      so I can't have a single execute covering both occasions

        Actually, it's not too difficult to use the same execute statement:

        $SQL = "select name from customers where 1=1"; if ($fl_have_cust_id) { $SQL .= " and custid=?"; push @args, $cust_id; } if ($fl_have_sales_id) { $SQL .= " and salesid=?"; push @args, $sales_id; } $ST = $DB->prepare($SQL); $ST->execute(@args);

        We have a hardcoded 1=1 condition so we can just add " and <condition>" to $SQL for each new condition we want.

        ...roboticus

        When your only tool is a hammer, all problems look like your thumb.

        Construct placeholders queries with hard coded options on the fly, no injection possible then.

        edit

        If you cache them after first use in a hash, you won't have a performance problem.

        Eg create a custom function getting a hash ( field =>entry,... ) and preparing only if new and returning result.

        Cheers Rolf
        (addicted to the Perl Programming Language and ☆☆☆☆ :)
        Je suis Charlie!

Re: Dynamic SQL
by erix (Prior) on Apr 06, 2015 at 15:39 UTC

    What DBMS are you building this on?

      Mysql,does it make a difference?

        Different DBMSen have different capabilities, yes. (Not to mention that people have varying interest in different systems)

A reply falls below the community's threshold of quality. You may see it by logging in.

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: perlquestion [id://1122563]
Front-paged by GotToBTru
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others about the Monastery: (6)
As of 2024-04-16 06:53 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found