Beefy Boxes and Bandwidth Generously Provided by pair Networks
go ahead... be a heretic
 
PerlMonks  

Input on Lightweight SQL Query Storage?

by Xenofur (Monk)
on Apr 25, 2009 at 13:27 UTC ( [id://760006]=perlmeditation: print w/replies, xml ) Need Help??

Note: I edited the title to reflect my intent better. This module doesn't focus on the SQL creation, but on the storage of it and on the access on said storage.

Hello monks,

Today i decided would be a good day to remove the SQL code mingled in with my Perl code. I wanted this to happen in the form of having all queries in a list in their own module, with the main code only executing a single call with their name to get the query for DBI functions.

However that leaves a problem: Some queries, like inserts, need to be generated programmatically. As such there was need for a templating system, but i wanted to keep it lightweight. So i tagged on the system used in wakaba, which allows scalars or arrays of hashes to be forwarded to the template.

I'm pretty happy with how it works now and am considering submitting it to CPAN. Before trying to do that however I'd like to get some general input. Am i needlessly duplicating other efforts? Is my interface crap? Could i do things better? Did i overlook something entirely? Any feedback along these lines is greatly welcome. :)

And now, the code.

Let's assume an App, which stores the queries in App::Queries. A possible structure would look like this:

App/Queries.pm :
package App::Queries; use base 'Exporter'; our @EXPORT_OK = (qw( get_query_hash )); sub get_query_hash { return \%queries; } my %queries = ( get_reprocessing_list => { type => 'simple', source => q( SELECT typeID FROM typeAM; ), }, get_blueprintTypeID => { type => 'complex', source => q( SELECT [% var $extract %], bpID FROM invBpT; ), } );

App.pm :
package App; use App::Queries qw( get_query_hash ); use SQL::Template qw( load_query_hash get_query ); load_query_hash( get_query_hash() ); my $simple_query = get_query( 'get_reprocessing_list' ); my $input = { extract => 'productTypeID' }; my $complex_query = get_query( 'get_blueprintTypeID', $input ); print "$simple_query\n$complex_query";

and result in this:
SELECT typeID FROM typeAM; SELECT productTypeID, bpID FROM invBpT;

The code for SQL::Template is as follows:
package SQL::Template; use 5.010; use strict; use warnings; use base 'Exporter'; our @EXPORT_OK = qw( load_query_hash get_query ); use Carp qw( cluck confess ); my $queries; my $eval_methods = { simple => \&eval_simple, complex => \&eval_complex, }; sub load_query_hash { my ($import) = @_; for my $name ( keys %{$import} ) { cluck "Existing query template redefined: $name" if ( $queries->{$name} ); $queries->{$name} = $import->{$name}; } return; } sub get_query { my ( $name, $input ) = @_; my $error; my $query = $queries->{$name}; $error = "Query template unknown for: $name" if ( !$query ); $error = "Eval method $query->{type} unknown for: $name" if ( !$error and !$eval_methods->{ $query->{type} } ); if ($error) { confess $error; return; } $eval_methods->{ $query->{type} }->( $query, $input ); return $query->{target}; } sub eval_simple { my ($query) = @_; return if ( $query->{target} ); $query->{target} = $query->{source}; return; } sub eval_complex { my ( $query, $input ) = @_; $query->{sub} = compile_template( $query->{source} ) if ( !$query- +>{sub} ); $query->{target} = $query->{sub}->( %{$input} ); return; } sub compile_template { my ($str) = @_; my $code; local $SIG{__WARN__} = sub { confess "Template Error: ", @_; }; $str =~ s/^\s+//; $str =~ s/\s+$//; while ( $str =~ m@(.*?)(\[% (/?)(var|const|if|loop)(?:|\s+(.*?[^\\])) %\]|$)@s +g ) { my ( $content, $tag, $closing, $name, $args ) = ( $1, $2, $3, +$4, $5 ); $content =~ s/(['\\])/\\$1/g; $code .= "\$res.='$content';" if ( length $content ); $args =~ s/\\>/>/g if ( defined $args ); if ($tag) { if ($closing) { if ( $name eq 'if' ) { $code .= '}'; } elsif ( $name eq 'loop' ) { $code .= '$$_=$__ov{$_} for(keys %__ov); } } }'; } } else { if ( $name eq 'var' ) { $code .= '$res.=eval{' . $args . '};'; } elsif ( $name eq 'const' ) { my $const = eval $args; $const =~ s/(['\\])/\\$1/g; $code .= '$res.=\'' . $const . '\';'; } elsif ( $name eq 'if' ) { $code .= 'if (eval{' . $args . '} ) {'; } elsif ( $name eq 'loop' ) { $code .= '{' . 'my $__a=eval{ ' . $args . '}; ' . 'if ($__a) { ' . ' for (@$__a) { ' . ' my %__v=%{$_}; ' . ' my %__ov; ' . ' for (keys %__v) { ' . ' $__ov{$_}=$$_; ' . ' $$_=$__v{$_}; ' . ' }'; } } } } $code = 'no strict; ' . 'sub { ' . ' my %__v=@_; ' . ' my %__ov; ' . ' for (keys %__v) { ' . ' $__ov{$_}=$$_; ' . ' $$_=$__v{$_}; ' . ' }' . ' my $res = "";' . $code . ' $$_=$__ov{$_} for(keys %__ov);' . ' return $res; ' . '} '; my $sub = eval $code; confess "Template format error: $@" if ( !$sub ); return $sub; } 1;

Replies are listed 'Best First'.
Re: Input on Lightweight SQL Query Templating?
by merlyn (Sage) on Apr 25, 2009 at 14:07 UTC
    I'd probably never use this. With a package like Rose::DB::Object, I never have to think about the SQL at all for 95% of the common cases, and it's a lot more flexible.

    -- Randal L. Schwartz, Perl hacker

    The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC 2119.

      Yeah, i considered using ORM, but honestly: SQL is a good language. It is well suited for what it's meant to do. I personally don't see a reason to introduce a lot of overhead on top of that only to have the benefit of writing SQL in Perl. (And the disadvantage of having to learn a completely new syntax only to express a language i already know.)

      As such this is meant for people who actually like SQL for what it is and simply want to keep it seperate from their Perl.
        You're misunderstanding the point of an ORM. It's not to avoid writing SQL, it's to avoid writing incredibly tedious SQL and DBI code that you've done a million times before. You still write the interesting SQL (reports and complex stuff where performance hinges on a specific SQL formulation) by hand.
Re: Input on Lightweight SQL Query Storage?
by Arunbear (Prior) on Apr 25, 2009 at 22:28 UTC
    Ok, but why not use Template Toolkit (or another templating module) to do the actual template processing ?
      Just personal preference. I use TT in other projects, but when i need something light-weight AND powerful i fall back to the sub routine you see there, as it allows me high control over templates (substitution, if statements, looping through arrays of hashes, ability to use perl directly in each of these) without having any sort of strings attached.
      placeholder support - look at SQL::Interp for an idea of what you need to get done.
Re: Input on Lightweight SQL Query Templating?
by zby (Vicar) on Apr 25, 2009 at 16:11 UTC
    I don't have any comment on this - but you might have a look at Fey.
      After some chat in IRC i realized that i was communicating my intent wrongly. The purpose of this is not to create SQL queries. The main focus here is on how to store and access them in a lightweight and elegant manner.

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: perlmeditation [id://760006]
Approved by Corion
Front-paged by Arunbear
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others perusing the Monastery: (5)
As of 2024-04-19 13:34 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found