Beefy Boxes and Bandwidth Generously Provided by pair Networks
There's more than one way to do things
 
PerlMonks  

Suggestions for handling a complex set of data

by sweetblood (Prior)
on Mar 02, 2016 at 22:32 UTC ( [id://1156700]=perlquestion: print w/replies, xml ) Need Help??

sweetblood has asked for the wisdom of the Perl Monks concerning the following question:

It's been quite a while since I've done any significant database work in perl, needless to say I'm a bit rusty both with the SQL side as well as the perl data structure side.

I need to look for certain records while summarizing some others basically I need all records by voucher_number and procedure_code, then walk through and take the records that have the right transaction_type.

my $sql = qq/select TOP 500 "vwGenSvcInfo"."Voucher_Number", "vwGenPatInfo"."Patient_Number", "vwGenPatInfo"."Patient_First_Name", "vwGenPatInfo"."Patient_Last_Name", "vwGenSvcInfo"."Service_Date_From", "vwGenSvcInfo"."Procedure_Code", "vwGenSvcPmtInfo"."Transaction_Type", "vwGenSvcPmtInfo"."Transfer_To_Carrier_Abbr" from ("Ntier_Training"."PM"."vwGenSvcPmtInfo" "vwGenSvcPmtInfo" INNER JOIN "Ntier_Training"."PM"."vwGenSvcInfo" "vwGenSvcInfo" + ON "vwGenSvcPmtInfo"."Service_ID"="vwGenSvcInfo"."Service_ID") INNER JOIN "Ntier_Training"."PM"."vwGenPatInfo" "vwGenPatInfo" + ON "vwGenSvcInfo"."Patient_ID"="vwGenPatInfo"."Patient_ID" order by Voucher_Number /; my $dbh = DBI->connect('DBI:ODBC:DATABOX'); my $sth = $dbh->prepare($sql); $sth->execute(); my %pats_hoh; while (my $pats = $sth->fetchrow_hashref) { #next unless exists($pats->{'Transaction_Type'}); my $voucher = $pats->{'Voucher_Number'}; my $proc_code = $pats->{'Procedure_Code'}; $pats_hoh{$voucher}{$proc_code}{Patient_Number} = $pats->{'Patient +_Number'}; $pats_hoh{$voucher}{$proc_code}{Voucher_Number} = $pats->{'Voucher +_Number'}; $pats_hoh{$voucher}{$proc_code}{Patient_Last_Name} = $pats->{'Pati +ent_Last_Name'}; $pats_hoh{$voucher}{$proc_code}{Transaction_Type} = $pats->{'Trans +action_Type'};

What I have now above crushes too much of the data. I need to see all the records for a couche_number and procedure_code, as it is now I only see one for each unique voucher/procedure. Maybe a hash is the wrong structure.

Any thoughts? - Help!

Thanks!!!

Sweetblood

Replies are listed 'Best First'.
Re: Suggestions for handling a complex set of data
by NetWallah (Canon) on Mar 02, 2016 at 22:45 UTC
    Doing the filteration and summarization in SQL will be far more efficient (and maintainable) than doing that in perl code.

    Use the "WHERE" clauses for filteration, and "GROUP BY" for summarization.

    You will need to share your SQL schema and objectives with a SQL professional to get answers.

    Alternatively, you can do an"OFF TOPIC" (OT) post here with that info, and try your luck.

            "Think of how stupid the average person is, and realize half of them are stupider than that." - George Carlin

      Yeah, that's is what SQL is made for.

      First, some comments on the SQL itself:

      Remove the quotes. Without spaces or case sensitivity, there's no point. Worse, if the query is run in a case sensitive environment, it will fail. Quotes preserve case, but databases usually uppercase all names. So, unless the name was created with quotes and something other than uppercase, you probably want to avoid using them. There's also the clutter they create.

      Keep like columns together. For ease of reading, keep columns from the same table together. Otherwise, it can get hard to identify which columns come from which table. As the order the columns are used in is how you use the variable in your code, there's no reason to order the columns in any specific way. So, i'd go for ease of reading.

      Alias normally. Why people use coded names for table is beyond me. Regardless, they do it. But if you're going to alias the names in the query, use words that make sense and are easily identified.

      Qualify all column names. When there is more than one table in the query, make sure all names are qualified. This avoids errors, and helps the reader understand what is going on.

      Just say no to inner joins. ANSI SQL Join syntax is just plain wrong. While many disagree with my opinion, i have to mention how hard that query is to discern. Moved into the WHERE clause, makes it so much cleaner and obvious.

      Taking those notes into consideration, the query can be rewritten as follows:

      SELECT TOP 500 Service.Voucher_Number, Service.Service_Date_From, Service.Procedure_Code, -- Patient.Patient_Number, Patient.Patient_First_Name, Patient.Patient_Last_Name, -- Payment.Transaction_Type, Payment.Transfer_To_Carrier_Abbr FROM Ntier_Training.PM.vwGenSvcInfo Service, Ntier_Training.PM.vwGenPatInfo Patient, Ntier_Training.PM.vwGenSvcPmtInfo Payment WHERE Patient.Patient_ID = Service.Patient_ID AND Payment.Service_ID = Service.Service_ID ORDER BY Service.Voucher_Number;

      Isn't that easier to read and understand? :)

      As for the aggregation, you probably want to use GROUP BY. Without knowing exactly what is required, it is hard to suggest a specific query.

Re: Suggestions for handling a complex set of data
by poj (Abbot) on Mar 03, 2016 at 15:52 UTC
    Maybe a hash is the wrong structure.

    You could use a hash of arrays

    my %pats_hoa=(); while (my $row = $sth->fetchrow_hashref){ my $voucher = delete $row->{'Voucher_Number'}; my $proc_code = delete $row->{'Procedure_Code'}; push @{$pats_hoa{$voucher}{$proc_code}},$row; } for my $voucher (sort keys %pats_hoa){ for my $proc (sort keys %{$pats_hoa{$voucher}}){ for my $rec (@{$pats_hoa{$voucher}{$proc}}){ if ($rec->{Transaction_Type} eq '???'){ # do something; } } } }
    poj

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: perlquestion [id://1156700]
Approved by Paladin
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others avoiding work at the Monastery: (3)
As of 2024-04-25 22:04 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found