Beefy Boxes and Bandwidth Generously Provided by pair Networks
The stupid question is the question not asked
 
PerlMonks  

Re: Splitting the records into multiple worksheets

by NetWallah (Canon)
on Feb 20, 2021 at 04:28 UTC ( [id://11128587]=note: print w/replies, xml ) Need Help??


in reply to Splitting the records into multiple worksheets

Excel is the wrong tool if you are processing > 10k rows.

I would recommend a sqlite database.
It has perl interfaces via the DBI and DBD::Sqlite perl modules.

There are plenty of snippets to get you started - check out this cookbook.

                "Avoid strange women and temporary variables."

  • Comment on Re: Splitting the records into multiple worksheets

Replies are listed 'Best First'.
Re^2: Splitting the records into multiple worksheets
by chandantul (Scribe) on Feb 20, 2021 at 04:43 UTC

    Thanks for the suggestion but Excel is the requirement as we need to generate the report with pivot chart with another sheet in the same excel file. The above solution will change complete code design as well. Do you have any other solution?

      After creating the raw data table in SQLite, you could use SQL to generate your pivoted data.

      The SQL "GROUP BY" clause can do the pivot functions to summarize.

      The result can then easily export into a csv file suitable for import into Excel.

                      "Avoid strange women and temporary variables."

        Hello, Do you have any specific example of pulling the raw data into DB? Please check below how i am pulling raw data and let me know next step if possible. How i should import into csv and then excel if possible.

        my $urlstringlog = $apiurllog . $sortor . $filter . $target . $appID . + $date2; $strExcelFilename = "C:/" . "Analysis-Details-automatic-bk" . $date . +".xlsx"; do { # Run get the users run_api_call($urlstringlog); @responsetext = parse_json ($client->responseContent()); push @responsetextall, @responsetext; $linkheader = $client->responseHeader("link"); if ($linkheader=~ m/next/) { (my $link1 = $linkheader) =~ s/ .*? (self)/$1/gx; (my $link2 = $link1) =~ s/self/$1/g; print "Post Pagination: $link10"; $urlstringlog = $link2; } } while ($linkheader=~ m/next/); for my $i (0..$#responsetextall) { $responsetextall[$i] =~ s/]\[/,/g; for my $j (0..$#{$responsetextall[$i]}) { @responsests1 = $responsetextall[$i][$j]{tat}; $responseid = $responsetextall[$i][$j]{act}{id}; $responsdisp = $responsetextall[$i][$j]{act}{Name}; $responsalter = $responsetextall[$i][$j]{act}{Id}; $responseclientipadd = $responsetextall[$i][$j]{cln}{ipAd +}; $responseappsdebug = $responsetextall[$i][$j]{deb}{Data}; + for my $m (0..$#responsests1) { for my $n (0..$#{$responsests1[$m]}) { $responseapps2id = $responsests1[$m][$n]{id}; $responseapps2 = $responsests1[$m][$n]{ty}; $responseapps1 = $responsests1[$m][$n]{Name}; if ( $responseapps2 eq 'AppInstance' ){ @responseapps3 = $responsests1[$m][$n]{displayName}; $responseapps5 = $responseapps3[0]; print "Its Matches Appinstances"; $responseapps4 = $responsests1[$m][$n]{Name}; if ( $responseapps2 eq 'AppUser'){ $responseapps3id = $responsests1[$m][$n]{a +Id}; } } push @responseapps3id,$responseapps3id; push @responseapps5 , $responseapps4; push @responseapps7, $responseapps7; } } push @responsalter, $responsalter; push @responseclientipadd,$responseclientipa +dd; } }

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: note [id://11128587]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others goofing around in the Monastery: (1)
As of 2024-04-25 04:12 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found