Beefy Boxes and Bandwidth Generously Provided by pair Networks
P is for Practical
 
PerlMonks  

Re^5: Uncompress streaming gzip on the fly in LWP::UserAgent/WWW::Mechanize

by pmqs (Friar)
on Dec 19, 2018 at 17:17 UTC ( [id://1227475]=note: print w/replies, xml ) Need Help??


in reply to Re^4: Uncompress streaming gzip on the fly in LWP::UserAgent/WWW::Mechanize
in thread Solved: Uncompress streaming gzip on the fly in LWP::UserAgent/WWW::Mechanize

If you have concatenated the chunks received & could uncompress the composite buffer, it sounds like the sub that gets triggered in the add_header is being passed a part of the same gzipped data stream every time it is invoked. You can push the compressed data a buffer at a time through Compress::Zlib. Something like this
use Compress::Zlib; my $gunzip = inflateInit(WindowBits => 16 + MAX_WBITS) or die "Cannot create a inflation stream\n" ; ... $mech->add_handler( response_data => sub { my ( $response, $ua, $h, $data ) = @_; my ($buffer, $status) = $gunzip->inflate($data); # uncompressed data in $buffer # return true to get called again for same response. 1; } ;
  • Comment on Re^5: Uncompress streaming gzip on the fly in LWP::UserAgent/WWW::Mechanize
  • Download Code

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: note [id://1227475]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others surveying the Monastery: (7)
As of 2024-04-16 06:50 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found