Beefy Boxes and Bandwidth Generously Provided by pair Networks
No such thing as a small change

PerlIO::gzip can't handle appended stuff

by Skeeve (Parson)
on Apr 16, 2021 at 11:46 UTC ( #11131366=perlquestion: print w/replies, xml ) Need Help??

Skeeve has asked for the wisdom of the Perl Monks concerning the following question:

I'm wondering what I'm missing.

Every hour I receive a bunch of data which I write into a compressed (gzip) text file.

I collect all the 24 data deliveries into one file by simply appending to the file created in the first hour.

When I do a gunzip -c myfile.gz I can see all my data.

Whenn I open the same file with PerlIO::gzip I only get the first set of data.

Here is a small script to reproduce:

use PerlIO::gzip; open my $out, ">:gzip", "tst.gz"; print $out "111111111111111111111111\n" x 100; close $out; open my $out, ">>:gzip", "tst.gz"; print $out "222222222222222222222222\n" x 100; close $out; open my $in, "<:gzip", "tst.gz"; print while <$in>; close $in;

Question is: How can I convince perl to continue reading data?


Replies are listed 'Best First'.
Re: PerlIO::gzip can't handle appended stuff
by haukex (Archbishop) on Apr 16, 2021 at 11:56 UTC

    AFAICT from Bug #90480, I think PerlIO::gzip may not support this. I usually use the core modules IO::Compress::* and IO::Uncompress::*. Their objects can also be used like filehandles - this works for me:

    use IO::Uncompress::Gunzip (); my $in = IO::Uncompress::Gunzip->new('tst.gz', MultiStream=>1 ) or die "$IO::Uncompress::Gunzip::GunzipError"; print while <$in>; close $in;

      I already tried that as well and it also just prints the first part.

      It works! Thanks!

      Just have to add Multistream like so:

      my $in = new IO::Uncompress::Gunzip "tst.gz", Multistream => 1 or die "IO::Uncompress::Gunzip failed: $GunzipError\n"; print while <$in>;


Log In?

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: perlquestion [id://11131366]
Approved by marto
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others contemplating the Monastery: (2)
As of 2022-12-03 22:06 GMT
Find Nodes?
    Voting Booth?

    No recent polls found