Dear monks:

I have a (very) large archive of processed reports stored in the filesystem. Those reports are actually Perl objects that have been stored using code resembling this (heavy editing for simplification):

use strict;
use warnings;
use Storable qw/nstore/;

my $rep = bless { phrase => 'All your base belong to us' }, 'SomeClass
+';
my $file = 'my_file_name';

eval     { nstore($rep, $file) || warn "Storable::nstore failed with $
+!\n"; };
if ($@) { warn "Failed to store the object: $@\n"; }
[download]

By using gzip/Compress::Zlib/IO::Zlib we could be saving, on average, 24% - 30% of the disk space according to the testing we've done. Then I thought that it would be very simple to ask Storable to use IO::Zlib to read the compressed, serialized objects.

So, here I am writing this Template::Plugin that can read and present those objects... And banging my head against this code:

    if ($path =~ m/\.gz$/)    { $fh = new IO::Zlib }
    else            { $fh = new IO::File }

    $ctx->throw('Abuse.open',
        "Problem opening Abuse report: $!")
    unless $fh->open($path, "r");

    $rep = fd_retrieve($fh);

    close $fh;
[download]

This code is throwing an exception that reads: Not a GLOB reference at /usr/lib/perl5/site_perl/5.8.5/IO/Zlib.pm line 566. when taking the IO::Zlib path, but works beautifully when taking the IO::File branch.

This is the list of things that I've tested, without success:

• Rewriting the $fh in fd_retrieve in various ways (\*{$fh}, *{$fh}, ...). The IO::File branch always work, the IO::Zlib always throws the same exception, although sometimes it happens within Storable, likely due to the crazy things I've tried so far.
• Using IO::Scalar to first read the serialized object to a scalar and then, feeding fd_retrieve.

I can't believe that I am the only one trying to use those two modules together, although I am ready to admit that I am the only one failing :)

I know that likely, something along these lines...

my $fh = IO::File->new("gunzip $file |");
[download]

...would work, but I want to avoid calling external programs and all the issues that may come out of it. Specially when all the required code is already within Perl's module library. (Yes, I know that IO::Zlib will fallback to more or less that when no Compress::Zlib is around, but then I don't have to support IO::Zlib).

For the record, these are the relevant versions of what I am using:

$ find /usr/lib/perl* -type f -name Zlib.pm -o -name Storable.pm | xar
+gs egrep -i '\$version = '
/usr/lib/perl5/5.8.5/i386-linux-thread-multi/Storable.pm:$VERSION = '2
+.15';
/usr/lib/perl5/5.8.5/Memoize/Storable.pm:$VERSION = 0.65;
/usr/lib/perl5/site_perl/5.8.5/IO/Zlib.pm:$VERSION = "1.04";
/usr/lib/perl5/site_perl/5.8.5/i386-linux-thread-multi/Compress/Zlib.p
+m:$VERSION = "1.41" ;
[download]

Update: Although I failed to mention this originally, I did look the code of both IO::File and Storable as suggested by frodo72 and samtregar...

sub AUTOLOAD
{
    my $self = shift;

    $AUTOLOAD =~ s/.*:://;
    $AUTOLOAD =~ tr/a-z/A-Z/;

    return tied(*{$self})->$AUTOLOAD(@_); # line 566
}
[download]

Update: Prompted by frodo72 and samtregar, I probed deeper in the IO::Zlib and Storable code...

sub fd_retrieve {
    my ($file) = @_;
    my $fd = fileno($file); # <== THIS IS THE CALL
    logcroak "not a valid file descriptor" unless defined $fd;
    my $self;
    my $da = $@;                            # Could be from exception 
+handler
    eval { $self = pretrieve($file) };        # Call C routine
    logcroak $@ if $@ =~ s/\.?\n$/,/;
    $@ = $da;
    return $self;
}
[download]

Update: Following advice from sauoq, I dropped IO::Zlib in favor of PerlIO::gzip. The now fully working code looks similar to this...

    $ctx->throw('Abuse.open',
        "Problem opening Abuse report: $!")
    unless $fh->open($path, ($path =~ /\.gz$/? "<:gzip" : "<"));
[download]