http://qs321.pair.com?node_id=578539

cengineer has asked for the wisdom of the Perl Monks concerning the following question:

Hello, I would like to use Compress::Bzip2 and Archive::Tar on a directory of .tar.bz2 files to extract the contents. I am able to use Compress::Bzip2 to uncompress the files, but I am unable to then use Archive::Tar to extract the files after by using the uncompressed tar data that is in a variable. Is this possible to do this without having to first write the uncompressed tar data to a file and then read that in using Archive::Tar? Thank you

Replies are listed 'Best First'.
Re: Using Archive::Tar on tar data already in memory
by BrowserUk (Patriarch) on Oct 16, 2006 at 16:33 UTC

    If you open the variable containing the the decompressed tar data as a ramfile (Search for  # in memory files in perlfunc (5.8.x)), you should be able to pass the resultant filehandle to Archive::Tar. Something like (untested):

    ... my $uncompressed = memBunzip( $compressed ); open my $ramFH, '<', \$uncompressed; my $tar = Archive::Tar->new; my @ATFobjects = $tar->read( $ramFH, 0, { extract => 1 } ); for my $ATFobject ( @ATFObjects ) { my $content = $ATFObject->getContent; ## Do something with the content. }

    Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
    Lingua non convalesco, consenesco et abolesco. -- Rule 1 has a caveat! -- Who broke the cabal?
    "Science is about questioning the status quo. Questioning authority".
    In the absence of evidence, opinion is indistinguishable from prejudice.
Re: Using Archive::Tar on tar data already in memory
by shmem (Chancellor) on Oct 16, 2006 at 16:42 UTC

    Use IO::Scalar and pass it a reference to your unbzip2'ed data (tested):

    #!/usr/bin/perl -w use strict; use Compress::Bzip2; use Archive::Tar; use IO::Scalar; use Data::Dumper; $Data::Dumper::Indent = 1; my $file = shift or die "usage: $0 filename.tar.bz2\n"; my $bz = bzopen($file,"r") or die; my ($out, $buf); $out .= $buf while $bz->bzread($buf) > 0; my $fh = IO::Scalar->new(\$out); my $tar = Archive::Tar->new; $tar->read($fh) or die "Cannot read from \$fh"; my @fileinfo = $tar->list_files([qw(name size)]); print Dumper(\@fileinfo);

    --shmem

    _($_=" "x(1<<5)."?\n".q·/)Oo.  G°\        /
                                  /\_¯/(q    /
    ----------------------------  \__(m.====·.(_("always off the crowd"))."·
    ");sub _{s./.($e="'Itrs `mnsgdq Gdbj O`qkdq")=~y/"-y/#-z/;$e.e && print}

      Suppose we were to take this one step further and have this completely in memory without reading in a file at all.

      I'm using LWP::Simple and pull down a tar.gz from a cgi that prints the binary data.
      $tarInput is a scalar which contains the tar data. Since gzread (or bzread) can't read in
      scalars or filehandles created from IO::Scalar, need to use a combination of the above 2 posts.
      I pass $tarInput to Compress::Zlib::memGunzip and it returns another scalar with the uncompressed
      data. Take that data and create a filehandle with the IO::Scalar module and pass that to
      $tar->read. I had been working on finding a solution for this and thought this thread would be appropriate
      to post my findings. As always there could be more than one way to do this, this just happens to
      work for me.

      use Archive::Tar; use LWP::Simple; use IO::Scalar; use Compress::Zlib; use strict; my $device = shift; my $tarInput = get "http://somehost/cgi-bin/getTar.cgi?device=$device" +; my $uncomptar = Compress::Zlib::memGunzip($tarInput); my $sh = new IO::Scalar \$uncomptar; my $tar = Archive::Tar->new; $tar->read($sh); print join "\n", $tar->list_files;
Re: Using Archive::Tar on tar data already in memory
by blazar (Canon) on Oct 16, 2006 at 16:20 UTC
    Hello, I would like to use Compress::Bzip2 and Archive::Tar on a directory of .tar.bz2 files to extract the contents. I am able to use Compress::Bzip2 to uncompress the files, but I am unable to then use Archive::Tar to extract the files after by using the uncompressed tar data that is in a variable.

    How are you "unable" to do that? I don't know Archive::Tar, but reading its documentation I find that

    $tar->read ( $filename|$handle, $compressed, {opt => 'val'} ) Read the given tar file into memory. The first argument can either be the name of a file or a reference to an already open filehandle (or an IO::Zlib object if it's compressed) The second argument indicates

    and also that

    Note that you can currently not pass a "gzip" compressed filehandle, which is not opened with "IO::Zlib", nor a string containing the full archive information (either compressed or uncompressed). These are worth while features, but not currently implemented. See the "TODO" section.

    However you should be able to trick it with an in-memory filehandle - suppose you full archive is in $tarcontent:

    # untested open my $fh, '<', \$tarcontent or die "D'oh! $!\n"; my $tar=Archive::Tar->new($fh);