Beefy Boxes and Bandwidth Generously Provided by pair Networks
Keep It Simple, Stupid
 
PerlMonks  

Download only part of a file from the web

by funstence (Initiate)
on May 10, 2007 at 03:59 UTC ( #614534=perlquestion: print w/replies, xml ) Need Help??

funstence has asked for the wisdom of the Perl Monks concerning the following question:

Hi all, I have a script which downloads FLV files (using wget). I've read about LWP::UserAgent, and HTTP::Request::Common but I couldn't find a way to download just part of the file. Any help would be appreciated. Thanks,
  • Comment on Download only part of a file from the web

Replies are listed 'Best First'.
Re: Download only part of a file from the web
by Anonymous Monk on May 10, 2007 at 04:49 UTC
      Any update? Or does anyone know of any tool that allows restricting the download size?

      I am also trying to write a tool to automate analysis of flv files on the web. The clip analysis can be done with a header preview, so I want similar functionality to avoid full download.

      Best regards,
      -Avid (seeker of perl wisdom)
        LWP::UserAgent has a max_size method that can limit the size that will be retrieved to a fixed number of bytes.

        Or you can be more ambitious and pass a :content_cb callback to the get method. This will give you the response in chunks and you can then accumulate them in a variable. When your analysis says that you have enough you can die to end the request, and catch that at a higher level with an eval. This will be a lot more complex, but avoids having to guess how much of the file to download.

Re: Download only part of a file from the web
by DigitalKitty (Parson) on May 10, 2007 at 04:24 UTC
    Hi funstence.

    I wrote a simple program that you might find useful.
    #!/usr/bin/perl use warnings; use strict; use LWP::Simple; my $file = 'test.txt'; my $content = get('http://www. google.com'); open FH, " >", "$file" or die "Error: $!\n"; print FH $content until -s $file >= 25; # I routinely include code to check for a successful closing # of the filehandle since a hardware error could occur. close FH or die "Error: $!\n";


    Hope this helps,
    ~Katie

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: perlquestion [id://614534]
Approved by varian
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others imbibing at the Monastery: (1)
As of 2021-04-18 00:47 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found

    Notices?