Beefy Boxes and Bandwidth Generously Provided by pair Networks
No such thing as a small change
 
PerlMonks  

Where to GET the pieces

by geoffleach (Scribe)
on Feb 18, 2016 at 05:03 UTC ( #1155522=perlquestion: print w/replies, xml ) Need Help??

geoffleach has asked for the wisdom of the Perl Monks concerning the following question:

I'm using LWP::UserAgent to GET a large file. The server is returning it in pieces. I've retrieved the first piece. A subsequent GET merely retrieves the first piece. Where do I find the next?

Just to be clear. I can discover that the first GET is short by comparing content_length with length(content). Do I need to use a different URI for the next GET? If so, where in the response do I find that puppy?

Replies are listed 'Best First'.
Re: Where to GET the pieces
by Athanasius (Archbishop) on Feb 18, 2016 at 06:53 UTC

    Hello geoffleach,

    I think you need to provide more information. What is the web page you’re trying to retrieve? And what exactly is the command you issue via LWP::UserAgent to send the GET request?

    In the meantime, lwpcook#LARGE-DOCUMENTS has a code snippet showing how to use LWP::UserAgent to process a large document in chunks, as it arrives. That may be what you’re looking for.

    Hope that helps,

    Athanasius <°(((><contra mundum Iustus alius egestas vitae, eros Piratica,

      Thanks. That is exactly what I needed.

      For anyone who might need this in future, the secret is to use the content callback ($content_cb) in the call to $ua->request().
Re: Where to GET the pieces
by Anonymous Monk on Feb 18, 2016 at 07:40 UTC

    The server is returning it in pieces. ...

    What does that mean?

        Thanks for correcting my technical verbiage :-)

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: perlquestion [id://1155522]
Approved by Athanasius
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others about the Monastery: (3)
As of 2022-05-17 20:42 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?
    Do you prefer to work remotely?



    Results (68 votes). Check out past polls.

    Notices?