Beefy Boxes and Bandwidth Generously Provided by pair Networks
P is for Practical
 
PerlMonks  

Re: Determining Content-Length when there is no Content-Length header

by calin (Deacon)
on Sep 30, 2007 at 01:26 UTC ( [id://641741]=note: print w/replies, xml ) Need Help??


in reply to Determining Content-Length when there is no Content-Length header

You can try to use a partial GET (byte range) to seek around the file to find its end by trial (and avoid downloading 20GB), but I'm pretty sure 99% of the resources supporting a partial GET will also report a Content-Length in the header.

Short answer: no, AFAIK

Update:Here's what I found in the standard. I don't quite understand the verbiage, but take a look at this.

Replies are listed 'Best First'.
Re^2: Determining Content-Length when there is no Content-Length header
by erroneousBollock (Curate) on Sep 30, 2007 at 03:25 UTC
    You can try to use a partial GET (byte range) to seek around the file to find its end by trial (and avoid downloading 20GB), but I'm pretty sure 99% of the resources supporting a partial GET will also report a Content-Length in the header.
    They're definitely not required to be present together.

    Furthermore, the server doesn't necessarily have to tell you that it doesn't support range requests at all (let alone in a useful manner).
    The RFC says:

    Note: clients cannot depend on servers to send a 416 (Requested range not satisfiable) response instead of a 200 (OK) response for an unsatisfiable Range request-header, since not all servers implement this request-header.
    The definitive answer is no.

    Now, for hacker...

    Alternatively, why not use the ':content_cb' callback of LWP::UserAgent. With that callback, you can implement your own semantics for max-content-length; if you decide you no-longer want to fetch the file when you reach say 2k, just abort the request by die()ing.

    -David

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: note [id://641741]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others goofing around in the Monastery: (5)
As of 2024-03-29 10:36 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found