Beefy Boxes and Bandwidth Generously Provided by pair Networks
Keep It Simple, Stupid
 
PerlMonks  

Re^2: Perl LWP Can handle client-transfer-encoding = chunked encoding?

by sam_bakki (Pilgrim)
on Jan 16, 2015 at 09:38 UTC ( #1113464=note: print w/replies, xml ) Need Help??


in reply to Re: Perl LWP Can handle client-transfer-encoding = chunked encoding?
in thread Perl LWP Can handle client-transfer-encoding = chunked encoding?

Hi FloydATC

Thanks for the details. IMHO, I am suffering from the "chunk" mode of transfer. Basically, Sever sends the data in multiple chunks, So the LWP need to make a multiple requests to get full data. As you have stated, LWP might not care (or dont need to care) about the data and encoding / chunked or non chunked.

Looks like Net::SSL can handle chunks properly where as IO::Socket::SSL is not handling it. I am not sure how to disable this chunking by using right http headers so server always sends data in one single chunk.

Thanks & Regards,
Bakkiaraj M
My Perl Gtk2 technology demo project - http://code.google.com/p/saaral-soft-search-spider/ , contributions are welcome.

  • Comment on Re^2: Perl LWP Can handle client-transfer-encoding = chunked encoding?
  • Download Code

Replies are listed 'Best First'.
Re^3: Perl LWP Can handle client-transfer-encoding = chunked encoding?
by FloydATC (Deacon) on Jan 16, 2015 at 10:01 UTC

    No, that's usually not what "chunked" encoding means. It usually means that the response contains more than one file/document and you have to use the appropriate method to separate them.

    What you are referring to is a partial response, which the server is only allowed to send if you explicitly ask for it. This is commonly used when resuming a large download or seeking in streamed media. (See "range request".)

    -- FloydATC

    Time flies when you don't know what you're doing

      Transfer encoding 'chunked' was introduced with HTTP version 1.1 and is used when the length of the content is not known up-front, so that no 'Content-length' header can be given. Instead the content is sent in several chunks, each prefixed by the length of the chunk.
      It is not related to content-encoding like gzip and both can be used together. It is also not related to a content consisting of multiple parts. In this rarely used cases instead a Content-type of multipart/whatever is used and the client has to split the content by the given boundary, similar to multipart MIME messages in mails.
      Any HTTP client claiming support for version 1.1 (like LWP) must support this transfer encoding and there is no way to tell the server not to use it except switching to HTTP/1.0. The Accept-Encoding header only affects the accepted values for Content-Encoding (i.e. compression), not Transfer-Encoding.

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://1113464]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others making s'mores by the fire in the courtyard of the Monastery: (8)
As of 2020-11-24 21:31 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found

    Notices?