Re^2: Perl LWP Can handle client-transfer-encoding = chunked encoding?

Hi FloydATC

Thanks for the details. IMHO, I am suffering from the "chunk" mode of transfer. Basically, Sever sends the data in multiple chunks, So the LWP need to make a multiple requests to get full data. As you have stated, LWP might not care (or dont need to care) about the data and encoding / chunked or non chunked.

Looks like Net::SSL can handle chunks properly where as IO::Socket::SSL is not handling it. I am not sure how to disable this chunking by using right http headers so server always sends data in one single chunk.

Thanks & Regards,
Bakkiaraj M
My Perl Gtk2 technology demo project - http://code.google.com/p/saaral-soft-search-spider/ , contributions are welcome.

Comment on Re^2: Perl LWP Can handle client-transfer-encoding = chunked encoding? Download Code

Replies are listed 'Best First'.
Re^3: Perl LWP Can handle client-transfer-encoding = chunked encoding? by FloydATC (Deacon) on Jan 16, 2015 at 10:01 UTC
No, that's usually not what "chunked" encoding means. It usually means that the response contains more than one file/document and you have to use the appropriate method to separate them. What you are referring to is a partial response, which the server is only allowed to send if you explicitly ask for it. This is commonly used when resuming a large download or seeking in streamed media. (See "range request".) -- FloydATC Time flies when you don't know what you're doing	[reply]
Re^4: Perl LWP Can handle client-transfer-encoding = chunked encoding? by noxxi (Pilgrim) on Jan 17, 2015 at 08:07 UTC
Transfer encoding 'chunked' was introduced with HTTP version 1.1 and is used when the length of the content is not known up-front, so that no 'Content-length' header can be given. Instead the content is sent in several chunks, each prefixed by the length of the chunk. It is not related to content-encoding like gzip and both can be used together. It is also not related to a content consisting of multiple parts. In this rarely used cases instead a Content-type of multipart/whatever is used and the client has to split the content by the given boundary, similar to multipart MIME messages in mails. Any HTTP client claiming support for version 1.1 (like LWP) must support this transfer encoding and there is no way to tell the server not to use it except switching to HTTP/1.0. The Accept-Encoding header only affects the accepted values for Content-Encoding (i.e. compression), not Transfer-Encoding.	[reply]


XP is just a number
	PerlMonks