Re^2: LPW $response->decoded_content different on different servers ?

>Compare ddumps

Thanks for the suggestion. I'm not familiar with ddump yet...this is a part of something called GHC ? That is a little over my head it seems. I would probably need a system administrators help installing that so I prefer looking at other methods first.

Running variations of the test script I found the following that might provide additional clues:

When I add the line below $charset does return the correct charset for the problem URL on the 'good' server (windows-874):

my $charset = $response->content_charset;
[download]

On the other two servers $charset remains empty for some reason. Bundle::LWP (which includes HTTP::Message that provides the content_charset method ) is identical on the three servers though.

When I specify 'windows-874' as a default_charset or charset the test works on all three servers as expected, but the test URLs are just example cases and in the live situation the charset is unknown.


##these lines would both work on all three servers 
my $dc = $response->decoded_content( raise_error => 1 , charset => 'wi
+ndows-874');
my $dc = $response->decoded_content( raise_error => 1 , default_charse
+t => 'windows-874');
[download]

Comment on Re^2: LPW $response->decoded_content different on different servers ? Select or Download Code

Replies are listed 'Best First'.
Re^3: LPW $response->decoded_content different on different servers ? by Anonymous Monk on Apr 29, 2015 at 22:18 UTC
Thanks for the suggestion. I'm not familiar with ddump yet...this is a part of something called GHC ? That is a little over my head it seems. I would probably need a system administrators help installing that so I prefer looking at other methods first. Um, no, I was on flip phone, so hard to type, I meant Data::Dump::dd()umpering to visualize your data (lesson courtesy of Basic debugging checklist I guess I should have said hexdump :)	[reply]
Re^4: LPW $response->decoded_content different on different servers ? by Transalp (Sexton) on Apr 30, 2015 at 08:05 UTC
>I meant Data::Dump::dd Ah.. glad I asked then :) I compared output of the good server to one bad server (see below), for the failing test URL http://www.ranks.nl/images/test/intrend4kids.htm and noticed the _content fields in $response are indentical, but the headers are different. The good server seems to use an older version of libwww-perl judging by the default User Agents. Also it seems something is wrong with headers like "x-meta-description" => "X-Meta-DESCRIPTION" on the bad server. I formatted the output a little for readability, and left out most of the _content field data because it would clutter the page. bad server do { require MIME::Base64; my $a = bless({ _content => MIME::Base64::d +ecode("ICAKICAKPCFET0NUWVBFIGh0bWw+PGh0bWwgPjxoZWFkPj...PC9odG1sPgog" +) , _headers => bless({ "::std_case" => { "client-date" => "Client-Date" , "client-peer" => "Client-Peer" , "client-response-num" => "Client-Response-Num" , "content-base" => "Content-Base", imagetoolbar => "Imagetoolbar" , "link" => "Link", title => "Title" , "x-meta-alexaverifyid" => "X-Meta-AlexaVerifyID" , "x-meta-author" => "X-Meta-Author" , "x-meta-contact-addr" => "X-Meta-Contact-Addr" , "x-meta-copyright" => "X-Meta-Copyright" , "x-meta-description" => "X-Meta-DESCRIPTION" , "x-meta-distribution" => "X-Meta-Distribution" , "x-meta-geo.position" => "X-Meta-Geo.Position" , "x-meta-geo.region" => "X-Meta-Geo.Region" , "x-meta-google-site-verification" => "X-Meta-Google-Site-Verificatio +n" , "x-meta-icbm" => "X-Meta-ICBM" , "x-meta-keywords" => "X-Meta-KEYWORDS" , "x-meta-language" => "X-Meta-Language" , "x-meta-rating" => "X-Meta-Rating" , "x-meta-robots" => "X-Meta-Robots" , "x-meta-viewport" => "X-Meta-Viewport" , "x-ua-compatible" => "X-UA-Compatible", } , "accept-ranges" => "bytes" , "client-date" => "Thu, 30 Apr 2015 07:35:44 GMT" , "client-peer" => "85.214.111.153:80" , "client-response-num" => 1, connection => "close" , "content-base" => "http://www.intrend4kids.com/template/" , "content-length" => 179664 , "content-type" => "text/html", date => "Thu, 30 Apr 2015 07:35:44 GM +T", expires => "Never", imagetoolbar => "no" , "last-modified" => "Sat, 25 Apr 2015 09:57:53 GMT" , "link" => [ "; media=\"print\"; rel=\"stylesheet\"; type=\"text/css\ +"" , "; rel=\"canonical\"" , "; rel=\"stylesheet\"" , "; rel=\"stylesheet\"" , "; rel=\"stylesheet\"; type=\"text/css\"" , "; rel=\"stylesheet\"; type=\"text/css\"", ], server => "Apache/2.2" +, title => pack("H","aad8b4c7e8d2c2b9e9d3e0b4e7a12c20e0cad7e9cdbce9d +2e0b4e7a1b9d3e0a2e9d22c20a2cda7e0c5e8b9e0b4e7a1"), vary => "Accept-En +coding" , "x-meta-alexaverifyid" => "QHNj33aO6KQmcqiV_jK9Ymtp6Jc" , "x-meta-author" => "Intrend4Kids" , "x-meta-contact-addr" => "Intrend4Kids\@yahoo.com" , "x-meta-copyright" => "Intrend4Kids" , "x-meta-description" => MIME::Base64::decode("qNPLuejSwiAtIKrYtMfo0s +K56dPgtOehLCDgytfpzbzp0uC056G50+Ci6dIsIMrUuaTp0uC056EsIKLNp+DF6LngtOe +hLCCizafgxei55MHpLCCizafgxei5ure60rfKwcHYtdQsIKLNp+DF6LngysPUwb7RsrnS +odLDIC0gw9Kk0srYtKTY6cE=") , "x-meta-distribution" => "Global" , "x-meta-geo.position" => "13.681508;100.60699" , "x-meta-geo.region" => "TH" , "x-meta-google-site-verification" => "myAUzDVw80kwrha0IdLVY__MjBuXZh +v3xy5nfvYP-po" , "x-meta-icbm" => "13.681508, 100.60699" , "x-meta-keywords" => MIME::Base64::decode("qti0x+jSwrnp0+C056EsIODK1 ++nNvOnS4LTnobnT4KLp0iwgytS5pOnS4LTnoSwgos2n4MXoueC056EsIKLNp+DF6Lnkwe +ksIKLNp+DF6LngysPUwb7RsrnSodLDLCCizafgxei5ure60rfKwcHYtdQsIKLNp+Oq6eC +056E=") , "x-meta-language" => "en-th" , "x-meta-rating" => "General" , "x-meta-robots" => "all" , "x-meta-viewport" => "width=device-width, initial-scale=1.0 user-sca +lable=no" , "x-ua-compatible" => "", } , "HTTP::Headers"), _msg => "OK", _protocol => "HTTP/1.1", _rc => 200, + _request => bless({ _content => "", _headers => bless({ "user-agent" + => "libwww-perl/5.837" } , "HTTP::Headers"), _method => "GET", _uri => bless(do{\(my $o = "http +://www.ranks.nl/images/test/intrend4kids.htm")} , "URI::http"), _uri_canonical => 'fix', } , "HTTP::Request"), } , "HTTP::Response"); $a->{_request}{_uri_canonical} = \${$a->{_request +}{_uri}}; $a; } [download] good server* do { require MIME::Base64; my $a = bless({ _content => MIME::Base64::d +ecode("ICAKICAKPCFET0NUWVBFIGh0bWw+PGh0bWwgPjxoZWFkPj...PC9odG1sPgog" +) , _headers => bless({ "accept-ranges" => "bytes" , "client-date" => "Thu, 30 Apr 2015 07:34:41 GMT" , "client-peer" => "85.214.111.153:80" , "client-response-num" => 1, connection => "close" , "content-base" => "http://www.intrend4kids.com/template/" , "content-length" => 179664 , "content-type" => "text/html", date => "Thu, 30 Apr 2015 07:34:39 GM +T", expires => "Never", imagetoolbar => "no" , "last-modified" => "Sat, 25 Apr 2015 09:57:53 GMT" , "link" => [ "; media=\"print\"; rel=\"stylesheet\"; type=\"text/css\ +"" , "; rel=\"canonical\"" , "; rel=\"stylesheet\"" , "; rel=\"stylesheet\"" , "; rel=\"stylesheet\"; type=\"text/css\"" , "; rel=\"stylesheet\"; type=\"text/css\"", ], server => "Apache/2.2" +, title => pack("H*","aad8b4c7e8d2c2b9e9d3e0b4e7a12c20e0cad7e9cdbce9d +2e0b4e7a1b9d3e0a2e9d22c20a2cda7e0c5e8b9e0b4e7a1"), vary => "Accept-En +coding" , "x-meta-alexaverifyid" => "QHNj33aO6KQmcqiV_jK9Ymtp6Jc" , "x-meta-author" => "Intrend4Kids" , "x-meta-contact-addr" => "Intrend4Kids\@yahoo.com" , "x-meta-copyright" => "Intrend4Kids" , "x-meta-description" => MIME::Base64::decode("qNPLuejSwiAtIKrYtMfo0s +K56dPgtOehLCDgytfpzbzp0uC056G50+Ci6dIsIMrUuaTp0uC056EsIKLNp+DF6LngtOe +hLCCizafgxei55MHpLCCizafgxei5ure60rfKwcHYtdQsIKLNp+DF6LngysPUwb7RsrnS +odLDIC0gw9Kk0srYtKTY6cE=") , "x-meta-distribution" => "Global" , "x-meta-geo.position" => "13.681508;100.60699" , "x-meta-geo.region" => "TH" , "x-meta-google-site-verification" => "myAUzDVw80kwrha0IdLVY__MjBuXZh +v3xy5nfvYP-po" , "x-meta-icbm" => "13.681508, 100.60699" , "x-meta-keywords" => MIME::Base64::decode("qti0x+jSwrnp0+C056EsIODK1 ++nNvOnS4LTnobnT4KLp0iwgytS5pOnS4LTnoSwgos2n4MXoueC056EsIKLNp+DF6Lnkwe +ksIKLNp+DF6LngysPUwb7RsrnSodLDLCCizafgxei5ure60rfKwcHYtdQsIKLNp+Oq6eC +056E=") , "x-meta-language" => "en-th" , "x-meta-rating" => "General" , "x-meta-robots" => "all" , "x-meta-viewport" => "width=device-width, initial-scale=1.0 user-sca +lable=no" , "x-ua-compatible" => "", } , "HTTP::Headers"), _msg => "OK", _protocol => "HTTP/1.1", _rc => 200, + _request => bless({ _content => "", _headers => bless({ "user-agent" + => "libwww-perl/5.833" } , "HTTP::Headers"), _method => "GET", _uri => bless(do{\(my $o = "http +://www.ranks.nl/images/test/intrend4kids.htm")} , "URI::http"), _uri_canonical => 'fix', } , "HTTP::Request"), } , "HTTP::Response"); $a->{_request}{_uri_canonical} = \${$a->{_request +}{_uri}}; $a; } [download]	[reply] [d/l] [select]
Re^5: LPW $response->decoded_content different on different servers ? (Bundle::LWP is dead) by Anonymous Monk on Apr 30, 2015 at 08:29 UTC
The good server seems to use an older version of libwww-perl judging by the default User Agents. Well, there is your problem :) When I came back I forgot to look at the bundle version stuff, look at this `libwww-perl-5.837 20 Sep 2010 last appearance of Bundle::LWP libwww-perl-6.13 14 Feb 2015 Latest Release` [download] In short, Bundle::LWP is dead, Changes doesn't explain why it went dead, but it did :) Get the latest with `cpan LWP LWP::Protocol::https`...	[reply] [d/l] [select]
Re^6: LPW $response->decoded_content different on different servers ? (Bundle::LWP is dead) by Transalp (Sexton) on Apr 30, 2015 at 09:04 UTC
Re^7: LPW $response->decoded_content different on different servers ? by Anonymous Monk on Apr 30, 2015 at 09:12 UTC
Some notes below your chosen depth have not been shown here


Don't ask to ask, just ask
	PerlMonks