Beefy Boxes and Bandwidth Generously Provided by pair Networks
Clear questions and runnable code
get the best and fastest answer

Re: Using WWW::Scripter with NTLM authentication

by rowdog (Curate)
on Feb 02, 2011 at 17:47 UTC ( #885804=note: print w/replies, xml ) Need Help??

in reply to Using WWW::Scripter with NTLM authentication

my $w = WWW::Scripter->new( keep_alive => 1);

WWW::Scripter is a subclass of WWW::Mechanize which is a subclass of LWP::UserAgent which is where you'll find the explaination of the keep_alive option. Basically, it's the number of objects to be stored in the LWP::ConnCache. You need a lot more than 1.

Replies are listed 'Best First'.
Re^2: Using WWW::Scripter with NTLM authentication
by LambethBoy (Initiate) on Feb 03, 2011 at 10:55 UTC

    Sorry, my example wasn't very helpful in that respect. Yes, you are right about it controlling the cache capacity. However, even when I set it to large values (100, 1000, 100000 etc) it makes no difference.

    After some digging around in the code for LWP::UserAgent (5.835) I found this

    sub clone { my $self = shift; my $copy = bless { %$self }, ref $self; # copy most fields delete $copy->{handlers};
        delete $copy->{conn_cache};
    # copy any plain arrays and hashes; known not to need recursive co +py for my $k (qw(proxy no_proxy requests_redirectable)) { next unless $copy->{$k}; if (ref($copy->{$k}) eq "ARRAY") { $copy->{$k} = [ @{$copy->{$k}} ]; } elsif (ref($copy->{$k}) eq "HASH") { $copy->{$k} = { %{$copy->{$k}} }; } } if ($self->{def_headers}) { $copy->{def_headers} = $self->{def_headers}->clone; } # re-enable standard handlers $copy->parse_head($self->parse_head); # no easy way to clone the cookie jar; so let's just remove it for + now $copy->cookie_jar(undef); $copy; }

    So it looks like the conn_cache data is not inherited when the UserAgent is cloned.

    If I comment this line out (delete $copy->{conn_cache};), my problems go away. Could a more enlightened member tell me whether or not this looks like a bug, or am I just barking mad and fiddling with something with terrible unintended consequences?


      I don't see where or why you're cloning the ua but it looks pretty easy to get the ConnCache from the original.

      my $ua2 = $ua->clone; $ua2->conn_cache($ua->conn_cache);

        I'm not explicitly cloning, it happens when WWW::Scripter starts reading referenced JavaScript files (when using the JavaScript plugin). The oroginal HTML document loads fine, I get the NTLM authorisation errors when the .js files are being read.


Log In?

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://885804]
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others chilling in the Monastery: (4)
As of 2020-10-01 13:40 GMT
Find Nodes?
    Voting Booth?
    My favourite web site is:

    Results (13 votes). Check out past polls.