Beefy Boxes and Bandwidth Generously Provided by pair Networks
Pathologically Eclectic Rubbish Lister
 
PerlMonks  

Rest::Client Headers not getting through

by Random_Walk (Prior)
on May 17, 2018 at 21:36 UTC ( #1214801=perlquestion: print w/replies, xml ) Need Help??

Random_Walk has asked for the wisdom of the Perl Monks concerning the following question:

Oh wise and splendiferous monks of the Perly persuasion

I am trying to use REST::Client to update a wiki from some text files. Although I can connect and read data, when I try to POST a page edit, it complains I have not sent a 'token'. The token is meant to go in the headers, the API docco https://www.mediawiki.org/wiki/API:Edit shows it as part of the URL such as

/mediawiki/api.php?action=edit&title=Tst&summary=Tsummary&text=article +%20content&token=81750dcca64dc
Note:Token above cut short to stop line wrapping on my laptop screen

Now I have added the token in my URI such as above, and got the error:

The 'token' parameter was found in the query string, but must be in th +e POST body

I have tried adding the token as a custom header in the code:   $client->addHeader('token', $token);   then I get this error:

The token parameter must be set
Here is my code with an attempt to add token as a custom header
use strict; use warnings; use v5.10.0; use Data::Dumper; use MIME::Base64; use JSON; use REST::Client; use Getopt::Long; my $debug = 0; GetOptions ("debug" => \$debug) or die "Error in command line argumen +ts\n"; my $wiki = 'http://HeartOfGold.improbable.tea'; my $api = '/mediawiki/api.php'; my $request; my $JSON = JSON->new->allow_nonref; # Create rest client and set some options my $client = REST::Client->new(); $client->setHost($wiki); $client->addHeader('format', 'json'); # First we get a login token $request = 'action=login&lgname=Marvin&lgpassword=leftdiode'; my $result = post($request); my $token = $result->{login}{token}; print "Got Token: $token\n"; $client->addHeader('token', $token); $request = "action=edit&title=LookupAutomationTest&summary=test%20summ +ary&text=article%20content"; $result = post($request); sub post { my $query = shift @_; $query .= '&format=json'; my $req = "$api?$query"; print "Fetching data with: POST $req\n" if $debug; print Dumper $client; $client->POST($req); if ( $client->responseCode() == 200 ) { print "Got Data OK, decoding ... " if $debug; my $data = $JSON->decode( $client->responseContent() ); print Dumper $data; return $data; } else { print "Failed with code $client->responseCode()\n"; print Dumper $client->responseContent(); } }
The first call to log in and get the token works fine.
The second call to edit the page gives the missing token type errors.
As you see the code dumps the client just before posting, so here is the output. One can see the headers are added in the _headers key of the client object but Wiki is ignoring them. I had a similar problem with Service Now a short time back, but I just &stuffed=them in the URI and it was happy then.
rwalk@HeartOfGold~> ./LookupsToWiki.pl -d Fetching data with: POST /mediawiki/api.php?action=login&lgname=Marvin +&lgpassword=leftdiodes&format=json $VAR1 = bless( { '_config' => { 'host' => 'http://HeartOfGold.improbab +le.tea', 'useragent' => bless( { 'protocols_all +owed' => undef, 'no_proxy' => +[], 'ssl_opts' => +{ + 'verify_hostname' => 1 +}, 'max_redirect' + => 7, 'requests_redi +rectable' => [ + 'GET', + 'HEAD' + ], 'handlers' => +{ + 'response_header' => bless( [ + { + 'owner' => 'LWP::UserAgent::parse_h +ead', + 'line' => '/usr/lib/perl5/vendor_pe +rl/5.18.2/LWP/UserAgent.pm:683', + 'm_media_type' => 'html', + 'callback' => sub { "DUMMY" } + } + ], 'HTTP::Config' ) +}, 'local_address +' => undef, 'proxy' => {}, 'show_progress +' => undef, 'use_eval' => +1, 'timeout' => 1 +80, 'max_size' => +undef, 'def_headers' +=> bless( { + 'user-agent' => 'REST::Client/273' + }, 'HTTP::Headers' ), 'protocols_for +bidden' => undef }, 'LWP::UserAge +nt' ) }, '_headers' => { 'format' => 'json' } }, 'REST::Client' ); Got Data OK, decoding ... $VAR1 = { 'login' => { 'cookieprefix' => 'bitnami_mediawiki', 'sessionid' => 'ltmfbpjfgj5il28d9k5105h8kc2a74d +b', 'result' => 'NeedToken', 'token' => 'bfbd1a1bdf1d1e0167e07ccdd6bdd4675af +df46d+\\' }, 'warnings' => { 'login' => { '*' => 'Fetching a token via ac +tion=login is deprecated. Use action=query&meta=tokens&type=login ins +tead.' } } }; Got Token: bfbd1a1bdf1d1e0167e07ccdd6bdd4675afdf46d+\ Fetching data with: POST /mediawiki/api.php?action=edit&title=LookupAu +tomationTest&summary=test%20summary&text=article%20content&format=jso +n $VAR1 = bless( { '_config' => { 'host' => 'http://HeartOfGold.improbab +le.tea', 'useragent' => bless( { 'protocols_all +owed' => undef, 'no_proxy' => +[], 'ssl_opts' => +{ + 'verify_hostname' => 1 +}, 'max_redirect' + => 7, 'requests_redi +rectable' => [ + 'GET', + 'HEAD' + ], 'handlers' => +{ + 'response_header' => bless( [ + { + 'owner' => 'LWP::UserAgent::parse_h +ead', + 'line' => '/usr/lib/perl5/vendor_pe +rl/5.18.2/LWP/UserAgent.pm:683', + 'm_media_type' => 'html', + 'callback' => sub { "DUMMY" } + } + ], 'HTTP::Config' ) +}, 'local_address +' => undef, 'proxy' => {}, 'show_progress +' => undef, 'use_eval' => +1, 'timeout' => 3 +00, 'max_size' => +undef, 'def_headers' +=> bless( { + 'user-agent' => 'REST::Client/273' + }, 'HTTP::Headers' ), 'protocols_for +bidden' => undef }, 'LWP::UserAge +nt' ) }, '_headers' => { 'token' => 'bfbd1a1bdf1d1e0167e07ccdd +6bdd4675afdf46d+\\', 'format' => 'json' }, '_res' => bless( { '_content' => '{"warnings":{"login +":{"*":"Fetching a token via action=login is deprecated. Use action=q +uery&meta=tokens&type=login instead."}},"login":{"result":"NeedToken" +,"token":"bfbd1a1bdf1d1e0167e07ccdd6bdd4675afdf46d+\\\\","cookieprefi +x":"bitnami_mediawiki","sessionid":"ltmfbpjfgj5il28d9k5105h8kc2a74db" +}}', '_protocol' => 'HTTP/1.1', '_rc' => '200', '_msg' => 'OK', '_headers' => bless( { 'content-le +ngth' => '295', 'cache-cont +rol' => 'private, must-revalidate, max-age=0', 'x-frame-op +tions' => 'DENY', '::std_case +' => { + 'x-frame-options' => 'X-Frame-Options', + 'client-response-num' => 'Client-Response-Num', + 'client-date' => 'Client-Date', + 'client-peer' => 'Client-Peer', + 'x-powered-by' => 'X-Powered-By', + 'x-content-type-options' => 'X-Content-Type-Options', + 'set-cookie' => 'Set-Cookie' + }, 'date' => ' +Thu, 17 May 2018 21:30:21 GMT', 'client-pee +r' => '10.255.64.83:80', 'x-powered- +by' => 'PHP/5.6.21', 'set-cookie +' => 'bitnami_mediawiki_session=ltmfbpjfgj5il28d9k5105h8kc2a74db; pat +h=/; httponly', 'connection +' => 'close', 'x-content- +type-options' => 'nosniff', 'server' => + 'Apache/2.4.18 (Unix) OpenSSL/1.0.2h PHP/5.6.21 mod_perl/2.0.8-dev P +erl/v5.16.3', 'content-ty +pe' => 'application/json; charset=utf-8', 'client-dat +e' => 'Thu, 17 May 2018 21:30:21 GMT', 'client-res +ponse-num' => 1 }, 'HTTP::Hea +ders' ), '_request' => bless( { '_uri_canon +ical' => bless( do{\(my $o = 'http://HeartOfGold.improbable.tea/media +wiki/api.php?action=login&lgname=Marvin&lgpassword=leftdiodes&format= +json')}, 'URI::http' ), '_headers' +=> bless( { + '::std_case' => { + 'format' => 'Format' + }, + 'user-agent' => 'REST::Client/273', + 'content-length' => 0, + 'format' => 'json' + }, 'HTTP::Headers' ), '_content' +=> '', '_method' = +> 'POST', '_uri' => $ +VAR1->{'_res'}{'_request'}{'_uri_canonical'} }, 'HTTP::Req +uest' ) }, 'HTTP::Response' ) }, 'REST::Client' ); Got Data OK, decoding ... $VAR1 = { 'error' => { 'code' => 'notoken', 'info' => 'The token parameter must be set', '*' => 'See http://HeartOfGold.improbable.tea/m +ediawiki/api.php for API usage' } };

Cheers,
R.

Pereant, qui ante nos nostra dixerunt!

Replies are listed 'Best First'.
Re: Rest::Client Headers not getting through
by bliako (Prior) on May 18, 2018 at 09:28 UTC

    Hi Random Walk, try to separate urlpath from POST params in the POST() call.

    The manpage says:

    POST ( $url, [$body_content, %$headers] )

    [] meaning optional

    I suggest you do something like this and see how it goes (untested):

    my $client = REST::Client->new(); $client->setHost($wiki); ... # got token # preparing a request: $request = "action=edit&title=LookupAutomationTest&summary=test%20summ +ary&text=article%20content"; $request .= '&format=json'; $request .= "&token=$token"; $client->POST($api, $request); # << separate path (e.g. "/mediawiki/ap +i.php") from the request-params e.g. the ? part which is now in strin +g $request.

    'token' must be the last item in the params list according to the MediaWiki.

    there is a pass in your code!

      Hi bliako,
      Thanks for taking the time to answer me. I tried your suggestion splitting the host and API path from the request and I also put the '&format=json' into each request string before calling post, so the "&token=$token" is then put last in the edit request.

      I'm afraid the result is a step back. Now I get an API help page in HTML from the first post. This I get at the first log attempt to get the token, which worked fine before. I updated the post sub like this

      sub post { my $query = shift @_; # my $req = "$api?$query"; # This removed print "Fetching data with: POST $api $query\n" if $debug; print Dumper $client; $client->POST($api, $query); if ( $client->responseCode() == 200 ) { print "Got Data OK, decoding ... " if $debug; print Dumper $client->responseContent() ; my $data = $JSON->decode( $client->responseContent() ); print Dumper $data; return $data; } else { print "Failed with code $client->responseCode()\n"; print Dumper $client->responseContent(); } }
      If Is switch this line:
      $client->POST($api, $query);
      To This
      $client->POST("$api?$query");
      Then I get back to where I was before. I get the Token, but can't get wiki to accept it for the edit

      Cheers,
      R.

      Pereant, qui ante nos nostra dixerunt!

        Hi Random Walk,

        The API you are using is pretty clear about this:

        The token parameter was found in the query
        string, but must be in the POST body

        .

        So, the token must go to the body. Which is by using POST($api, $body_content);

        I would add also the header "Content-type", "application/x-www-form-urlencoded" to become this:

        POST($api, "...token=the_token". {"Content-type" => "application/x-www +-form-urlencoded"});

        Now the above is for whenever you are making a request involving a token. The 1st param is the api path, the 2nd param is the post-body as a query string (you can use $client->buildQuery([a=>$a,b=>$b...]) for this) and the 3rd param is a hashref of headers.

        If the login phase worked somehow differently, then keep using that method and ... end up having two different post() - what can I say?

        Let me know if there is a public url to test this from my side.

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: perlquestion [id://1214801]
Approved by haukex
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others scrutinizing the Monastery: (3)
As of 2020-12-01 00:42 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found

    Notices?