Beefy Boxes and Bandwidth Generously Provided by pair Networks
The stupid question is the question not asked
 
PerlMonks  

Re^8: Paging with REST::Client?

by Argel (Prior)
on Jan 14, 2020 at 23:23 UTC ( [id://11111416]=note: print w/replies, xml ) Need Help??


in reply to Re^7: Paging with REST::Client?
in thread Paging with REST::Client?

Paging in this context refers to pulling in a subset of data, then the next subset (page) of data, etc. Here's some documentation (not hosted by the vendor or the company I work for):

Elda Taluta; Sarks Sark; Ark Arks
My deviantART gallery

Replies are listed 'Best First'.
Re^9: Paging with REST::Client?
by Anonymous Monk on Jan 15, 2020 at 01:03 UTC

    Paging in this context refers to pulling in a subset of data, then the next subset (page) of data, etc.

    Yeah, thats a loop

    Here's some documentation (not hosted by the vendor or the company I work for):

    Ok, ask yourself, where is "Infoblox" mentioned in REST, REST::Client or LWP?

    Or why it took take until Re^8 to link Infoblox ?

    Vacation time or something

    #!/usr/bin/perl -- use strict; use warnings; our @PAIRS; our %JSON; Main( @ARGV ); exit( 0 ); sub Main { %JSON = fromPairs(); my $startUrl = $PAIRS[0][0]; Paginate( $startUrl, sub { my( $json, $url ) = @_; ## real work here print "## callback @_ \n"; print $json->{result}[0]{name}, "\n\n"; }, ); return; } sub getAll { my @jsons; Paginate( $startUrl, sub { my( $json, $url ) = @_; push @jsons, $json; }, ); return \@jsons; } sub Paginate { my( $url, $callback )= @_; my @gets = $url; while( @gets ){ my $url = shift @gets; my $json = GET( $url ); if( my $id = $json->{next_page_id} ){ push @gets, urlNext( $url, $id, $json ); } $callback->( $json, $url ); } } sub urlNext { my( $url, $id, $json ) = @_; ## logic here return "https://127.0.0.1/wapi/v2.7.3/record:a?_page_id=$id"; } sub GET { ## real network request, REST::Client or whatever return $JSON{ $_[0] }; } sub fromPairs { @PAIRS = ( [ "https://127.0.0.1/wapi/v2.7.3/record:a?_max_results=2&_paging=1&_ +return_as_object=1&_return_fields=name&name~=test1.com", { next_page_id => "789c5590...4efc1732", result => [ { _ref => "record:a/ZG5zLmJpbmRfYSQuXY29tLn3DEwLjEuMC4x:a1.tes +t1.com/default", name => "a1.test1.com", }, { _ref => "record:a/ZG5zLmJpbmRLnRlc3QxLGE1LDEwLjEuMC41:a5.tes +t1.com/default", name => "a5.test1.com", }, ], }, ], [ "https://127.0.0.1/wapi/v2.7.3/record:a?_page_id=789c5590...4efc17 +32", { next_page_id => "789c5590...3e113c3d4d", result => [ { _ref => "record:a/ZG5zLmJpbmRfYSQc3QxLGE0LDEwLjEuMC40:a4.tes +t1.com/default", name => "a4.test1.com", }, { _ref => "record:a/ZG5zLmuY29tLnRlc3QxLGEzLDEwLjEuMC4z:a3.tes +t1.com/default", name => "a3.test1.com", }, ], }, ], [ "https://127.0.0.1/wapi/v2.7.3/record:a?_page_id=789c5590...3e113c +3d4d", { result => [ { _ref => "record:a/ZG5zLmJpbmRfYSQuX2RLGEyLDEwLjEuMC4y:a2.tes +t1.com/default", name => "a2.test1.com", }, ], }, ], ); return map { @$_ } @PAIRS; }
      I thought this kind of paging was common in WAPIs, so didn't think the specific one mattered. And I do know other WAPIs provide similar functionality for large data sets. So I was hoping support was more baked in. Basically, a generic question looking for a generic answer.

      As for not mentioning the product, I work in a large corporation where it's generally considered prudent to avoid giving out more information than necessary. So I pretty much try and keep those kinds of details out of my posts as much as possible. Sorry if it tripped you up!!

      Elda Taluta; Sarks Sark; Ark Arks
      My deviantART gallery

        ...So I was hoping support was more baked in. Basically, a generic question looking for a generic answer.

        Ok, so what would interface look like? Fill in the blanks  use REST::Client; my $url ; ...

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: note [id://11111416]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others sharing their wisdom with the Monastery: (4)
As of 2024-04-19 13:29 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found