Beefy Boxes and Bandwidth Generously Provided by pair Networks
No such thing as a small change
 
PerlMonks  

Not an ARRAY reference

by chandantul (Scribe)
on Nov 06, 2020 at 17:40 UTC ( [id://11123454]=perlquestion: print w/replies, xml ) Need Help??

chandantul has asked for the wisdom of the Perl Monks concerning the following question:

Hello All, My code has worked before but i could not able to figure out the reason its not working now. I have Array reference and that i am pushing by following:

push @responsetextall, @responsetext;
but while i am trying to loop 2nd time its complaining about the following line:
for my $j (0..$#{$responsetextall[$i]}) {
Please check below code and advise the probable solution on this.

run_api_call($urlstringapp); @responsetext = parse_json ($client->responseContent()); push @responsetextall, @responsetext; $linkheader = $client->responseHeader("Link"); print $linkheader; if ($linkheader=~ m/next/) { (my $link1 = $linkheader) =~ s/ .*? (self)/$1/gx; (my $link2 = $link1) =~ s/self/$1/g; (my $link3 = $link2) =~ s/</$1/g; (my $link4 = $link3) =~ s/>/$1/g; (my $link5 = $link4) =~ s/;/$1/g; (my $link6 = $link5) =~ s/ /$1/g; (my $link7 = $link6) =~ s/=\"/$1/g; (my $link8 = $link7) =~ s/relnext/$1/g; (my $link9 = $link8) =~ s/"/$1/g; (my $link10 = $link9) =~ s/,/$1/g; print $link10; #print $link10; #$linkheader =~ s/next/$1/g; $urlstringapp = $link10; #print $urlstringapp; #break(); #print $linkheader; } } while ($linkheader=~ m/next/); #print @responsetextall; for my $i (0..$#responsetextall) { ## for my $i (0..@$responsetextall) { # for my $i (@responsetextall) { #print Dumper @responsetextall; $responsetextall[$i] =~ s/]\[/,/g; #print Dumper @responsetextall; ## for my $j (0..$#{$responsetextall[$i]}) { for my $j (0..$#{$responsetextall[$i]}) { # for my $j (@$responsetextall) { my $responseid = $responsetextall[$i][$j]{id}; my $responsests = $responsetextall[$i][$j]{status}; #$responselastlogin = $responsetextall[$i][$j]{lastLogin};

Replies are listed 'Best First'.
Re: Not an ARRAY reference
by Fletch (Bishop) on Nov 06, 2020 at 18:23 UTC

    Repeatedly using $1 on the RHS of a substitution without any capturing parens on the LHS looks extremely suspicious to begin with, to say nothing of trying to parse (SG|X|HT)ML with that chain of substitutions. But that never modifies anything in @responsetextall so it's probably moot.

    But you don't provide any sample of what the contents of @responsetext or @responsetextall ever contains so who knows other than the obvious answer that appparently none of them have any array references where you think they do. Perhaps you were at one time pushing a new array ref each time through (e.g. you might have had push @responsetextall, [ @responsetext ] or push @responsetextall, \@responsetext (the latter though depends on how/where it was declared otherwise you might get multiple copies of references to the same data)).

    The cake is a lie.
    The cake is a lie.
    The cake is a lie.

Re: Not an ARRAY reference (updated)
by AnomalousMonk (Archbishop) on Nov 07, 2020 at 00:48 UTC
    ...
    (my $link1 = $linkheader) =~ s/ .*? (self)/$1/gx;
    (my $link2 = $link1) =~ s/self/$1/g;
    ...
    Repeatedly using $1 on the RHS of a substitution without any capturing parens on the LHS looks extremely suspicious ...

    Further to Fletch's post:   Suspicious indeed. That code is almost certainly not doing what you expect.

    Win8 Strawberry 5.8.9.5 (32) Fri 11/06/2020 18:48:09 C:\@Work\Perl\monks >perl -Mstrict -Mwarnings use Data::Dump qw(pp); my $linkheader = 'axbyczSelfaxbycz'; printf "A: '$linkheader' (\$1 is %s) \n", pp $1; (my $link1 = $linkheader) =~ s/ .*? (Self)/$1/gx; printf "B: '$link1' (\$1 is %s) \n", pp $1; (my $link2 = $link1) =~ s/a/$1/g; printf "C: '$link2' (\$1 is %s) \n", pp $1; ^Z A: 'axbyczSelfaxbycz' ($1 is undef) B: 'Selfaxbycz' ($1 is "Self") Use of uninitialized value in substitution iterator at - line 9. C: 'Selfxbycz' ($1 is undef)
    After the first substitution s/ .*? (Self)/$1/gx (point B in the demo code), $1 has the value of 'Self' as expected.

    After the second substitution s/a/$1/g (point C), $1 is no longer defined because a successful match was made, but nothing was captured; there's no capture group. There are no captures in any of the following substitutions in the OPed code, so $1 never again has a defined value. (BTW: I'm surprised you didn't get a ton of "... uninitialized value ..." warnings. You do have warnings enabled, don't you? Along with strict?)

    Update: If you need to use the value of a capture variable in a subsequent regex, it's usually best to capture the value of the capture variable in an ordinary variable (with a nice, self-documenting name) immediately after the regex in which the capture variable is populated. E.g.:

    $string =~ m{ ... (pattern) ... }xms; my $you_can_trust_this = $1; ... ... m{ ... }xms; ... ... s{ ... (foo) ... }{$1...$you_can_trust_this}xms;
    You can even do this without any direct reference to a particular capture variable:
    my ($you_can_trust_this) = $string =~ m{ ... (pattern) ... }xms;
    Note the parentheses around the lexical to impose list context on the match.

    As Fletch mentioned, it looks like you might be trying to parse or transform some kind of ML with regexes. Don't. If this is your task, describe the task and ask for recommendations for appropriate modules; there are many well-tested and useful modules available.

    As a general note, if you're trying to parse or transform non-ML text with regexes, this can be very successfully done for one-to-one | one-to-one replacement applications, even for quite large blocks of text. See haukex's article Building Regex Alternations Dynamically. As an example:

    Win8 Strawberry 5.8.9.5 (32) Fri 11/06/2020 19:28:21 C:\@Work\Perl\monks >perl use strict; use warnings; my $string = 'foo + bar == baz'; my %xlate = ( 'foo' => 'wibble', 'bar' => 'wobble', 'baz' => 'boff', '+' => 'plus', '==' => 'equals', ); my ($rx_token) = map qr{ $_ }xms, join ' | ', map quotemeta, reverse sort keys %xlate ; print "\$rx_token $rx_token \n"; # for debug print "before: '$string' \n"; $string =~ s{ ($rx_token) }{$xlate{$1}}xmsg; print "after: '$string' \n"; ^Z $rx_token (?msx-i: foo | baz | bar | \=\= | \+ ) before: 'foo + bar == baz' after: 'wibble plus wobble equals boff'
    The %xlate translation table, a hash, is easily maintained and automatically produces the corresponding regex.


    Give a man a fish:  <%-{-{-{-<

Re: Not an ARRAY reference
by BillKSmith (Monsignor) on Nov 06, 2020 at 23:49 UTC
    Your for loops expect your complex data structure @responsetextall to be an array-of-arrays-of-structures something like this:
    use strict; use warnings; my @responsetextall = ( [ { id => 'id00', status => 'status00' }, { id => 'id01', status => 'status01' }, ], [ { id => 'id10', status => 'status10' }, { id => 'id11', status => 'status11' }, ], ); for my $i ( 0 .. $#responsetextall ) { #$responsetextall[$i] =~ s/]\[/,/g; for my $j ( 0 .. $#{ $responsetextall[$i] } ) { my $responseid = $responsetextall[$i][$j]{id}; my $responsests = $responsetextall[$i][$j]{status}; print "$responseid $responsests\n"; } }

    In your code, $responsetextall[$i] are not array references because you push array arrays (rather than array references)into @responsetextall. The following change will fix your immediate problem. I doubt that it is the only correction required.

    #push @responsetextall, @responsetext; push @responsetextall, [@responsetext];
    Bill

      Thanks Bill, I was trying to do pagination in a do/while loop and that works. The data structure was json responses Can you please let me know what could be the better pagination here? The code works eventually.

Re: Not an ARRAY reference
by AnomalousMonk (Archbishop) on Nov 07, 2020 at 06:57 UTC
    I have Array reference ...

    Very often solving a problem boils down to answering a basic question like "What is my actual data?" This and other basic debugging questions are covered in toolic's Basic debugging checklist.


    Give a man a fish:  <%-{-{-{-<

      Thanks for responding. Please check the data structure that i am looping for pagination. There are 200 same kinds of links like below

      <https://ssoqa.abcoffice.com/api/v1/users?after=200abcdefghijklmno55&l +imit=200>; rel="next"https://ssoqa.abcoffice.com/api/v1/users?after=2 +00abcdefghijklmno5&limit=200https://ssoqa.abcoffice.com/api/v1/users? +after=200abcdefghijklmno55&limit=200<https://ssoqa.abcoffice.com/api/ +v1/users?after=200abcdefghijklmno55&limit=200>; rel="self",

      There are data structure that i am writing for report and looping for loop . Please check below.

      [ { 'status' => 'ACTIVE', 'activated' => '2020-04-29T17:58:43.000Z', 'created' => '2020-04-29T17:58:42.000Z', '_links' => { 'self' => { 'href' => 'https://ssoqa.abcoffi +ce.com/api/v1/users/abcdefghijklmno55' } }, 'passwordChanged' => undef, 'lastUpdated' => '2020-11-06T19:09:08.000Z', 'id' => '00abcdefghijklmn6', 'lastLogin' => '2020-10-05T15:16:53.000Z', 'profile' => { 'email' => 'soula.olla@abc.com', 'middleName' => 'Y', 'countryCode' => 'IN', 'login' => '123456789@ADSYF.SYFBANK.COM', 'streetAddress' => '14-45,idhaian', 'organization' => 'ABC Organization', 'employeeNumber' => '123456789', 'division' => 'ABC - IT', 'AWSrole' => [], }, 'type' => { 'id' => 'omstdyc' }, 'credentials' => { 'provider' => { 'type' => 'AD', 'name' => 'AD.COM' } }, 'statusChanged' => '2020-06-01T08:56:35.000Z' }, { 'id' => '00mnopqrstq6', 'lastLogin' => ${\$VAR1->[0]{'passwordChanged'}}, 'profile' => { 'managerId' => '2456789', 'userType' => 'Contract', 'city' => 'Stamford', 'zipCode' => '00009', 'JobFunction' => 'IT', 'manager' => 'Kal, R', 'department' => 'IT - IN - CS&T - Client Ec +om', 'displayName' => 'Bh, M', 'lastName' => 'Bht', 'streetAddress' => '7 Lo Ri Rd', 'email' => 'R.Kal@syf.com', }, 'type' => { 'id' => 'obcdefghijklmn5' }, 'credentials' => { 'provider' => { 'type' => 'AD', 'name' => 'AD.ABC.COM' } }, 'statusChanged' => '2020-04-29T17:58:49.000Z', 'status' => 'ACTIVE', 'activated' => '2020-04-29T17:58:49.000Z', 'created' => '2020-04-29T17:58:47.000Z', '_links' => { 'self' => { 'href' => 'https://ssoqa.abc.com +/api/v1/users/00abcdefghijkl6' } }, 'passwordChanged' => ${\$VAR1->[0]{'passwordChanged'}}, 'lastUpdated' => '2020-11-06T19:09:10.000Z' }, ]

      I am getting an error after few times running the code - "JSON error at line 1, byte 1/156: Unexpected character 'C' parsing initial state: expecting whitespace: 'n', '\r', '\t', ' ' or start of string: '"' or digit: '0-9' or minus: '-' or start of an array or object: '{', '[' or start of literal: 't', 'f', 'n' at All-users-Apps.pl line 158." - Please let me know is this is a code issue or network issue.

      do { # Run get the users run_api_call($apiurlapplog1); @responsetext = parse_json ($client->responseContent()); push @responsetextall, @responsetext; $linkheader = $client->responseHeader("Link"); print $linkheader; if ($linkheader=~ m/next/) { (my $link1 = $linkheader) =~ s/ .*? (self)/$1/gx; (my $link2 = $link1) =~ s/self/$1/g; (my $link3 = $link2) =~ s/</$1/g; (my $link4 = $link3) =~ s/>/$1/g; (my $link5 = $link4) =~ s/;/$1/g; (my $link6 = $link5) =~ s/ /$1/g; (my $link7 = $link6) =~ s/=\"/$1/g; (my $link8 = $link7) =~ s/relnext/$1/g; (my $link9 = $link8) =~ s/"/$1/g; (my $link10 = $link9) =~ s/,/$1/g; print $link10; print $link10; #$linkheader =~ s/next/$1/g; $apiurlapplog1 = $link10; } } while ($linkheader=~ m/next/); for my $i (0..$#responsetextall) { $responsetextall[$i] =~ s/]\[/,/g; for my $j (0..$#{$responsetextall[$i]}) { my $responseid = $responsetextall[$i][$j]{id}; my $responsests = $responsetextall[$i][$j]{status}; if ($responsests ne "Deactivated") { + run_api_call($apiurluser. $responseid . $apiurl2); my @responsetext4 = parse_json ($client->responseContent()); my $responsecode1 = $client->responseCode() ; for my $m (0..$#responsetext4) { for my $n (0..$#{$responsetext4[$m]}) { + $response1 = $responsetext4[$m][$n]{id}; $resplable = $responsetext4 [$m][$n]{l +abel}; push @mouapps, $resplable; } } my $newurl = ($apiurluser . $responseid ); + run_api_call($newurl); + my @responsetext1 = parse_json ($c +lient->responseContent()); print Dumper @responsetext1 ; for my $l (0..$#responsetext1) { my $response2 = $responsetext1[$l]{id}; my $response3 = $responsetext1[$l]{profile}{firstName}; my $response4 = $responsetext1[$l]{profile}{lastName}; my $response5 = $responsetext1[$l]{profile}{email}; my $response6 = $responsetext1[$l]{credentials}{provide +r}{name}; ## Wrting to the reporting file. $worksheet->write(0, 0, 'OKTA-ID'); $worksheet->write(0, 1, 'FIRST NAME'); $worksheet->write(0, 2, 'LAST NAME'); $worksheet->write(0, 3, 'EMAIL-ID'); $worksheet->write(0, 4, 'Profile-Master'); $worksheet->write(0, 5, 'APPS'); $worksheet->write($r, 0, $response2); $worksheet->write($r, 1, $response3); $worksheet->write($r, 2, $response4); $worksheet->write($r, 3, $response5); $worksheet->write($r, 4, $response6); foreach (@mouapps) { local $" = '|'; my $response7 = join ("|","@mouapps"); $worksheet->write($r, 5, $response7); } $r += 1; } } else { print "Do Nothing" ; } } }

      2020-11-08 Athanasius fixed over-long line.

        chandantul:

        I am getting an error after few times running the code
        "JSON error at line 1, byte 1/156: Unexpected character 'C' parsing in +itial state: expecting whitespace: 'n', '\r', '\t', ' ' or start of s +tring: '"' or digit: '0-9' or minus: '-' or start of an array or obje +ct: '{', '[' or start of literal: 't', 'f', 'n' at All-users-Apps.pl +line 158."
        Please let me know is this is a code issue or network issue.

        With an error like this, it's not likely to be either a code or network issue. The error message is telling you that it's having trouble parsing the JSON data, so it's almost certainly a data issue. Since it tells you where in the data the error is, you should look at the JSON data to see what it looks like, and see if there's an obvious data problem.

        Normally when I see a JSON error, it comes down to one of the following:

        • There was an error on the server and instead of it sending JSON, it sent an error message and my code failed to check for errors before parsing the JSON.
        • Someone created the JSON by just pasting strings together rather than a JSON library, and they created some bogus JSON with an error like:
          • A text string contained a newline, so the JSON record was unterminated.
          • A text string contained a quote breaking the JSON quoting rules.
          • The sender got some trace/debug messages mixed up in the JSON string.
          • . . . other cases I don't recall off hand . . .
        • There's a mismatch in the encoding/decoding (for unicode, HTML entities, or other) that's butchered the JSON. This is one of the cases where it's more likely to be a code issue than a data issue, as the code steps on the data.
        • Congratulations! you've found a bug in a JSON routine... Note: Very unlikely if you're using a decent JSON package, as it would be used by many thousands of people in many applications. I won't say impossible as I've found a couple compiler bugs in several decades of programming, but I'd take a hard look at the incoming JSON code, my own code that processed the data long before I'd start suspecting a bug in a well-used / well-tested JSON library. I've never actually encountered this situation, I just included it for completeness.

        ...roboticus

        When your only tool is a hammer, all problems look like your thumb.

        2020-11-09 Athanasius fixed over-long line.

Re: Not an ARRAY reference
by tobyink (Canon) on Nov 07, 2020 at 08:52 UTC
      This solution is slightly better then my [@responsetext], but will not work at all unless @responsetext is declared with 'my' within the do/while loop.
      Bill

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: perlquestion [id://11123454]
Approved by LanX
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others browsing the Monastery: (4)
As of 2024-04-25 14:42 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found