Beefy Boxes and Bandwidth Generously Provided by pair Networks
Just another Perl shrine
 
PerlMonks  

WWW::Mechanize::Chrome doesn't load/use cookies

by tunerooster (Acolyte)
on May 23, 2020 at 11:18 UTC ( [id://11117158]=perlquestion: print w/replies, xml ) Need Help??

tunerooster has asked for the wisdom of the Perl Monks concerning the following question:

When I run the command:
/usr/bin/chromium --user-data-dir=/home/rwk/chrome-session-for-walmart + https://grocery.walmart.com/orders
from a shell, chrome properly logs me in (from walmart cookies) and gets/displays my order page. However when I run the code below (same data dir, same chrome binary, same page) it *fails* to read the walmart login cookies and instead gets the walmart login page (as I can see in /tmp/out.html). The login page uses captchas, so the cookie approach is the only option to gain login status, but it is not working.

I have been searching the web and the perl docs for days with no success. How can I make WMC load my order page?

use Log::Log4perl qw(:easy); use WWW::Mechanize::Chrome; Log::Log4perl->easy_init($ERROR); # Set priority of root logger to ER +ROR my $mech = WWW::Mechanize::Chrome->new( headless => 1, start_url => 'https://grocery.walmart.com/orders', data_directory => '/home/rwk/chrome-session-for-walmart', incognito => 0, launch_exe => '/usr/bin/chromium', ); $mech->allow(javascript => 1); my $txt = $mech->content; open($f, '>', '/tmp/out.html'); print $f $txt; close $f;

P.S. From another post, I am told that without headless mode, two windows open (a known bug), with WMC attached to an incognito window (which understandably does not read the cookies - no account). I have not found a way around this.

P.P.S. My understanding is that using: data_directory => '/home/rwk/chrome-session-for-walmart' should allow WMC to save/load cookies automatically. Is that correct?

Replies are listed 'Best First'.
Re: WWW::Mechanize::Chrome doesn't load/use cookies
by nysus (Parson) on May 23, 2020 at 14:40 UTC
      Thanks...

      Both the shell command and perl script are using /usr/bin/chromium.

      How is the "profile" option different from: data_directory => '/home/rwk/chrome-session-for-walmart'.

      In the chromiun man page it says:

      --user-data-dir=DIR Specifies the directory that user data (your "profile") +is kept in. Defaults to ~/.config/chromium . Separate instances of Chromium +must use separate user data directories; repeated invocations of chromium-browser + will reuse an existing process for a given user data directory.

      In the WMC doc it says:

      profile profile => '/path/to/profile/directory' # set the profile directory By default, your current user profile directory is used. Use this sett +ing to change the profile directory for the browsing session. data_directory data_directory => '/path/to/data/directory' # set the data directory By default, the current data directory is used. Use this setting to ch +ange the base data directory for the browsing session.

      What's the difference between "profile directory" and "data directory"?

        The profile option is used for the Chrome --profile-directory option, but that doesn't change anything for me. What works for getting the "current" cookies is the data_directory option.

Re: WWW::Mechanize::Chrome doesn't load/use cookies
by JediMasterT (Sexton) on May 24, 2020 at 12:34 UTC
    Random debugging thought: Try starting the instance normally (removing the "start_url" option) and navigating to the url you're looking for.

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: perlquestion [id://11117158]
Approved by davies
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others musing on the Monastery: (3)
As of 2024-04-26 05:56 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found