I've written a scraper with WWW::Mechanize.

It logs in to a website every time I run it and returns a table of data.

So, now I'm thinking, why does it have to log in every time?

My browser doesn't have to log in every time. There's a timeout of an hour.

I've got this:

my $mech = WWW::Mechanize->new(
    cookie_jar => {
        file     => "/path/to/cookies.txt",
        autosave => 1
      }

);
[download]

in my code, and that file is being written to every time the script runs, but it's empty except for "#LWP-Cookies-1.0" at the top.

There's no "remember me" option on this website, so they're only for-this-session type cookies, but why doesn't Mech save those for me? That way the website would let my script return five minutes later and see it as still logged in, just like it does with my browser.

Maybe there's something about session cookies I don't understand? Any help gratefully received.

---

---

• Are you posting in the right place? Check out Where do I post X? to know for sure.
• Posts may use any of the Perl Monks Approved HTML tags. Currently these include the following:

  <code> <a> <b> <big> <blockquote> <br /> <dd> <dl> <dt> <em> <font> <h1> <h2> <h3> <h4> <h5> <h6> <hr /> <i> <li> <nbsp> <ol> <p> <small> <strike> <strong> <sub> <sup> <table> <td> <th> <tr> <tt> <u> <ul>

• Snippets of code should be wrapped in <code> tags not <pre> tags. In fact, <pre> tags should generally be avoided. If they must be used, extreme care should be taken to ensure that their contents do not have long lines (<70 chars), in order to prevent horizontal scrolling (and possible janitor intervention).
• Want more info? How to link or How to display code and escape characters are good places to start.