Beefy Boxes and Bandwidth Generously Provided by pair Networks
We don't bite newbies here... much
 
PerlMonks  

Re: Faster locking

by oiskuu (Hermit)
on Apr 03, 2017 at 17:30 UTC ( [id://1186857]=note: print w/replies, xml ) Need Help??


in reply to Faster locking

I'll assume the lock_retrieve() and lock_nstore() both obtain a lock and then operate on the cachefile.

First off, you have some problematic races. There is one between -e $lockfile and open (LOCK, ">$lockfile"); where multiple script instances can fall through to the non-cached path (and then sequentially update the cache file).

Secondly, I think there's a race between close LOCK and unlink $lockfile. After the close, another process can obtain the lockfile, only for this to be promptly deleted, allowing a third process to grab the lock as well...

Thirdly, the locking will needlessly hinder cached reads when it happens.

Now about the fix. 1) Open the cache-file. 2) Stat the open file. 3) If age is less than 2*thresh, use the cached content. 4) If age is more than 1*thresh, trigger $update_needed. 5) When update is needed, launch the updater, or, if access delays are acceptable, perform the locked-update right there. 6) The updater can LOCK_EX|LOCK_NB the cache file, write a new tmpfile and rename that. Coded this way, the cached fast path needs no locking at all. HtH.

Edit. added LOCK_NB above, probably the simplest way to ensure a single updater is run. flock+fork should be acceptable for running the update.

Replies are listed 'Best First'.
Re^2: Faster locking
by bbs2web (Acolyte) on Apr 03, 2017 at 20:44 UTC

    Thank you! I dropped the .lock file and am now locking the script itself. It appears I can lock it again, with no apparent side effect, if it was locked earlier. Found a neat way of referencing the running script without using DATA but now can't find the site to reference it.

    The lock_retrieve and lock_nstore were intended to keep other processes waiting, whilst the updated data is being flushed to the cache file.

    Herewith the resulting script snippet:

    my $cache_ttl = 60; my $cachefile = '/tmp/mtapi.'.$router.'.cache'; my %bgp_peer; my $cached = 0; my $modified = (stat($cachefile))[9]; open our $lockfile, '<', $0 || die $!; if (defined $modified) { if (($modified >= (time-$cache_ttl)) || (!flock $lockfile, LOCK_EX | + LOCK_NB)) { if (%bgp_peer = %{lock_retrieve($cachefile)}) { $cached = 1 }; } } if ($cached == 0) { if (!flock $lockfile, LOCK_EX | LOCK_NB) { print STDERR "Another pro +cess is already using MikroTik API.\n"; exit 6 }; $api->connect(); $api->login(); %bgp_peer = $api->get_by_key('/routing/bgp/peer/print', 'name'); $api->logout(); lock_nstore \%bgp_peer, $cachefile; chmod 0660, $cachefile; flock $lockfile, LOCK_UN; }

      Could someone please confirm that Perl would skip trying to acquire a lock when the first 'if' condition is true?

      Should I rather place the test in to a nested condition, completely outside of the first?

      if ( (1 == 1) || (!flock $lockfile, LOCK_EX | LOCK_NB) )

      PS: Things appear to be working as expected, but I would still like to know, for my own peace of mind.

        Okay, so it works as expected:

        Test code:

        #!/usr/bin/perl use strict; use warnings; use Fcntl qw(:flock); open our $lockfile, '<', $0 || die $!; if ( (1 == 1) || (!flock $lockfile, LOCK_EX | LOCK_NB) ) { print "in loop\n"; sleep 30; if (flock $lockfile, LOCK_EX | LOCK_NB) { print "locked\n"; } else { print "not locked\n"; } sleep 30; }

        First instance:

        in loop locked

        Second instance:

        in loop not locked

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: note [id://1186857]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others having a coffee break in the Monastery: (2)
As of 2024-04-24 17:03 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found