Beefy Boxes and Bandwidth Generously Provided by pair Networks
good chemistry is complicated,
and a little bit messy -LW
 
PerlMonks  

Re: Will these functions for use with Storable properly lock my file?

by afoken (Canon)
on Aug 03, 2021 at 17:13 UTC ( #11135586=note: print w/replies, xml ) Need Help??


in reply to Will these functions for use with Storable properly lock my file?

use Storable; use Fcntl qw(:flock) sub _store { my $data = shift; my $data_file = 'data/plugin_portfolio'; store $data, $data_file; close LOCK; } sub _retrieve { my $data_file = 'data/plugin_portfolio'; return 0 if !-f $data_file; my $lock = $data_file . '.lck'; open (LOCK, "> $lock") or die "Can't open lock file"; flock(LOCK, LOCK_EX); retrieve $data_file; }

Not pretty.

  • Two-argument open was already mentioned in Re: Will these functions for use with Storable properly lock my file?. Use three-argument open.
  • Don't use bareword file handles. They are truely global and may collide with other (legacy) code using bareword file handles. Use lexical file handles.
  • _retrieve() leaks the bareword file handle LOCK, the lock file is kept locked until some other code explicitly closes that handle.
  • _store() closes a bareword file handle never opened there.
  • Are the previous two points intentional? Spooky action at a distance?

Regarding the last point: I did not look up the Storable API, so maybe Storable explictly requires this behaviour? Does Storable guarantee that _retrieve() and _store() are always in this order, and never alone? If so, the code lacks a clear comment indicating that behaviour.

I would expect Storable to call either _store() or _retrieve(), but not both. In that case, your code effectively locks only when you read first, but then until you end the process, read again, or write.

If you have two instances running in parallel, and one chooses to read, then do a lot of other stuff, the other instance will fail to get the lock even if the first instance has finished reading long ago. At least, in this situation you won't loose data. But you effectively have only one working process at any time.

If you have two instances running in parallel, and one chooses to write (calling _store()) without prior read, while the other chooses to read (calling _retrieve()), the writer will simply damage the data on disk while the reader assumes to be safe because it holds a lock. The writer doesn't even get an error, because close lacks error checks. Instant loss of data. (And trust me in that regard, Murphy will make sure that your data is damaged at the most inconvienient moment in time, causing the maximum damage.)

davido++ explains in Re: Will these functions for use with Storable properly lock my file? that Storable already has code for file locking. If that was not so, you should lock both reading and writing, each time using a lexical file handle that is implicitly unlocked and closed when you leave the reading and writing routines. That way, only one process can ever hold a lock for the data file, and it will hold the lock only as long as absolutely needed.

Alexander

--
Today I will gladly share my knowledge and experience, for there are no sweeter words than "I told you so". ;-)

Replies are listed 'Best First'.
Re^2: Will these functions for use with Storable properly lock my file?
by nysus (Parson) on Aug 04, 2021 at 01:43 UTC

    _retrieve() leaks the bareword file handle LOCK, the lock file is kept locked until some other code explicitly closes that handle. _store() closes a bareword file handle never opened there.

    Yes, I realize this is a litle precarious. It relies on my code always calling _retrieve() followed by _store(() at some point. It also counts on the lock getting removed when the process. Wasn't sure how else I could get around this.

    If you have two instances running in parallel, and one chooses to write (calling _store()) without prior read, while the other chooses to read (calling _retrieve()), the writer will simply damage the data on disk while the reader assumes to be safe because it holds a lock. The writer doesn't even get an error, because close lacks error checks. Instant loss of data. (And trust me in that regard, Murphy will make sure that your data is damaged at the most inconvienient moment in time, causing the maximum damage.)

    Yes. Bu tin my case, the processes are always upating the existing data in the file.

    I tried Storable's lock functions first. They didn't work in my case (though it entirely possible I wasn't using them right. It seemed to me that something like this was happening:

    1) Process A open data file to read, puts lock on it then removes lock + when done 2) Process B reads data file, puts lock on it, remove it when done. 3) Process C does the same. Now we have three different process with same data. Fine. But: 1) Process A finishes and writes to file. 2) Process D reads file. 3) Process B finishes and writes data file, overwriting A's work. 4) Process E reads file (saved by Process B). Now Process D and E have totally different starting points.

    But again, maybe I was using the lock functions wrong. Not sure.

    $PM = "Perl Monk's";
    $MCF = "Most Clueless Friar Abbot Bishop Pontiff Deacon Curate Priest Vicar";
    $nysus = $PM . ' ' . $MCF;
    Click here if you love Perl Monks

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: note [id://11135586]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others studying the Monastery: (3)
As of 2021-11-27 22:54 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found

    Notices?