Beefy Boxes and Bandwidth Generously Provided by pair Networks
Think about Loose Coupling
 
PerlMonks  

Safely read/write a file simultaneously

by golux (Chaplain)
on May 17, 2018 at 00:40 UTC ( #1214701=CUFP: print w/replies, xml ) Need Help??

I'm working on a Client/Server project where a "progress file" gets updated (potentially quite often) and read from a separate (CGI) process. I was pretty sure having the writer reopening the progress file could cause the reader to have occasional problems, and also fairly sure that writing to a tempfile instead (and then moving the tempfile over the progress file) would be much safer (ie. atomic).

But why Google it when you can write code to test it instead? ;-)

Here's the result, which indicates I was correct on both counts, and happily the latter seems to be atomic enough that an error never occurs. Set or clear the value of $unsafe to try the different algorithms.

#!/usr/bin/perl ############### ## Libraries ## ############### use strict; use warnings; use File::Copy; use Function::Parameters; use IO::File; ################## ## User-defined ## ################## my $file = 'file.txt'; my $rdelay = 0.03; # Read delay: 3/100th of a second my $wdelay = 0.01; # Write delay: 1/100th of a second my $unsafe = 1; # Set to zero to call the "safe" write algorit +hm ################## ## Main Program ## ################## $| = 1; if (fork) { writer($unsafe); } else { reader(); } ################# ## Subroutines ## ################# # # Writes to a file many times per second. # fun writer($unsafe) { my $count = 0; while (1) { if ($unsafe) { writefile1($file, $count++); } else { writefile2($file, $count++); } # Sleep for 1/100th of a second select(undef, undef, undef, $wdelay); } } # # Reads from the file, displaying number of total errors # (each time the $count was undefined). # fun reader() { sleep 1; # Give the writer time to create the file initiall +y my $nerrs = 0; # How many total errors did we get? while (1) { my $count = readfile($file); if ($count) { printf "%8d, ", $count; } else { printf "\nTotal errors = %d\n", ++$nerrs; sleep 1; } } select(undef, undef, undef, $rdelay); } # # Algorithm 1 # # Writes the $value directly to the file # This turns out to be quite prone to error when the file is read. # fun writefile1($file, $value) { my $fh = IO::File->new; open($fh, '>', $file) or die "Can't write '$file' ($!)\n"; print $fh "$value\n"; close($fh); } # # Algorithm 2 # # Writes the $value to a temp file, then moves the tempfile over the # actual destination. This turns out to be quite safe for reading. # fun writefile2($file, $value) { my $fh = IO::File->new; my $tmp = 'tmp.txt'; open($fh, '>', $tmp) or die "Can't write '$file' ($!)\n"; print $fh "$value\n"; close($fh); move($tmp, $file); } # # Reads the $value from the $file # fun readfile($file) { my $fh = IO::File->new; open($fh, '<', $file) or die "Can't read '$file' ($!)\n"; my $value = <$fh>; defined($value) or return 0; chomp($value); close($fh); return $value; }
say  substr+lc crypt(qw $i3 SI$),4,5

Replies are listed 'Best First'.
Re: Safely read/write a file simultaneously
by haukex (Bishop) on May 17, 2018 at 08:01 UTC

    Whether or not rename(2) will be atomic depends very much on the operating and file systems. Unfortunately File::Copy's move adds another layer of uncertainty, since it doesn't guarantee to use rename under the hood. But then again, when the rename operation is atomic, it's a great way for updating a file with one writer and many readers (multiple writers still need to coordinate). You might be interested in my module File::Replace.

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: CUFP [id://1214701]
Approved by Athanasius
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others browsing the Monastery: (6)
As of 2020-12-03 20:02 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?
    How often do you use taint mode?





    Results (57 votes). Check out past polls.

    Notices?