|
blahblah has asked for the wisdom of the Perl Monks concerning the following question:
Hello,
I wrote some code that needs to open a storable file and keep a lock on it until I say to let go (because I am modifying the data while I have it open). I thought the best way to do this would be to write a routine to open and lock a filehandle, which would then pass that filehandle into the routine that would read it through storable. Unfortunately I seemed to have coded myself into a circle and can't get this to quite work right. Any ideas? Many thanks!
#!/usr/bin/perl -w
use strict;
use Fcntl qw(:DEFAULT :flock);
use Storable qw(store_fd fd_retrieve);
use Data::Dumper;
$Data::Dumper::Deepcopy=1;
$Data::Dumper::Purity=1;
$Data::Dumper::Sortkeys=1;
sub db_load {
my $fh = $_[0];
return fd_retrieve($fh);
}
sub db_save {
my ($fh, $db_hashref) = @_;
store_fd($db_hashref, $fh);
return $fh;
}
sub fileopenandlock {
my ($fh, $file) = @_;
sysopen($fh, $file, O_RDWR|O_CREAT, 0777) or
die("I can't open the filehandle $fh for file $file in order to
+lock it\n");
flock($fh, LOCK_EX) or
die("I can't get an exclusive lock LOCK_EX on the file handle $f
+h\n");
return $fh;
}
sub filecloseandunlock {
my $fh = $_[0];
flock($fh, LOCK_UN) or
croak("I can't unlock LOCK_UN the file handle $fh\n");
close($fh) or
croak("I can't close the filehandle $fh\n");
return 1;
}
my %data = ( 1 => 'FEE',
2 => 'FYE',
3 => 'FOH',
4 => 'FUM', );
my $db_file = "./giant.db";
# open and read db
my $db_fh = do { local *DB };
my $db = db_load(fileopenandlock($db_fh,$db_file));
print "First read dump:\n";
print Dumper($db) . "\n\n"; # should dump empty
# fill it up with data
$db->{$_} = $data{$_} for sort keys %data;
print "Dump before write:\n";
print Dumper($db) . "\n\n"; # should dump full
# save db
filecloseandunlock(db_save($db_fh,$db));
# test read - did it work?
my $db_fhnew = do { local *DBNEW };
my $dbnew = db_load(fileopenandlock($db_fhnew,$db_file));
print "Second read dump:\n";
print Dumper($dbnew) . "\n\n"; # should dump full
filecloseandunlock($dbnew);
Thanks |
Re: Storable and passed filehandles
by TedPride (Priest) on Aug 14, 2005 at 22:37 UTC
|
Some comments. First of all, I don't know what system you're using, but my Unix hosting account will wait on flock rather than returning an error if another flock is currently running. Second, it's very inconvenient to flock the file you're trying to process. What if you want to open a file, read its contents, close it, open it for writing, write some new or edited data, and close? If you're flocking the primary file, the flock will be gone between the first close and the second open. What you need to do instead is create a temporary file based off the name of the file you're trying to lock, lock that, do whatever processing needs to be done on the primary file, then unlock. This will give you much more flexibility. Thirdly, you shouldn't have your flock routine also be your open routine, and your f(un)lock be your close. Separate the processes so you don't have to change your routines every time you want to do something different to the files you're editing.
As for your code, any hints as to where you think the problem(s) lie? | [reply] |
Re: Storable and passed filehandles
by Roger (Parson) on Aug 15, 2005 at 06:03 UTC
|
The flock is only advisory - it is not guarranteed.
The best way to coordinate your programs on the same server is to use a proper interprocess locker, such as IPC::Semaphore (I assume that you are working on Unix systems), create a public lock, and have all the processes that want to read the file to obtain that lock before accessing the file.
If the file you are trying to access is on the network, then use a network locking mechanism, such as, say, IPC::Locker.
| [reply] |
|
|
I am working on a unix system (FreeBSD), and thanks for the semaphore pointer (further explained here).
I guess I'm just surprised that the above doesn't work. As I test and test and re-test I am completely duped that even this simple code doesn't work...
#!/usr/bin/perl -w
# start with no existing giant.db file
use strict;
use Fcntl qw(:DEFAULT :flock);
use Storable qw(store_fd fd_retrieve retrieve);
use Data::Dumper;
$Data::Dumper::Deepcopy=1;
$Data::Dumper::Purity=1;
$Data::Dumper::Sortkeys=1;
my %data = ( 1 => 'FEE', 2 => 'FYE',
3 => 'FOH', 4 => 'FUM', );
sysopen(*DB, "giant.db", O_RDWR|O_CREAT, 0666) or die("sysopen: $!\n")
+;
flock(*DB, LOCK_EX) or die("flock: $!\n");
my $hashref = fd_retrieve(*DB);
# modify the data, do work, etc..
$hashref->{$_} = $data{$_} for sort keys %data;
store_fd($hashref, *DB) or die("store_fd: $!\n");
truncate(*DB, tell(*DB));
close(*DB);
I'm totally spent and frustrated. Does anyone have any idea why this doesn't work? I must be missing something. I'm just trying to block access to a file while I change it. The above SHOULD work, shouldn't it?....shouldn't it?....please?
Thanks | [reply]
[d/l] |
|
|
Try the flock implemented by Storable first...
#!/opt/kalm/bin/perl -w
use strict;
use Storable qw(lock_store lock_retrieve);
use Data::Dumper;
$Data::Dumper::Deepcopy=1;
$Data::Dumper::Purity=1;
$Data::Dumper::Sortkeys=1;
my %data = ( 1 => 'FEE', 2 => 'FYE',
3 => 'FOH', 4 => 'FUM', );
my $file = 'test.dat';
lock_store \%data, $file;
my $hashref = lock_retrieve $file;
print Dumper($hashref);
| [reply]
[d/l] |
|
|
|
|
In general, passing in \*DB is better than just *DB.
Looking at the docs for Storable I see both ways documented.
However, when reading the source, it looks like using just *DB would get you a fatal "not a ref" thrown, though I don't see you describe the failure beyong "doesn't work" (the universal useless problem description) in your node.
| [reply] |
|
|
|
|
|
|
|
