Things you should test:
- Double check that you're indeed using unbuffered IO everywhere.
- If I understand you correctly, you're forking children to read:
- use strace -f instead for the _real_, _complete_ trace.
- consider to explicitly open the files separately in each child (my bet - fixes issues both for the pointer AND for the locking).
- keywords: perldoc -f fork, man 2 fork, file-descriptor/file-handle, race-condition
Concerning inherited filehandles and the file pointer
Note that I use open/print with autoflush instead due to laziness.
1> perl -e 'use IO::Handle; open(FH,">out"); FH->autoflush(1);
if(fork){
print FH "aaaa"; sleep 2; print FH "bbb";
close FH
}else{
sleep 1; print FH "1111"; seek FH,0,0; print FH "4";
close FH
}'
Inherited file handle is shared between parent and child, thus we should find 4bbb1111 in ./out on a reasonably idle host.
2> perl -e 'use IO::Handle; open(FH,">out"); FH->autoflush(1);
if(fork){
print FH "aaaa"; sleep 2; print FH "bbb";
close FH
}else{ close FH;open(FH,"+<out") or warn "err"; FH->autofl
+ush(1);
sleep 1; print FH "1111"; seek FH,0,0; print FH "4";
close FH1
}'
Same file but opened separately: 4111bbb - the pointer is no longer shared.
Concerning locking
- Perl's flock flushes the handle ( not an issue if you use sys*/... consistently :))
- Perl's flock uses one of flock/lockf/fcntl as base mechanism, which have somewhat
differing semantics ranging from required read or write modi to possible nfs issues.
While Perl's choice is usually sane, I'd still prefer to simplify what I'd use
flock on: Here I'd suggest to try explicit independent open's of the file we try to lock.
Consider writing a simplified test case for the locking with warn() on either side of
the flock plus another set around the close()(/unlock). Note that unlocking with other
than close() may allow a race when using buffered IO.
- a quick test of flock wrt shared / separate FD:
- (got me again:( ) strictures and checking the flock return: A missing qw/LOCK_EX/ when using use POSIX
is a nice way to waste some time when omitting use strict;use warnings; for a quick test.
check that LOCK_EX==2.
-
Results are just as my finely honed paranoia suspected: flock() works in case 2, but is "idempotent"
as long as the same file descriptor is used (case 1). man 2 lock offers this explanation for
the situation on Linux (double-check for POSIX/your OS...):
A single file may not simultaneously have both shared and exclusive locks.
Locks created by flock() are associated with an open file table entry. This
means that duplicate file descriptors (created by, for example, fork(2) or
dup(2)) refer to the same lock, and this lock may be modified or released using
any of these descriptors. Furthermore, the lock is released either by an
explicit LOCK_UN operation on any of these duplicate descriptors, or when all
such descriptors have been closed.
If a process uses open(2) (or similar) to obtain more than one descriptor for
the same file, these descriptors are treated independently by flock(). An
attempt to lock the file using one of these file descriptors may be denied by a
lock that the calling process has already placed via another descriptor.
A process may only hold one type of lock (shared or exclusive) on a file. Sub‐
sequent flock() calls on an already locked file will convert an existing lock to
the new lock mode.
HTH
Peter