Do You have long running applications, which write logfiles and do rotate them when they reach a maximum size? Do you need to
'tail -f' them?
As I could not find a solution, I try to solve this with a module using tie. I use ActiveState on Windows, the latter gives me some additional problems with file handling.
The following is more a proof of concept than a solution, even if it seems to work for me.
The idea is to use normal file io-functions like open, readline or seek etc and tie this to a module, which transparently switches the files after rotation. The file must not stay open, otherwise the rotation of the locked file fails.
My logfile is written by an aplication with the great
Log::Agent module combined with
Log::Agent::Rotate. The important part of my reader is the standard solution for perl tail:
my $logfile = "logfile";
open( LOG, "<$logfile" ) or die "$logfile: $!";
while (!$end_condition)
{
while ( my $line = <LOG> ) {
print $line;
}
sleep(1); # do something usefull
seek( LOG, 0, 1 );
}
close(LOG);
This way, it does not allow the logfile to rotate, as it is open. My solution inserts just a
tie before the
open-statement:
tie *LOG, "Tie::Tailrotate", "$logfile";
open ...
The module enabling the rotation is as follows (only main parts, see below for open questions)
package Tie::Tailrotate;
use strict; # ;-) learned that
use Carp;
sub TIEHANDLE {
my $class = shift;
my $fname = shift;
# need $fname to open file and initialize filehandle
if ( $fname =~ /[>+|]/) {
croak "Only read support for $fname\n"; }
open my $self, $fname
or croak "Open $fname failed:$!";
# $self is filehandle,
# but created is $$$self, @$$self and %$$self
# see Camel Book: 14.4. Tying Filehandles
$$self->{filename} = $fname;
$$self->{curpos} = 0;
return bless $self, $class; # $self is a glob ref
}
sub OPEN { # needs to be covered
my $self = shift;
my $fname = shift;
$self->CLOSE;
if ( $fname =~ /[>+|]/) {
croak "Only read support for $fname\n"; }
open $self, $fname
or croak "Open $fname failed:$!";
$$self->{curpos} = 0;
$$self->{filename} = $fname;
close($self);
return 1;
}
sub READLINE {
my $self = shift;
my $line;
my $size = (stat $$self->{filename} )[7];
# if the last position I read from is bigger than
# actual file size, the file rotated. Continue
# reading from last file, last position.
my $use_active_file = ($$self->{curpos} <= $size );
if ($use_active_file)
{ # continue with active logfile
open $self, $$self->{filename}
or croak "Open $$self->{filename} failed:$!";
seek ($self, $$self->{curpos}, 0 );
$line = <$self>; # undef if eof
$$self->{curpos} = tell($self) if ($line);
}
else { # File rotated, Log::Agent naming convention
my $rot_fname = "$$self->{filename}.0";
$size = (stat $rot_fname )[7];
open $self, $rot_fname
or croak "Open $rot_fname failed:$!";
seek ($self, $$self->{curpos}, 0 );
$line = <$self>; # undef if eof
if ($line) { $$self->{curpos} = tell($self); }
else
{ # reset position, to continue with active file
$$self->{curpos} = 0;
}
}
close($self); # unlock file
return $line;
}
I left out the
seek,
close etc. subs, the above should illustrate my concept. Remember, this is a first hit and needs lot of fine-tuning (or throw it away?) Your comments please.
And it has problems:
- there is still achance, that the reader blocks the file, or hit a locked file (remove the sleep from the reader to see it).
- the tie staement needs the filename to open the file and initialize $self with the filehandle (could/should be created manually)
- better checks for readonly opening of files
- implement a better seek for positioning
- implement a better close
- implement initial start position, e.g. start with last 10 lines instead of last 1000s from the beginning of a big logfile.
- more general solution
And it came to pass that in time the Great God Om spake unto Brutha, the Chosen One: "Psst!"
(Terry Pratchett, Small Gods)