Beefy Boxes and Bandwidth Generously Provided by pair Networks
Problems? Is your data what you think it is?
 
PerlMonks  

FIFO logfile done with flat file?

by u914 (Pilgrim)
on Jul 28, 2002 at 19:45 UTC ( #185843=perlquestion: print w/replies, xml ) Need Help??

u914 has asked for the wisdom of the Perl Monks concerning the following question:

So... i have some scripts which do some things on sets of files in directories. In order for them not to clobber each other, i need to tell one script what the last 50 actions of the other have been.

My best guess is that this could be most easily/efficiently accomplished with a flat text file, one that never grows larger than it's indicated max size (like 50 or 100 lines).

i've searched here, Super Search, CPAN, googled perlmonks.thepen.com and asked in the chatterbox, but i cannot find the previous node (saw it a month or so ago) that answered this very question.

If there's a better way to do this, and/or a module already that accomplishes something equivalent, i'd be happy to know!

Replies are listed 'Best First'.
Re: FIFO logfile done with flat file?
by clintp (Curate) on Jul 28, 2002 at 20:22 UTC
    You could do it with a flat file. There's lots of ways. One script I had used a simple DBM file and went something like this (hope this is cut down right):
    { use Fcntl qw(:flock); my %h; my $size=5; sub ringopen { open(F, ">ring.lock") || die; flock(F, LOCK_EX); dbmopen(%h, "ring", 0666) || die; } sub ringclose { dbmclose(%h); close(F); } sub ringadd { ringopen(); local $_=$h{next}; delete $h{$_-$size} if (defined $_); $h{$h{next}++}=$_[0]; ringclose(); } sub ringdump { ringopen(); my @a=sort { $a <=> $b } grep /^\d+$/, keys %h; @a=@h{@a}; ringclose(); return @a; } }
    And you used it like this:
    for(0..9, 'a'..'z') { ringadd($_); } print ringdump();
    To initialize the ring buffer, remove the ring* files.

    YMMV, TIMTOWDI, etc..

Re: FIFO logfile done with flat file?
by tstock (Curate) on Jul 28, 2002 at 20:48 UTC
    my $filepath = "/tmp/fifo"; logInFIFO('log line example'); sub logInFIFO { my $logline = shift; my $fifolines = 5; my @log; if (-e $filepath) { open(IN, "< $filepath") || die $!; @log = (<IN>); close IN; } # transform and validate $logline here push @log, "$logline\n"; shift @log while ($#log > $fifolines-1); eval { open(OUT, ">$filepath") || die $!; print OUT @log; close(OUT) || die $!; }; return $@ ? warn($@) && 0 : 1; }
    no file locking... buyer beware.

    Tiago
Re: FIFO logfile done with flat file?
by u914 (Pilgrim) on Jul 28, 2002 at 22:52 UTC
    Well, i hope it's not bad form to answer your own question...

    Two (so far) very good replies, but in the meantime i found something even easier|better, i think.

    This node is the one i was looking for, and it's all about Tie::File, which essentially turns any flat-file into an array manipulable with Perl in the usual fashion. Absolutely perfect for what i needed.

    Here's a stripped-down version of the code i ended up generating:

    tie @log, 'Tie::File', $logfile or displayError("could not open logfil +e"); while ($some_condition) { #do some other stuff, and unshift @log, $info_to_be_logged; } # find out how long the logfile is, and truncate it $length = @log; $truncate = $length - 150; if ($truncate > 0) { $#log -= $truncate; } untie @log;
    By using unshift, each new entry is made at the beginning of the array/file. By truncating the array/file at a fixed number of records before closing, the file essentially becomes FIFO. Of course this isn't perfect, the logfile can swell up a lot while in use, before it gets truncated and there isn't any locking (unless it's built into Tie::File, it seems to be one mightily sophisticated module), but i'm not too concerned.

    For the curious, here's a copy of the actual sub that it's used in.... yes, this is my 'production' code, and i know it's pretty horrible ;-)

    sub delete_threads_from_file { $logfile = "$htdocs${bbs_location}/$removed_log"; tie @log, 'Tie::File', $logfile or displayError("could not open lo +gfile"); local($file, $delete_hash) = @_; local(*FILE); local($new_content); if ((open(FILE, "$file"))) { while($line = <FILE>) { if ($line =~ /<!-- mid="(.*?)" -->/) { local($id) = ($line =~ /<!-- mid="(.*?)" -->/)[0]; $should_delete = 0; foreach $delete_id (keys(%{$delete_hash})) { # look for exact id matches if ($id eq $delete_id) { $should_delete = 1; # add this id to the removed.log unshift @log, $delete_id; } # if the delete_id(message) is a topic message, # remove all its responses local($d_thread_id,$d_message_id) = split(/_/, $de +lete_id); local($thread_id,$message_id) = split(/_/, $id); if (($d_thread_id == $d_message_id)&&($thread_id = += $d_thread_id)) { $should_delete = 1; # if this is a response to a removed topic, # add it to the removed.log if ($id ne $delete_id) { $should_delete = 1; unshift @log, $id; } } } if (! $should_delete) { # this id is NOT in our list of ids to delete, so +keep it $new_content .= $line; } } else { $new_content .= $line; } } close(FILE); # find out how long the logfile is, and truncate it $length = @log; $truncate = $length - 150; if ($truncate > 0) { $#log -= $truncate; } untie @log; } ### ### modify the file ### if ((open(FILE, "> $file"))) { print FILE $new_content; close(FILE); } }

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: perlquestion [id://185843]
Approved by CubicSpline
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others avoiding work at the Monastery: (2)
As of 2021-12-01 02:55 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found

    Notices?