http://qs321.pair.com?node_id=1197014

ytjPerl has asked for the wisdom of the Perl Monks concerning the following question:

Hi Folks, I am working on a script to check whether application started by matching a string in its log file. I intend to populate an array with last lines (20lines) of the log file instead of the entire file then match a string which stands for starting successfully. But if application restarts multiple time within a short period, I cannot know if it is the match string from last start up. My code is still searching in entire file. If anyone knows how to correct my code, please do. Thanks 
 use File::ReadBackwards;
use strict;
use autodie;
use File::stat;
my $file = 'D:/service_console.log';
my $bw = File::ReadBackwards->new($file);
while (defined ( my $log_line = $bw->readline))
{  for( my $n=0; $<=3; $n++)
   {
    my $size = stat($file)->size;
    sleep(60);
    my $size = stat($file)->size;
    last if ( $size1 == $size2);
    }
    push (my @array, $log_line);
   for (@array)
   { last if (/ running mode/);}
}$bw->close();

[download]

Replies are listed 'Best First'.
Re: how to populate array with last 20 lines of file
by talexb (Chancellor) on Aug 08, 2017 at 17:28 UTC

    I can highly recommend the File::Tail module as part of a solution to this problem.

    Alex / talexb / Toronto

    Thanks PJ. We owe you so much. Groklaw -- RIP -- 2003 to 2013.

[reply]
      I tried it. 
      use strict;
use File::tail;

my $file =  File::Tail->new('C:/Users/tiang/Documents/Perl/daily.20170
+731142932.txt');

while(defined(my $line = $file->read))
{

   print "$line\n";
}

      [download]
      the file is only 8KB, but it just got stuck. nothing in progress..
[reply]
[d/l]

        Yeah .. not surprising, since you tail the file once, and then stop.

        Why not read it, sleep for a minute, then read it again, as your original script tried to do?

        Alex / talexb / Toronto

        Thanks PJ. We owe you so much. Groklaw -- RIP -- 2003 to 2013.

[reply]
[d/l]

        Try it again, but like this:

        #!perl -slw
use strict;
use File::Tail;

my $logfile = File::Tail->new(
  name        => 'logfile.txt' # daily logfile
  ,maxinterval => 5            # poll logfile every 5 seconds
  ,interval    => 3            # wait 3 seconds before initial 'tail'
  ,tail        => 5            # give me last 5 lines
);

print while(defined($_ = $logfile->read));

        [download]

        Best Regards,
        Shadowsong

[reply]
[d/l]
        the file is only 8KB

        Why not read the whole file then ?

[reply]
Re: how to populate array with last 20 lines of file
by dasgar (Priest) on Aug 08, 2017 at 18:56 UTC

    I don't know if this is the "best" way, but one route would be to use Tie::File. It basically "represents a regular text file as a Perl array". And since it does not store the entire file into memory, you can use this for large files without major concern about memory usage - although the 8KB size you mentioned in another post isn't too big to store in memory.

    If you use Tie::File, grabbing the last 20 lines is simply grabbing the last 20 elements of the array.

[reply]
      Hi, Thanks for your posting. The major concern of not using the whole file is not because of memory issue. I need to find a string in that file to determine whether application is running or not. If I restart the application multiple times, there will be multiple 'startup' string in this log, so that I only need last lines of the log when its size is not increasing anymore.
[reply]
Re: how to populate array with last 20 lines of file
by thanos1983 (Parson) on Aug 09, 2017 at 10:23 UTC

    Hello ytjPerl,

    There is an alternative solution to your problem using module File::ReadBackwards

    Sample of code bellow:

    Read more... (1295 Bytes)

    Sample of in.log for testing purposes:

    Read more... (366 Bytes)

    As a next step you can apply a grep or a regex on the lines that you have collected and do what ever you want with the process.

    Update: I would use flock to lock the file as it is important not get mixed on an instance that data are populated on the file. Sample of code bellow:

    Read more... (2 kB)

    Update2: In case you want to loop again and again the file in reverse mode you can apply something like that. Sample of code bellow:

    Read more... (2 kB)

    Update3 Last Update: Similar solution different way of writing it, just for fun and preference. Sample of code bellow:

    Read more... (311 Bytes)

    Hope this helps, BR.

    Seeking for Perl wisdom...on the process of learning...not there...yet!
[reply]
[d/l]
[select]
      It is definitely helpful. I will try them all to see which solution works best to meet my needs. I really appreciate that. very thoughtful.
[reply]

Back to Seekers of Perl Wisdom