http://qs321.pair.com?node_id=1226107


in reply to Extract Block Of Text From Log

Well, i'd use a classic state machine for this problem, then we can work line-by-line, without having to load the whole file into memory.

#!/usr/bin/env perl use strict; use warnings; use diagnostics; my @blocklines; my $inblock = 0; open(my $IFH, '<', 'blockextract.txt') or die($!); while((my $line = <$IFH>)) { chomp $line; if($line =~ /parameters\ after\ change/) { # Start of a block we want to read $inblock = 1; next; } # Skip handling line unless we are in a block next unless($inblock); if($line =~ /NRG\ location/) { # Block ends here # Do whatever you want to do to the block lines # stored in @blocklines. # I'm just dumping them to STDOUT print "*** START ***\n"; print join("\n", @blocklines), "\n"; print "*** END ***\n\n"; # Clean up block @blocklines = (); $inblock = 0; next; } # just some line within the interesting block # remember it for later in @blocklines push @blocklines, $line; next; } close $IFH; exit(0);

That way, we can even modify the program very slightly to make it work via pipes, working live on a stream of data generated by some other program. We just remove the open and close calls and change the while loop a bit:

while((my $line = <>)) {

Then we can use the program on an arbitrary stream of this kind of data, and it extracts each block as soon as it is pushed into the programs STDIN:

cat blockextract.txt | perl blockextract.pl

And the only thing the state machine has to hold in memory is the block it is currently working on and a single state variable...

perl -e 'use MIME::Base64; print decode_base64("4pmsIE5ldmVyIGdvbm5hIGdpdmUgeW91IHVwCiAgTmV2ZXIgZ29ubmEgbGV0IHlvdSBkb3duLi4uIOKZqwo=");'

Replies are listed 'Best First'.
Re^2: Extract Block Of Text From Log
by LanX (Saint) on Nov 21, 2018 at 12:22 UTC
    > Well, i'd use a classic state machine for this problem, then we can work line-by-line, without having to load the whole file into memory.

    Sorry for nitpicking, but ... :)

    Flip/Flop solutions are state machines and do it in one pass.

    I even demonstrated that you can safely nest them, hence the technology is "scalable".

    The advantage of your solution is that it's easily portable to other languages, i.o.W. it ignores Perl's possibilities. ;-)

    Cheers Rolf
    (addicted to the Perl Programming Language :)
    Wikisyntax for the Monastery FootballPerl is like chess, only without the dice

      I fully support your argument. I just like to write easy-to-read easy-to-port examples.

      When you write stuff that might still have to be supported in a couple of decades, then making it easy to read and understand is much more important than making it "nice and fast and only understandable by perlmonks" ;-)

      perl -e 'use MIME::Base64; print decode_base64("4pmsIE5ldmVyIGdvbm5hIGdpdmUgeW91IHVwCiAgTmV2ZXIgZ29ubmEgbGV0IHlvdSBkb3duLi4uIOKZqwo=");'
        > only understandable by perlmonks" ;-)

        I disagree because state-machines need a lot of documentations and flip/flop is well documented in perldoc.

        At worst you'll just need to add a link in the comments.

        > just like to write easy-to-read easy-to-port examples.

        So consequently you use goto instead of loop constructs, to keep it easily portable to assembler? ;-)

        Cheers Rolf
        (addicted to the Perl Programming Language :)
        Wikisyntax for the Monastery FootballPerl is like chess, only without the dice