Beefy Boxes and Bandwidth Generously Provided by pair Networks
Do you know where your variables are?
 
PerlMonks  

Getting lines in a file between two patterns

by darklord_999 (Acolyte)
on Jul 05, 2012 at 04:49 UTC ( #979946=perlquestion: print w/replies, xml ) Need Help??

darklord_999 has asked for the wisdom of the Perl Monks concerning the following question:

I have a file named input.txt which contains the following data

XXXX

YYYY

START

These are the first

set of lines

which are to be extracted

END

XXX

ZZZ

YYY

START

These are the second

set of lines

which are to be extracted

END

aasds

tteret

tertetr

......

.....

---------------

I want to extract only the data which is present in between the START and END tags while reading the file using a file handle.How to do this ?

  • Comment on Getting lines in a file between two patterns

Replies are listed 'Best First'.
Re: Getting lines in a file between two patterns
by Utilitarian (Vicar) on Jul 05, 2012 at 05:13 UTC
    One option would be to use the range operator eg.
    perl -e 'while (<>){print if (/^START/../^END/);}' input.txt
    print "Good ",qw(night morning afternoon evening)[(localtime)[2]/6]," fellow monks."
      This will also print START/END, if this is not desired, try:
      perl -ne 'if (/START/../END/) {print unless /START/ or /END/}'
Re: Getting lines in a file between two patterns
by GrandFather (Sage) on Jul 05, 2012 at 05:14 UTC

    What have you tried? How did it fail?

    True laziness is hard work
Re: Getting lines in a file between two patterns
by johngg (Canon) on Jul 05, 2012 at 09:45 UTC

    Another solution, if the printing of "START" and "END" is not desired, would be to keep flipping the value of $/.

    $ perl -Mstrict -Mwarnings -E ' > open my $inFH, q{<}, \ <<EOD or die $!; > ssdfif > START > Line 1 > Line 2 > END > iufifhu > wieuhwi > START > Line 3 > Line 4 > END > wkwwef > wefwef > EOD > > { > my $inWanted = 0; > local $/ = qq{START\n}; > while ( not eof $inFH ) > { > $_ = <$inFH>; > if ( $inWanted ) > { > chomp; > print; > $inWanted = 0; > $/ = qq{START\n}; > } > else > { > $inWanted = 1; > $/ = qq{END\n}; > } > } > }' Line 1 Line 2 Line 3 Line 4 $

    I hope this is of interest.

    Cheers,

    JohnGG

Re: Getting lines in a file between two patterns
by Yates (Beadle) on Jul 05, 2012 at 08:33 UTC
    You could use a counter, which is set to 1 when you reach START then reset to 0 when you reach END. As the counter will only be 1 between START and END, you can simply print out any lines where this is the case.
    use strict; my $count = 0; open (IN, "input.txt"); while (<IN>) { if (/START/) { $count = 1; } elsif (/END/) { $count = 0; } elsif ($count) { print; } } close IN;
    Output:
    These are the first set of lines which are to be extracted These are the second set of lines which are to be extracted
    I wasn't sure if the blank lines were intentional, but if you don't want them, change to
    elsif ($count && $_ !~ /^$/)
Re: Getting lines in a file between two patterns
by awohld (Hermit) on Jul 05, 2012 at 06:22 UTC
    First I changed the default line delimiter to 'END'. Then I went through each record and did a multi-line regex to match what is between START and END.
    #!/usr/bin/perl use strict; use warnings; use Data::Dumper; # change line delimter $/ = "END"; while (<DATA>) { my $line = $_; # match REGEX across multiple lines $line =~ /START(.*)END/s; # get the match my $contents = $1; print Dumper $contents if defined $contents;; } __DATA__ XXXX YYYY START These are the first set of lines which are to be extracted END XXX ZZZ YYY START These are the second set of lines which are to be extracted END aasds tteret tertetr
      Thanks everyone. And Utilitarian's solution worked :)
Re: Getting lines in a file between two patterns
by Marshall (Canon) on Jul 07, 2012 at 01:01 UTC
    I'll show you another common way that this parsing problem is handled. May be useful in other situations...

    When you see the "start-of-record", call a subroutine to process the record. In this case there is no state variable to say that "we are in the record", the fact that you are in the process_record() subroutine serves that purpose.

    #!/usr/bin/perl -w use strict; while (<DATA>) { process_record() if /^START/; } sub process_record { my $line; while ($line = <DATA>, $line !~ /^END/) { print "$line" } print "\n"; #a printout spacer for next record } =prints These are the first set of lines which are to be extracted These are the second set of lines which are to be extracted =cut __DATA__ XXXX YYYY START These are the first set of lines which are to be extracted END XXX ZZZ YYY START These are the second set of lines which are to be extracted END aasds tteret tertetr
      Thanks Marshall. I agree this is actually a much better solution and suits my need perfectly.

      I made some experiment with these lines of codes and maybe I find an issue (I'm not a perl expert...). If input file is like below, with two START-END sections in succession, perl script seems to skip the "START" line after the previous "END" ones, and so it doesn't print the content of the paragraph. I can't say which can be the reason for this behaviour

      __DATA__ XXXX YYYY START These are the first set of lines which are to be extracted END START New line And new Will be extracted? END XXX ZZZ YYY START These are the second set of lines which are to be extracted END aasds tteret tertetr

        I am unable to reproduce your findings. Here are the results with perl 5.20.3:

        $ cat 980408.pl #!/usr/bin/perl -w use strict; while (<DATA>) { process_record() if /^START/; } sub process_record { my $line; while ($line = <DATA>, $line !~ /^END/) { print "$line" } print "\n"; #a printout spacer for next record } __DATA__ XXXX YYYY START These are the first set of lines which are to be extracted END START New line And new Will be extracted? END XXX ZZZ YYY START These are the second set of lines which are to be extracted END aasds tteret tertetr $ ./980408.pl These are the first set of lines which are to be extracted New line And new Will be extracted? These are the second set of lines which are to be extracted $
        In order to reproduce your symptoms, if one of the START tokens doesn't start in column 1 (right at the beginning of the line), then then it wouldn't be recognized and that record would be skipped.

        I made some slight changes to allow for START and END not being at the beginning of the line. Also, I eliminated the use of the "comma operator" in the sub's while statement in favor of an "and" statement. This should handle the case of a malformed record missing an "END" at the end of the DATA a bit better.

        Your DATA as posted does indeed work as hippo demonstrates. This code should work on any Perl >= 5.6 - nothing "fancy" about it.

        This is a very exacting business and when making a bug report, please post exactly the DATA that causes the problem, verbatim.

        #!/usr/bin/perl -w use strict; while (<DATA>) { process_record() if /^\s*START/; } sub process_record { my $line; while (defined ($line = <DATA>) and $line !~ /^\s*END/) { print "$line" } print "\n"; #a printout spacer for next record } __DATA__ XXXX YYYY START These are the first set of lines which are to be extracted END START New line And new Will be extracted? END XXX ZZZ YYY START These are the second set of lines which are to be extracted END aasds tteret tertetr
        As a comment, this malformed "START" record problem applies to all of the code in this thread. This is not unique to my code. "  START" vs "START" is my best guess as what you did wrong in your DATA.

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: perlquestion [id://979946]
Approved by GrandFather
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others chilling in the Monastery: (5)
As of 2020-11-30 23:24 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found

    Notices?