Beefy Boxes and Bandwidth Generously Provided by pair Networks
Come for the quick hacks, stay for the epiphanies.
 
PerlMonks  

Re^2: Regular expressions across multiple lines

by abcd (Novice)
on Apr 24, 2016 at 17:44 UTC ( [id://1161375]=note: print w/replies, xml ) Need Help??


in reply to Re: Regular expressions across multiple lines
in thread Regular expressions across multiple lines

Thanks, i will try this but I am new to programming so not sure i understand your code. What I was doing in my code was to simply chomp every line and append it to the end of a string so I get a single string containing everything without any newlines which I then search.
  • Comment on Re^2: Regular expressions across multiple lines

Replies are listed 'Best First'.
Re^3: Regular expressions across multiple lines
by Discipulus (Canon) on Apr 24, 2016 at 19:00 UTC
    you welcome, even if i'm not sure to understand your issue.

    Basicly a\n?b\n?c means match a followed by, perahps ? a newline \n followed by a b followed by, perahps ? a newline \n and a c

    The m regex modifier (probably unneeded in my example) stands for multiline and the g one means globally ie all occurences are returned.

    $count=()=$string=~/pattern/g idiom is used to count the occurences of pattern in $string infact $string=~/pattern/g with the g returns a list and the generic list () is provided and it's scalar value (ie the number of elements) is returned to the scalar $count

    For shortness i put your example data into a doublequoted string using qq operator qq(abcdefab\ncdefa\nbcdef)

    the rest is only print stuffs.

    If you want to slurp a file into a string you can play with $/ aka input record separator, see perlvar and How do I read an entire file into a string?

    L*

    There are no rules, there are no thumbs..
    Reinvent the wheel, then learn The Wheel; may be one day you reinvent one of THE WHEELS.
Re^3: Regular expressions across multiple lines
by Marshall (Canon) on Apr 24, 2016 at 18:01 UTC
    Please clarify crystal clear: "but for some reason doesnt work with my actual txt file which is several hundred mb", I am presuming that "slow", maybe many,even tens of minutes is NOT the issue?
      No it is not the time. If I output the string to a txt file it creates the file in a few seconds but when I open the file in a text editor it seems corrupt with characters displaying one on top of another.

        Create a hex dump of the problematic file, and post a relevant part of it.

        If you are on unix (Linux, BSD, Mac OS X), try the command od -tx1 -c filename.txt:

        >find . -name '*.txt' -print -exec od -tx1 -c {} \; ./linux-file.txt 0000000 41 20 73 69 6d 70 6c 65 20 66 69 6c 65 20 67 6 +5 A s i m p l e f i l e g +e 0000020 6e 65 72 61 74 65 64 0a 6f 6e 20 4c 69 6e 75 7 +8 n e r a t e d \n o n L i n u +x 0000040 20 77 69 74 68 20 55 6e 69 78 0a 6c 69 6e 65 2 +0 w i t h U n i x \n l i n e 0000060 65 6e 64 69 6e 67 73 2e 0a e n d i n g s . \n 0000071 ./windows-file.txt 0000000 41 20 73 69 6d 70 6c 65 20 66 69 6c 65 20 67 6 +5 A s i m p l e f i l e g +e 0000020 6e 65 72 61 74 65 64 0d 0a 6f 6e 20 57 69 6e 6 +4 n e r a t e d \r \n o n W i n +d 0000040 6f 77 73 20 77 69 74 68 20 57 69 6e 64 6f 77 7 +3 o w s w i t h W i n d o w +s 0000060 0d 0a 6c 69 6e 65 20 65 6e 64 69 6e 67 73 2e 0 +d \r \n l i n e e n d i n g s . \ +r 0000100 0a \n 0000101 ./mac-file.txt 0000000 41 20 73 69 6d 70 6c 65 20 66 69 6c 65 20 67 6 +5 A s i m p l e f i l e g +e 0000020 6e 65 72 61 74 65 64 0d 6f 6e 20 57 69 6e 64 6 +f n e r a t e d \r o n W i n d +o 0000040 77 73 20 77 69 74 68 20 4f 6c 64 20 4d 61 63 0 +d w s w i t h O l d M a c \ +r 0000060 6c 69 6e 65 20 65 6e 64 69 6e 67 73 2e 0d l i n e e n d i n g s . \r 0000076

        A plain ASCII file should not contain any bytes outside the range 0x20 to 0x7E, except for 0x0D and / or 0x0A for newlines. Any other byte value below 0x20 is very fishy, as is 0x7F. Bytes from 0x80 to 0xFF should not appear in ASCII files, they may indicate some other encoding, like UTF-8 and various legacy encodings.

        If (nearly) every second byte is 0x00, it is very likely a text file encoded in UTF-16 or UCS-2; if only every fourth byte is not 0x00, the file is probably encoded in UTF-32.

        Alexander

        --
        Today I will gladly share my knowledge and experience, for there are no sweeter words than "I told you so". ;-)
        This "corrupt characters" sounds like what I suspect, that this file contains something other than ASCII. My experience in this regard is very limited, but other Monks know a lot about this. I highly suspect that a character set problem is causing the issue, but I don't know how to assist you further myself.
Re^3: Regular expressions across multiple lines
by pryrt (Abbot) on Apr 24, 2016 at 19:33 UTC

    assuming you're really only looking for a short string, and given the size of your file, I would be tempted to only concatenate the new line with the last few nonspace characters from the previous line(s), and do the comparison every loop.

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: note [id://1161375]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others goofing around in the Monastery: (5)
As of 2024-04-25 16:59 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found