Beefy Boxes and Bandwidth Generously Provided by pair Networks
Keep It Simple, Stupid

Extracting a parenthesized fragment from a string

by fce2 (Sexton)
on Sep 11, 2009 at 10:32 UTC ( #794740=perlquestion: print w/replies, xml ) Need Help??

fce2 has asked for the wisdom of the Perl Monks concerning the following question:

Hi monks,

I'm currently fixing a few bugs in Mail::IMAPClient that for whatever reason have gone unnoticed. One of these is that its fetch_hash() method can't handle a parenthesized value if contains parentheses. In other words, its not dealing with nested parentheses correctly.

A typical server response to a FETCH command might look something like this:

* 2 FETCH (UID 2 FLAGS (\Answered \Seen) INTERNALDATE "05-Jun-2008 0 +7:19:01 +1000" RFC822.SIZE 5144 BODYSTRUCTURE ("text" "plain" ("chars +et" "ISO-8859-1") NIL NIL "7bit" 2053 46 NIL ("inline" NIL) NIL NIL))

fetch_hash() seeks to turn that into the following hash:

"UID" => "2", "FLAGS" => "\Answered \Seen", "INTERNALDATE" => "05-Jun-2008 07:19:01 +1000", "RFC822.SIZE" => "5144", "BODYSTRUCTURE" => "text" "plain" ("charset" "ISO-8859-1") NIL NIL " +7bit" 2053 46 NIL ("inline" NIL) NIL NIL",

In other words, the parenthesized data consists of key/value pairs. The values are either simple strings (matched by \S+), quoted strings (matched by "[^"]*") or a parenthesized fragment, which may include parentheses itself.

What's inside the parentheses is actually completely opaque as far as this method is concerned. The original code used \([\)]*\) to try and match it, which works great until there is a closing paren inside.

My attempt matching/capturing the parenthesized fragment is based on mjd's regex to match balanced parentheses. Its actually part of a larger regex that matches the other stuff described above, but this doesn't work in isolation either so I don't think those things are affecting it:

(?: (?{ local $d=0 }) # set depth to 0 ( # start capture (?: \( # opening paren (?{$d++}) # increment the depth | \) # or closing paren (?{$d--}) # decrement the depth (?(?{$d==0}) # if we're back to the start (*ACCEPT) # we're done ) | (?>[^()]*) # or there's just some normal text )* (?(?{$d!=0}) # done. did we reach a matching closing paren (?!) # nope, failed ) ) # end capture )

Running a basic test program with -Mre=debug seems to suggest that this is doing something close to what I want, stopping when it gets to the correct closing paren. The only problem is that nothing gets captured, even though the description of (*ACCEPT) in perlre makes it sound like it should.

So, what's wrong here? And is there an easier way to do this? Note that I'm restricted to Perl 5.6, which sucks as I'd really like to try the PARNO stuff that came with 5.10.


Replies are listed 'Best First'.
Re: Extracting a parenthesized fragment from a string
by almut (Canon) on Sep 11, 2009 at 11:38 UTC

      I'd forgotten about this module, but yes, it could probably do quite well. I'll try a couple of implementations based on the different ideas that have been presented here and see how they compare for readability, speed, and so forth. Cheers!

Re: Extracting a parenthesized fragment from a string
by ELISHEVA (Prior) on Sep 11, 2009 at 11:23 UTC
    Though you might be able to parse this with a fancy regex, a simple state machine would in my opinion be easier to read and less complicated to write and debug. If I understand your post correctly, your string has four tokens:
    • quoted strings - opaque between the quotes
    • runs of non-whitespace/non-close parenthesis that begin with anything but an open parenthesis or double quote, /[^)("][^)\s]*/.
    • runs of whitespace used as a separator between the first two types of tokens
    • parenthesized strings that may contain any of the first three types of tokens.

    Assuming there are no parenthesized tokens within parenthesized tokens, you could use something like this:

    If you also need parenthesized tokens within parenthesized tokens, they the loop is only slightly more complicated. You would need to change the flag $bParen to a counter that was incremented for each '(' and decremented for each ')' found. You would then build the token until $iParenCount returned to 0. Parentheses within quotes will have no effect on this count because the "[^"] run insures that only parentheses outside of quotes will get parsed into separate tokens:

    Best, beth

    Update: added some discussion about handling nested parenthesized tokens.

    Update: Fixed overly greedy regex

      I very much like your analysis and approach to this. I'd spent so long trying to twist this regex to my will that I'd forgotten that there's other tools in the shed!

      I'll take this and fit it into my code and see how it goes. Thanks so much!

Re: Extracting a parenthesized fragment from a string
by Anonymous Monk on Sep 11, 2009 at 11:12 UTC

    Could you elaborate on what that (*ACCEPT) is supposed to be doing?

Log In?

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: perlquestion [id://794740]
Approved by Bloodnok
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others lurking in the Monastery: (1)
As of 2023-06-06 06:42 GMT
Find Nodes?
    Voting Booth?
    How often do you go to conferences?

    Results (26 votes). Check out past polls.