Extracting a parenthesized fragment from a string

fce2 has asked for the wisdom of the Perl Monks concerning the following question:

Hi monks,

I'm currently fixing a few bugs in Mail::IMAPClient that for whatever reason have gone unnoticed. One of these is that its fetch_hash() method can't handle a parenthesized value if contains parentheses. In other words, its not dealing with nested parentheses correctly.

A typical server response to a FETCH command might look something like this:

  * 2 FETCH (UID 2 FLAGS (\Answered \Seen) INTERNALDATE "05-Jun-2008 0
+7:19:01 +1000" RFC822.SIZE 5144 BODYSTRUCTURE ("text" "plain" ("chars
+et" "ISO-8859-1") NIL NIL "7bit" 2053 46 NIL ("inline" NIL) NIL NIL))
[download]

fetch_hash() seeks to turn that into the following hash:

  "UID"           => "2",
  "FLAGS"         => "\Answered \Seen",
  "INTERNALDATE"  => "05-Jun-2008 07:19:01 +1000",
  "RFC822.SIZE"   => "5144",
  "BODYSTRUCTURE" => "text" "plain" ("charset" "ISO-8859-1") NIL NIL "
+7bit" 2053 46 NIL ("inline" NIL) NIL NIL",
[download]

In other words, the parenthesized data consists of key/value pairs. The values are either simple strings (matched by \S+), quoted strings (matched by "[^"]*") or a parenthesized fragment, which may include parentheses itself.

What's inside the parentheses is actually completely opaque as far as this method is concerned. The original code used $[$]*\) to try and match it, which works great until there is a closing paren inside.

My attempt matching/capturing the parenthesized fragment is based on mjd's regex to match balanced parentheses. Its actually part of a larger regex that matches the other stuff described above, but this doesn't work in isolation either so I don't think those things are affecting it:

  (?:
    (?{ local $d=0 }) # set depth to 0
    (                 # start capture
      (?:                  
        \(            # opening paren
        (?{$d++})     # increment the depth
        |  
        \)            # or closing paren
        (?{$d--})     # decrement the depth
        (?(?{$d==0})  # if we're back to the start
          (*ACCEPT)   # we're done
        )
        |
        (?>[^()]*)    # or there's just some normal text
      )*
      (?(?{$d!=0})    # done. did we reach a matching closing paren
        (?!)          # nope, failed
      )
    )                 # end capture
  )
[download]

Running a basic test program with -Mre=debug seems to suggest that this is doing something close to what I want, stopping when it gets to the correct closing paren. The only problem is that nothing gets captured, even though the description of (*ACCEPT) in perlre makes it sound like it should.

So, what's wrong here? And is there an easier way to do this? Note that I'm restricted to Perl 5.6, which sucks as I'd really like to try the PARNO stuff that came with 5.10.

Cheers,
Rob.

Comment on Extracting a parenthesized fragment from a string Select or Download Code

Replies are listed 'Best First'.
Re: Extracting a parenthesized fragment from a string by almut (Canon) on Sep 11, 2009 at 11:38 UTC
Maybe you could use Text::Balanced instead? (in particular extract_bracketed()) Module::CoreList says it's a core module since: `$ corelist Text::Balanced Text::Balanced was first released with perl 5.007003` [download] (IOW, for 5.6 you'd have an additional dependency...)	[reply] [d/l]
Re^2: Extracting a parenthesized fragment from a string by fce2 (Sexton) on Sep 11, 2009 at 12:49 UTC
I'd forgotten about this module, but yes, it could probably do quite well. I'll try a couple of implementations based on the different ideas that have been presented here and see how they compare for readability, speed, and so forth. Cheers!	[reply]
Re: Extracting a parenthesized fragment from a string by ELISHEVA (Prior) on Sep 11, 2009 at 11:23 UTC
Though you might be able to parse this with a fancy regex, a simple state machine would in my opinion be easier to read and less complicated to write and debug. If I understand your post correctly, your string has four tokens: quoted strings - opaque between the quotes runs of non-whitespace/non-close parenthesis that begin with anything but an open parenthesis or double quote, `/[^)("][^)\s]/`. runs of whitespace used as a separator between the first two types of tokens parenthesized strings that may contain any of the first three types of tokens. Assuming there are no parenthesized tokens within parenthesized tokens, you could use something like this: Read more... (1358 Bytes) If you also need parenthesized tokens within parenthesized tokens, they the loop is only slightly more complicated. You would need to change the flag `$bParen` to a counter that was incremented for each '(' and decremented for each ')' found. You would then build the token until `$iParenCount` returned to 0. Parentheses within quotes will have no effect on this count because the `"[^"]` run insures that only parentheses outside of quotes will get parsed into separate tokens: Read more... (1232 Bytes) Best, beth Update: added some discussion about handling nested parenthesized tokens. Update*: Fixed overly greedy regex	[reply] [d/l] [select]
Re^2: Extracting a parenthesized fragment from a string by fce2 (Sexton) on Sep 11, 2009 at 12:42 UTC
I very much like your analysis and approach to this. I'd spent so long trying to twist this regex to my will that I'd forgotten that there's other tools in the shed! I'll take this and fit it into my code and see how it goes. Thanks so much!	[reply]
Re: Extracting a parenthesized fragment from a string by Anonymous Monk on Sep 11, 2009 at 11:12 UTC
Could you elaborate on what that `(*ACCEPT)` is supposed to be doing?	[reply] [d/l]


Keep It Simple, Stupid
	PerlMonks