Hi monks,

I'm currently fixing a few bugs in Mail::IMAPClient that for whatever reason have gone unnoticed. One of these is that its fetch_hash() method can't handle a parenthesized value if contains parentheses. In other words, its not dealing with nested parentheses correctly.

A typical server response to a FETCH command might look something like this:

  * 2 FETCH (UID 2 FLAGS (\Answered \Seen) INTERNALDATE "05-Jun-2008 0
+7:19:01 +1000" RFC822.SIZE 5144 BODYSTRUCTURE ("text" "plain" ("chars
+et" "ISO-8859-1") NIL NIL "7bit" 2053 46 NIL ("inline" NIL) NIL NIL))
[download]

fetch_hash() seeks to turn that into the following hash:

  "UID"           => "2",
  "FLAGS"         => "\Answered \Seen",
  "INTERNALDATE"  => "05-Jun-2008 07:19:01 +1000",
  "RFC822.SIZE"   => "5144",
  "BODYSTRUCTURE" => "text" "plain" ("charset" "ISO-8859-1") NIL NIL "
+7bit" 2053 46 NIL ("inline" NIL) NIL NIL",
[download]

In other words, the parenthesized data consists of key/value pairs. The values are either simple strings (matched by \S+), quoted strings (matched by "[^"]*") or a parenthesized fragment, which may include parentheses itself.

What's inside the parentheses is actually completely opaque as far as this method is concerned. The original code used \([\)]*\) to try and match it, which works great until there is a closing paren inside.

My attempt matching/capturing the parenthesized fragment is based on mjd's regex to match balanced parentheses. Its actually part of a larger regex that matches the other stuff described above, but this doesn't work in isolation either so I don't think those things are affecting it:

  (?:
    (?{ local $d=0 }) # set depth to 0
    (                 # start capture
      (?:                  
        \(            # opening paren
        (?{$d++})     # increment the depth
        |  
        \)            # or closing paren
        (?{$d--})     # decrement the depth
        (?(?{$d==0})  # if we're back to the start
          (*ACCEPT)   # we're done
        )
        |
        (?>[^()]*)    # or there's just some normal text
      )*
      (?(?{$d!=0})    # done. did we reach a matching closing paren
        (?!)          # nope, failed
      )
    )                 # end capture
  )
[download]

Running a basic test program with -Mre=debug seems to suggest that this is doing something close to what I want, stopping when it gets to the correct closing paren. The only problem is that nothing gets captured, even though the description of (*ACCEPT) in perlre makes it sound like it should.

So, what's wrong here? And is there an easier way to do this? Note that I'm restricted to Perl 5.6, which sucks as I'd really like to try the PARNO stuff that came with 5.10.

Cheers,
Rob.