Perhaps if you showed us the change you made we could see what you did wrong. At a random shout in the dark you inserted stuff inside the recursive part of the pattern. PCRE now supports the reusable sub-expression syntax.

The problem of matched parentheses is the most commonly quoted example of using this construct.

my $bal = qr/
    (?<bal>            # Name the rule (optional)  
    \{                 # Open brace
    (?>                # Possessive subgroup
        (?> [^{}]+ )   #  Grab all the non braces
    |                  #    or
        (?&bal)        #  Recurse
    )*                 # Zero or more times
    \}                 # Close brace
    )                  # End named rule
/x;

if ('{x{x}y{x}x}' =~ /^$bal$/ ){
    print "It's balanced\n";
}

$_= 'XXXX function xxx() {x{x}y{x}x} XXXX';

while ( /\bfunction\s+(\w+)\(\)\s*($bal)/g ){
    print "function: $1\nbody: $2\n";
}
[download]

This, of course, is Perl code - adapting it to PHP is left as a exercise for the reader.

I should add, this only checks for nested {} it doesn't know to ignore {} within quotes - that too is possible by hybridising the above with the tricks for parsing CSV.