Hi,
I'll add some considerations which looks needed. This will also be an answer to the 'Anonymous' below, who thinks he or she can hide and insult people without even disturbing him ore herself to register into the community...

Stricly speaking, Perl regex are realy much more powerfull than those described in the wonderfull books you refer to. To understand regex as they are used in perl (but also in other langages & tools) I'd rathere refer to

• Mastering Regular Expressions by Jeffrey E. F. Friedl and Andy Oram

A basic thing that one always see written about regex is that the can not count. Meaning that you must know the maximum number of times the <em>...</em> will be embedded..

While this is true of 'standard' regex, this is not true for Perl regex. By using carefull combination of the /e modifier and of the (?{}) programmatic pattern you can do using regex, everithing a parser will do.

IMHO, using a regex or another approach is a matter of taste, and a careful crafted and optimized regex will be more efficent than a sloppy written rec descent parser.

Just my 5 (euro) cents.

Cheers

By using carefull combination of the /e modifier and of the (?{}) programmatic pattern you can do using regex, everithing a parser will do.

My guess is that you probably mean the (??{...}) assertion.

(?{...}) merely executes, whereas
(??{...}) executes and interpolates.

(A possibly confusing mnemonic would be that one ? would be like one q, which doesn't interpolate. Double ? would be like double q, which interpolates. It's different types of interpolations (one interpolates into the construction, one interpolates its result), so ignore this if it doesn't make sense to you.)

A bit generalized you may say that:
(?{...}) is used for debugging and/or setting state.
(??{...}) is used for generating patterns at "match-time".

Beware of using =~ inside either of these assertions though. The engine is known to often blow upon that.

Update:
A good example that uses both these assertion is to be found at Re: Capturing brackets within a repeat group [plus dynamic backreferences].

Hope I've helped,
ihb

I am the previous anonymous monk.

Perl 6's regex syntax will make using it for parsing reasonable. But while you theoretically can do that with a lot of care and using constructs that very people know about, the odds are strongly that the average programmer who thinks that they can just misunderstands and underestimates the difficulties.

Therefore on the odds I stand by my previous comments.

You're right, and you're wrong... I'm fairly certian that while ordinary regular expressions aren't up to parsing HTML, even on a theorical basis. Perl regular expressions are a whole 'nother breed. Regular expressions with backreferences are NP-complete; it's been proven at least twice. (Well, three times, but one of them is buggy.) I suspect I'm missing somthing here... if anybody knows what (other then my mind), I'd love to hear it.

NP-completeness is a property of an algorithm. It implies that no algorithm is known to solve the problem in polynomial time.
This means that if you increase the length of the input for the problem, the execution time will increase exponentially. (Of course there are input cases which are polynomial, but many of interest are not). Essentially, it means that brute force is the only known method to tackle the problem exactly.

The question is on the relation between the behavior of an algorithm to decide on a language and the class to which this language belongs. For regular languages and context free languages polynomial time algorithms are known, but does this necessarily mean that since regular expressions with backreferences are proven to be NP-complete that the language they describe are a superset of regular and context free languages?

It certainly means it is hard to decide whether or not a certain string is an element of the language described by a regular expression with backreferences. But what does it tell us about the expressive power?

The expression /^(.*)\1$/ defines the language {ww | w in sigma*}, known neither to be regular, nor context free. On the other hand, regular expressions with backreference can't describe {a^n b^n | n >= 0} which is definitely context free.

So on the one hand, regular expressions with backreference describe languages that are not context free, but can't describe all context free languages either! This example illustrates that one has to be very careful when judging expressive power from algorithmic complexity. A high complexity is a sign that the expressive power must be high in some cases, but doesn't guarantee that everything can be done.

Incidently, the code below shows two Perl regular expressions that describe non-regular languages:

{ a^n b^n | n >= 0}

      /^
        (a*)
        (??{sprintf("b{%d}", (length($1)))})
       $/x
[download]

which is context free as mentioned above and

{ a^n b^n c^n | n >= 0 }

      /^
        (a*)
        (??{sprintf("b{%d}", (length($1)))})
        (??{sprintf("c{%d}", (length($1)))})
       $/x
[download]

which is context sensitive.

Just my 2 cents, -gjb-

NP-completeness is a property of an algorithm. It implies that no algorithm is known to solve the problem in polynomial time.
This means that if you increase the length of the input for the problem, the execution time will increase exponentially. (Of course there are input cases which are polynomial, but many of interest are not). Essentially, it means that brute force is the only known method to tackle the problem exactly.

I think that this may be a little misleading. Right now (as 6 years ago), NP-completeness of a problem means that no polynomial-time algorithm is known, but that statement may eventually become false ^*. Maybe it's better to say “Computer scientists believe that, if a problem is NP-complete, then there is no polynomial-time algorithm to solve it”?

Also, I'm not sure that it's fair to say that NP-completeness of a problem means that the time-complexity of the problem grows exponentially in the input. Again, we think that NP-completeness correlates with exponential time-complexity, but that could change ^*. For that matter, can't NP-complete problems have super-exponential complexity (like 2^(n^2))—or are you using ‘exponential’ in the generic sense of ‘faster-growing than polynomial’?

^* Although we all know that it won't really. :-)