Hopefully to be somewhat clearer, $`, $&, and $' are especially bad because once they've been seen by Perl, all regexes will do extra work.

Capturing in a regex imparts a performance hit because it means that a copy will be made of the string that the regex is being applied to (which makes it a worse performance hit when matching against really large strings -- one of the worst cases being running a lot of little regexes with capturing against the same huge string, something a parser is likely to do). The performance hit of capturing only applies to the regex that does capturing and most of the time it isn't enough that you'd notice.

@- and @+ are always set by regexes and whether you use them or not doesn't have any performance impact at all.

I'm curious why the regex engine doesn't just copy out the parts that were captured and only copy them after the regex has finished. That seems like a "best of both worlds" solution. Though, I think /k would be really cool, especially if you could use it with:

@matches= $string =~ /$regex/gk;
[download]

to make it so it doesn't matter whether $regex contained capturing parens or not. (:

Finally, this benchmark shows that the per-regex performance hit from $`, $&, and $' is the same hit from capturing parens:

#!/usr/bin/perl -w
use strict;
use Benchmark qw( timethis cmpthese );

my %t;
my $pref= "no&";
my $str= "match" . ( "garbage" x 500 );
for( 0, 1 ) {
    $t{$pref."?:"}= timethis( -3, sub {
        $str =~ /(?:match)+/;
    } );
    $t{$pref."()"}= timethis( -3, sub {
        $str =~ /(match)+/;
    } );
    eval '$&';
    $pref= "amp";
}
cmpthese( \%t );
__END__
           Rate amp() no&() amp?: no&?:
amp()  439813/s    --   -0%   -1%  -81%
no&()  441612/s    0%    --   -0%  -81%
amp?:  443563/s    1%    0%    --  -81%
no&?: 2354611/s  435%  433%  431%    --
[download]

But, remember: $`, $&, and $' are evil while capturing parens are usually the best solution when you need them and usually don't cause a noticeable performance penalty.