The issue I have isn't so much with the core Perl behavior, so much as the plethora of built-in global variables in Perl, and the general abuse of globals. For example, in one of my @INC directories I ran ack '^\s*\$\w+::.*=' and read plenty of CPAN modules who set global variables for other packages and don't localize those changes. I've had my code break as a result of other people doing that and it's very hard to track down these bugs.
When you combine the potential for mysterious action at a distance along with the relatively inscrutable punctuation variables, it's a double-whammy in my book. Thus, I used to use local $" = ...;, but I've stopped doing that and recommend join (yes, even with array slices) because it's clear and there's less risk of mysterious side effects.