I don't discount at all that perception is the main problem. I think Perl is a wonderful language overall myself. The perceptions of it being complex, deep, broad, and somewhat arcane start somewhere, though. They are also reinforced when people come to the language and have such issues as did WoodyWeaver.
Should the rough spots be enough to keep people away from Perl? No, I don't think so. However, the rough spots do little to discourage the negative perceptions of incantations via line noise. Smaller, more regular, and more orthogonal languages are less susceptible to these perceptions and to some extent to these realities. One of the ways for Perl to really answer critics is to overcome the perceptions, not allow things that reinforce them when someone ventures past the anti-Perl hype.