If the webapp wasn't on CPAN, you could do the same thing. Or, you could just bundle the modules you need and, that way, you're guaranteeing the proper versions. Just because you require v1.34+ doesn't mean that 1.35 didn't break something you need.
As for everything else you're talking about, these things don't require CPAN. They are installation process issues. Don't worry about "diversification of Perl" issues. Forcing everyone to come to CPAN just to get your app means you're limiting your userbase to Perl users. I shouldn't be forced to care about what language you've implemented in. I only know what language Trac is implemented in because I had to patch it once. I don't know what language Jira, gVim, Firefox, or iTerm are written in ... and I don't care.
My criteria for good software:
- Does it work?
- Can someone else come in, make a change, and be reasonably certain no bugs were introduced?