I see no reason not to put your application on CPAN, as long as it's published under an open source license.

About an, App:: namespace. Is that getting off the ground at all? One of the issues this will raise for me is that I'll have to change the name of all of the modules the webapp uses from, ProgramName::Utility to, App::ProgramName::Utility

One of the nice sides of CPAN is that it doesn't have many restritctions. Nobody forces you to publish your modules and the distribution namespace. So you can publish a distribution App::AppName and let the modules all start with AppName::. It's not really nice because it could lead to module name clashes one day, but it's certainly a possibility. (I hope that monks with more experience comment on that ;)