Thanks for your thoughts!
The problem with having a separate installation of Perl for every chroot program on your system is that maintainability becomes difficult. In particular, instead of relying on your distribution to let you know when there are Perl-related security updates available, now you need a way to track all of those installations for security updates yourself. In my experience the likelihood of getting that wrong outweighs the security advantages of using chroot to begin with.
At any rate, most chroot programs don't require large installations of software systems and libraries to work. For example, many programs chroot into /var/empty, so they have access to nothing at all. They just make sure to load up everything they need beforehand.
One of the reasons I like to use Perl is that generally I can follow this strategy: load all the resources up front, chroot into a minimal environment, then be confident that my security risks are minimized. This particular program has run that way for several years without any issues.
Really, what I would like to do is find a way to load all of that unicode stuff up front, or else disable it for this program.