In response to points 1, 2, 3, 4, AND 5, there's an array of solutions any of which solve or partially solve all five of them.

If you want to ensure compatible dependencies, build and test an image/package (in tar, zip, Docker, kvm disk image, RPM, APT/deb, flatpak, perlbrew, Carton, or whatever) and ship the image/package to production as a first-class semantically versioned entity.

If you want idempotence of installations, make the image or package removable and reinstallable as a whole.

If you want consistent upgradability, upgrade things in your build system and ship an updated, versioned image or package.

If you want to know your security posture and be consistent with it, test, evaluate, inspect, and otherwise poke at your dependencies before or during build before shipping them to production. Yes, at this point fewer, smaller dependencies can be a boon if you don't trust upstream very much. You might find a bug in upstream code, be it security-related or not. But better that than shipping new bugs to production without looking.

Don't depend on the OS's perl for your Perl application. Re-vendor your dependencies. Specifically, put your application and its dependencies, again, in an image or package for self-contained, repeatable installations. You can use perlbrew or another tool for this. We're currently evaluating replacing our own RPM builds of perl and modules with just throwing a perlbrew with all our needed modules into a repo or container. In any case, we ship our own perl and our own copy of all relevant modules.

You don't necessarily need to go hog wild with high-end orchestration systems like Kubernetes, but the modern DevOps/DevSecOps/GitOps/application deployment movement has solved many problems associated with long-lived hand-rolled server systems. They mostly solve that with the very simple philosophy of "well, don't do that then". Just don't have unmanaged systems, manually upgraded systems, or systems with periodic configuration management runs in production when you're building new applications. Just don't, and your problems 1 through 5 are largely solved, mitigated, or simplified.