in reply to cpan2rpm and Jenkins (not so perfect together?)

not a puppet user (cfengine shop here), but what we do is create our own repository of rpm packages and distribute the repo information to all clients, so you can just use yum install yourpackage from the cfengine policies. It should not be very different using puppet/ansible/whatever.

For creating repos we use mrepo (another battle tested tool) but many people use pulp or cobbler. You can even just use createrepo.

What we do in jenkins is clone the production nfs volume where the repo data lives, we update the repos in this clone which is offered to the test clients (so patches are first tested there), and if after x days everything is fine then the cloned nfs volume gets promoted to production nfs volume. We have Netapp filers but you can easily achieve this with any zfs enabled filer.