Best way to build dependencies in a CI?

Dallaylaen has asked for the wisdom of the Perl Monks concerning the following question:

Hello dear esteemed monks,

We have a project with a lot of CPAN dependencies. So we need to install them, meeting the following conditions:

Exactly the same set of modules is installed on all of our production servers;
The developers are able to add dependencies at will, without ops interaction (after code review, of course);
The build happens at every commit and it doesn't take forever.

What we currently do is:

unpack dependency cache;
install missing dependencies with cpanm;
run test suite against the updated library;
update dependency cache;
pack libraries together with the product itself into a huge tarball;
distribute the tarball to the servers, if needed.

What I'm currently pushing for is to build the dependency cache from scratch from time to time (say weekly) and only install the missing modules (we don't add new dependencies every day after all). This would increase the resilience to changes in upstream and also allow to purge unused dependencies.

But maybe there's a better way to achieve the stated requirements?

Comment on Best way to build dependencies in a CI?

Replies are listed 'Best First'.
Re: Best way to build dependencies in a CI? by afoken (Chancellor) on Apr 10, 2019 at 20:38 UTC
Maybe a local CPAN mirror could help? You could update the mirror only when you really want updates (instead of updating daily or hourly). Then, point all cpan(m) instances to the local mirror. Or, if all servers and dev machines use the same OS, build an OS-specific package (.deb, .rpm, whatever ...) with the required modules, and use the OS to install and update the packages from a local package repository. Alexander -- Today I will gladly share my knowledge and experience, for there are no sweeter words than "I told you so". ;-)	[reply]
Re^2: Best way to build dependencies in a CI? by Dallaylaen (Chaplain) on Apr 12, 2019 at 11:30 UTC
Well I'm thinking of both, leaning towards the second. Also thinking about trying out NIX package manager as we don't really need tight integration with the underlying system.	[reply]


Clear questions and runnable code get the best and fastest answer
	PerlMonks