Beefy Boxes and Bandwidth Generously Provided by pair Networks
Clear questions and runnable code
get the best and fastest answer

Creating a Bundle:: with all deps?

by dragonchild (Archbishop)
on Feb 01, 2006 at 16:24 UTC ( [id://527097] : perlquestion . print w/replies, xml ) Need Help??

dragonchild has asked for the wisdom of the Perl Monks concerning the following question:

I've been given the task of configuring a client's machine to run our "Cures-The-Common-Cold" web application. Since this is a very large task, we ended up using a lot of code from CPAN. Except, the client is paranoid and won't let this machine talk to the "Big Bad Outside World". Only my machine is allowed to talk to it from outside their DMZ.

While I am more than willing to hand-follow all the dependencies for the various CPAN modules we need, I'm positive that someone has run into this before and has a solution. What I need is a script that will create a tarball of all the CPAN distros I list (given a Perl version), including all their dependencies, then provide a script within that tarball to install them in the right order (dependencies first).

My criteria for good software:
  1. Does it work?
  2. Can someone else come in, make a change, and be reasonably certain no bugs were introduced?

Replies are listed 'Best First'.
Re: Creating a Bundle:: with all deps?
by glasswalk3r (Friar) on Feb 01, 2006 at 17:24 UTC

    Suposing that you're using your workstation to develop (you're not deploying directly to the production enviroment, are you?) you can download all modules and dependencies that you need. When you think the stuff is ready for deploy, you can use the module CPAN::Distro Builder to create a tarball with those modules. If you have many modules that needs to be shipped to the production server, then you probably will want to search at CPAN for the various modules that generates a list of dependencies (CPAN::Unwind is a candidate for that).

    Other solution is to use the repository that you have at your machine and copy it to the production server. Once there, you just need to setup CPAN to use a file as an URL and points to the repository that you had copied in the filesystem.

    Alceu Rodrigues de Freitas Junior
    "You have enemies? Good. That means you've stood up for something, sometime in your life." - Sir Winston Churchill
Re: Creating a Bundle:: with all deps?
by xdg (Monsignor) on Feb 01, 2006 at 16:48 UTC

    For a standalone application like this, have you considered/tried using PAR?


    Code written by xdg and posted on PerlMonks is public domain. It is provided as is with no warranties, express or implied, of any kind. Posted code may not have been tested. Use of posted code is at your own risk.

Re: Creating a Bundle:: with all deps?
by rhesa (Vicar) on Feb 01, 2006 at 17:04 UTC
    Wouldn't it be easier to configure your client's cpan shell to use your machine as a http/ftp proxy? I might consider that a reasonable compromise.

    Another suggestion I can offer is to look into alias's Task namespace and ideas.
    Only thing I couldn't figure out yet is whether cpan/cpanplus can download dependent dists automatically.

Re: Creating a Bundle:: with all deps?
by brian_d_foy (Abbot) on Feb 02, 2006 at 09:31 UTC

    In these cases, I give our clients a minicpan on a CD (maybe even with their stuff added with CPAN::Mini::Inject). They configure their (or whatever) to grab files off of the disk rather than the network. Everything works peachy.

    brian d foy <>
    Subscribe to The Perl Review
Re: Creating a Bundle:: with all deps?
by toma (Vicar) on Feb 01, 2006 at 19:07 UTC
    We have a similar challenge with our build system. Our builds are independent of whatever happens to be on CPAN at the moment. We do a lot of testing, and we don't want to introduce CPAN changes or availability as a variable.

    We use a perl program that builds all our modules, and it has to manage the dependencies. It uses the dependencies listed in the module as a starting point, but many modules do not list their dependencies correctly, so we have to refine the list by hand. Of course, the dependencies themselves depend on the version of perl, since more modules are brought into the core in higher versions.

    It should work perfectly the first time! - toma
Re: Creating a Bundle:: with all deps?
by jbrugger (Parson) on Feb 01, 2006 at 20:30 UTC
    We had a similar problem once, and we had no way we were allowed to install any external module at all.
    What we did, is give the it-department of the company a list of prerestiqued modules, and let them do the installation of the server. Next, if the program failed due to the miss of one or more modules, they were responsible (and took that serious luckily)
    I do think, if you explain and negotiate, a lot will be possible, and let your client participate. (and make him responsible)

    "We all agree on the necessity of compromise. We just can't agree on when it's necessary to compromise." - Larry Wall.
Re: Creating a Bundle:: with all deps?
by Anonymous Monk on Feb 02, 2006 at 12:29 UTC
    If all your distros are the same, (we have standardized on RHEL4) you can simply copy the perl distro and the perl libraries and install it identically on the other machine. All is identical even the cpan status.

    We develop a pretty tricky mod_perl2/mason app with a supertuned mysql (all compiled with intels c++ compiler for best performance - around 25% faster). And we create a big rpm of everything for its proper place, we only check the kernel release.

Re: Creating a Bundle:: with all deps?
by exussum0 (Vicar) on Feb 02, 2006 at 16:03 UTC
    Except, the client is paranoid and won't let this machine talk to the "Big Bad Outside World". Only my machine is allowed to talk to it from outside their DMZ.
    As a security advocate, I wouldn't call the person paranoid. Yes, too much security, and you can't get anything done. But it is common place to have production machines, if that is what this is, to do very VERY specific things. I'm sorry it sounds like I'm jumping down your throat, but I am asking people who read what you wrote and think of some of the consequences of not DMZing.

    webservers normally have connections only comming in from a certain set of ips, if yer doing nat, or a load balancer, this may be small. If the device is just packet forwarding, it's huge! The connections going out from the same server would normally be really small, since as a web server, its duty is to serve pages, not to be a resource to access other foreign resources.

    That being said, anything going onto the machine, normally is verified as the required set of changes for auditing and quality purposes. If one day, CPAN was hacked, as public repositories have had happen, and you tried to use it, that new machine will have, "bad code".

    If you package you modules, and it went down a pipe to get to production, directly from dev or through QA, 1) The code you downloaded has had a justation period for like people to download the code and say, "HOLY CRAP!", 2) What everyone has validated and audited that is going into production, really is just that. No suprise upgrades.

    Easiest suggestion I can recommend, have a target that is, "pristine." Install a tool like tripwire and run it against the pristine target. Now run CPAN. Run a tripwire report to see what has changed. That will be your list of things to export.

    A nicer way would be to have two copies of production on a dev box, then run cpan targetting one. Do a diff. Enjoy! :)

    Both solutions would work even if you weren't using CPAN and can't easily figure out the differences for modules, configuration files, beer.. stuff.. yeah.

      Yes! Thank you Sporty! I don't think the client's requirements are unreasonable at all..


        Absolutely, I don't think anyone here feels that the client was unreasonable. I'm pretty sure dragonchild used the word "paranoid" only to make it crystal-clear that the client is very strict about security.

        The only thing that made me wonder is the emphasis on outside connectivity, rather than on the code being introduced to the machine. I have the feeling that

        1. Only allowing the cpan shell to go out and download code, while closing everything else down, is pretty simple to arrange with a simple set of firewall rules
        2. The risk of unwanted traffic to the machine is way lower than the inherent risk of installing foreign code

        Personally, I'd rather trust my firewall rules than the new code. If I were paranoid about security, I'd prefer to audit each and every newly installed module in favor of worrying about network traffic during the installation. Of course, that's a much harder problem, and for practical reasons alone I'd be inclined to trust CPAN code to be secure. But then I'd no longer be paranoid :-)