http://qs321.pair.com?node_id=1145143

Update: Excellent points made in responses. A rewrite is in the works. Keep them coming.

Perhaps this is a tutorial, or perhaps something that just belongs long term in Meditations, but my intent with this node is to document the arguments related to modifying the system's Perl installation, so that I don't have to lay out the argument every time it comes up in SoPW. There are some pro, and some con, but it seems to be, in my experience, that "Can't touch this", if given the choice, is the correct path to take.

Some of the previous comments I have made on this topic can be found here: SuperSearch Preload

So, without further ado, here are some arguments in favor of installing your own Perl (in no particular order):

There are also some cons to installing your own Perl:

In summary, unless you are writing an OS-level tool specifically for this platform, install a Perl version specifically for your own application's runtime stack.

Anything else? (ed: updates tagged with author)

--MidLifeXis

Replies are listed 'Best First'.
Re: RFC: (Do Not) Modify the System Perl
by Corion (Patriarch) on Oct 16, 2015 at 18:44 UTC

    When using the system Perl, you cannot upgrade modules via CPAN. If you do that, a system update may revert (parts of) the module chain back to older, vendor-supplied versions. This usually happens on OSX, where some XS module gets upgraded by the user and then Apple distributes their own, patched old version XS library again, overwriting the XS file but leaving the Perl file upgraded.

      When using the system Perl, you cannot upgrade modules via CPAN.

      This is a reason for not using CPAN to upgrade modules within the system perl installation - not a reason for not using the system perl at all. Each (reasonable) OS provides its own packager, and each packager has means to install modules elsewhere, apart from the system's locations.
      For some reason, there is:

      • archlib
      • vendorlib
      • sitelib

      and so on. Then PERL5LIB, lib.pm, FindBin.pm et al.

      perl -le'print map{pack c,($-++?1:13)+ord}split//,ESEL'
Re: RFC: (Do Not) Modify the System Perl
by Your Mother (Archbishop) on Oct 16, 2015 at 22:41 UTC

    The cons both strike me as partly to mostly false. If you use *anything* other than pure vanilla, feature-free Perl you’ll need to include the distribution or build it. RedHat’s brain dead 5.8 that had a slowdown of a thousand-fold on some kinds of object creation because they used their own non-standard build flags comes to mind. The packaging requirements are already more complex if you’re doing things right and if you’re already doing things right… then it’s not more complex. :P

      If you use *anything* other than pure vanilla, feature-free Perl you’ll need to include the distribution or build it.

      There's no such thing as a pure vanilla, feature-free Perl. Perl's Configure is full of featuritis and hints, and some features may well not be turned on in your vendor's perl. If I need e.g. -DDEBUGGING on my platform, yes, I have to rebuild perl - but I rebuild it as close to the platforms procedures as possible, i.e. only altering that switch in the build process. Otherwise I would just be asking for problems - asking the platform for them, that is, not perl.

      perl -le'print map{pack c,($-++?1:13)+ord}split//,ESEL'
Re: RFC: (Do Not) Modify the System Perl
by shmem (Chancellor) on Oct 17, 2015 at 13:26 UTC
    So, without further ado, here are some arguments in favor of installing your own Perl (in no particular order):

    I strongly disagree with almost all of them, because they all start from the same point. Why?

    short answer

    If it ain't broke, don't fix it. You should probably NOT mix your custom libraries with the system libraries, but that is a whole different point. Perl has PERL5LIB, lib, FindBin and more ways to include paths unknown to the system within perl's search path.

    long answer

    All of the arguments in favour of installing your own perl don't hold if the system's perl isn't suspicious in one or another way. I rarely found a seriously broken perl core installation - well yes, RedHat is doing such a terrible job here sometimes that one might suspect they want to discourage the use of perl all together. But in general, the perl core installation is - just perl core fine-tuned to the capabilities and requirements of the system you are using. Why would you be smarter configuring a suitable perl than those which oversee the whole system?

    If the perl core of your system isn't too much outdated for your needs, why not just use it, along with the pre-compiled packages provided by your vendor? Your system probably provides a toolchain to build perl packages, and with minor tweaks, you can install any additional or newer version of needed modules into a custom path, even application-wise, and prepend that location to your @INC. That way you are only responsible for those packages which you absolutely need with newer features or custom bug fixes.

    Some reasons why your system's perl might be better than a custom built perl:

    • the systems's perl is always up to date with system libraries
    • your vendor might provide you with new packages containing securtity/bug fixes for problems you are not aware of
    • the vendor's approach is almost always conservative - they are not interested in breaking perl (but see RedHat) as part of the system
    • using a reasonable well built perl including packages built with that perl and it's toolchain saves you work and reduces your technical debt
    • chances are higher that problems within your system's perl are known out there, and that there is a fix - you might get better support and hotfix packages

    Of course, if building your essential modules turns out to be a nightmare with your vendor's perl - then that perl build is broken. Roll your own, complain, and if possible, provide upstream patches.

    perl5porters, module authors and the perl community at large as well as vendor software maintainers are generally doing an awfully good job at providing a perl out of the box which just works, and deficiencies of older UNIXes like AIX or Solaris, b0rken stuff like RedHat's perl v5.8 or braindead update procedures like on MacOS are no reason to degrade perl's fame of being way better in backward compatibility, write once - run everywhere and robustness than Java, Python or PHP.

    Generally advocating always roll your own perl is a disservice to perl users and programmers.

    tl;dr

    If you know what you are doing, and why - build your own perl. But always disregard the system's perl and build your own MUST NOT become another of those recommended Perl Best Practices.
    Summary:

    In summary, unless you are writing an OS-level tool specifically for this platform, install a Perl version specifically for your own application's runtime stack.

    If the perl shipped with your OS is broken, get another OS. Why should the OS be any less broken than perl on that OS?

    perl -le'print map{pack c,($-++?1:13)+ord}split//,ESEL'
Re: RFC: (Do Not) Modify the System Perl
by bitingduck (Chaplain) on Oct 17, 2015 at 03:33 UTC

    I'd maybe point out in the "you can break the OS" part that while the OS may include a version that has what are now known bugs, sometimes there are features that effectively depend on those because the OS counts on the bad behavior for some reason. All the more reason to wait for the system Perl to get updated with system updates.

    Another pro is that if you're trying to either test someone else's code that depends on the version, or test a claimed bug (e.g. posted at PM) it's much easier to do if you're set up to switch among Perl installs at will, and can install new ones safely and cleanly with something like Perlbrew.

    It's easier to upgrade to a new Perl if you don't have to do it all at once. You can install the latest and greatest, write new code with it, etc. without worrying about breaking any of your existing code and having to spend a bunch of unplanned time fixing it. This is especially useful for "intermittent" developers like me. (FWIW, I do find that Perl tends to have very good backward compatibility and can't recall the last time I ran into a conflict, unlike Ruby, which sometimes makes me crazy with incompatible dependencies.

      ...sometimes there are features that effectively depend on those because the OS counts on the bad behavior for some reason.

      Which are proliferated bugs, and those strongly merit to be complained about.

      perl -le'print map{pack c,($-++?1:13)+ord}split//,ESEL'
Re: RFC: (Do Not) Modify the System Perl
by KeighleHawk (Scribe) on Oct 21, 2015 at 13:18 UTC
    I'm inclined to agree with shmem, though perhaps not quite so vehemently. I believe your reasoning is fine, but does not stand on its own. There are other issues, if ignored could lead you to these conclusions much faster than you should.

    First and foremost, as shmem stated, "if it ain't broke, don't fix it." In the old days, when distros first started packaging extras like PERL with their own package managers, odds were, it was broken except for the most trivial needs. Again as already stated, those days are **mostly** past. Therefore, the exercise of asking yourself if you are really on one of those few, still busted up distros, should be asked BEFORE all the other questions you list.

    Second, and almost as important, never, ever, never, ever mix and match maintenance paths. The vendor provided perl distro should never have CPAN invoked on it. While that may not seem like an issue from a PERL developer standpoint it is a nightmare from a System Administrator standpoint. It breaks anything but the most trivial maintenance processes and positively does not scale. If you must inject non-vendor supplied packages, it should be done along side the vendor supported mechanism (eg. for Red Hat that would be by installing an rpm) from a locally managed library. This allows your solution to scale using things like Puppet and other products without having to apply a sledge hammer and crowbar to the mix.

    It helps if you clarify for yourself what you intend to support. If you support the OS, then you should install to that OS in the manner expected by the OS. If you support an Application, then that application is limited to what is supplied by the OS (and it's support mechanism). If you install both PERL and an Application, then you are supporting BOTH! That's why when companies like Oracle or Google install a product, the supporting products they need are usually buried in the application directories. No one else is expected to use or monkey with them. As far as the Sys Admin is concerned, Application SpiffySoft was just installed. Not SpiffySoft and its own PERL distro. Again, that distro should never be managed by CPAN as that invites disaster. It is managed by patches/updates from SpiffySoft Corp. If you must manage multiple PERL installations, it should be due to "reasonable" incompatibilities between products and even then, in today's world, you are probably better off creating virtual or chroot'd environments to keep them separated.

    In short, before committing to the path of maintaining separate PERL distros on the same machine, first examine why you've gotten yourself into that pickle to begin with. Make sure you have not committed some other fatal mistake that should be fixed first. Second, examine your application feature set. Are those requirements needed for your app/platform (eg. due to a bug) or just because you want to use that slick cool module? Remember, the harder your application is to install and maintain, the less cool it becomes over time...

Re: RFC: (Do Not) Modify the System Perl
by dsheroh (Monsignor) on Oct 17, 2015 at 09:25 UTC
    I think you could structure this a little more clearly. The title and introduction seem to present this as "pros and cons of modifying the system Perl", and then the lists are pros and cons of installing your own Perl. If someone (like, say, me...) isn't reading carefully, they may start on the "pro" list and think "'Modifying the system Perl can break the OS.'? How the hell is that a point in favor of modifying the system Perl?" before noticing that it's a list of reasons to build your own, not a list of reasons to modify the system Perl.

    Also, I'm not clear on how "You are (safely) limited to the module versions provided in packages by your vendor" fits into the "pros of building your own" list. If you build your own Perl, you aren't limited (safely or otherwise) to module versions provided by the vendor.

Re: RFC: (Do Not) Modify the System Perl
by karlgoethebier (Abbot) on Oct 17, 2015 at 14:11 UTC
    ..."that 'Can't touch this', if given the choice, is the correct path to take"

    Yes.

    "...install a Perl version specifically for your own application's runtime stack"

    Yes.

    "Anything else?"

    IMHO yes. Add a functional user and install perlbrew under it's UID. You might need to add this UID to the group wheel.

    Regards, Karl

    «The Crux of the Biscuit is the Apostrophe»

      Would you please explain the reasons for your yesses?

      • why are other paths incorrect?
      • aren't the limits of "your own stack" arbitrary? where does it begin, where does it end, and why? which system libraries, tools and applications are part of that stack? where does your trust begin, where does it end?
      • is adding a functional user always (and the best) way to go? why?

      perl -le'print map{pack c,($-++?1:13)+ord}split//,ESEL'

        I wonder why you kick up such a shindy about this issue. ("so ein Fass aufmachen...?").

        If you got an application, it needs to have a home and a UID an so on.

        It is simply so that Linux/Unix works like this.

        It's a proven practice/concept that worked well for decades. Having everything at one place has many benefits.

        And i agree to all the points the OP mentioned.

        Best regards, Karl

        P.S.: You ever had a broken Perl?

        «The Crux of the Biscuit is the Apostrophe»

Re: RFC: (Do Not) Modify the System Perl
by sundialsvc4 (Abbot) on Oct 20, 2015 at 03:56 UTC

    Pragmatically speaking, you probably can at-least tolerate “the system Perl,” provided (a) that it is of a sufficiently recent version, and (b) that you can cleanly ensure that “your app” will always encounter “your version” of those libraries which actually matter to “your app,” superseding any system alternatives.

    If you are stuck on a shared-host service whose Perl is unacceptably ancient, then, yeah, you have to install your own Perl.   (But, some hosts do not allow you to install binary executables.   Catch-22.™)

    Otherwise, you may be perfectly all right setting up an application-specific library directory, ensuring that it is first on PERL5LIB (or use lib), ensuring also that cpan[m] points to it, and then installing(!) the app-important libraries (via cpan[m](!) ...) into that location.   The module-installation process will see the entire @LIB list ... including “the System Perl” on this particular system ... and will Do The Right Thing.™   But this only works because cpan[m] can see the System Perl environment as it lurks just-behind your app-specific directory in the module search order, and because of the total set of work that each package’s installation scripts do.

    All of this, of course, is enormously influenced by ... XS.   (Such that, really, this one concern rules them all.)   Any [Perl] application is likely to make many references to .so or .DLL libraries that are present on the system, and there is unfortunately a very good chance that you cannot supersede nor replace too-old versions of those!   :-X

    And all of this presupposes that your app has a thoroughly-complete Test::Most suite, and that you run it to verify that the application does run correctly in its target environment at the conclusion of your installation sequence.

Re: RFC: (Do Not) Modify the System Perl
by sundialsvc4 (Abbot) on Oct 21, 2015 at 15:17 UTC

    In practice, this is very much like “installing Perl packages as a non-root user,” i.e. the shared-hosting situation.   I used to try to engineer the value of PERL5LIB to contain a certain value, but most recently I’ll use a stub script that issues use lib statements, then instantiates an object representing the main routine and invokes it, doing all of these things within an eval{} error-trap block.

    As stated earlier, I will explicitly install whatever is needed by a particular application, into a private library-directory, always using cpan so that all installation scripts and dependencies are correctly handled.   If the system-Perl environment is updated, you may need to re-install one or more packages.   Only once did I encounter a too-old .so-file issue, and for various arcane reasons I had to switch hosting-companies in order to solve it.   (Shared-hosting providers are becoming increasingly restrictive, for obvious reasons.)