http://qs321.pair.com?node_id=673605

xevian has asked for the wisdom of the Perl Monks concerning the following question:

Here's the setup:

I have a system with a bunch of default packages installed.
I allow users to drop in newer versions of these packages to a well-known location on the disk (not in @INC).

I've defined a universal autoloader to redirect specific function calls to these newer versions... for example:

My::Package::foo() is a subroutine in the default install.
My::Package::foo() is a subroutine in the new version location.

The autoloader (simplified for this example):
sub AUTOLOAD { my $f = $AUTOLOAD; if( $f =~ /^(My(::[a-zA-Z0-9_]+)*)(::|->)([a-zA-Z0-9_]+)$/ ) { my $pkg = $1; eval "require $pkg;"; } if( $f =~ /^NEW::(.*)$/ ) { my $func = $1; unshift( @INC, "/newpkgs" ); return &{$func}(@_); } else { goto \&{$f}; return; } }
There are two problems with this, and I'm grasping at straws now :)

Problem 1: If anything causes My::Package to load before they call NEW::My::Package::foo(), it will use the older version because it was already loaded.

Problem 2: If anything causes the My::Package module to load from the /newpkgs location, subsequent calls to My::Package::foo() (without the NEW:: prefix) will use the new package and not the old one.

What I really need is a way to local the entire symbol table and then clear it out so that calling "NEW::Package::foo()" will load whatever requirements it has first from the /newpkgs location and falling back to the other paths in @INC... and after the call is over ensuring that the symbol table is back to the "before-I-called-NEW::My::Package::foo()" state.

Any suggestions? Many thanks in advance!

Replies are listed 'Best First'.
Re: How can I (safely) use packages of the same name but different versions?
by samtregar (Abbot) on Mar 11, 2008 at 20:31 UTC
    Problem 0: your technique doesn't work at all. Modifying @INC won't change what function is called, it only affects require and use.

    I think you need to slow down - what's the real problem you're trying to solve? Are you trying to create a dynamic plugin system?

    -sam

      I'm sorry I forgot to mention that ALL function calls go through this universal autoloader, and if the function can not be found in the symbol table it figures out the package name with a regular expression, requires the package, and then calls the function:
      # figure out package name as $pkg eval "require $pkg;";
      The real problem is similar to a dynamic plugin system, but I do not want the residual effects of loading a package (permanently modifying the symbol table) that exists elsewhere.

        The real problem is similar to a dynamic plugin system, but I do not want the residual effects of loading a package (permanently modifying the symbol table) that exists elsewhere.

        How do you intend to avoid permanently modifying the symbol table? Why would you want that, anyway? If you don't permanently modify the symbol table won't you have to reload the entire module on each function call?

        -sam
      I don't know anything about xevian's situation, but one place where I can see something kind of like this come up is when there are backward compatibility problems. For example, let's say DBD::XYZ version 1 only supports XYZ database version 1, whereas DBD::XYZ version 2 only supports XYZ database version 2. If your code, for whatever reason, might have to connect to instances of different versions of XYZ within a single program, you're going to have a problem.
        I've heard rumors that Perl 6 will solve this problem. Personally I don't think it's going to be all that helpful. New versions are released for a reason - leaving old code using old versions of their dependencies is only going to get you so far. Your example is a good one - if XYZ v1 and v2 are both being used to access the same physical database then you've likely got a problem that Perl alone supporting multiple version loading can't solve.

        I haven't had a chance to try it yet, but I thought Erlang's facility for running multiple versions of code long enough to do a seemless cut-over sounded smart. Since everything in Erlang is a networked server the system starts up the new version and starts sending all new requests to it. Once the old version is done handling any lingering requests it gets shutdown.

        -sam

Re: How can I (safely) use packages of the same name but different versions?
by aufflick (Deacon) on Mar 12, 2008 at 03:44 UTC
    I'm not convinced that "safely" and a global autoload belong in the same post. Seriously this is going to kill your performance and bite you with unexpected side effects.

    Like sam I suggest you find a simpler solution. Just because you can change the symbol table doesn't mean it should be your first option.

Re: How can I (safely) use packages of the same name but different versions?
by xevian (Sexton) on Mar 12, 2008 at 18:12 UTC
    For those who were interested in the problem, I was able to have the "new" code run in a "safe" way by forking... forking copies the symbol table and changes to it in the child process are not mirrored to the parent process.
    Here's a quick snippet showing the method...
    use Data::Dumper; sub foo { if ( open(FROM, "-|" ) ) { # PARENT local($/); my $output = <FROM>; # from the child my $retval = eval "$output"; my @values = @$retval; return $values[0] if( @values == 1 ); return @values; } else { # CHILD unshift( @INC, "/path/to/NEW"); findAndRequire( 'My::Package' ); my @output = My::Package::func(); # local not really needed here, but who is counting local $Data::Dumper::Purity = 1; local $Data::Dumper::Terse = 1; print Dumper( \@output ); # to the parent exit; } } sub findAndRequire { my ($pkgName) = @_; $pkgName =~ s/::/\//g; # search @INC for a file which matches foreach my $dir (@INC) { my $file = "${dir}/${pkgName}.pm"; if( -e $file ) { require "$file"; last; } } } # Call our "proxied" function! my $output = foo(); local $Data::Dumper::Purity = 1; local $Data::Dumper::Terse = 1; print "OUTPUT = " . Dumper( $output );
      That should work fine as long as all your code is fork-safe. Beware though, DBI connections will cause you all manner of problems when forking, even if the children don't use DBI at all.

      But before you go, would it kill you to tell us why you want to do this in the first place? Inquiring minds want to know!

      -sam

        Thanks for the word of warning about the DBI stuff, I'll be sure to spend some extra time testing that when this actually gets integrated :)

        The reason for doing this is because we have a product which supports adding entities that are essentially sandboxed codebases. The catch here is that these entities can be different versions of the product. (I am not a fan of this, but it's water under the bridge at this point).

        Each codebase has at least one well-known method across all versions, and there are methods in the base product that will iterate over all entities and call this method on each one (to initialize it).

        The snags happen when the codebases are different versions, not just from eachother but also from the main product (which by happenstance also has a definition of this method).

        So, in effect, this solution is necessary to compensate for the requirement that these entities support different versions running on the same machine.
Re: How can I (safely) use packages of the same name but different versions?
by alpha (Scribe) on Mar 12, 2008 at 09:46 UTC
    This is completely crazy. Just update your modules and use them system-wide. And if they're broken, report bugs or write patches :/ This is how it works, not byt modifying @INC and symbol-table-vodoo-ing
Re: How can I (safely) use packages of the same name but different versions?
by Animator (Hermit) on Mar 12, 2008 at 20:19 UTC

    2.5 years ago I was thinking about a similar approach to solve a particular problem. (I eventually walked away from it.) (This was for a mod_perl envirnoment)

    Let me try to explain the approach I used/was playing with:

    • at startup build a list of all the modules that are being used.
    • unshift @INC with a custom code ref:
      That code ref checked if the package that is being loaded is in the list of modules. If it is it does not load that file but instead loads a default file (LoadModule.pm) in which the name of the package (LoadModule) gets replaced with the name of the package that is being loaded.
    • The LoadModule file contains an AUTOLOAD subroutine which checks what version of the file should be loaded. (In my case it did this by looking at the initial caller.)
      Before loading it it changes the name of the package.
    • It loads that file (if it isn't loaded already) and dispatches all methods to the 'correct' version.

    (I also used some code to make sure importing/exporting worked as expected.)

    Attached you can find the code I used. Big note: that code is/was not complety finished and tested. The codes posted is just as is. I did not test it, nor did I clean it up. (Nor do I fully remember how it worked)