Freeze @INC?

Surge has asked for the wisdom of the Perl Monks concerning the following question:

Summary: I would like to capture a complex @INC, so it does not have to be re-built at runtime.

Goals:

avoid complex configuration
speed up module load
separate module management from development
enable use of multiple module versions from one perl install
empower code stability with respect to module version changes

Discussion:

I have a scenario, where many module versions are available in many directories. While there exist facilities to add these to @INC, they are only for runtime configuration. I have seen similar issues in the past, where modules where installed in a variety of locations and keeping track of them was too complex. In addition, there is always a risk of a module upgrade breaking existing code.

I would like to either find or write a perl prog, that can take the compiled @INC, delta it with anything brought in by PERL5LIB/PERLLIB, various 'use's and any other @INC hackery (which would be separately defined) and create a perisistent snapshot of the config, for example, in a single directory tree of symlinks. Subsequently, I would be able to run perl with PERL5LIB set to this directory and forget all the @INC hackery, as well as achieve the other goals above.

Is this a reasonable idea?

Does something like this already exist?

Is the suggested implementation approach of collecting symlinks in a single tree a reasonable one?

The only shortcoming to the above may be that it would only have utility on *nix and the like due to symlinks.

Thanks for the forthcoming wisdom.

Comment on Freeze @INC?

Replies are listed 'Best First'.
Re: Freeze @INC? by almut (Canon) on Jul 24, 2008 at 20:00 UTC
Is the suggested implementation approach of collecting symlinks in a single tree a reasonable one? If it helps, why not? :) In case you have a runnable program, you can dump the values in the %INC hash after all modules have been loaded. Those are the full paths to the actual modules used. Then, simply create a symlinks directory with that particular snapshot of modules, and put it in PERL5LIB (or make it the first/only directory in @INC). Architecture specific parts would have to go in an `auto` subdirectory, as usual. To list those, use PERL_DL_DEBUG, e.g. $ PERL_DL_DEBUG=1 perl -MList::Util -e'print join"\n",sort values %INC +' ... DynaLoader::bootstrap for List::Util (auto/List/Util/Util.so) /usr/lib/perl5/5.8.8/AutoLoader.pm /usr/lib/perl5/5.8.8/Carp.pm /usr/lib/perl5/5.8.8/Exporter.pm /usr/lib/perl5/5.8.8/List/Util.pm /usr/lib/perl5/5.8.8/strict.pm /usr/lib/perl5/5.8.8/vars.pm /usr/lib/perl5/5.8.8/warnings.pm /usr/lib/perl5/5.8.8/warnings/register.pm /usr/lib/perl5/5.8.8/x86_64-linux-thread-multi/Config.pm /usr/lib/perl5/5.8.8/x86_64-linux-thread-multi/DynaLoader.pm /usr/lib/perl5/5.8.8/x86_64-linux-thread-multi/XSLoader.pm [download]	[reply] [d/l] [select]
Re: Freeze @INC? by gaal (Parson) on Jul 24, 2008 at 19:58 UTC
I read this several times and it's not clear to me if you mean you want to cache the value of `@INC`, the variable, or snapshot the state on disk. If it's the former, and you only have data in your computed @INC (not code), you can indeed just serialize to a file and load it in a BEGIN block. You'd probably want your own serialization as Storable might be on the list of things you want to control! If the latter, I'd suggest avoiding symlink hell and just using PAR.	[reply] [d/l]
Re^2: Freeze @INC? by Surge (Initiate) on Jul 24, 2008 at 21:40 UTC
I apologize for lack of clarity. I do not want to snapshot the disk - the contents will not change. As thinking about it makes me realize, I wanted to do two things at once: do the work of computing @INC in advance reduce the number of computed elements in @INC by consolidating contents of multiple locations into one with symlinks The first is primarily for manageability and stability, the second is primarily for performance. I intend to follow your advice and snapshot a serialized @INC after it is computed. Data::Dumper seems to be the right answer, as it is readily available, if 5.8 or later can be assumed. Then I can do a few tests to see whether the performance component is actually a real concern in any of my cases. If so, I can add a sepearate step for consolidating many @INC components into one by caching symlinks. Thanks!	[reply]
Re^3: Freeze @INC? by diotalevi (Canon) on Jul 25, 2008 at 15:55 UTC
Data::Dumper is available for many versions into the past. I'd be surprised if it wasn't available in 5.0. ⠤⠤ ⠙⠊⠕⠞⠁⠇⠑⠧⠊	[reply]


laziness, impatience, and hubris
	PerlMonks