do [...], @INC subs, 3-args open to heredoc refs (oh my)

ForgotPasswordAgain has asked for the wisdom of the Perl Monks concerning the following question:

Someone (Maddingue) in #perlfr mentioned that they couldn't understand the following test code in bleadperl/t/op/incfilter.t:

do [$fh, sub {s/s/ss/gs; s/([\nS])/$1$1$1/gs; return;}] or die;
[download]

After looking at it, I gave up, so I was wondering if someone could explain it in gory detail. Thanks.

I'll explain what I figured out.

First, what does do [...] do? Does it use something like "ARRAY(0x814bc28)" as the filename to open? If you look at the test file, at the top is:

unshift @INC, sub {
    no warnings 'uninitialized';
    ref $_[1] eq 'ARRAY' ? @{$_[1]} : $_[1];
};
[download]

If you look in perldoc perlvar for the docs on @INC, you'll find that it says

You can also insert hooks into the file inclusion system by putting Perl code directly into @INC. Those hooks may be subroutine references, array references or blessed objects. See "require" in perlfunc for details.

So look also in perldoc perlfunc, grep for require VERSION followed by hooks. By this point my head is already spinning, but then note that before the do comes a 3-arg open of a reference to a heredoc:

open $fh, "<", \<<'EOC';
BEGIN {prepend_block_counting_filter};
pas("SSS make s fast SSS");
EOC
[download]

The docs for that are in perldoc -f open:

open file handles to "in memory" files held in Perl scalars via:

    open($fh, ’>’, \$variable)
[download]

Then note that \"foo" makes a reference to a string, so \<<'EOC';... is just a funny reference to a string that happens to've been made by a heredoc.

Now's where I get lost. Anyone fill in the details? (What's the $fh doing in do [$fh, sub {...}], for example?)

Comment on do [...], @INC subs, 3-args open to heredoc refs (oh my) Select or Download Code

Replies are listed 'Best First'.
Re: do [...], @INC subs, 3-args open to heredoc refs (oh my) by Maddingue (Sexton) on May 01, 2006 at 16:48 UTC
In fact, I partially understand how it works, but not completely. As I told you on IRC, it's a two steps trick. First, a hook is installed inside `@INC`. What this sub does is to check whether it's second argument is an arrayref, and derefence it, hence returning the contained list. Second, `do()` is passed an arrayref with a filehandle and a coderef inside. As `do()` also uses `@INC`, the hook that was installed is invoqued and returns the filehandle which is read and `eval`'d by `do()`. The last thing I don't understand is when and how the coderef (the second element of the arrayref given to `do()` is called. It is obviously called as it plays the role of a filter, but I don't understand how. And the use of the 3-args `open()` with a scalarref as file name is just a convenient way to avoid having real files everywhere in the hierarchy (plus, it also avoid path-related problems). But I think that this test script uses a few features that are not documented anywhere except in the source code.	[reply] [d/l] [select]
Re^2: do [...], @INC subs, 3-args open to heredoc refs (oh my) by vkon (Curate) on May 01, 2006 at 18:13 UTC
you understood most essential part. Last, and simpliest part is, once hooked by @INC subroutine gets anonymous array, it could do with it whatever it decides to: call its subs, use first argument as file handle, call any found CODE ref, and so on... pretty simple, IMO.	[reply]
Re^3: do [...], @INC subs, 3-args open to heredoc refs (oh my) by ForgotPasswordAgain (Priest) on May 03, 2006 at 11:40 UTC
I don't agree that it's simple. You haven't explained how the subroutine reference in @INC gets called. Yes, it could in principle do whatever it wants, but it specifically does `unshift @INC, sub { no warnings 'uninitialized'; ref $_[1] eq 'ARRAY' ? @{$_[1]} : $_[1]; };` [download] which doesn't call the sub. The sub is somehow passed to the filter when the `do` reads in the heredoc in the 3-argument open that creates the $fh. The docs I pointed to above in `perldoc perlfunc` say: Subroutine references are the simplest case. When the inclusion system walks through @INC and encounters a subroutine, this subroutine gets called with two parameters, the first being a reference to itself, and the second the name of the file to be included (e.g. "Foo/Bar.pm"). The subroutine should return "undef" or a filehandle, from which the file to include will be read. If "undef" is returned, "require" will look at the remaining elements of @INC. It says that one value, the filehandle, is returned, but the @INC hook above appears to me to return two values; because the "filename" passed to `do` is an array ref, and the @INC hook returns `@{$_[1]}`, those are the items inside the array ref (`$fh, sub {...}`) passed to `do`. As the docs say, the @INC hook should return a single $fh, but how is the second return value (`sub {...}`) called?	[reply] [d/l] [select]


Keep It Simple, Stupid
	PerlMonks