I'm the author of Type::Tiny which internally does a lot of inlining. I mean, seriously a lot.

As an example, there is code defined for Int to check a value is an integer, and code defined for ArrayRef to check a value is an arrayref. A lot of Type::Tiny is about efficiently being able to compile those into checks like ArrayRef | Int (check a value is an ArrayRef or an integer), ArrayRef[Int] (check a value is an arrayref of integers), and Int | ArrayRef[Int] (check a value is an integer or an arrayref of integers). If the arrayref is long, you really want to avoid calling a sub to do the integer check for every single element.

So here are my thoughts on what you've said:

Your benchmark is a little too simple. You'll notice the "plain" case and the "do" case run at the same speed. This is because Perl is able to optimize the do case to the plain case. If your do block did something like load a lexically scoped pragma or declare lexical variables, it could not be optimized this way and would run slower than the plain case. Though still a lot faster than a sub call. If a do block can be avoided, you'll get better performance.

Your module implementation is mostly good, but I don't like how you configure it with global variables. What if two modules by different authors try to alter $declmatch? It would be better to allow people to pass these as parameters to import.

You don't seem to do anything to cover the case where the inlined sub closes over variables which are declared later than when it is called.

I have a module called Sub::Block which is similar in aim to your module, though takes a pretty different approach. You may be interested in borrowing the _check_coderef function though. Given a coderef, it inspects the optree for the sub body, searching for any use of return or wantarray and throws an error if it finds them. As noted in your documentation, using return in inlined code is bad. (You may also want to note wantarray and caller in your documentation as also being promlematic.

Your benchmark is a little too simple. You'll notice the "plain" case and the "do" case run at the same speed.

Of course, but there's that drop for the "do_vars" sub in which the arguments passed to a subroutine are assigned as lexicals inside the "do" block. In that case optimizing away the scope isn't possible, which of course happens with loading pragmas.

Your module implementation is mostly good, but I don't like how you configure it with global variables. What if two modules by different authors try to alter $declmatch? It would be better to allow people to pass these as parameters to import.

Thank you. - Indeed that's work to be done yet (and easy enough): making the package globals into package private lexicals as defaults, which can be overridden by named import parameters. This definitely makes more sense. update: done in the OP.

You don't seem to do anything to cover the case where the inlined sub closes over variables which are declared later than when it is called.

This is on purpose, because that would get me into trouble, into the egg vs. hen problem: I would need to check for outer frames and inspect the nature and existence of variables in those, which are used inside the inlined block. But only constants in closures are suitable for inlining, dynamic closures aren't. So I would need to compile the whole stuff first to munge the source after the fact... no. This module is purely for textual replacement, think cpp.

As a related note, perl4 had the -P command line switch, which caused the source being run through cpp. When did that go away in perl 5? and why?

perl -le'print map{pack c,($-++?1:13)+ord}split//,ESEL'

The -P switch was deprecated in 5.10 and removed in 5.12. (This was before the current two-stable-versions deprecation cycle.)

It was replaced with Filter::cpp (though it's only on CPAN, not in core).

toby döt ink

Hi Toby

> I'm the author of Type::Tiny which internally does a lot of inlining. I mean, seriously a lot.

May I ask how you solved this?

> This is because Perl is able to optimize the do case to the plain case.

Are you sure?

  DB<17> sub test { print do { 5+4 } }

  DB<18> print B::Deparse->new("-p")->coderef2text(\&test)
{
    use feature 'current_sub', 'say' ...       # yadda shortened
    print(do {                                 # still here
        9
    });
}
[download]

update

this is the mentioned do-case

{
    use warnings;
    use strict;
    my($total);
    foreach $_ (1 .. $end) {
        ($total += do {
            (1 / $_)
        });
    }
    $total;
}
[download]

  DB<27> print $]
5.024001
[download]

hence the do-opcode is still present.

I suppose you meant to say that it has no performance penalty.(?)

Cheers Rolf
_{(addicted to the Perl Programming Language :)
Wikisyntax for the Monastery FootballPerl is like chess, only without the dice}

Each type constraint knows how to generate a string of Perl code for validating a value, given a variable name for the value.

For example:

$ perl -MDevel::Hide=Ref::Util::XS,Type::Tiny::XS -MTypes::Standard=-t
+ypes -E'say HashRef->inline_check(q/$myvar/)'
Devel::Hide hides Ref/Util/XS.pm, Type/Tiny/XS.pm
(ref($myvar) eq 'HASH')
[download]

So the ArrayRef[HashRef] type constraint can use that when generating a longer bit of code to validate an arrayref of hashrefs:

$ perl -MDevel::Hide=Ref::Util::XS,Type::Tiny::XS -MTypes::Standard=-t
+ypes -E'say ArrayRef->of(HashRef)->inline_check(q/$myvar/)'
Devel::Hide hides Ref/Util/XS.pm, Type/Tiny/XS.pm
do { (ref($myvar) eq 'ARRAY') and do { my $ok = 1; for my $i (@{$myvar
+}) { ($ok = 0, last) unless (ref($i) eq 'HASH') }; $ok } }
[download]

toby döt ink

Oh, and see also GFUJI's macro which takes a similar approach to your module, but not as nice a syntax.

Also this old github gist of mine which is kind of similar to GFUJI's module (less polished), but uses the Perl 5.14 keyword API instead of source filters.

Source filtering? No!

Quick, throw away your C pre-processor! and along with that, all that autoconf and automake rubble.
Seriously, why should source filtering be bad for perl, but good for C?

I have explained in the op why in this case source filtering makes sense, and I see no point against that if it is done sensibly. If I don't do it sensibly, please point out why.

You seem to have missed Damians latest(?) stroke of genius resulting in Keyword::Declare and PPR.

I missed them only slightly. I am aware of those beasts, but I didn't delve much into them because I currently haven't any use for them. Please don't compare me to Damian, that's unfair to both of us.

perl -le'print map{pack c,($-++?1:13)+ord}split//,ESEL'

I think there's a misunderstanding here, I am not comparing anything. I wanted to express that the Keyword::Declare module is the tool you need to implement your module without source filters or rolling your own perl parsing regexes.


holli

You can lead your users to water, but alas, you cannot drown them.

Despite such vehement discouragement, there's nothing (IMHO) intrinsically wrong with source filtering. It's just requires being very careful when using it. Also, it can make a program it using harder to debug.

Anyway, I can suggest how you might use Keyword::Declare to accomplish your inlining module. Though with a modified syntax.

Although Keyword::Declare will let you redefine sub (and other keywords), it won't do what you want. Instead, I suggest adding another keyword, maybe inlinable to mark a subroutine as suitable for inlining. Your handler for inlinable would save the body of the sub for later use by inline, then return the sub definition without the inlinable keyword.

For inline sub definitions, save the body, then instead of returning the sub definition, use the name to define a new keyword. When perl later encounters the name of the sub, it will be treated as a plug-able keyword. then your handler can inline the routine.

Caveat: Keywords defined by Keyword::Declare are only recognized at the beginning of a statement. If you want to inline in the middle of a statement, you have to enclose the "call" in a do { } block.

There might be a plus side to this caveat. Since you know the inlining will always take place at the start of a statement, you might not have to inline it as a do { } block, but as a plain block. This is because if your inlined code isn't "returning" a value, you shouldn't need the do { } block. But if it does "return" a value and you want that value, the "call" will already be inside a do { } block (as long as it is the last (or only) statement in the do { } block).

Summary: A Keyword::Declare based module may be easier to fully implement, but a source filter version probably will result in a cleaner syntax. The Keyword::API might be able to install keywords that work in the middle of expressions, but is not as easy to use as Keyword::Declare.

Disclaimer: I have not tested any of this. YMMV.

Edited to correct grammer errors.

Seriously, why should source filtering be bad for perl, but good for C?

In both cases the compiler might report an error where the mistake was inserted, rather than where it actually is. Also in perl, source filters screw up the line numbering (insert two lines in the code, and the line numbers for every compilation message, or warn or die afterwards will be off by two). A call to warn on the last line of your test will indicate an issue on line 35 of a 26 lines file...

The #line directive can solve most of those issues though, so if you turn

sub capitalize_next {
    my ($thing) = @_;
    uc inline increase($thing);
}
sub increase {
    my ($foo) = @_;
    ++$foo;
}
[download]

into

sub capitalize_next {
    my ($thing) = @_;
    uc do { local @_ = ($thing);
#line 6
      my ($foo) = @_;
      ++$foo;
#line 3
    };
}
sub increase {
    my ($foo) = @_;
    ++$foo;
}
[download]

Mistakes in the parameter list (eg increase(£thing)) will be marked as coming from line 3 (the line of the call), and mistakes from the function definition (eg ++£foo) will be reported as coming from inside the function definition.

local @_ makes it possible to access $_[0] and other values inside @_, while making line numbering easier (if ($foo) = @_ is replaced by ($foo) = $thing, the LHS comes from the function body but the RHS from the call, so where do you report an erreor?), but it probably slows things down... And since $foo = shift; uses @_ inside a function, but @ARGV elsewhere, it might introduce some interresting bugs.

Also note that the C preprocessor will only expand macros in code, not in comments nor in strings. Try this in your example:

sub capitalize_next {
    my ($thing) = @_;
    # uc inline increase($thing);
}
[download]

and you'll get some confusing errors. It's pretty easy to correct that one, but things might get difficult with strings and pod.

I don't know much about source filtering, and perhaps Filter::Util::Call does some whitespace munging, but does this regexp forbid sub declarations from being indented?

qr{^inline\s+sub\s+(\w+)\s*(?:;|\{)}ms
[download]

Obviously, which is a bug arguably. But then, indented sub declarations are (within my coding standards) closures and such. I will use

qr{^\s*inline\s+sub\s+(\w+)\s*(?:;|\{)}ms
[download]

if I can be convinced that this is a sensible default for inlineable subs.

perl -le'print map{pack c,($-++?1:13)+ord}split//,ESEL'

Indenting some sub definitions is useful for applying a lexical pragma like no warnings qw(redefine) but then again pragmata wouldn't survive inlining. (Though you could set pragma inside the sub.) I do also use them for grouping code occasionally, but the biggest reason to support indented sub definitions is:

package Foo {
   sub bar {
      ...;
   }
}
[download]

toby döt ink