http://qs321.pair.com?node_id=881827

Anonymous Monk has asked for the wisdom of the Perl Monks concerning the following question:

I was going through the Perl 5 interpreter code, I find there are simply too many macros to deal with. Often to figure out the meaning of a macro I have to go almost six to seven levels of more macros below it to understand. While I understand it might have been done to hide the underlying implementation which occurs frequently, its just too many of them to deal with.

I wanted to know what techniques do the maintainers use to work around this? Also sometime all there is to a scoping block are macros, there fore its very difficult to find the logical meaning of that block in such a case.

Any technique some one is using to work around this, tool or documentation?

Replies are listed 'Best First'.
Re: Perl 5 interpreter
by moritz (Cardinal) on Jan 12, 2011 at 08:15 UTC

    Regarding documentation, illguts (illustrated guts) is certainly worth looking, as well as perlhack (there's a tour through the interpreter) and perlxs.

    The macros do make it hard to read, but writing portable and efficient C code basically requires excessive macro usage.

    As for searching things in code files, I found App::Ack to be more convenient than grep.

      but writing portable and efficient C code basically requires excessive macro usage.

      Most, if not all of the macros in perl5 could be replaced by inline functions and be as if not more efficient.


      Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
      "Science is about questioning the status quo. Questioning authority".
      In the absence of evidence, opinion is indistinguishable from prejudice.

        Except that Perl also is supposed to compile on C89 (or is that ANSI C, or even K&R?) compilers that don't support inline. And then you'd either get non-inlined functions (which would bring us back to inefficient) or no Perl at all...

        Definitely not all. Some macros ("dXXX;") create and initialise variables used by one or more other macros. Those macros and the macros that use the variables declared by them couldn't be made into functions without a cost.

        In fact, these are the only macros that gave me any confusion. The documentation isn't clear as ti what declares what and what uses what. The only other time I've had to look into the macros was to see what exactly is an acceptable input. (e.g. Is NULL ok?)

        I'm not sure how much more readable inlinable functions would be. Looking down seven levels of functions is no easier than looking down seven level of macros. I imagine most debuggers are better at handling inline functions than macros, though.

Re: Perl 5 interpreter
by Corion (Patriarch) on Jan 12, 2011 at 08:13 UTC

    To expand a macro there is expand-macro.pl.

    That said, in my opinion, you're better off learning the "language" that Perl is written in, that is, learning the macros instead of trying to learn the C code behind them.

Re: Perl 5 interpreter
by eyepopslikeamosquito (Archbishop) on Jan 12, 2011 at 09:56 UTC

    Any technique some one is using to work around this, tool or documentation?
    Some Perl 5 pumpkings recommend playing Jenga:

    The perl5 internals are a complete mess. It's like Jenga - to get the perl5 tower taller and do something new you select a block somewhere in the middle, with trepidation pull it out slowly, and then carefully balance it somewhere new, hoping the whole edifice won't collapse as a result.

    -- Nicholas Clark

    Incidentally, you may not be surprised to learn that Larry Wall won the Grand prize in most well-rounded in confusion in the 1986 International Obfuscated C Code Contest. You may be surprised to learn that he did not submit the Perl C source code (I believe the size limit of the contest prevented him from doing that).

      And relevant to this thread, that entry into the obfuscation contest can only be compiled with gcc if you use -E -traditional in a first step to get the macros out. And sadly produces a segmentation fault on my machine (even when compiled with -m32).
Re: Perl 5 interpreter
by tilly (Archbishop) on Jan 12, 2011 at 08:10 UTC
    I believe that the intended solution is to learn to use the high level macros, and not worry too much about the details.

    That said, I fully agree that there are a lot of them to learn.

      Actually most(Almost all) don't have documentation regarding what they do. So inorder to figure out the logical equivalent of code, the sub macros have to be hand assembled into the main macro.

      Then that macro put back to the original calling scope of the program. To make complete sense of the scope, each and every macro needs to be assembled.

      That kind of puts the complexity out of bounds for human interpretation.

Re: Perl 5 interpreter
by syphilis (Archbishop) on Jan 12, 2011 at 08:19 UTC
    While I understand it might have been done to hide the underlying implementation...

    I've seen it claimed that the real aim was to write an interpreter using *only* C macros.
    (The attempt failed ... but not by much ;-)

    Cheers,
    Rob
Re: Perl 5 interpreter
by Anonyrnous Monk (Hermit) on Jan 12, 2011 at 08:35 UTC

    Then again, as a last resort, there's always the compiler's -E option — to spit out the fully expanded C code instead of producing an object file.  (But be sure to run the compiler command with the otherwise same options that it would be run during a normal build, in order to get the correct results.)

    For example, from gcc's man page:

    -E Stop after the preprocessing stage; do not run the compiler pr +oper. The output is in the form of preprocessed source code, which is sent to the standard out +put.

      That throws text like this for example

      (void)( { while (PL_curstack != PL_mainstack) { Perl_dounwind(-1); + (void)( { SV **sp = PL_stack_sp; PERL_SI * const prev = PL_curstacki +nfo->si_prev; if (!prev) { fprintf((PL_stderrgv && (((svtype)((PL_std +errgv)->sv_flags & 0xff)) == SVt_PVGV) && (((PL_stderrgv)->sv_u.svu_g +p)->gp_io) && ((XPVIO*) ((((PL_stderrgv)->sv_u.svu_gp)->gp_io))->sv_a +ny)->xio_ofp ? ((XPVIO*) ((((PL_stderrgv)->sv_u.svu_gp)->gp_io))->sv_ +any)->xio_ofp : stderr), "panic: POPSTACK\n"); Perl_my_exit(1); } (vo +id)( { ((XPVAV*) (PL_curstack)->sv_any)->xav_fill = sp - PL_stack_bas +e; PL_stack_base = ((prev->si_stack)->sv_u.svu_array); PL_stack_max = + PL_stack_base + ((XPVAV*) (prev->si_stack)->sv_any)->xav_max; sp = P +L_stack_sp = PL_stack_base + ((XPVAV*) (prev->si_stack)->sv_any)->xav +_fill; PL_curstack = prev->si_stack; } ); PL_curstackinfo = prev; } ) +; } } );

      That is unreadable

        Which is why Perl programmers use macros, to hide the complexity.

        I guess this is code from the preamble to an XS sub that marks the Perl stack, or maybe code from the end of an XS sub that restores the previous stack state. If you show the original macros, it's maybe easier to explain.

Re: Perl 5 interpreter
by sundialsvc4 (Abbot) on Jan 12, 2011 at 14:44 UTC

    This somewhat-curious style of programming is partly intended to simplify the task of porting the interpreter to other (perhaps radically different) execution environments.   The programmers work in the “language” that is defined by those macros, so that they can ignore, insofar as possible, how this-or-that macro will expand on any particular computer.

    It is “an acquired taste.”

    And it runs like the Devil’s own (furry, winged) bat.

Re: Perl 5 interpreter
by Anonymous Monk on Jan 12, 2011 at 08:10 UTC
      Considering how many lines of C some of the macros expand to, and how little the English version will assist you, I strongly expect that not to help very much.

      Assuming, that is, that you could easily let it know about all of the layers of macros that were the original problem.

        I think even there were to be a tool that could extract English out of such macros. The English would be an encyclopedia of C terms, in dense texts under many scopes. Such a thing would be more difficult to understand than actual macros themselves.