Beefy Boxes and Bandwidth Generously Provided by pair Networks
Pathologically Eclectic Rubbish Lister
 
PerlMonks  

Serious Exporter Problems

by PetaMem (Priest)
on Jul 09, 2005 at 14:26 UTC ( [id://473679]=perlquestion: print w/replies, xml ) Need Help??

PetaMem has asked for the wisdom of the Perl Monks concerning the following question:

Dear Monks,

I'm experiencing serious and unexplainable problems with exporting/importing symbols across modules. I'm using Perl 5.8.6 and am not far away to claim to have found a weird bug. Before I do so, I'd like to make sure I'm not missing something. Your assistance is greatly appreciated.

I've got this project with around 30 modules. These modules make frequent mutual use of some of the subroutines and - sometimes - global variables they declare.

My problem is twofold: 1) I still do not exactly know the best practice to distribute these symbols (subs and variables) across the modules, hence I might have some kludges in the code and 2) things that did work and things that should further keep working, suddenly stop working with no obvious reason at compile time and the error messages are IMHO frankly wrong.

Let's start with 2). Right now my code doesn't even compile and bails out with this error message:

Undefined subroutine &PMLS::Help::globalconf called at /home/proj/pmls +/modules/PMLS/Help.pm line 41.
Yeah. That's ok, as there is no &PMLS::Help::globalconf defined. Looking at the source in Help.pm we see at line 41:
my $sections = &dir2array(&globalconf('pmls_help')); # rea +d in all section help files
But OTOH we have at line 23:
use PMLS::System qw(log globalconf);
So why doesn't the compiler see this globalconf? Yes, it works elsewhere, yes it worked before.

"Before", I hear you mumble, "What do you mean by before?"

Well - e.g. adding

use PMLS::Help qw($help);
to PMLS::System. And it's even worse, if I simply comment out this globalconf, then the compiler stops because one of the other modules suddenly doesn't even see the $help from
use PMLS::Help qw($help);
So it is the same problem and again it occurs in just only one module. All others work. What is so special about it? I don't know.

Now - ok: Probably best would be to build something, so the weird behaviour/error would be reproducible. This is no easy task, as this app is around 50k LOC. I will try to set up a bung of module files and try to reproduce that. But am I right in that if I export some variable/subroutine in some module and import it in another module the standard way, that it ALWAYS should be visible (provided no nameclashes etc.). Or do I have to take something else into account? Are some "use cycles" dangerous? e.g having files A B C:

A: use B qw(bloh); use C qw(bleh); export = qw(blah); <can use bloh and bleh> B: use C qw(bleh); use A qw(blah); export = qw(bloh); <can use blah and bleh> C: use B qw(bloh); use A qw(blah>; export= qw(bleh); <can use blah and bloh>
Should not matter - should it?

update:

I did a manual expansion of the "use module" chronology in the project. The "*" Lines show the first occurence of an use.

From the tracestack of the error message
Uncaught exception from user code: Undefined subroutine &PMLS::Help::globalconf called at /home/p +roj/pmls/modules/PMLS/Help.pm line 41. Compilation failed in require at /home/proj/pmls/modules/PMLS/System.p +m line 42. BEGIN failed--compilation aborted at /home/proj/pmls/modules/PMLS/Syst +em.pm line 42. Compilation failed in require at /home/proj/pmls/modules/GDCF.pm line +35. BEGIN failed--compilation aborted at /home/proj/pmls/modules/GDCF.pm l +ine 35. Compilation failed in require at /home/proj/pmls/modules/PMLS/FS.pm li +ne 27. BEGIN failed--compilation aborted at /home/proj/pmls/modules/PMLS/FS.p +m line 27. Compilation failed in require at /home/proj/pmls/modules/PMLS/Reposito +ry.pm line 39. BEGIN failed--compilation aborted at /home/proj/pmls/modules/PMLS/Repo +sitory.pm line 39. Compilation failed in require at /home/proj/pmls/modules/PMLS/Doc.pm l +ine 27. BEGIN failed--compilation aborted at /home/proj/pmls/modules/PMLS/Doc. +pm line 27. Compilation failed in require at /home/proj/pmls/modules/PMLS/Server.p +m line 43. BEGIN failed--compilation aborted at /home/proj/pmls/modules/PMLS/Serv +er.pm line 43. Compilation failed in require at /home/proj/pmls/bin/pmls line 16. BEGIN failed--compilation aborted at /home/proj/pmls/bin/pmls line 16. at /home/proj/pmls/bin/pmls line 16
I suspect, the point where it makes BOOM is here:
*use PMLS::Server * use PMSuite::File qw(safe_save); * use PMLS::Constants; * use PMLS::Doc; use PMSuite::File qw(write_file); use PMLS::Constants; * use PMLS::Pmts qw(getname_iso639); use PMLS::Constants; * use PMLS::Repository; use PMSuite::File qw(file2array); * use PMSuite::List qw(inref isect_union); use PMLS::Constants; use PMLS::Doc; * use PMLS::FS qw(ensure_home); * use GDCF; use PMLS::Constants; * use PMLS::System qw(log message); use PMLS::Constants; * BOOM! use PMLS::Help qw($help); use PMSuite::File qw(dir2array dir2hash); use PMLS::System qw(log globalconf); use PMLS::FS qw(ensure_home); * use PMLS::Protocol; use PMLS::Constants; use PMSuite::File qw(dir2array file2array); use PMSuite::List qw(columns_list); use PMLS::Constants; use PMLS::Help qw($help); use PMLS::System qw(globalconf message); use PMLS::Help qw($help); * use PMLS::Lexicon; use PMSuite::File qw(safe_save); use PMSuite::List qw(inref); use PMLS::Constants; * use PMLS::Meaning; use PMLS::Constants; use PMLS::System qw(log message); * use PMLS::Property qw(get_proparg get_proptype prop +dist trim); use PMLS::Constants; use PMLS::System qw(message); * use PMLS::Morphology qw(procrule); use PMLS::Constants; use PMLS::Meaning; use PMLS::Repository; use PMLS::System qw(log message); use PMLS::Pmts qw(valid_iso639); use PMLS::Property qw(synmod); use PMLS::Repository; use PMLS::System qw(log globalconf message pb_fillup); * use PMLS::Text; use PMSuite::File qw(dir2array file2scalar); * use PMLS::SNLP::Analyzer qw(analyse filter_analysis) +; * use PMLS::SNLP::CharServer; use PMLS::System; * use PMLS::SNLP::Comparator qw(compare); * use PMLS::T9::Common; use PMLS::System qw(log); * use PMLS::T9::cz use PMLS::System qw(globalconf); * use PMLS::T9::de use PMLS::System qw(globalconf); * use PMLS::T9::es use PMLS::System qw(globalconf); * use PMLS::T9::fr use PMLS::System qw(globalconf); * use PMLS::T9::it use PMLS::System qw(globalconf); * use PMLS::T9::sk use PMLS::System qw(globalconf); use PMLS::Constants; use PMLS::FS qw(ensure_home); use PMLS::Help qw($help); * use PMLS::Phrase qw(dia2asc compat); use PMLS::Constants; use PMLS::FS qw(ensure_home); use PMLS::System qw(message); use PMLS::Repository; use PMLS::System qw(log message globalconf); use PMLS::T9::Common; use PMLS::Morphology qw(alias concat procrule); use PMLS::Phrase qw(dia2asc is_regex); use PMLS::System qw(globalconf log message miter); use PMLS::Server; use PMLS::System qw(globalconf message procinfo); use PMLS::Text qw(_tident tident_out_suplang); use PMLS::FS; use PMLS::Help qw($help); * use PMLS::NLP; use PMLS::Constants; * use PMLS::MT; use PMSuite::List qw(get_maxval get_minval inref); use PMLS::Constants; use PMLS::Help qw($help); use PMLS::Lexicon; use PMLS::Morphology qw(build_ruleset manal); use PMLS::Pmts qw(valid_iso639); use PMLS::Repository; use PMLS::System qw(kv_out log message); use PMLS::Text; use PMLS::T9::Common; * use PMLS::NLA; use PMSuite::List qw(get_maxval); use PMLS::Constants; use PMLS::FS qw(ensure_home); use PMLS::Help qw($help); use PMLS::Meaning; use PMLS::Morphology qw(build_ruleset manal); use PMLS::System qw(log message globalconf); * use PMLS::NLG; use PMLS::Constants; use PMLS::FS qw(ensure_home); use PMLS::Help qw($help); use PMLS::Pmts qw(valid_iso639); use PMLS::Repository; use PMLS::System qw(log message globalconf); use PMLS::Phrase; use PMLS::Repository; use PMLS::System qw(message); use PMLS::Text; use PMLS::Protocol qw(msg_compose msg_verify); use PMLS::System qw(globalconf log message num_cpus procin +fo cmdloop); * use PMLS::User; use PMLS::Constants; use PMLS::FS qw(ensure_home); use PMLS::Help qw($help); use PMLS::System qw(globalconf message);

Bye
 PetaMem
    All Perl:   MT, NLP, NLU

Replies are listed 'Best First'.
Re: Serious Exporter Problems
by etcshadow (Priest) on Jul 09, 2005 at 16:15 UTC
    This is a problem that originates from the difference between compile-time and run-time, and how the compile-time of one module is the run-time of modules it uses. Think about it: if module A uses module B, then module A's compile-time is not complete until after module B is both compiled *and* run. So... what if A uses B and B uses A? Then A's runtime wants to get tangled up in its *own* compile-time.

    I'll skip the suspense and just tell you the specific answer to your problem, though: you have to wrap your @ISA = 'Exporter' and @EXPORT = ... bits of code in a BEGIN block. Look at the example below (to which I've added some verbose information, so that you can really see what's happening):

    ... Now, just do this:
    me@host> perl -c XXA.pm Beginning XXA compile Beginning XXB compile Beginning XXA compile Just used XXB in XXA compile Bareword "xxb" not allowed while "strict subs" in use at XXA.pm line 2 +6. BEGIN not safe after errors--compilation aborted at XXA.pm line 29. BEGIN failed--compilation aborted at XXB.pm line 15. BEGIN failed--compilation aborted at XXA.pm line 15. me@host:>

    That's basically the situation you are in currently. Now, go and un-comment the BEGIN keyword before the Exporter blocks in those modules...

    me@host> perl -c XXA.pm Beginning XXA compile Beginning XXB compile Beginning XXA compile Just used XXB in XXA compile Finishing XXA compile XXA->import called Just used XXA in XXB compile Finishing XXB compile XXB->import called Just used XXB in XXA compile Finishing XXA compile XXA.pm syntax OK me@host>

    Voila! Problem fixed. This should really be in some sort of FAQ, somewhere... I've gotten into arguments with VERY experienced perl programmers about this very topic before (that you must *ALWAYS* but your @EXPORT and @ISA in begin blocks (or use base.pm for your @ISA declarations)).

    ------------ :Wq Not an editor command: Wq
      You have gotten into arguments before, and you're about to again.

      You have set up a bad design and found one way out of a problem that it causes, and then you think that way is how everyone should do it. This is wrong.

      The real moral is that circular dependencies are a Bad Thing. They cause lots of complexity, and compilers don't like dealing with them. That is true in virtually any language! Avoid having circular dependencies and you'll avoid running into many subtle and strange issues.

      Secondly there is another way out of your issue. Rather than putting manipulations of @ISA into BEGIN, you could switch from use to require. This not only solves you problem, it also gives you a shot of successfully handling the situation where one module actually calls functions in the other in setting up class variables. OK, it doesn't guarantee that (see what I said above about circular dependencies), but you have a chance.

      Thirdly there are other issues with BEGIN blocks that you're ignoring. People disagree on them, but if you read (tye)Re: Supersplit carefully you'll see that there are subtle issues with, for instance, error reporting which make it desirable to avoid playing subtle BEGIN games.

      So avoid circular dependencies and don't play subtle BEGIN games. Your code will be less complex and it will work better. Furthermore if you move into other languages, this habit will avoid your having to worry about nasty corners of how they deal with circular dependencies. (A few handle it smoothly, some break, some handle it but you get fragility as a result. Perl falls into the last category.)

        You have set up a bad design and found one way out of a problem that it causes, and then you think that way is how everyone should do it.
        To be pedantic, it was Petamem that set up a bad design; etcshadow was just suggesting a way out that didn't involve redesign (redesign probably being a suggestion that would be ignored). Sometimes bad design just has to be coped with.
        Well, as to the bit about the arguments... they've really not been about questions of circular dependancies, but rather about the specific question at hand (importing from a module that imports from you), and the necessity of setting up of your exports at compile-time. That's why I happened to have a fairly detailed example of this imediately at hand.

        As to the outright condemnation of circular-uses, I have to respectfully disagree. In managing a complex project, it can become a fairly arbitrary line to draw between your modules. It forces you to structure your code-containers around your code's calling-graph, rather than around the functional nature of your code.

        I spend most of my time writing code for healthcare, so here is a simplified example: I want to be able to group code for patients in a Patient module, and code for claims in a Claim module. There are perfectly reasonable cases which require the processing of a claim to have to access patient functions, and vice-versa. Patient's have claims, and the description of a patient is not complete without his/her claim-history. When displaying a claim, I need to access the patient's information, as it is a fundamentally important part of the claim. If I understand your "circular uses are bad, 'mkay" correctly, then I should not be allowed to organize my code this way.

        Also, as to the comment that other languages do not allow circular dependancies, that's just not true (in my experience). Certainly *some* other languages will not have a means to allow for it... but *some* other languages do all sorts of crazy things and impose limitations that I don't ever want to deal with as a user of that language. Even a language which provides as little in the way of facilities to the programmer as C has a means for dealing with circularly dependant libraries. First, there is a separation between headers and code... that goes a long way, on its own. Second, a fairly simple method for preventing pre-compiler infinite-loops is employed:

        #ifndef some_token_i_use_to_represent_this_file #define some_token_i_use_to_represent_this_file 1 /* your code goes here */ #endif

        So, C, and any language which inherits it's basic separation of header files and code files from C, along with a pre-compiler as capable as C's is set. It can handle circular dependancies.

        I guess that gets somewhat to a larger point about this question (as well as some of the issues you raised with circular uses in perl): different languages do things differently with respect to this, and in many of them (all, maybe), there is a necessary discipline to build atop the language, itself, to protect yourself from problems. In perl, this discipline includes understanding the difference between BEGIN time and run-time, and using that understanding appropriately. Maybe perl requires a little bit too much in the way of requiring discipline from its programmers, in this regard... on that I won't comment. It certainly wouldn't be the first time perl's been accused of requiring self-discipline of its programmers, rather than enforcing discipline upon them.

        However, I actually don't think that there is anything all that wrong with publishing some best practices as a way to encourage that self-discipline in people who haven't yet attained the understanding of why that self-discipline is necessary. In my mind, telling people to always put their @ISA and @EXPORT setup in a BEGIN block is kind of like telling people to use strict. Actually, a better comparison would be like telling people to NEVER put a "my" operator in a conditional. The reason why it's important to do this is tricky to divine on your own, and may not bite you imediately. Still, there's no harm in doing it, and it may (proably will) prevent errors, later.

        ------------ :Wq Not an editor command: Wq
Re: Serious Exporter Problems
by tphyahoo (Vicar) on Jul 09, 2005 at 16:41 UTC

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: perlquestion [id://473679]
Approved by Tanktalus
Front-paged by etcshadow
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others goofing around in the Monastery: (3)
As of 2024-04-25 08:33 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found