Beefy Boxes and Bandwidth Generously Provided by pair Networks
Pathologically Eclectic Rubbish Lister
 
PerlMonks  

Avoiding loading modules

by jfrm (Monk)
on May 04, 2007 at 16:16 UTC ( #613610=perlquestion: print w/replies, xml ) Need Help??

jfrm has asked for the wisdom of the Perl Monks concerning the following question:

Good morrow scholars. I have a script that loads a whole lot of files and modules at the front and this takes a long time 1-2 seconds which has become disgruntling. I was just naively wondering if there is a way to only load modules if one needs them? i.e. to put some of the require (I assume use is no use (lol)) statements into certain subroutines so that those files/modules are loaded only if subroutines are actually called.

Or is this doomed to failure because everything is loaded and compiled at compile time so no time savings are feasible?

thanks for any tips.

Brother James.

Replies are listed 'Best First'.
Re: Avoiding loading modules
by derby (Abbot) on May 04, 2007 at 16:38 UTC

    Through judicious use of require and import you can shift the burden of module loads from compile time to run time ... but ... a large number of 'use' directives is a huge red flag that your app is becoming a big ball of mud. I'd look into ways of redo-ing the app so there's not a huge number of use directives (or a huge number that impacts load time). And before I shifted to the require/import method, I would probably use Autoloader (I like seeing the use statements at the top of the file).

    -derby
      ...a large number of 'use' directives is a huge red flag... I'd look into ways of redo-ing the app
      Could you expand on that? I often end up with "a large number of 'use' directives" to, amongst other things, _avoid_ a big ball of mud. :-)

      How many is large? How would you go about "redo-ing the app"?

      I, too, "like seeing the use statements at the top of the file". But perhaps for the wrong reasons?

        It would depend on the type of app but for me, it's mainly web apps. If one of my web apps get's huge, I take a hard look at the namespace (http://www.foo.com/webapp) and see if I can break it out into parts. For example, for a typical CRUD database app, I may do:

        • http://www.foo.com/webapp/create
        • http://www.foo.com/webapp/retrieve
        • http://www.foo.com/webapp/update
        • http://www.foo.com/webapp/delete
        and then instead of all the logic (model,view,controller) being in one big-old-module (webapp), I would transition as much as I could to new modules which hold the logic for each piece of the app (with appropriate commonality in a webapp superclass). So the object hierarchy may look something like this in the file system:
        • /path/to/foo/webapp
          • Controller.pm
          • Model.pm
          • View.pm
          • create
            • Controller.pm
            • Model.pm
            • View.pm
          • retrieve
            • Controller.pm
            • Model.pm
            • View.pm
          • update
            • Controller.pm
            • Model.pm
            • View.pm
          • delete
            • Controller.pm
            • Model.pm
            • View.pm

          As for when to do this -- I cannot say. For me it's more a comfort level decision. I tend to apply cohesion and coupling principles at the module level - when the module is doing too many things and I cannot reasonably reuse it, then I break it down.

          -derby
Re: Avoiding loading modules
by RL (Monk) on May 04, 2007 at 16:56 UTC
    use --> compile time
    require --> run time

    You may also consider to use either Selfloader or Autoloader in your own modules in order to load needed methods at run time.

    If you have the option to run mod_perl (and your own app is prepared for mod_perl) than you'll speed up things a lot switching to mod_perl (or FastCGI) because the modules are only compiled once per Apache process opposed to a CGI-environment where all modules need to be compiled for each call to your app.

    RL
Re: Avoiding loading modules
by ChemBoy (Priest) on May 04, 2007 at 18:48 UTC

    Depending on your context, you might find the autouse module helpful.



    If God had meant us to fly, he would *never* have given us the railroads.
        --Michael Flanders

Re: Avoiding loading modules
by tsee (Curate) on May 05, 2007 at 18:42 UTC

    As others have pointed out, the two prominent methods of reducing start-up time are using require()+import() to load modules on-demand and Autoloader/Selfloader-like techniques. (Autoloader usually being faster than Selfloader, see Autoloader docs.)

    My addition to this is mostly a remark about replacing use() with require()+import(): If run in a context where several instances of the same program are run in parallel and which can share memory, this reduces the amount of memory that can be shared. The most prominent case where this matters is if you have a CGI application which you later port to mod_perl. There is a solution to this, however. Please have a look at Adam Kennedy's "prefork" module on CPAN. The documentation explains things in detail.

    Furthermore, you may reduce the time it takes to load a module by making sure the modules you load are found in the first directory in the list of search paths (@INC). As you probably know, perl will search for the .pm modules source files in several directories by default. If many of the modules are found in paths that are at the end of the @INC array, this may take some time depending on the speed of your file system. (Perhaps there is some caching involved to speed things up, but I'm not aware of that.)

    Cheers,
    Steffen

      Expanding on this, you should make sure your modules are loaded on a fast (usually local) drive, and that the fast drive directories are mentioned first in @INC. Startup can spend a lot of time executing stat(1) on networked directories, trying to locate '.pm' files. Ideally you should avoid having networked directories in @INC altogether.

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: perlquestion [id://613610]
Approved by Corion
Front-paged by Corion
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others about the Monastery: (2)
As of 2022-05-19 03:03 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?
    Do you prefer to work remotely?



    Results (71 votes). Check out past polls.

    Notices?