RE: Memory usage in Perl.

If this is running via CGI, and not in the mod_perl environment, i would certainly have a series of concerns. Not neccesseray so much memory usage, but mostly load time, especially the population of these 100 or so accesible variables. While that is not a GREAT deal of memory, I have been programming for a long time, and I have never encountered a situation with requirements for that type of environment, especially in one monolithic 3000 line module.

My suggestion to you is:

Take a step back, think to yourself, what does this module actually do?
Think to yourself, am I doing a proper job of abstracting data from logic?
Ask: does all of this data need to be loaded at all times? Or is this the type of thing where I can make 3 or 4 dbm's and tie them at run time, so as to not have to repopulate them all the time?
Do I have so much data, and I foresee this growing and needing more computation, that it really should be in a relational model of some sort?
Does this really need to be 1 3000 line module? or should this be 1 50 line module and a series of modules that inherit the interface and special case?

I think once you've answered those questions, you will know if you have made the right decisions. Realistically, 100 object variables ain't gonna kill you... however that may be the symptoms of a improper design. Make sure you attack the cause, and not the symptoms!

Good luck! and keep asking away...

Comment on RE: Memory usage in Perl.

Replies are listed 'Best First'.
RE: RE: Memory usage in Perl. by Speedfreak (Sexton) on Aug 07, 2000 at 11:36 UTC
Well, to justify some of my points. This module is huge because the data it processes is really complex. Its handling strings from another program which report up to 200 different enviromental conditions in a working environment. The string can report 0-200 of these conditions at any one time and all of the values need to be available. Therefore, say someone wanted to know the oxygen content of the environment, they would take the environmental data string from the test equipment and send it to the module, which decodes it and returns all the factors as object variables. If there was no oxygen data, then the variable containing its content value would be undef and there program using the module would take appropriate action to report that. I have looked and looked again and this module and made two or three small programs, optimised to do certain tasks and then combined. The big problems is that the whole problem requires the decoding and handling of various text strings, something I find Perl unmatched for, however the nature of the data is what is forcing the size of the module! I'd like to use mod_perl, but never used it before and cant find any clear details on implementing it. - Jed	[reply]
RE: RE: RE: Memory usage in Perl. by eduardo (Curate) on Aug 07, 2000 at 16:47 UTC
whenever I hear about complex parsing, and I think about module size, performance, scalability, maintainability... I always come to the same exact conclusion... someone should check out Parse::RecDescent There have been some awsome comments on how to use Parse::RecDescent here in the monestary, look at some of merlyn's posts... however, the quick overview is simple (if you've never worked with parsers...) Parse::RecDescent will allow you to write a grammar that describes what it is that the data should look like, and the mechanisms by which it should be dissected and manipulated. If you are having to do a lot of conditional matching (eg If this match this else match that) then it may be a perfect job for a recursive descept parser... Give it a look, the docs are awsome, and if you have questions on it, I am sure some of the more knowledgable monks (talk to maverick) will be able to give you a hand on how to dissect Parse::RecDescent. Hope it goes well!	[reply]


Perl Monk, Perl Meditation
	PerlMonks