Re^6: Isolating dynamically loaded modules with Thread::Isolate.

in reply to Re^5: Isolating dynamically loaded modules with Thread::Isolate.
in thread Isolating dynamically loaded modules with Thread::Isolate.

I suspect 100 forked processes that don't load any modules would use a lot less than 40MB, and it only gets worse for threads when you start loading enough modules to do something useful. The code and variable storage used by the modules will all be duplicated into every thread. With forking, copy-on-write prevents this. Forking is not great, but at this point it's a lot lighter than threads.

Comment on Re^6: Isolating dynamically loaded modules with Thread::Isolate.

Replies are listed 'Best First'.
Re^7: Isolating dynamically loaded modules with Thread::Isolate. by BrowserUk (Patriarch) on Feb 01, 2005 at 01:59 UTC
The code and variable storage used by the modules will all be duplicated into every thread. Not so, if you defer the loading of the modules and do it only within the thread or threads where it is required. That is what my post was demonstrating. By requireing those modules required by each thread from within that thread, only that thread gets that code and data. If the thread then terminates, the memory used by those things it loaded is returned to the process' memory pool. Indeed, it is entirely possiible to have a single process that has HTTP::Daemon loaded in one thread; LWP in another; CGI in another; with none of them duplicated. It is the fork-like "replicate everything currently loaded" behaviour that creates the problem. As I said elsewhere, if the choice of what was duplicated was left entirely to the programmer--life would be much simpler and iThreads would be considerably faster to start and lighter in use. With forking, copy-on-write prevents this If your OS supports fork-with-COW, then you're laughing--provided you do not need for the separate parts of your overall task to communicate. Under those circumstances, I too would use fork, and I do advocate that. Examine what is said, not who speaks. Silence betokens consent. Love the truth but pardon error.	[reply]

Replies are listed 'Best First'.

Re^7: Isolating dynamically loaded modules with Thread::Isolate.
by BrowserUk (Patriarch) on Feb 01, 2005 at 01:59 UTC

The code and variable storage used by the modules will all be duplicated into every thread.

Not so, if you defer the loading of the modules and do it only within the thread or threads where it is required. That is what my post was demonstrating.

By requireing those modules required by each thread from within that thread, only that thread gets that code and data. If the thread then terminates, the memory used by those things it loaded is returned to the process' memory pool. Indeed, it is entirely possiible to have a single process that has HTTP::Daemon loaded in one thread; LWP in another; CGI in another; with none of them duplicated.

It is the fork-like "replicate everything currently loaded" behaviour that creates the problem. As I said elsewhere, if the choice of what was duplicated was left entirely to the programmer--life would be much simpler and iThreads would be considerably faster to start and lighter in use.

With forking, copy-on-write prevents this

If your OS supports fork-with-COW, then you're laughing--provided you do not need for the separate parts of your overall task to communicate. Under those circumstances, I too would use fork, and I do advocate that.

Examine what is said, not who speaks.

Silence betokens consent.

Love the truth but pardon error.

[reply]

In Section Meditations