Felow monks,

Recently I was working a lot with threads, producing a group of services that should work 24h per day, 7 days per week. This services are multithread since they access the internet doing some tasks, so they need to split the tasks in many threads to do the job faster and use better the hardware and link of our servers. But we have the problem that each thread in Perl uses more and more memory, so I want to share my experience on trying to fix that.

To do this tasks over the internet we use WWW::Mechanize that make our life much more easy, specially when we need to login in some site.

Today Perl 5.8.6 is a real solution for multithread services, but we have a big problem, memory usage. Our servers don't run only one service, but many services, and each thread of WWW::Mechanize uses a lot of memory! So, we can't start much more than 5 threads per process, since with 5 threads the service uses 60Mb+-!

Looking for a solution I have created Thread::Isolate, where the main idea is to create a thread than can be called externally, so, I can load modules from outside and call them. With that I can organize my programs in some threads, each of them with different resources and only duplicate threads/resources that will need to run in parallel.

Here's a simple sample of Thread::Isolate:

  use Thread::Isolate ;
  
  my $thi = Thread::Isolate->new() ;

  $thi->use('Data::Dumper') ;
  
  print $thi->call('Data::Dumper::Dumper' , [123 , 456]) ;
[download]

Also I can do detached (unsynchronous) calls:

  
  use Thread::Isolate ;
  
  $|=1 ;
  
  my $thi = Thread::Isolate->new() ;

  $thi->use('LWP::Simple');
  
  my $job = $thi->call_detached('LWP::Simple::get' , 'http://www.perlm
+onks.com/') ;
  
  while( $job->is_running ) {
    print "." ;
    sleep(1);
  }
  
  $html = $job->returned ;
  
  die( $thi->err ) if $thi->err ;
  
  print $html ;
[download]

Looking the code above we can see that we are not only isolating LWP::Simple, but we also can run a code until get() returns something.

Thread::Isolate was inspirated on Thread::Tie::Thread by Elizabeth Mattijsen (liz), that did something like that to make Thread::Tie. Than I saw that I could use that to organize my multithread programs, so I created a new module with more resources and an intuitive interface, enjoy! ;-P

Like any new module it still needs tests, and a module that works with multithread need more tests, so, I'm asking for you monks to test it and report your experience with it here. Thanks in advance.

CPAN should put it soon at:
http://www.linorg.usp.br/CPAN/authors/id/G/GM/GMPASSOS/