This sounds like a perfect application for threading, lots of IO wait states and (I assume) little or no interaction required between the threads.

I would also concur that a thread pool is the right mechanism, though that's not ane endorsement of Thread::Pool if in fact such a module exists, as I (obviously) have never tried it.

The type of model I would use for this goes something like the following (untested, pseudo-code).

#! perl -w
use strict;
require 5.008;
use threads;
use Thread::Queue;

# Adjust (slowly) till you find the optimum
use constant MAX_THREADS => 10;

my $dispatchQ = Thread::Queue->new();
my $resultsQ  = Thread::Queue->new();

sub worker {
    # While there's stuff to be done
    while( $dispatchQ->pending() ) {

        # Get the next serverb
        my $server = $dispatchQ->dequeue();

        # creat a new connection (A guess!!)
        my $smtp = Net::SMTP->new( $server );

        # do whatever is required
        my $result = 'Whatever';

        # Tack the server name on the front of the results
        # for later identification. The normal perl idiom of
        # using an (anonymous) array has the problem that it
        # causes rapid memory leaks. Using this simplistic
        # approach seems to be most reliable and scalable.

        $resultsQ->enqueue( join( $; , $server, $result ) );
    }
    return; # When there is nothing left to do, die.
}

# Create the pool
my @threads = map{ threads->create( \&worker ) } 1 .. MAX_THREADS;

# Read the list of servers
my @servers = do{ local (*ARGV, $/) = 'servers.list'; <> };

# Put the list on the queue
$dispatchQ->enqueue( @servers );

# While there is still work to be done
while( $dispatchQ->pending() ) {

    # Get the results as they become available
    my ($server, $result) = split $;, $resultsQ->dequeue();

    # Record the results somewhere.
}
# DispatchQ is empty, each thread will terminate once it completes its
+ last task

$_->join for @threads; # Wait till they finish

# Record the last few results
while( my ($server, $result) = @{ $resultsQ->dequeue } ) {
    # Record the results somewhere.
}

__END__
[download]

This is extremely speculative code, but derived from some stuff I am playing with that seems to work quite well. I'd love some feedback (either way) if you use it.

By "Multi-process" do you mean forking??

Yes. There's a good example of doing this with shared data in merlyn's column.

use Paralle::ForkManager;

my $pm = new Parallel::ForkManager(x);
# where x = number of forks

foreach ($item) @items){
    $pm->start and next;

    #do stuff in parallel here

    $pm->finish;
}
$pm->wait_all_children;
[download]

Neil Watson
watson-wilson.ca

How often are you planning on upgrading? If this is a one-time thing, don't get yourself buiried in the complexities of threading. If this is something that might happen once or twice a month, and if it takes longer than a single night to upgrade everything, then you can probably get a speed boost with parrellizaing. Alternatively, you can make one program that can take a portion of the list of hosts and run it multiple times, each run with a different section of the list.

----
I wanted to explore how Perl's closures can be manipulated, and ended up creating an object system by accident.
-- Schemer

Note: All code is untested, unless otherwise stated

I am planning on running the upgrade once a week. In mutiple locations. I have not used parrellizaing before. How do I do it? :)
My current script uses thread pool, but I notice a occasional problem, thats why I was asking about the recommend style for this type of project.

You're already doing parrellization. Any program that has portions running together is using parrellization. Though it isn't true "parrellisim" unless you're on a multi-processor machine, a cluster, or one of those new-fangled hyperthreading processors, but that's a pesky detail that I'm ignoring for the moment.

Recommended style here is "whatever works for you" :)

----
I wanted to explore how Perl's closures can be manipulated, and ended up creating an object system by accident.
-- Schemer

Note: All code is untested, unless otherwise stated