http://qs321.pair.com?node_id=98396

Anonymous Monk has asked for the wisdom of the Perl Monks concerning the following question:

I'm doing about 30 snmpwalk/snmpget commands in one perl script to find the config on Cisco boxes and throw them into an html page. The problem is, each router/switch takes about 13 minutes to complete because of the amount of time these walks/gets take to complete. Having about 40-50 pieces of Cisco network equipment means that it would take this script about 10 hours to complete (yeah, unexceptable).

I've been reading up on the "system" function, but it looks like it "waits until that process is done" before it moves on to the next command.

I just need a way to multithread this script in a very simple manner -- none of the threads _ever_ have to communicate with each other -- they just should run simultaneously and return what they get.

Any simple ways to do this? Am I misunderstanding the "system" function?


Thanks for any help I can get!
Justin

Replies are listed 'Best First'.
Re: Multithreading
by Masem (Monsignor) on Jul 20, 2001 at 17:23 UTC
    You probably want to use the fork function, which basically creates a child process using the same perl code. There's more details on the perldoc pages as well as here on PM, but the general idea is that your parent process can create however many instances it needs and wait for them to finish via waitpid. The children should write data to files, which the parent process can then collate and put into the final HTML file(s) that you want.

    -----------------------------------------------------
    
    Dr. Michael K. Neylon - mneylon-pm@masemware.com || "You've left the lens cap of your mind on again, Pinky" - The Brain
      What works even better, is to have the children write to one file, but use an excluse lock. Then the parent can sort the file if you need to.

      Another option, is to have the children write back to the parent.

      my $pid = open ($handle, "-|"); if ($pid){ my $return = <$handle>; close $handle; # Do something with the return value } else { # Do something interesting print $handle "Something interesting"; close $handle; exit; }
      Note, I'm on a windows box so I wasn't able to test the above code, but it should be close. Now for mulitple children you'll need to use IO::Select.
Re: Multithreading
by merlyn (Sage) on Jul 20, 2001 at 19:06 UTC
Re: Multithreading
by MZSanford (Curate) on Jul 20, 2001 at 17:24 UTC
    There are several way for this to be done. Since the entire process is in Perl, an presumably in Perl, there are forking, and threads. Threads are somewhat (very) experimental and should not be used for production work, so, fork.
    I would read up on the fork function, and maybe take a look at super search, or the book "Network Programming with Perl".
    remeber the immortal word's of Socrates who said, "I drank what ?"
Re: Multithreading
by PetaMem (Priest) on Jul 20, 2001 at 19:06 UTC
    Some monks here already adviced fork instead of threading.
    I found the Parallel::ForkManager most useful for
    spaning up independent processes. However: If you need
    communication between the processes youŽll have to stick with IPC.

    But I think fork is the ideal candidate for your purpose.

    Ciao