Hello,
I have created a script that SSHs to multiple nodes (~150) and issues commands and then process the output.
I have used this script successfully in the past but the cli commands produced a short output.
Now the cli command I am using produces ~70MB output.
Initially I tested using 3 nodes and it was successful. However, when I run it on all nodes, the output (as seen from logs) is cut short in most nodes.
You can see that I have set the ssh output timeout at 240s but it doesnt take that long in reality.
I am not sure in what issue I am running into but seems to be performance related since it doesn't happen when the number of nodes (and children processes) is low.
foreach my $node (@nodes) {
#sleep(1);
my $pid = $pm->start and next;
my $nodeip = $Configuration::node_list_nmnet{$node};
$datestamp = strftime("%Y%m%d%H%M", localtime);
my $ssh = Net::SSH::Expect->new (
host => $nodeip,
password=> "*****",
user => "******",
timeout => 5,
raw_pty => 1,
log_stdout => 0,
exp_debug => 0,
log_file => "/home/logs/$node/diam.$node.$datestamp.log"
);
my $login_output = $ssh->login();
$ssh->waitfor("#", 30);
$ssh->send("show diameter peers full debug");
$ssh->waitfor("#", 240);
my $output;
$output = $ssh->before;
if ($output =~ /Peers in CLOSED state\S*\s*(\d*)/ ){
$peersvalue = $1;
if ($peersvalue > 0){
print "$node\n";
push @emaillog, "$node\n\n";
$sendemail = 1;
}
}
$datestamp = strftime("%Y%m%d%H%M", localtime);
$pm->finish(0);
}
print "Waiting for Children...\n";
$pm->wait_all_children;
