Hi again,

I made an improved version including worker sending the error message. Replace system with syscmd in your script. That sends the command along with any arguments to the background worker. The worker receives the command to run and any arguments. The status and error message are sent to the main process after running the command.

syscmd:

use strict;
use warnings;

use MCE::Child;
use MCE::Channel;

my $chnl = MCE::Channel->new( impl => 'Simple' );

# spin up worker early before creating big hash
mce_child {
    local $SIG{__WARN__} = sub {};
    while ( my ($cmd, @args) = $chnl->recv ) {
        local ($?, $!);
        system($cmd, @args);
        $chnl->send2($?, $!);
    }
};

sub syscmd {
    my $cmd = shift;
    return unless $cmd;

    $chnl->send($cmd, @_);
    my ($status, $errmsg) = $chnl->recv2;
    
    if ($status == -1) {
        print "SYSTEM: failed to execute ($cmd): $errmsg\n";
    }
    elsif ($status & 127) {
        printf "SYSTEM: $cmd died with signal %s, %s coredump\n",
            ($status & 127), ($status & 128) ? 'with' : 'without';
    }
    else {
        printf "SYSTEM: $cmd exited with status %d\n", $status >> 8;
    }
}

# create big hash
my %big_hash;

# pass command and optionally args
syscmd('ls');

# attempt to run a command not found
syscmd('something');

# sleep for 2 seconds
syscmd('sleep', '2');

# notify no more work, then reap worker
$chnl->end;
MCE::Child->waitall;
[download]

output:

ls output from syscmd
SYSTEM: ls exited with status 0
SYSTEM: failed to execute (something): No such file or directory
SYSTEM: sleep exited with status 0
[download]

Regards, Mario

Using the memory gobbling technique by Corion here, let's do some testing. I divided by 2 to reflect the actual memory consumption desired.

My virtual CentOS 7 machine has 4 GB of RAM allocated to it. This creates a 3 GB scalar key-value pair to consume 75%. The system command called by the main process fails, no ls output on the 2nd time. Calling syscmd succeeds due to the background worker spun early.

syscmd

use strict;
use warnings;

use MCE::Child;
use MCE::Channel;

my $chnl = MCE::Channel->new( impl => 'Simple' );

# spin up worker early before creating big hash
mce_child {
    local $SIG{__WARN__} = sub {};
    while ( my ($cmd, @args) = $chnl->recv ) {
        local ($?, $!);
        system($cmd, @args);
        $chnl->send2($?, $!);
    }
};

sub syscmd {
    my $cmd = shift;
    return unless $cmd;

    $chnl->send($cmd, @_);
    my ($status, $errmsg) = $chnl->recv2;
    
    if ($status == -1) {
        print "SYSTEM: failed to execute ($cmd): $errmsg\n";
    }
    elsif ($status & 127) {
        printf "SYSTEM: $cmd died with signal %s, %s coredump\n",
            ($status & 127), ($status & 128) ? 'with' : 'without';
    }
    else {
        printf "SYSTEM: $cmd exited with status %d\n", $status >> 8;
    }
}

# My CentOS VM has 4 GB of RAM

# create big hash
my $memory_eaten = 3 * 1024*1024*1024 / 2; # 3 GB, adjust to fit
my %memory_eater = (
    foo => scalar( ' ' x $memory_eaten ),
);

# pass command and optionally args
syscmd('ls');  # this one works; see status that it succeeded
system('ls');  # this one fails; no ls output the 2nd time

# attempt to run a command not found
syscmd('something');

# sleep for 2 seconds
syscmd('sleep', '2');

# busy loop, see top output in another terminal
# notice the memory consumption (i.e. RES)
# press Ctrl-C to exit or let it finish
1 for 1..3e8;

# notify no more work, then reap worker
$chnl->end;
MCE::Child->waitall;
[download]

output:

ls output from syscmd
SYSTEM: ls exited with status 0
SYSTEM: failed to execute (something): No such file or directory
SYSTEM: sleep exited with status 0
[download]

Regards, Mario

Hi again,
I tested your example and it seems to work great, so the problem is solved!
Thank you so much!

I quickly tested your script on the Centos, and it works perfectly! I will implement it in my script tomorrow, hopefully it will work as good. You can send me your name, I would like to thank you when the tool will be published.

Wow thanks so much for your help! I will test it tomorrow for sure! I thought I had to give up the system call, would be amazing if it works!