Re^2: Looking for alternative for IPC::Shareable (or increase size)

Dear stevib,

I already discovered some strings with a length of 84_217 characters, which I try to insert into the IPC::Shareable array @sender, but I do this very often, but only on this I get lists of arrays, I can't predict of length...

This is a snipped:

my $allowed_childs    = 10;
<other code>
        ## Main update agent
        my @agency    = ();    # Agents (forks)
        my $recID    = &gen_random(4);    # IPC key as $transID, but o
+nly visible for childs of the update agent
        # Separate transfer array for agents (sub-forks of fork)
        my $update_knot    = tie(my @receiver, 'IPC::Shareable', $recI
+D, {
            create    => 1,
            mode    => 0600,
            destroy    => 1,
            size    => IPC::Shareable::SHM_BUFSIZ(),
            #size    => 131072 * 2,
            });
        # Array to store what was transfered through @receiver, becaus
+e IPC-tied arrays are very small
        my @storage    = ();
        # Transfer child elements to @storage
        my $_sub_gatherer    = sub {
            my $allowed_childs    = shift;
            if ( &wait_children($allowed_childs, \@agency) ) {
                my $knot    = tie(my @cleaner, 'IPC::Shareable', $recI
+D);
                $knot->shlock;
                push(@storage, splice(@cleaner, 0, scalar(@cleaner) - 
+1));
                $knot->shunlock;
                return(1);
                }
            return(0);
            };

<other code>

                        # Get region dependend type ids
                        foreach my $region ( @regions ) {
                            my $agent        = fork;
                            # Parent
                            if ( defined($agent) && $agent ) {
                                push(@agency, $agent);
                                &debug_out("get_region_types(): Starte
+d download agent $agent");
                                #Time::HiRes::sleep(0.01);        # WO
+RK Maybe no longer needed because of dl_lock()
                                #&{$_sub_gatherer}(sprintf("%.0f", $ma
+x_forks * 0.5));
                                &{$_sub_gatherer}(1);

                                # Report to main window
                                $perc_count    += 15 / scalar(@regions
+);
                                $ac_knot->shlock;
                                $agent_carrier{progress_percent}    = 
+$perc_count / &{$_sub_percent_base}();
                                $ac_knot->shunlock;
                                }
                            # Child
                            elsif ( defined($agent) ) {
                                srand();
                                my @region_market_types    = &download
+er(0, "/markets/$region/types/");

                                if ( @region_market_types ) {
                                    my $knot    = tie(my @sender, 'IPC
+::Shareable', $recID);
                                    $knot->shlock;
                                    push(@sender, encode_json(\@region
+_market_types)); # Here the failure occures: Length of shared data ex
+ceeds shared segment size at ./myprogram.pl line 1210.
                                    $knot->shunlock;
                                    }
                                exit(0);
                                }
                            else {
                                &debug_out(
                                    "update_types(): Can't fork",
                                    );
                                &pprop_exit();
                                }
                            }

                        # Wait for children
                        &wait_children(1, \@agency);

<other code>

sub wait_children {
    my $left    = shift;
    my $childs    = shift;
    my $waiting    = 0;

    if ( $left < 1 ) {
        $left    = 1;
        }

    while ( scalar(@{$childs}) >= $left ) {
        $waiting    = 1;
        Time::HiRes::sleep(0.1);
        @{$childs}    = grep { kill(0 => $_) } @{$childs};
        }

    return($waiting);
    }
[download]

Comment on Re^2: Looking for alternative for IPC::Shareable (or increase size) Download Code

Replies are listed 'Best First'.

Re^3: Looking for alternative for IPC::Shareable (or increase size)
by jeffenstein (Hermit) on Aug 07, 2020 at 11:24 UTC

The man page for shmctl(2) doesn't give a way to resize an existing shared memory segment, so you can't expand it with any of the shared memory modules, since it's a limitation of the OS.

It might be possible with mmap(2), but if other process have already mapped the memory area, then they would have to be notified of the change and unmap/map the memory again. This implies that it would have to be a file on disk so that other processes can unmap/map the same data.

Otherwise, it's has to be something that copies the data between the processes.

[reply]


There's more than one way to do things
	PerlMonks