AnyEvent - sequence of async operations without recursion?

sectokia has asked for the wisdom of the Perl Monks concerning the following question:

Hi Monks,

I have list of async jobs that need to happen one after the other. I can do that as per example below. However it has a deep recursion problem that limits the size of the queue. Also it has to store all the async instances (in this example in @times) which is silly as only 1 is really active at a time.

What would be the correct way to do this without recurssion?

Thanks!

use AnyEvent;

my @queue = (9,8,7,6,5,4,3,2,1);
my @timers;

my $cv = AnyEvent->condvar;

DoQueue();

# Do other things here...

$cv->recv;

sub DoQueue
{
    my $i = pop @queue;
    
    if (!defined $i)
    {
        $cv->send;
        return;
    }

    print "Starting async operation $i\n";
    $timers[$i] = AnyEvent->timer(after => 1, interval => 0, cb => sub
+ {
        print "Finish async operation $i\n";
        DoQueue();
    });
}
[download]

Comment on AnyEvent - sequence of async operations without recursion? Download Code

Replies are listed 'Best First'.
Re: AnyEvent - sequence of async operations without recursion? by kschwab (Vicar) on Feb 10, 2021 at 22:38 UTC
AnyEvent::FIFO maybe? The source code is pretty short, seems like AnyEvent::Util::guard is what's used to implement the FIFO	[reply]
Re^2: AnyEvent - sequence of async operations without recursion? by sectokia (Pilgrim) on Feb 11, 2021 at 01:49 UTC
Thanks that was exactly what I was after! Works great: use strict; use warnings; use AnyEvent; use AnyEvent::FIFO; my $cv = AnyEvent::condvar; my $fifo = AnyEvent::FIFO->new( max_active => 1, # max "concurrent" callbacks to execute per slot ); my @q1 = (9,8,7,6,5,4,3,2,1); my @q2 = qw(a b c d e f g h i); foreach my $item (@q1) { $fifo->push( "number_queue", \&callback1, $it +em); } foreach my $item (@q2) { $fifo->push( "letter_queue", \&callback2, $it +em); } #do other async stuff here... $cv->recv; my ($q1t,$q2t); sub callback1 { my ($guard,$item) = @_; $q1t = AnyEvent->timer(after=>1, interval => 0, cb => sub { print "Got $item\n"; undef $guard; }); } sub callback2 { my ($guard,$item) = @_; $q2t = AnyEvent->timer(after=>1, interval => 0, cb => sub { print "Got $item\n"; undef $guard; }); } [download]	[reply] [d/l]
Re^3: AnyEvent - sequence of async operations without recursion? by Anonymous Monk on Feb 11, 2021 at 16:36 UTC
Glad it worked out for you!	[reply]
Re^4: AnyEvent - sequence of async operations without recursion? by Anonymous Monk on Feb 11, 2021 at 16:50 UTC
Re:AnyEvent - sequence of async operations without recursion? by bliako (Monsignor) on Feb 10, 2021 at 22:35 UTC
`no warnings 'recursion';` The above will turn "deep recursion" warnings off (and can be localised). After that Perl will happily recurse to exhaustion. I think the limit before complaining (and not bailing out) is 100. Obviously with your queue of 10 this limit is not reached. Since you said that your jobs are not really asynchronous why not run them in a sequence? Edit: removed `use` from the 1st line, thanks AnomalousMonk	[reply] [d/l] [select]
Re^2: AnyEvent - sequence of async operations without recursion? by ikegami (Patriarch) on Feb 11, 2021 at 06:34 UTC
Yeah, it's 100 by default. That's on the low end of acceptable. I bump it to 1000 when I build `perl` (`-DPERL_SUB_DEPTH_WARN=1000`). The key is that it's just a warning to help debug and/or detect infinite recursion before Perl uses up your system's memory. Because Perl will happily recurse until you run out of memory. Seeking work! You can reach me at ikegami@adaelis.com	[reply] [d/l] [select]
Re: AnyEvent - sequence of async operations without recursion? by Anonymous Monk on Feb 11, 2021 at 14:31 UTC
AnyEvent and its ecosystem are outside my area of expertise, but ... If the original question is really about tail recursion, the way to do that in Perl is `goto &DoQueue;`. The gone-to subroutine receives as its arguments whatever was in `@_` at the time the `goto` is executed.	[reply] [d/l] [select]


XP is just a number
	PerlMonks