Beefy Boxes and Bandwidth Generously Provided by pair Networks
Perl Monk, Perl Meditation
 
PerlMonks  

Implementing AnyEvent::Ping multiple times

by mmoorreett (Acolyte)
on Jun 10, 2020 at 07:59 UTC ( [id://11117894]=perlquestion: print w/replies, xml ) Need Help??

mmoorreett has asked for the wisdom of the Perl Monks concerning the following question:

Hi All I have been developing relatively simple perl programs on and off over the years, mainly for integrations using synchronous operations. I have since been tasked with building a "simple" ICMP and SNMP polling engine (yes re-inventing the wheel...) to allow for asynchronous/parallel polling. In researching the various options, i have learnt of the event loop options in perl. AnyEvent, IO:Async etc. Given AnyEvent has Ping and SNMP modules I have started using these and learning the ropes of AnyEvent. In testing the use cases required for the solution, I need to be able to ping different sets of hosts on different timers. So I have started developing the below code. The problem I am having is i'm getting the "AnyEvent::CondVar: recursive blocking wait attempted" error. I understand why, $cv_ping->recv; is blocking the event loop. However I am not how I work around this, or if my approach is actually going to work as I don't have much experience with AnyEvent. I have trawled the interwebs for various materials and tried different things but haven't been able to get it to work. My code is below, help/advice is much appreciated.

use strict; use AnyEvent; use AnyEvent::Ping; use IO::Async::Timer::Periodic; use IO::Async::Loop; use Data::Dumper; my @todoList = map { "192.168.100." . $_ } (1 .. 254); my $ping_timeout=2; #seconds to wait for response my $ping_attemps=4; #number of pings to issue, my $ping_pktsize=32; #packet size of ping my $start_run = time(); main(); sub main{ my $loop = IO::Async::Loop->new; my $timer = IO::Async::Timer::Periodic->new( interval => 5, first_interval => 2, reschedule => 'hard', on_tick => sub { print "before ping\n"; \&ping_hosts(@todoList); print "after ping\n"; }, ); my $timer2 = IO::Async::Timer::Periodic->new( interval => 7, first_interval => 2, reschedule => 'hard', on_tick => sub { print "before ping2\n"; \&ping_hosts(@todoList); print "after ping2\n"; }, ); $timer->start; $timer2->start; $loop->add( $timer ); $loop->add( $timer2 ); $loop->run; } sub ping_hosts{ my @hosts=@_; my $callback=shift; my $result; my $cv_ping = AnyEvent->condvar; my $ping = AnyEvent::Ping->new(packet_size => $ping_pktsize); $ping->timeout($ping_timeout); #$cv_ping->begin(sub { shift->send($_[0]) }); foreach my $host (@hosts) { $cv_ping->begin; my $request; $request = $ping->ping($host, $ping_attemps, sub { print "\n$host state:".Dumper($_[0]); undef $request; $cv_ping->end; }); } $cv_ping->recv; $cv_ping->end; #$cv_ping->cb(sub { $_[0]->recv }); $ping->end; }

Replies are listed 'Best First'.
Re: Implementing AnyEvent::Ping multiple times
by hippo (Bishop) on Jun 11, 2020 at 11:43 UTC

    The code as presented is confusing to me. That may be my fault as I am no authority on event loops. However, since nobody else has replied yet it may be that they are confused as well. If you could maybe address these questions I have then it might encourage others to chip in too.

    1. You are using both IO::Async and AnyEvent but not using either of the glue modules I would expect to see (AnyEvent::Impl::IOAsync or IO::Async::Loop::AnyEvent). Could you explain this?
    2. The code appears to want to ping the same set of hosts twice on different schedules. I cannot think of any reason to do this. What is your intention here?
    3. Within each timer you are pinging 254 addresses 4 times with a 2 second timeout on each. Any target which drops the packets will therefore take 8 seconds to complete. And yet your timers are each on a hard reschedule of less than 8 seconds. Even if your code worked without blocking you could end up with a ping bomb. Again, without knowing your intention it is hard to advise further.

    If you could give a clear statement of your intended algorithm it would help me (and maybe others) to be in a position to help you more.

    Update: removed reference to IO::Async::Loop::AnyEvent as newer versions of AnyEvent explicitly prevent its use.

      Thanks for the reply, sorry if this seems confusing.

      This is my first go at using event loops so I am slowly grasping the concepts and how to implement it, but i'm struggling a little, so I'm not even sure if i'm doing this properly. The main reason I started with this, was to develop a simple perl ICMP monitoring plugin for another application that would be able to ping different sets of many hosts, very quickly, across different, possibly overlapping schedules. E.g 192.168.1.x every 2 minutes and 192.168.2.x every 5 minutes. I've tried various methods and the AnyEvent::Ping module seems to be the best option for performance for asynchronous. To answer your questions:

      1. I used IO::Async::Timer::Periodic to give me an extra option on scheduling the code execution. I didn't realise that i needed AnyEvent::Impl::IOAsync, i've done some more reading and understand this now, so thanks.

      2. This is just test code, so in reality the intention is to invoke AnyEvent::Ping on different sets of hosts and different schedules, however I'm not sure how I get around the fact that AnyEvent::Ping is blocking. I just didn't bother to create another subnet as I'm still trying to understand how I will get this to work. So just ignore that for now.

      3. Yes I'm aware of that, thats me testing on how the reschedule option in Periodic works, i.e. the ping bomb you're talking about, so just ignore that for now. If this is not the best approach to achieve this then I'm open to alternatives. I also looked at using Threads but its very CPU/memory intensive compared to AnyEvent::Ping. I'm testing this on Windows using Strawberry Perl if thats relevant.

      Thanks! Mike

        Thanks for the clarifications - that helps a lot. Having played around with this now I think that the problem you are having essentially boils down to the fact that the event loop is blocking because you are telling it to and that there's no reason for you to do so.

        Firstly, in order to simplify things I have decided to concentrate solely on AnyEvent (because that's what I'm less unfamiliar with) and ditch IO::Async for this illustration. That gets rid of the need for the glue and stops the chances of 2 different event loops coming to blows. As a starting point, here are two timers running on 5 and 7 second intervals implemented purely with AnyEvent. There's a third timer there just so the thing doesn't carry on forever.

        #!/usr/bin/env perl use strict; use warnings; use AnyEvent; my $cv = AnyEvent->condvar; my @timers = ( AnyEvent->timer ( after => 2, interval => 5, cb => sub { warn "Timer 1 at " . AnyEvent->now . "\n"; } ), AnyEvent->timer ( after => 2, interval => 7, cb => sub { warn "Timer 2 at " . AnyEvent->now . "\n"; } ), AnyEvent->timer ( after => 20, cb => sub { warn "Timer 3 at " . AnyEvent->now . "\n"; $cv->send; } ), ); $cv->recv;

        If you run this you will see something along these lines:

        Timer 1 at 1591953440.37264 Timer 2 at 1591953440.37264 Timer 1 at 1591953445.37213 Timer 2 at 1591953447.37533 Timer 1 at 1591953450.36854 Timer 2 at 1591953454.38475 Timer 1 at 1591953455.36656 Timer 3 at 1591953458.37977

        So the timers seem to be working on their correct schedules. Now we can add in some pings. I've reduced the numbers here so we don't get bucketloads of output, but the principle should be the same.

        #!/usr/bin/env perl use strict; use warnings; use AnyEvent; use AnyEvent::Ping; use Data::Dumper; my @todoList = map { "192.168.100." . $_ } (1 .. 5); my $cv = AnyEvent->condvar; my @timers = ( AnyEvent->timer ( after => 2, interval => 5, cb => sub { warn "Timer 1 at " . AnyEvent->now . "\n"; ping_hosts (1, @todoList); } ), AnyEvent->timer ( after => 2, interval => 7, cb => sub { warn "Timer 2 at " . AnyEvent->now . "\n"; ping_hosts (2, @todoList); } ), AnyEvent->timer ( after => 20, cb => sub { warn "Timer 3 at " . AnyEvent->now . "\n"; $cv->send; } ), ); $cv->recv; sub ping_hosts { my ($num, @hosts) = @_; my $ping_timeout = 2; #seconds to wait for response my $ping_attempts = 4; #number of pings to issue, my $ping_pktsize = 32; #packet size of ping my $ping = AnyEvent::Ping->new (packet_size => $ping_pktsize); $ping->timeout ($ping_timeout); #$cv_ping->begin(sub { shift->send($_[0]) }); foreach my $host (@hosts) { $ping->ping ( $host, $ping_attempts, sub { print "Pinger $num: $host state:" . Dumper ($_[0]) . " +\n"; } ); } }

        I've simplified your ping loop/sub a bit and you'll notice that there is now no counter and importantly no cond_var. This is because you have no need to block here if all you are doing is firing off the pings. Now when we run this it runs to completion with the pings sent on the intended schedule.

        This is my first go at using event loops so I am slowly grasping the concepts and how to implement it, but i'm struggling a little, so I'm not even sure if i'm doing this properly.

        The concepts are tough - there's no doubt about that. I don't do much functional programming and even less on event loops so it was interesting to approach this from an equally inexpert standpoint. My simplistic approach with these is just not to block unless it's really necessary. That's not always easy to do when you are coming from a procedural background but it has helped me through when tackling this sort of task.

        I hope this is useful for you and wish you luck with the rest of your project.

Re: Implementing AnyEvent::Ping multiple times -- MCE
by Discipulus (Canon) on Jun 15, 2020 at 20:29 UTC
    Hello mmoorreett

    another rope to learn.. but you can be interested in ping_tcp.pl from MCE. The example uses MCE::Hobo which provide its own timeout.

    marioroy's effort to support windows is commendable.

    L*

    There are no rules, there are no thumbs..
    Reinvent the wheel, then learn The Wheel; may be one day you reinvent one of THE WHEELS.
      cool thanks for that, I haven't seen this one before. i'll check it out.
Re: Implementing AnyEvent::Ping multiple times
by bliako (Monsignor) on Jun 14, 2020 at 12:05 UTC

    Perhaps AnyEvent::FastPing can be of help. Alternatively ...

    Being unfamiliar, as well, with AnyEvent I could be wrong but how about using a global table (shared among the timer callbacks - is this allowed? It seems OK.) to store the state of each host like: my $state{'192.168.100.1' => [0, ['success at timexxx with packetsxxx', 'failed at ...', ...]]}; where the first array element is a flag indicating whether a ping is in progress right now or not. And the second element stores the history of ping results, possibly in a finite-size data structure (Queue?).

    With the above idea and modifying slightly hippo's code:

    use strict; use warnings; use AnyEvent; use AnyEvent::Ping; use Data::Dumper; # separate the hosts wrt their timer groups my @todoList1 = map { "192.168.100." . $_ } (1 .. 2); my @todoList2 = map { "192.168.100." . $_ } (3 .. 4); # a global hash keyed on IPs, => [0,[]]: no pinging now, =>[1,[]]: pin +g in progress # the array ref storing ping history can be a Queue of finite size my $pingState = { map { $_ => [0,[]] } @todoList1, @todoList2 }; my $cv = AnyEvent->condvar; my @timers = ( AnyEvent->timer ( after => 2, interval => 5, cb => sub { warn "Timer 1 at " . AnyEvent->now . "\n"; ping_hosts (1, @todoList1); } ), AnyEvent->timer ( after => 2, interval => 7, cb => sub { warn "Timer 2 at " . AnyEvent->now . "\n"; ping_hosts (2, @todoList2); } ), AnyEvent->timer ( after => 20, cb => sub { warn "Timer 3 at " . AnyEvent->now . "\n"; $cv->send; } ), ); $cv->recv; print "DONE.\n".Dumper($pingState); sub ping_hosts { my ($num, @hosts) = @_; my $ping_timeout = 2; #seconds to wait for response my $ping_attempts = 4; #number of pings to issue, my $ping_pktsize = 32; #packet size of ping my $ping = AnyEvent::Ping->new (packet_size => $ping_pktsize); $ping->timeout ($ping_timeout); #$cv_ping->begin(sub { shift->send($_[0]) }); foreach my $host (@hosts) { if( $pingState->{$host}->[0] == 1 ){ print "ping in progress already to host '$host', skipp +ing till the next timer ...\n"; next } $pingState->{$host}->[0] = 1; # set the "ping in progress" fla +g $ping->ping ( #mock_ping( # emulate the ping $host, $ping_attempts, sub { print "Pinger $num: $host state:" . Dumper ($_[0]) . " +\n"; # store the ping results in global history and remove +the "ping-in-progress" flag push @{$pingState->{$host}->[1]}, $_[0]; $pingState->{$host}->[0] = 0; # ping finished }, # mock_ping extra parameters # $ping_timeout, # 50/100 # probability of a failed ping ); } } # code below is # optional for testing without ping'ing sub mock_ping { my ($host, $attempts, $cb, $timeout, $failure_prob) = @_; # no $attempts, just fail or succeed so many % of times my $started = AnyEvent->time; if( rand()<$failure_prob ){ # simulate a failed ping with lots of delay # so as to coincide with next timer event AEsleep($timeout*3); $cb->([$host, "$started -> fail -> ".AnyEvent->time]); return 0; } else { # a successful ping safely within the timeout AEsleep($timeout/2); # simulate a ping $cb->([$host, "$started -> succ -> ".AnyEvent->time]); return 1; # success ping } } # this is the closest I came to emulating spawning a blocking # ping. Note that using Perl's sleep() is no-go with AnyEvent's timers # AFAIU, re: "It is usually a mistake to intermix alarm and sleep call +s" # and sadly there is no AE::sleep() AFAIK sub AEsleep($) { `sleep $_[0]` }

    btw, in my Linux the pinging code needs to be run as root.

    bw, bliako

      I did try AnyEvent::FastPing but it won't compile via cpan... thanks for the tip, i'll give it a go

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: perlquestion [id://11117894]
Front-paged by Corion
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others musing on the Monastery: (2)
As of 2024-04-19 22:16 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found