http://qs321.pair.com?node_id=688019

druidmatrix has asked for the wisdom of the Perl Monks concerning the following question:

I have a daemon process that handles a number of signals along with HUP. Basically the parent disassociates itself using POSIX::setsid, and then spawns a number of children that it communicates with using USR1 and USR2 signals. The parent also implements HUP where it kills the children, re-reads the configuration, and respawns the children according to the new configuration. I am reinstalling the signal handlers after every call, and all seem to work fine except HUP. The HUP handler runs fine the first time, but ceases to do so after that, almost as if the handler is not re-installed. However, I print out the SIG hash just before entering the "endless loop", and I see that the handler has a target (something like HUP => CODE(0x8da8428)). This is perl 5.8.8 running on RHEL 5. I tried using POSIX handlers, but see the exact same thing. What more, once the first HUP is received, all other signal handlers also seem to stop working. Is there a notable difference between the HUP signal and others? Does anyone have any advice regarding how to debug this sort of issue? Thanks.

Replies are listed 'Best First'.
Re: handling HUP signals
by almut (Canon) on May 22, 2008 at 19:13 UTC

    Could you post some trimmed-down code sample that allows interested monks to play with / reproduce the effect without requiring them to write test code from scratch?

      Almut, thank you for your reply. Here is some test code, I am sorry it is not a 10-liner, but I wanted to maintain the concept of a global object that I have in my application. :)

      #!/usr/bin/perl use strict; use Getopt::Std; use Mgr; use Global1; my %OPT; my $ok = getopts("D", \%OPT); my $BASE = "."; daemonize(); start(); sub start { eval { $SIG{HUP} = \&catchHup; $SIG{CHLD} = "IGNORE"; $SIG{INT} = \&catchInt; $SIG{USR1} = \&catchUsr1; if ($MANAGER) { undef $MANAGER; } $MANAGER = new Mgr; $MANAGER->monitor(); }; if ($@) { Global1::error ("Got fatal error:\n$@\n"); catchInt(); } } sub daemonize { require POSIX; open STDOUT, '>/dev/null' or die "Can't write to /dev/null: $!"; open STDERR, '>/dev/null' or die "Can't write to /dev/null: $!"; defined(my $pid = fork) or die "Can't fork: $!"; exit if $pid; POSIX::setsid() or Global1::error ( "Could not setsid\n$!\n" ); `echo $$ > tun_mgr.pid`; } sub catchHup { Global1::error ( "\n\t!!!Got HUP!!!\n" ); $MANAGER->killAll(); Global1::error ( "\n\tRestarting application\n" ); &start(); } sub catchInt { $MANAGER->killAll(); `rm -rf tun_mgr.pid`; Global1::error ( "Parent process $$ exiting\n" ); exit; }

      package Mgr; use strict; use Global1; sub new { my $this = {}; $this->{children} = {}; bless $this; return $this; } sub killAll { my $this = shift; $SIG{CHLD} = "IGNORE"; foreach (keys %{$this->{children}}) { my $cpid = $this->{children}->{$_}->{pid}; if (kill INT => $cpid) { Global1::error ("Killed child $cpid\n"); } delete $this->{children}->{$_}; } } my $usr1 = 0; my $usr2 = 0; sub monitor { my $this = shift; my $num = 0; my $pid; my $ppid = $$; while ($num < 2) { unless ($pid = fork()) { # Child $SIG{USR1} = \&doPing; $SIG{INT} = \&gotINT; $SIG{HUP} = 'IGNORE'; $SIG{USR2} = 'IGNORE'; CHILD_START: while (! $usr1) { Global1::error ( "Child $$ sleeping for 10 seconds\n" +); sleep(10); } $usr1 = 0; eval { `echo 1 > $$`; my $numkill = kill 'USR2' => $ppid; Global1::error ( "\t$$ Sent SigUSR2 to parent proc +ess \n" ); goto CHILD_START; }; sub doPing { $usr1 = 1; } sub gotINT { Global1::error ( "$$ got INT signal - exiting\n" ); exit; } } # Parent Code $this->{children}->{$num}->{pid} = $pid; Global1::error ( "Spawned pinger process $pid child ID # $num\ +n" ); $num++; } foreach my $ch (keys %{$this->{children}}) { my $numkill = kill 'USR1' => $this->{children}->{$ch}->{pi +d}; Global1::error ( "Parent $$ sent SigUSR1 to process ".$thi +s->{children}->{$ch}->{pid}." \n" ); } $SIG{USR2} = \&catchUsr2; $SIG{HUP} = \&main::catchHup; $SIG{CHLD} = "IGNORE"; $SIG{INT} = \&main::catchInt; if ($DEBUG) { my $msg = ""; foreach (keys %SIG) { $msg .= "$_ => $SIG{$_}\n"; } Global1::error ( "Parent's SIG hash:\n$msg\n" ); } HANGOUT: while (! $usr2) { sleep 1; } $usr2 = 0; my $done = 0; $num = 0; foreach $num (keys %{$this->{children}}) { my $fc = $this->{children}->{$num}->{pid}; if (-e $fc) { Global1::error ( "Parent process $$ recieved USR2 sign +al from child $fc\n" ); unlink $fc; kill "USR1" => $fc; last; } } goto HANGOUT; sub catchUsr2 { $usr2 = 1; } } 1;

      package Global1; require Exporter; @ISA = qw(Exporter); @EXPORT = qw ( $MANAGER $DEBUG ); $MANAGER = ""; $DEBUG = 1; sub error { my $mes = shift; my $date = scalar (localtime (time)); open (LOG, ">>logfile") or die "Could not open logfile for write:\ +n$!\n"; print LOG "$date\t $mes\n"; close LOG; } 1;

        I haven't investigated this in every detail, but it looks to me like you're never exiting your catchHup() signal handler, once it got called. At the end of the handler you call start() which then calls $MANAGER->monitor();. At the end of that routine you have

        HANGOUT: while (! $usr2) { sleep 1; } ... goto HANGOUT;

        which looks like an endless loop to me...  i.e. you never leave that function, and thus you never leave the signal handler you called it from. And, as you probably know, signal handlers will not be reactivated before you have left them...

        Update: Generally, it's a good idea to not do too much within a signal handler routine. In most cases it's best to just set some flag to which you can react in the main program's loop.

Re: handling HUP signals
by traveler (Parson) on May 22, 2008 at 21:20 UTC
    The Description in the documentation for Sys::SigAction may provide a partial answer.
      Traveler, thank you for your reply. I did try to use POSIX handlers, but I see the same thing. So replacing the signal handlers with POSIX handlers in the code above doesn't seem to make a difference.