Beefy Boxes and Bandwidth Generously Provided by pair Networks
Syntactic Confectionery Delight
 
PerlMonks  

Re^2: Splitting one large timeout into few smaller ones..

by Eyck (Priest)
on Apr 07, 2005 at 14:56 UTC ( [id://445719]=note: print w/replies, xml ) Need Help??


in reply to Re: Splitting one large timeout into few smaller ones..
in thread Splitting one large timeout into few smaller ones..

Tasks cannot be reordered. Lets look at something like this
  1. Start processing/new transaction
  2. Get message from spool(requires contacting remote spool)
  3. verify signature (requires contacting remote keyservers)
  4. uncompress/convert/split
  5. Send required files(remote...)
  6. Request additional parts(remote..)
  7. End transaction

Or, better yet, think of pop3 client,

  1. Authenticate
  2. Get list of new mails
  3. Get mail body (repeat) ...

Imagine what happens when you implement it like this:

auth(); @uidls=getuidls(); foreach $uidl (@uidls) { fork || getmail($uidl); };

getting back to my original problem, you can easily put timeout protection inside getuidl and getmail, but what you created is either instant DOS against your provider, and yourself (try that code with 1000s of mails to download..).

Not to mention that now you have to devise a clever way for generating for example unique filename for every mail you're downloading and few similar headaches.

Of course if you're trying to write reasonable code, and use just one connection for downloading those mails, you need to devise a clever scheme of locking and passing socket fd around to your children etc...

Replies are listed 'Best First'.
Re^3: Splitting one large timeout into few smaller ones..
by Roy Johnson (Monsignor) on Apr 07, 2005 at 15:00 UTC
    I didn't suggest you reorder tasks. You fork and wait. If the wait times out, you kill the child and error out. You only have one spawned sub-process at a time.

    Update: you'd have to rig up the timeout yourself, depending on what your system supports. You want to poll and sleep until the process completes, or until you've waited long enough for it. Or you could set an alarm, do a wait, and if the wait returns, unset the alarm.


    Caution: Contents may have been coded under pressure.
      Please provide sample code/pseudocode, because I don't quite follow. Here's how it looks right now:
      Event->idle(min=>$mint,max=>$maxt, cb=> [$sth,"method"], max_cb_tm=>$timeout,);
      where $timeout is my large timeout value, roughly every $mint seconds $sth->method() is called, if it takes longer then $timeout it is killed off.

      I don't know where am I supposed to fork.

        You would fork inside your
        foreach $task (@tasks) { $task->do_one_task() };
        It might go like this (see fork for more error checking):
        foreach $task (@tasks) { use POSIX ':sys_wait_h'; my $child = fork(); if ($child) { for (my $polls = 30; waitpid($child, WNOHANG) == 0; --$polls) +{ if ($polls < 1) { print "Game over...killing $child\n"; kill 'ABRT', $child; # Maybe return some code, or exit here } sleep 1; } } else { $task->do_one_task() exit 0; } }
        or, probably better:
        my $child = fork(); if ($child) { $SIG{ALRM} = sub { print "Game over...killing $child\n"; kill 'ABRT', $child; # Maybe return some code, or exit here }; alarm 10; # Waiting for child or alarm waitpid($child, 0); } else { $task->do_one_task() exit 0; }

        Caution: Contents may have been coded under pressure.

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: note [id://445719]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others surveying the Monastery: (5)
As of 2024-03-29 10:36 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found