in reply to Re^3: Module for transparently forking a sub?
in thread Module for transparently forking a sub?

Ok, I took the bait and my faith in the general slowness of threading remains intact. Here's my results:

$ perl -N=100 Time taken w/ threads: 0.0077 seconds Time taken w/ forks: 0.0021 seconds

And here's my code:

#! perl -slw use strict; use threads; use Time::HiRes qw[ time ]; use Data::Dump qw[ dump ]; use Storable qw[ store_fd fd_retrieve ]; our $N ||= 10; sub stuff { my %hash = (ARGS => \@_, A => [1 .. 10], B => {'a' .. 'z'}, C => 'Just a big scalar' x 100,); return \%hash; } my $complexData; { my $start = time; for (1 .. $N) { ## "fork" the subroutine my ($thread) = async \&stuff, 1, 2.3, 'four'; ## Do other stuff sleep 1; ## Get the complex results $complexData = $thread->join; } printf "Time taken w/ threads: %.4f seconds\n", (time() - $start) +/ $N - 1; } my $complexData2; { my $start = time; pipe(READ, WRITE); for (1 .. $N) { my $pid = fork; if (!$pid) { # in the kid - do stuff() and send it back to parent store_fd(stuff(1, 2.3, 'four'), \*WRITE); exit; } ## Do other stuff sleep 1; ## Get the complex results $complexData2 = fd_retrieve(\*READ); waitpid($pid,0); } printf "Time taken w/ forks: %.4f seconds\n", (time() - $start) / +$N - 1; } # data should match if (dump($complexData) ne dump($complexData2)) { warn "Data did not match!"; }

As I was coding it I realized it's kind of a bizarre benchmark since it's not really testing any concurency. It's only testing how fast a single thread/process can be spawned and send back data. And really there's just no way Perl's threads are going to beat fork() at that test!

One neat thing I learned - I didn't realize you could use use Storable's store_fd() and fd_retrieve() to pass messages like this. I'd previously used nstore() and thaw() with a prefixed length() of the message so the other end would know how much to read. This is so much easier!


PS: I just noticed you're on Windows (or DOS, I guess)! You don't have a real fork() there, so I guess you're not going to be able to replicate my results. Oh well.

Replies are listed 'Best First'.
Re^5: Module for transparently forking a sub?
by BrowserUk (Pope) on Feb 14, 2009 at 20:08 UTC
      Nothing too fancy - it's a Core 2 Duo T7200 @ 2GHz running Fedora 8. It was top-o-the-line when I bought it 1.75 years ago, but time marches on...

      Frankly, I couldn't figure out what you were doing with dividing the time by $N-1, but I decided not to care since I could just do the same for threads and forks and they should be affected equally. Also, sleep(1) in a benchmark could create some interesting problems - the OS generally doesn't guarantee how long that will take, only that it will take at least 1 second.


        it's a Core 2 Duo

        Okay. That explains it. Two cores versus one.

        I couldn't figure out what you were doing with dividing the time by $N-1,

        Actually, I'm dividing by $N and then subtracting one.

        The total time will be N*1 + N*(time of interest); so dividing by N gives 1 + (time of interest); -1 leaves (time of interest).

        The idea of the sleep was to simulate doing something else in the interim--there being no point in multitasking it, if all your going to do is wait for the results.

        Not a great benchmark I admit, but it did get me a set of numbers to consider, rather than just anecdotal "forks are faster than threads on *nix". Thanks :)

        ps. Suggesting that win32 is "DOS" is very tacky. 'sides, even if it was DOS, it'd still be a decade newer than that Multics derivitive that you seem to like :)

        Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
        "Science is about questioning the status quo. Questioning authority".
        In the absence of evidence, opinion is indistinguishable from prejudice.