http://qs321.pair.com?node_id=1216491


in reply to Re^5: Why should any one use/learn Perl 6?
in thread Why should any one use/learn Perl 6?

Perl 6 has decided that it is not a good idea to shuffle the inherent problems of updating a data-structure like a hash from multiple threads at the same time, under the carpet. Tieing a hash with a Perl interface to make sure that all updates are done sequentially, is not a good step towards making a fully functional, and well performing threaded program without any bottlenecks. You, as a developer, need to be aware of the issues, and make adaptations to your program and/or the way you think about threaded programming.

Think about writing your solutions as a pipeline, or using react whenever using an event driven model.

As I understand this, there are two points you are making: shared hashes in Perl 6 are faster but not thread safe, and hashes are a bad choice of data structure for parallel programming.

In cases where shared data is a better choice (no judgement here), isn't this a step backwards towards pthreads? In Perl 5 you're sure that at least the underlying data structure can't be corrupted (each thread has it's own interpreter) but in Perl 6 you need to take specific steps to ensure that it is not corrupted (threads share the same interpreter). I would think that with Perl 6, even with locking, you would still have better performance here since you aren't copying data between interpreters.

Surely there is some middle ground between the Perl 5 threading model, pthreads, and the Python/Ruby GIL?

I'm sure there is some nuance that I'm missing here. You have a lot more experience with threading than me, so maybe I'm just oversimplifying it and missing the point?

Update: I found this blog post that discusses it. Still digesting it

Update^2: Ok, I'm more confused now. :(

In the blog post, I think I understand the reasoning: he wants to make subtle logic errors into hard failures:

... What achieving safety at the micro level will most certainly achieve, however, is increasing the time it takes for the programmer to discover the real problems in their program. If anything, we want such inevitably unreliable programs to reliably fail, not reliably pretend to work.

Ok, fair enough. I don't necessarily agree with it, but I'm certainly not on Jonathan's level, so my opinion doesn't count for much here. So, zeroing in on the "...unreliable programs to reliably fail...", I wrote this very bad snippet that does stuff very wrong:

#!/usr/bin/env perl6
#

my %shared_hash;

await do for 1..10 -> $thread {
    start {
        for 1..100_000 -> $loop {
            my $key = floor(rand * 10_000);
            my $ref := %shared_hash<$key>;
            if !$ref {
                $ref = 0;
            }
            %shared_hash{$key} = $ref + 1;
        }
    }
}

say "shared_hash has ", %shared_hash.keys.elems, " keys";
# Don't know how to do this easily.  Is there an average function some
+where?
my $sum = 0;
for %shared_hash.values -> $v {
    $sum += $v;
}
my $ave = $sum / %shared_hash.keys.elems;
say "Average value (~10 if threadsafe): ", $ave;

[download]

As expected, I get the wrong results from the hash contents:

$ perl6 killit.p6
shared_hash has 10001 keys
Average value (~10 if threadsafe): 0.999900

[download]

And about every 5th run through, this not so helpful error message:

$ perl6 killit.p6
*** Error in `/home/jeff/perl6/bin/moar': double free or corruption (!
+prev): 0x00002ad5b0147070 ***
======= Backtrace: =========
/lib/x86_64-linux-gnu/libc.so.6(+0x777e5)[0x2ad5a8b3c7e5]
/lib/x86_64-linux-gnu/libc.so.6(+0x8037a)[0x2ad5a8b4537a]
/lib/x86_64-linux-gnu/libc.so.6(cfree+0x4c)[0x2ad5a8b4953c]
//home/jeff/perl6/lib/libmoar.so(+0x2266af)[0x2ad5a838b6af]
[0x2ad5aa6f1ce8]
======= Memory map: ========
00400000-00402000 r-xp 00000000 fc:09 888145                          
+   /home/jeff/perl6/bin/moar
00602000-00603000 r--p 00002000 fc:09 888145                          
+   /home/jeff/perl6/bin/moar
00603000-00604000 rw-p 00003000 fc:09 888145                          
+   /home/jeff/perl6/bin/moar
00a4f000-039f1000 rw-p 00000000 00:00 0                               
+   [heap]
...

[download]

I'm completely out of my depth here, since I've never really written threaded code, but to me this seems to not be accomplishing the stated goal. It's not preventing the logic error by failing reliably, and the error when it does fail is completely ambiguous. I would probably have guessed that it's a bug in moar rather than a bug in my code.

I'm still going to try to learn the language, but I don't think it will ever become my primary language. Most of the code I write is glue anyway, so I don't think I'm the target audience

In Section Seekers of Perl Wisdom