Beefy Boxes and Bandwidth Generously Provided by pair Networks
The stupid question is the question not asked
 
PerlMonks  

Re^10: Amicable divorce

by jeffenstein (Friar)
on Jul 23, 2020 at 10:44 UTC ( #11119704=note: print w/replies, xml ) Need Help??


in reply to Re^9: Amicable divorce
in thread Amicable divorce

Named arguments, default values, parameter checking are all syntactic sugar, which sure, is somewhat nice to have, but is no possible way essential. Every time you go "I can do this using this little longer syntax" - you already disqualified it as a "long overdue must have feature".
Really, anything beyond binary can be considered syntactic sugar. It's up to you to decide how much you want, but I don't believe refusing a feature because it is "syntactic sugar" is reasonable; there are other, better criteria for this.
This is something I had to write 2 months ago. I can not write this in perl, and am not likely to ever be able to.
Ah, but this doesn't necessarily mean it can't be written in perl. You might just need to use a little longer syntax. ;)

Replies are listed 'Best First'.
Re^11: Amicable divorce
by ribasushi (Pilgrim) on Jul 24, 2020 at 14:18 UTC

    > > This is something I had to write 2 months ago. I can not write this in perl, and am not likely to ever be able to.

    > Ah, but this doesn't necessarily mean it can't be written in perl. You might just need to use a little longer syntax. ;)

    This is woefully incorrect. No amount of syntax will give me SMP-multithreading in Perl (preemptively: any type of async is not even close to multithreading). On top of that, in order to write the above I would need a proper, marshal-less, shared memory implementation.

    That said: I am behind the times with CPAN, so perhaps you will point me to an implementation of the above. I will be eternally grateful!

      No amount of syntax will give me SMP-multithreading in Perl

      Do you mean perl 5.005 threads?

      IIRC, the conclusion we got from there is that it is not doable without rewamping the runtime completely.

      Perl internal data structures were never designed for atomic access so locks need to be used to serialize their access. You could do as Python, have a global lock (bye bye SMP!) or require the programmer to lock by hand (that was the 5.005 approach), but that didn't work. Most modules failed to work in multithreading environments, usually because of the perl internals becoming corrupted... XS code was even worse.

      Perl 5.6 threads was a compromise where everything is cloned so that there is no interference between threads and whenever you want to share something, you have to say it explicitly and an intermediate (and quite inefficient) layer that does the locking and serializes access is set up by the runtime.

      So, in conclusion, do you want SMP in Perl 5? then write a new runtime! and forget about compatibility with any XS module!!!

      Would it be worth the trouble? IMO, well, maybe yes. There are so many things in Perl 5 that need fixing that could only be fixed by using a new runtime!!!

      But anyway, who is gonna do it?

      Update: oops, s/5\.6/5.005/g; s/5\.8/5.6/g;. I knew I had it wrong because I can still remember my excitment trying the threads support with LWP and getting SIGSEGVs all the time when perl 5.005 was released 22 years and two days ago! Just one of those things than randomly stick in your mind!

        > Perl 5.8 threads was a compromise where everything is cloned so that there is no interference between threads and whenever you want to share something, you have to say it explicitly and an intermediate (and quite inefficient) layer that does the locking and serializes access is set up by the runtime.

        I'm by far no expert here, but I remember seeing benchmarks where the Perl approach actually did beat Python's GIL in terms of performance.

        But benchmarks are like statistics, the author often only measures his own prejudices ... ;-)

        Cheers Rolf
        (addicted to the Perl Programming Language :)
        Wikisyntax for the Monastery

      I was not being entirely serious in my comment (see the smiley at the end). That said, I'll still give it a go to answer your statements

      No amount of syntax will give me SMP-multithreading in Perl (preemptively: any type of async is not even close to multithreading)

      use threads;? Unless I'm missing something, which is likely here since the answer seems so simple.

      On top of that, in order to write the above I would need a proper, marshal-less, shared memory implementation.

      I'm not sure what you mean by unmarshalled shared memory, but then I'm not a full-time programmer, so this is probably my fault. There is PerlIO::mmap, the SysV shared memory modules, or a bunch of mmap modules on CPAN that might be usable for a generic blob of memory. I would guess that at least one of these would be usable, if not necessarily efficient or pleasant.

      Of course, it would be crazy to try to literally translate between two different programming languages, much like literally translating between spoken languages, so YMMV.

      A purist could pop in and say to you "no, I want Perl to just be P.E.R.L. and only that, no fancy additions."

      I don't agree with that view but we should not forget what Perl was created for.

      Asking for shared memory alone is vague. Asking for *efficient* shared memory is much more accurate. Only then we can see who is "swimming naked" ("You only find out who is swimming naked when the tide goes out" hehe, and let the source remain nameless) when it comes to comparing to other systems. Although I do not dispute that Perl's shared memory is inefficient.

      For the realist, if a solution must involve Perl, perhaps it should not be as fine-grained as to require sharing the million variables of a parallel algorithm. A coarse-grain implementation could be more suitable for Perl (being further away from hardware, compared to C, and this is the reason we all enjoy other perks!). That is, limit the amount of what is shared or access it in larger chunks of raw data where the inefficient lock mechanism does ot matter much. On the other hand, MMAPing a raw-memory buffer and sharing it, is easy but each thread interpreting it as a Perl nested data structure or object will take some efficiency away. Interpreting it as 100 such data structures in 100 threads it will be much slower. And then locking each of these.

      Additionally, there is the other route to harnessing state-of-the-art computer science developments: by using extenral libraries. For example the excellent implementation of AI::MXNet by Sergey Kolychev. Which is still a well-kept secret (I think? Anybody using it? Although regularly update. Thanks Sergey). Also see his relevant post http://blogs.perl.org/users/sergey_kolychev/2017/02/machine-learning-in-perl.html and my brief introduction to using it for very simple Dataflow programming Dataflow programming on CPU and GPU using AI::MXNet even on a GPU.

      These are great doors opened from within Perl. Fair enough, not what exactly you are asking for: transparent shmem-smp. But just write your algorithm as a Graph and let MXnet worry about parallelising it or even having the GPU solve it.

      Perhaps setting a concrete programming challenge can show the distance between practice and my theoretical blabber above?

      bw, bliako

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://11119704]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others examining the Monastery: (10)
As of 2020-11-24 21:19 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found

    Notices?