Beefy Boxes and Bandwidth Generously Provided by pair Networks
We don't bite newbies here... much
 
PerlMonks  

comment on

( [id://3333]=superdoc: print w/replies, xml ) Need Help??

A purist could pop in and say to you "no, I want Perl to just be P.E.R.L. and only that, no fancy additions."

I don't agree with that view but we should not forget what Perl was created for.

Asking for shared memory alone is vague. Asking for *efficient* shared memory is much more accurate. Only then we can see who is "swimming naked" ("You only find out who is swimming naked when the tide goes out" hehe, and let the source remain nameless) when it comes to comparing to other systems. Although I do not dispute that Perl's shared memory is inefficient.

For the realist, if a solution must involve Perl, perhaps it should not be as fine-grained as to require sharing the million variables of a parallel algorithm. A coarse-grain implementation could be more suitable for Perl (being further away from hardware, compared to C, and this is the reason we all enjoy other perks!). That is, limit the amount of what is shared or access it in larger chunks of raw data where the inefficient lock mechanism does ot matter much. On the other hand, MMAPing a raw-memory buffer and sharing it, is easy but each thread interpreting it as a Perl nested data structure or object will take some efficiency away. Interpreting it as 100 such data structures in 100 threads it will be much slower. And then locking each of these.

Additionally, there is the other route to harnessing state-of-the-art computer science developments: by using extenral libraries. For example the excellent implementation of AI::MXNet by Sergey Kolychev. Which is still a well-kept secret (I think? Anybody using it? Although regularly update. Thanks Sergey). Also see his relevant post http://blogs.perl.org/users/sergey_kolychev/2017/02/machine-learning-in-perl.html and my brief introduction to using it for very simple Dataflow programming Dataflow programming on CPU and GPU using AI::MXNet even on a GPU.

These are great doors opened from within Perl. Fair enough, not what exactly you are asking for: transparent shmem-smp. But just write your algorithm as a Graph and let MXnet worry about parallelising it or even having the GPU solve it.

Perhaps setting a concrete programming challenge can show the distance between practice and my theoretical blabber above?

bw, bliako


In reply to Re^12: Amicable divorce by bliako
in thread Amicable divorce by ribasushi

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post; it's "PerlMonks-approved HTML":



  • Are you posting in the right place? Check out Where do I post X? to know for sure.
  • Posts may use any of the Perl Monks Approved HTML tags. Currently these include the following:
    <code> <a> <b> <big> <blockquote> <br /> <dd> <dl> <dt> <em> <font> <h1> <h2> <h3> <h4> <h5> <h6> <hr /> <i> <li> <nbsp> <ol> <p> <small> <strike> <strong> <sub> <sup> <table> <td> <th> <tr> <tt> <u> <ul>
  • Snippets of code should be wrapped in <code> tags not <pre> tags. In fact, <pre> tags should generally be avoided. If they must be used, extreme care should be taken to ensure that their contents do not have long lines (<70 chars), in order to prevent horizontal scrolling (and possible janitor intervention).
  • Want more info? How to link or How to display code and escape characters are good places to start.
Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others having a coffee break in the Monastery: (3)
As of 2024-04-24 03:28 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found