Beefy Boxes and Bandwidth Generously Provided by pair Networks
Keep It Simple, Stupid

Re^2: OpenMP from Perl?

by lin0 (Curate)
on Jun 22, 2007 at 06:35 UTC ( #622728=note: print w/replies, xml ) Need Help??

in reply to Re: OpenMP from Perl?
in thread OpenMP from Perl?

Hi BrowserUk,

There is some work related to using OpenMP with piddles by Darin McGill, the author of PDL::Parallel::MPI. However, I am not aware of any concrete result yet.



Replies are listed 'Best First'.
Re^3: OpenMP from Perl?
by BrowserUk (Pope) on Jun 22, 2007 at 08:10 UTC

    Thanks lin0, that looks interesting.

    The MPI standard appears to be distributed processes controlled through function calls and 'sharing' data via tcp. Whilst distinctly non-trivial to implement, in Perl's terms, calling functions that fork processes and pass chunks of data back and forth via sockets is at least 'known technology'. There is some mention of threads in MPI-2, but they (seem) to relate to whether the programs using an MPI implementation can be threaded, and whether they can call the library routines from more than one thread serially or concurrently. Ie. The work is done by forking processes, but those processes might be multi-threaded.

    Looking at the OpenMP standard, it appears (at a fairly cursory reading) to use threads and shared memory to do the work. It also specifies that the spawning of threads and the subdivision and sharing of data be done via directives embedded in comments. The idea appears to be that by disabling the directives, the same program shoudl be able to run to completion as a single threaded application. And by enabling them, it completes, but more quickly--assuming the availability of multiple cpus/cores.

    The directives bit is easily, if controversially, achievable using a source filter.

    The most problematic part is that the API assumes an implicitly-shared memory environment--unlike Perl's iThreads which are explicitly-shared only. The problem is that the directives are geared to indicating to the preprocessor which variables need to be explicitly protected and/or later accumulated.

    In Perl's explicitly shared environment, that makes the information provided by the programmer on the directives almost the exact opposite of what is required.

    All of that said, as piddles are opaque objects at the perl level may mean that they can be implicitly shared by threads? Also, the fact that piddle slices can be referenced in-place may make the implementation of the parallel for directive easier than it is for perl arrays. I guess the real question is whether the PDL libraries are thread-safe?

    Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
    "Science is about questioning the status quo. Questioning authority".
    In the absence of evidence, opinion is indistinguishable from prejudice.

      Hi BrowserUk,

      There is a group of PDL libraries that are not thread-safe. I am referring to those associated with dimension manipulation as indicated in the PDL::Internals documentation:

      Certain types of transformations can be optimized very efficiently obviating the need for explicit readdata and writebackdata methods. Those transformations are called pdl_vaffine. Most dimension manipulating functions (e.g., slice, xchg) belong to this class.

      The basic trick is that parent and child of such a transformation work on the same (shared) block of data which they just choose to interpret differently (by dusing different dims, dimincs and offs on the same data, compare the pdl structure above). Each operation on a piddle sharing data with another one in this way is therefore automatically flown from child to parent and back -- after all they are reading and writing the same block of memory. This is currently not perl thread safe --

      The same documentation goes on to indicate that "no big loss since the whole PDL core is not reentrant". Therefore, the PDL core would not be thread safe



Log In?

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://622728]
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others scrutinizing the Monastery: (6)
As of 2020-06-03 14:00 GMT
Find Nodes?
    Voting Booth?
    Do you really want to know if there is extraterrestrial life?

    Results (24 votes). Check out past polls.