I know absolutely nothing about OpenMP (update: but Google is my friend - it already tells me that ANSI C is enough, so already the water got warmer).

It appears that OpenMP is dissembled across embedded, environment variable and shared object facilities. To use Perl instead of C for all three of these requires then three separate techniques:

1) compiler directives: For this part of the interface you could translate Perl main programs to C main programs without knowledge of C using B::C (update: or perlcc) and then apply the OpenMP directives to that as directed by the OpenMP documentation.

2) Environment variables: For these, I tend to make a subroutine that gets them parametrically and then uses IPC::Open3 to create a session with whatever foreign program (could be your own C from Perl program in thes case) is being run with syntax something like:

use IPC::Open3;
sub NameOfCommandInterface { # subcommand, envvar, val, envvar, val...
    my $subcommand = shift;
    my $env = ''.;
    while ( @_ ) {
        $env .= 'export ' . shift();
        $env .= '="' . shift() . '";';
    }
    my $pid = open3 my $wh, my $rh, my $eh, $env . 'NameOfCommand' or 
+die $!;
    print $wh $subcommand;
    close $wh;
    my @er = <$eh> and die ( join ('',@er ));
    close $eh;
    my @rt = <$rh>;
    close $rh;
    waitpid $pid, 0;
    return @rt;
}
[download]

3) shared objects - load those into Perl using P5NCI::Library.

That at least completes the 3 requirements to build the interfacing I found by googling - if there are more, do tell and we'll have a look for you. But (update) to address more of your questions:

"Is Perl 5 multi-threading robust enough to take such a standard?" Nothing to do with robustness - the OpenMP standard is suitable for C. You could try to take Perl's multithreading at source level and interface that to OpenMP, but I suspect the water is too cold for you there. (Update: to be clear, for an OpenMP implementation you would have to achieve the multithreading via something like the three techniques I outline above, and not by using Perl's own multithreading) Will Perl 6 support OpenMP? I presume that the modules I have mentioned will get ported to Perl 6 so I presume yes insofar as you need to build your own main glue interfacing along lines such as above.

Thanks for that. Yes, it is the thought of FORTRAN that makes me shiver. I am considering my own implementation, but did not wish to reinvent the wheel. I had not considered B::C (is it still supported?) and that could be the easiest route.
Update: B::C must be still supported, since there is a 5.9.4 version, but I thought I read somewhere that it is going to be dropped? Maybe I was dreaming.

Come to think of it, for that part of the story you could also use perlcc
__________________________________________________________________________________

^M Free your mind!

• Does anyone know of an implementation of OpenMP with Perl?

  Looking around, the only references I found for Perl & OpenMP are those relating to Perl's use in running the OpenMP test and validation suite--so the answer to your first question appears to be: No.

• Could there be a Perl implementation the OpenMP API?

  I'm talking at the Perl level, rather than within the compiler which I think is a non-starter.

  On the basis of the examples I've seen, there would be little problem hand coding the types of constructs used.

  Making these turn-on and offable (with apologies to EE), via say, the inclusion of a module is probably possible.

  The module could use a source filter to expand/enact comment-embedded directives. Comment out the use statement, you have a single-threaded program. Uncomment the use statement and it becomes parallel.

  Smart::Comments works in a similar way.

• Would this be a useful thing to do?

  It would depend upon what you were using it for, but my gut feel is no.

  On the basis that all of the examples of using OpenMP I found, were for cpu intensive math. Usually on large, repetative data structures. And whilst Perl is fine for the odd bit of math here and there and can certainly handle the large data structures, it is never going to be good at performing intensively repetative math. Natively that is.

  However, PDL is great at performing this kind of thing, and the speed ups obtained from switching to using piddles rather than Perl arrays for this type of work will far exceed any benefits that you could derive from threading Perl native math.

• The next logical question is could OpenMP directives (at the Perl level) be used in conjunction with PDL and piddles to distribute piddle processing across multiple threads?

  On the basis that piddles are opaque black boxes at the Perl level, I would have to say I think not. Hopefully, lin0 will breeze by and give an answer with more authority on that than I can. Ie, Some authority.

Hi BrowserUk,

There is some work related to using OpenMP with piddles by Darin McGill, the author of PDL::Parallel::MPI. However, I am not aware of any concrete result yet.

Cheers,

lin0

Thanks lin0, that looks interesting.

The MPI standard appears to be distributed processes controlled through function calls and 'sharing' data via tcp. Whilst distinctly non-trivial to implement, in Perl's terms, calling functions that fork processes and pass chunks of data back and forth via sockets is at least 'known technology'. There is some mention of threads in MPI-2, but they (seem) to relate to whether the programs using an MPI implementation can be threaded, and whether they can call the library routines from more than one thread serially or concurrently. Ie. The work is done by forking processes, but those processes might be multi-threaded.

Looking at the OpenMP standard, it appears (at a fairly cursory reading) to use threads and shared memory to do the work. It also specifies that the spawning of threads and the subdivision and sharing of data be done via directives embedded in comments. The idea appears to be that by disabling the directives, the same program shoudl be able to run to completion as a single threaded application. And by enabling them, it completes, but more quickly--assuming the availability of multiple cpus/cores.

The directives bit is easily, if controversially, achievable using a source filter.

The most problematic part is that the API assumes an implicitly-shared memory environment--unlike Perl's iThreads which are explicitly-shared only. The problem is that the directives are geared to indicating to the preprocessor which variables need to be explicitly protected and/or later accumulated.

In Perl's explicitly shared environment, that makes the information provided by the programmer on the directives almost the exact opposite of what is required.

All of that said, as piddles are opaque objects at the perl level may mean that they can be implicitly shared by threads? Also, the fact that piddle slices can be referenced in-place may make the implementation of the parallel for directive easier than it is for perl arrays. I guess the real question is whether the PDL libraries are thread-safe?

---

Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.

"Science is about questioning the status quo. Questioning authority".

In the absence of evidence, opinion is indistinguishable from prejudice.

"Too many [] have been sedated by an oppressive environment of political correctness and risk aversion."

I know, I'm digging up an old post here, but for future reference the answer is, "Perl has supported MPI for many, many years." Search CPAN for Parallel::MPI and Parallel::MPI::Simple (I prefer the latter: much more perlish). In addition, the already mentioned PDL::Parallel::MPI is properly coded but has has a bad Makefile.PL. Fortunately, hand-tweaking the Makefile.PL to get it to work isn't hard at all if you know how to compile a C-based MPI program on your cluster.

Now, I just wish I could get a hold of Darin so I could fix his Makefile.PL and get it passing tests again...

Note from THE FUTURE: there is now an Alien::OpenMP which helps writing Makefile.PL or maybe PDL .pd files. OpenMP is for shared-memory systems; MPI is a separate, complementary technology for coordinating processing between non-shared-memory systems.

There are plans https://github.com/PDLPorters/pdl/issues/349 to use OpenMP or a similar technology in PDL, though at the time of writing they are only plans.