Beefy Boxes and Bandwidth Generously Provided by pair Networks
Perl Monk, Perl Meditation
 
PerlMonks  

Re: Inline::CUDA : looking for feedback

by kcott (Archbishop)
on Jul 29, 2021 at 06:23 UTC ( [id://11135466]=note: print w/replies, xml ) Need Help??


in reply to Inline::CUDA : looking for feedback

G'day bliako,

We had some discussions about CUDA a few weeks ago (Perl GPGPU Modules) so I was looking forward to trying this out. I downloaded and extracted the files from github without any problems.

I do have all of the prerequisite components (nvcc, special compiler, etc.). Just as an FYI for others, the SDK was about a 2Gb download (from memory). I'm running Cygwin on Win10: the version of SDK I got lives in "Program Files/NVIDIA GPU Computing Toolkit/CUDA". The spaces are a problem.

When running 'perl Makefile.PL', it correctly found '/cygdrive/c/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v11.4/bin/nvcc' but then immediately failed with 'Can't exec "/cygdrive/c/Program": No such file or directory ...".

I expect I could find a work-around myself (e.g. create a symlink, modify 'Inline::CUDA::Utils::find_dependencies', etc.); although, I haven't looked into this in any detail.

I suspect others running Strawberry Perl, ActivePerl, and so on, may run into exactly the same problem. The solution may be as simple as quoting one or more pathnames. It would be great if you could look into this; I'm more than happy to help out with testing.

— Ken

Replies are listed 'Best First'.
Re^2: Inline::CUDA : looking for feedback
by bliako (Monsignor) on Jul 29, 2021 at 10:08 UTC

    Hello kcott,

    I have added double quotes to the command spawning nvcc (which includes where the c compiler is via the --compiler-bindir option). And I have put the changes back to the repository. In any case these are lines 621 and 630 of lib/Inline/CUDA.pm and you can experiment with them to fit windows requirement.

    Thank you for offering to help

      The following is a set of running notes that I made as I worked through this.

      I checked through Makefile.PL: I had all prerequisites except for two. Math::Matrix installed without a hitch. The other, Test::CheckManifest, had a problem:

      Running make test for RENEEB/Test-CheckManifest-1.42.tar.gz PERL_DL_NONLAZY=1 "/home/ken/perl5/perlbrew/perls/perl-5.34.0/bin/perl +.exe" "-MExtUtils::Command::MM" "-MTest::Harness" "-e" "undef *Test:: +Harness::Switches; test_harness(0, 'blib/lib', 'blib/arch')" t/*.t t/ +extra/*.t t/xt/*.t t/00_load.t ........................ ok Bailout called. Further testing stopped: Cannot find a MANIFEST. Ple +ase check! FAILED--Further testing stopped: Cannot find a MANIFEST. Please check! make: *** [Makefile:874: test_dynamic] Error 255 (/usr/bin/make test exited with 512)

      I've encountered that problem on a variety of platforms over a number of years. There was a report about this in rt-cpan; the tracker has been moved to github and I can't find the report. However, I do recall that discussions in that report lead to this in the doco: "Test::CheckManifest - REPLACE THIS MODULE". I've been using ExtUtils::Manifest ever since (for both personal and $work modules) without any problems. Also, ExtUtils::Manifest is a core module, so no CPAN installation necessary. You may want to consider making this change too. Anyway, I force-installed Test::CheckManifest to avoid problems with prerequisite testing: actual tests using this may still fail, but I'll worry about that down the track.

      [On a side note, modules such as Test::Pod and Test::CheckManifest should be for "Author Only" tests. They are typically skipped unless something like $ENV{RELEASE_TESTING} has a true value. I question their inclusion under PREREQ_PM in Makefile.PL.]

      I manually made the changes you indicated for lib/Inline/CUDA.pm (from new code in github). Unfortunately, that didn't fix my reported problem:

      $ perl Makefile.PL Can't exec "/cygdrive/c/Program": No such file or directory at /home/k +en/local/opt/perl/inline_cuda/perl-inline-cuda-master/lib/Inline/CUDA +/Utils.pm line 182. Inline::CUDA::Utils::find_dependencies (via N/A) : failed to find vers +ion of /cygdrive/c/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v1 +1.4/bin/nvcc with this command: /cygdrive/c/Program Files/NVIDIA GPU +Computing Toolkit/CUDA/v11.4/bin/nvcc --version Makefile.PL : error, call to Inline::CUDA::Utils::find_dependencies() +has failed.

      I then made a manual change to Inline::CUDA::Utils::find_dependencies(). At line 181:

      #my $cmd = "${aexe} --version"; my $cmd = "\Q${aexe}\E --version";

      A Makefile was successfully created:

      $ perl Makefile.PL find_dependencies() : report: cc : /usr/bin/cc cxx : /usr/bin/c++ ld : /usr/bin/cc nvcc : /cygdrive/c/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v +11.4/bin/nvcc nvcc-release : 11.4 nvcc-version : V11.4.48 nvlink : /cygdrive/c/Program Files/NVIDIA GPU Computing Toolkit/CUDA +/v11.4/bin/nvlink nvlink-release : 11.4 nvlink-version : V11.4.48 Makefile.PL : configuration written to '/home/ken/local/opt/perl/inlin +e_cuda/perl-inline-cuda-master/config/Inline-CUDA.conf'. Generating a Unix-style Makefile Writing Makefile for Inline::CUDA Writing MYMETA.yml and MYMETA.json

      No problems with make:

      $ make cp /home/ken/local/opt/perl/inline_cuda/perl-inline-cuda-master/config +/Inline-CUDA.conf blib/lib/auto/share/dist/Inline-CUDA/Inline-CUDA.co +nf cp lib/Inline/CUDA/Utils.pm blib/lib/Inline/CUDA/Utils.pm cp lib/Inline/CUDA.pm blib/lib/Inline/CUDA.pm cp scripts/nvidia-ml-test.pl blib/script/nvidia-ml-test.pl "/home/ken/perl5/perlbrew/perls/perl-5.34.0/bin/perl.exe" -MExtUtils:: +MY -e 'MY->fixin(shift)' -- blib/script/nvidia-ml-test.pl Manifying 1 pod document

      I did have problems with 'make test'. There was a huge amount of output: too much to post here and I don't have time to look through this closely right now. Some look like previous problems caused by spaces and can probably be just as easily fixed. For instance:

      /bin/sh: /cygdrive/c/Program: No such file or directory make[1]: *** [Makefile:352: _011_basic_t_f5ac.o] Error 127

      The "Author Tests" were handled as I indicated above, so no issues with my potentially dodgy Test::CheckManifest:

      t/manifest.t ....................................... skipped: Author t +ests not required for installation t/pod-coverage.t ................................... skipped: Author t +ests not required for installation t/pod.t ............................................ skipped: Author t +ests not required for installation

      This further strengthens my argument for not including those Test::* modules as prerequisites. I also note that Test::Pod::Coverage is not mentioned in Manifest.PL but is used in t/pod-coverage.t.

      The final few lines of make test output were:

      Files=20, Tests=13, 59 wallclock secs ( 0.03 usr 0.01 sys + 14.46 cus +r 16.56 csys = 31.06 CPU) Result: FAIL Failed 15/20 test programs. 0/13 subtests failed. make: *** [Makefile:926: test_dynamic] Error 255

      It's Friday morning in my timezone and I have $work today. I can probably spend more time on this over the weekend.

      — Ken

        kcott, the immediate cause is that C/C++ compilers are not detected. And possibly errors because of more quoting problems.

        it's obvious that I need to handle quoting commands and parameters properly. I will follow anonmonk's suggestion if your \Q \E proves not robust enough.

        System C and C++ compiler must be detected properly. For locally-built perl.exe finding the default C-compiler is as easy as $Config{cc} but for C++ is not. For downloaded perl.exe, I have no idea where $Config{cc} will point to! I will also make a basic/preliminary test to check quotes and to check compilers.

        Secondly, I have no idea how to affect Makefile.PL vars in windows (for specifying CC,CXX,LD), in the same way I do it in bash:

        CC=/usr/local/gcc84/bin/gcc84 CXX=/usr/local/gcc84/bin/g++84 LD=/usr/l +ocal/gcc84/bin/gcc84 perl Makefile.PL

        A workaround is, after perl Makefile.PL, to try to modify config/Inline-CUDA.conf

        If all else fails I will have to resort to prompting the user during installation. Any opinions on that style?

        Thanks for your time spent and valuable feedback. I am not in a hurry, please work on it in your own pace.

        bw, bliako

        apropos Test::CheckManifest being a requirement, I was just scanning all files for external packages and I saw that in t/manifest.t created by module-starter

        #!perl use 5.006; use strict; use warnings; use Test::More; unless ( $ENV{RELEASE_TESTING} ) { plan( skip_all => "Author tests not required for installation" ); } my $min_tcm = 0.9; eval "use Test::CheckManifest $min_tcm"; plan skip_all => "Test::CheckManifest $min_tcm required" if $@; ok_manifest();

        this will be useful, thanks

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: note [id://11135466]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others making s'mores by the fire in the courtyard of the Monastery: (9)
As of 2024-03-28 09:39 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found