Beefy Boxes and Bandwidth Generously Provided by pair Networks
more useful options
 
PerlMonks  

Re^3: Inline::CUDA : looking for feedback

by kcott (Bishop)
on Jul 29, 2021 at 20:32 UTC ( #11135499=note: print w/replies, xml ) Need Help??


in reply to Re^2: Inline::CUDA : looking for feedback
in thread Inline::CUDA : looking for feedback

The following is a set of running notes that I made as I worked through this.

I checked through Makefile.PL: I had all prerequisites except for two. Math::Matrix installed without a hitch. The other, Test::CheckManifest, had a problem:

Running make test for RENEEB/Test-CheckManifest-1.42.tar.gz PERL_DL_NONLAZY=1 "/home/ken/perl5/perlbrew/perls/perl-5.34.0/bin/perl +.exe" "-MExtUtils::Command::MM" "-MTest::Harness" "-e" "undef *Test:: +Harness::Switches; test_harness(0, 'blib/lib', 'blib/arch')" t/*.t t/ +extra/*.t t/xt/*.t t/00_load.t ........................ ok Bailout called. Further testing stopped: Cannot find a MANIFEST. Ple +ase check! FAILED--Further testing stopped: Cannot find a MANIFEST. Please check! make: *** [Makefile:874: test_dynamic] Error 255 (/usr/bin/make test exited with 512)

I've encountered that problem on a variety of platforms over a number of years. There was a report about this in rt-cpan; the tracker has been moved to github and I can't find the report. However, I do recall that discussions in that report lead to this in the doco: "Test::CheckManifest - REPLACE THIS MODULE". I've been using ExtUtils::Manifest ever since (for both personal and $work modules) without any problems. Also, ExtUtils::Manifest is a core module, so no CPAN installation necessary. You may want to consider making this change too. Anyway, I force-installed Test::CheckManifest to avoid problems with prerequisite testing: actual tests using this may still fail, but I'll worry about that down the track.

[On a side note, modules such as Test::Pod and Test::CheckManifest should be for "Author Only" tests. They are typically skipped unless something like $ENV{RELEASE_TESTING} has a true value. I question their inclusion under PREREQ_PM in Makefile.PL.]

I manually made the changes you indicated for lib/Inline/CUDA.pm (from new code in github). Unfortunately, that didn't fix my reported problem:

$ perl Makefile.PL Can't exec "/cygdrive/c/Program": No such file or directory at /home/k +en/local/opt/perl/inline_cuda/perl-inline-cuda-master/lib/Inline/CUDA +/Utils.pm line 182. Inline::CUDA::Utils::find_dependencies (via N/A) : failed to find vers +ion of /cygdrive/c/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v1 +1.4/bin/nvcc with this command: /cygdrive/c/Program Files/NVIDIA GPU +Computing Toolkit/CUDA/v11.4/bin/nvcc --version Makefile.PL : error, call to Inline::CUDA::Utils::find_dependencies() +has failed.

I then made a manual change to Inline::CUDA::Utils::find_dependencies(). At line 181:

#my $cmd = "${aexe} --version"; my $cmd = "\Q${aexe}\E --version";

A Makefile was successfully created:

$ perl Makefile.PL find_dependencies() : report: cc : /usr/bin/cc cxx : /usr/bin/c++ ld : /usr/bin/cc nvcc : /cygdrive/c/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v +11.4/bin/nvcc nvcc-release : 11.4 nvcc-version : V11.4.48 nvlink : /cygdrive/c/Program Files/NVIDIA GPU Computing Toolkit/CUDA +/v11.4/bin/nvlink nvlink-release : 11.4 nvlink-version : V11.4.48 Makefile.PL : configuration written to '/home/ken/local/opt/perl/inlin +e_cuda/perl-inline-cuda-master/config/Inline-CUDA.conf'. Generating a Unix-style Makefile Writing Makefile for Inline::CUDA Writing MYMETA.yml and MYMETA.json

No problems with make:

$ make cp /home/ken/local/opt/perl/inline_cuda/perl-inline-cuda-master/config +/Inline-CUDA.conf blib/lib/auto/share/dist/Inline-CUDA/Inline-CUDA.co +nf cp lib/Inline/CUDA/Utils.pm blib/lib/Inline/CUDA/Utils.pm cp lib/Inline/CUDA.pm blib/lib/Inline/CUDA.pm cp scripts/nvidia-ml-test.pl blib/script/nvidia-ml-test.pl "/home/ken/perl5/perlbrew/perls/perl-5.34.0/bin/perl.exe" -MExtUtils:: +MY -e 'MY->fixin(shift)' -- blib/script/nvidia-ml-test.pl Manifying 1 pod document

I did have problems with 'make test'. There was a huge amount of output: too much to post here and I don't have time to look through this closely right now. Some look like previous problems caused by spaces and can probably be just as easily fixed. For instance:

/bin/sh: /cygdrive/c/Program: No such file or directory make[1]: *** [Makefile:352: _011_basic_t_f5ac.o] Error 127

The "Author Tests" were handled as I indicated above, so no issues with my potentially dodgy Test::CheckManifest:

t/manifest.t ....................................... skipped: Author t +ests not required for installation t/pod-coverage.t ................................... skipped: Author t +ests not required for installation t/pod.t ............................................ skipped: Author t +ests not required for installation

This further strengthens my argument for not including those Test::* modules as prerequisites. I also note that Test::Pod::Coverage is not mentioned in Manifest.PL but is used in t/pod-coverage.t.

The final few lines of make test output were:

Files=20, Tests=13, 59 wallclock secs ( 0.03 usr 0.01 sys + 14.46 cus +r 16.56 csys = 31.06 CPU) Result: FAIL Failed 15/20 test programs. 0/13 subtests failed. make: *** [Makefile:926: test_dynamic] Error 255

It's Friday morning in my timezone and I have $work today. I can probably spend more time on this over the weekend.

— Ken

Replies are listed 'Best First'.
Re^4: Inline::CUDA : looking for feedback
by bliako (Monsignor) on Jul 30, 2021 at 06:02 UTC

    kcott, the immediate cause is that C/C++ compilers are not detected. And possibly errors because of more quoting problems.

    it's obvious that I need to handle quoting commands and parameters properly. I will follow anonmonk's suggestion if your \Q \E proves not robust enough.

    System C and C++ compiler must be detected properly. For locally-built perl.exe finding the default C-compiler is as easy as $Config{cc} but for C++ is not. For downloaded perl.exe, I have no idea where $Config{cc} will point to! I will also make a basic/preliminary test to check quotes and to check compilers.

    Secondly, I have no idea how to affect Makefile.PL vars in windows (for specifying CC,CXX,LD), in the same way I do it in bash:

    CC=/usr/local/gcc84/bin/gcc84 CXX=/usr/local/gcc84/bin/g++84 LD=/usr/l +ocal/gcc84/bin/gcc84 perl Makefile.PL

    A workaround is, after perl Makefile.PL, to try to modify config/Inline-CUDA.conf

    If all else fails I will have to resort to prompting the user during installation. Any opinions on that style?

    Thanks for your time spent and valuable feedback. I am not in a hurry, please work on it in your own pace.

    bw, bliako

      There's 15 failed tests ("Failed 15/20 test programs."): I'll need to go through each to find the actual problems.

      If all are simply quoting uses, then the solution is straightforward (if a little tedious to implement across the board). Let's see how that pans out before worrying about config modifications and the like.

      I recall you specified a minimum Perl version of v5.6. I've checked https://perldoc.perl.org/5.6.0/functions/quotemeta and https://perldoc.perl.org/5.6.0/perlop; quotemeta, \Q and \E are all available in that version, so no issues there.

      "If all else fails I will have to resort to prompting the user during installation. Any opinions on that style?"

      As I said, let's wait and see if secondary options are necessary; however, my general "opinions on that style" are:

      • I don't, as a rule, like it. On a number of occasions, I've started what I expected to be a fairly long-running installation, gone off to do something else while it runs, then returned ten or so minutes later only to find that, two seconds after I walked away from the screen, it prompted: "Specify location of whatever [/usr/bin/whatever]?" or something similar.
      • There are cases where it simply can't be avoided.
      • I have (reluctantly) used it myself in production code.
      • Where it has been necessary, I have tried to indicate early on that user intervention will be required.
      "Thanks for your time spent and valuable feedback. I am not in a hurry, please work on it in your own pace."

      No worries. I am rather eager to see how this works.

      I have, in fact, spent most of this week (at $work) modifying modules and repeatedly going through the make cycle, tweaking tests to reflect the changes. While I'm happy to do this, "All work and no play makes Ken a dull boy.", so I need a break. As I said this morning, I can probably look into this further over the weekend. [Current lockdown restrictions means that I'm not even allowed guests in my home, so I'm certainly free to do this work.]

      — Ken

        I have updated the github repository with a new version which uses ShellQuote::Any to do (some of) the quoting. I have also added a test which tries to run some perl-scripts with space in their name. That's in t/002-spaces-spaces-spaces.t.

        I have also added another repository: https://github.com/hadjiprocopis/perl-nvidia2-ml which takes over nvidia::ml, fixes its compilation errors and adds 1 more binding. If you install this first, it will be easier to enquire the GPU in current module. It requires nvidia-ml binary library which must be in the toolkit you downloaded. If you can't install that then it's not a problem but 1 test will fail (t/001-enquire-nvidia-gpu.t).

        I suggest you run perl Makefile.PL and if successful, then inspect config/Inline-CUDA.conf to see if cc, cxx, nvcc, ld are all set to valid names (unquoted).

        Following that, run just a few tests: t/000-require-modules.t, t/002-spaces-spaces-spaces.t, t/008-config.t, t/011-basic.t . The last two are compiling CUDA code. If all goes well, then it's likely that all tests will succeed. 031-matrix-multiplication-pass-parameters.t does a matrix multiplication with data passed from the Perl script and results returned back. That's a good base for any CUDA code you may want to write.

        The prompting during installation is only for windows, the user must somehow indicate where a c-compiler is if not in path already.

        BTW, you will be using a newer version of nvcc than mine. So a lot can go wrong there too.

        bw, bliako

Re^4: Inline::CUDA : looking for feedback
by bliako (Monsignor) on Jul 30, 2021 at 08:06 UTC

    apropos Test::CheckManifest being a requirement, I was just scanning all files for external packages and I saw that in t/manifest.t created by module-starter

    #!perl use 5.006; use strict; use warnings; use Test::More; unless ( $ENV{RELEASE_TESTING} ) { plan( skip_all => "Author tests not required for installation" ); } my $min_tcm = 0.9; eval "use Test::CheckManifest $min_tcm"; plan skip_all => "Test::CheckManifest $min_tcm required" if $@; ok_manifest();

      The issue isn't so much that module-starter adds a use Test::CheckManifest to t/manifest.t; rather, it's that you added Test::CheckManifest under PREREQ_PM in Makefile.PL (module-starter doesn't do that).

      I was questioning the inclusion of Test::CheckManifest in Makefile.PL, not its presence in t/manifest.t. I could tell (by inspection) that Test::CheckManifest in Makefile.PL would be a problem: I force-installed Test::CheckManifest to circumvent that issue.

      Take a look at "Re: Perl tools for making code better" for an example of my equivalent of manifest.t, along with some related information.

      — Ken

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: note [id://11135499]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others contemplating the Monastery: (1)
As of 2021-10-18 01:22 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?
    My first memorable Perl project was:







    Results (72 votes). Check out past polls.

    Notices?