Beefy Boxes and Bandwidth Generously Provided by pair Networks
Clear questions and runnable code
get the best and fastest answer
 
PerlMonks  

Re^3: How to merge two different coverage reports remotely

by 1nickt (Abbot)
on May 26, 2021 at 09:44 UTC ( #11133050=note: print w/replies, xml ) Need Help??


in reply to Re^2: How to merge two different coverage reports remotely
in thread How to merge two different coverage reports remotely

Hi again,

"I test the code on different hosts, since it's depends on the environment."

That's one thing, and should be done for sure. But generating a test coverage report depends on the codebase, not the environment, unless you're using it in some way I've not encountered.


The way forward always starts with a minimal test.
  • Comment on Re^3: How to merge two different coverage reports remotely

Replies are listed 'Best First'.
Re^4: How to merge two different coverage reports remotely
by pryrt (Monsignor) on May 26, 2021 at 13:38 UTC
    But generating a test coverage report depends on the codebase, not the environment, unless you're using it in some way I've not encountered.

    I hadn't chimed in because I've never tried merging the coverage report. But I can think of lots of situations where the environment will influence which code can be covered in a given environment, and thus will influence the test coverage report.

    Here are three examples:

    # the coverage report on MSWin32 will be different on Win10 from Ubunt +u Linux. # I assume the OP would like to merge the two reports to make sure tha +t all the # functions are covered at least once between the two (or more) differ +ent # environments if($^O eq 'MSWin32') { run_this_function(); } else { run_a_different_function(); } # or # the coverage report is different in the environment where # MY_ENV_VAR is set true vs where it's false or undefined, # and the OP would like to merge the two reports to make sure that all + the # functions are covered at least once between the two (or more) differ +ent # environments if( $ENV{MY_ENV_VAR} ) { run_special_case(); } # different parts of the code are covered depending on whether some ot +her # executable is found, so that it can be run if( -x '/x/y/bin/my_executable' ) { my $ret = qx(/x/y/bin/my_executable); # cannot run on system X, wher +e repo is in /a/b/repo/run.pl, and /x/y/ hierarchy might not even exi +st! ... error processing ... $some_outer_variable += $ret; }

    In all of these situations, on system X the coverage will show certain functions or blocks of code run, but others not; and system Y may have a different group of functions or blocks that are run. So for example, maybe each system can run the same 80% base coverage, but system X covers 5% of code blocks that cannot run on Y and system Y has 15% of code blocks that cannot run on X; the individual reports say 85% coverage on X and 95% coverage on Y, when in reality, together they make 100% coverage, but neither report shows that. So the OP is asking how to merge those so that they can tell whether or not all lines of code are covered in at least one of the test systems.

      Personally, I would want to mock the environment differences and know that the tests were passing and the code covered before deploying.


      The way forward always starts with a minimal test.
        mock the environment differences before deploying

        I'm always afraid that something will be missed while mocking, or I'd get something wrong. So if I can run my tests on a system that's more like what the end user will have, I like to use it. For example, GitHub Actions, Travis-CI and AppVeyor allowing for testing on different OSs or on different environments with the same OS, even before you've deployed to the end-user machines. And in situations like that, I can see why merging the coverage reports would be nice (whether for verifying actual coverage, or for verifying that the mocked environment works the same as the real environment.)

        If mocking is sufficient for you, and you don't need to merge coverage reports; great. I still think it's an idea worth pursuing for how I understand the OP's situation.

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: note [id://11133050]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others meditating upon the Monastery: (2)
As of 2021-10-21 21:32 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?
    My first memorable Perl project was:







    Results (85 votes). Check out past polls.

    Notices?