It will be the module author's job to test everything in all the environments.

I can't imagine any author doing that - I certainly won't be.
Mostly, I just test locally on Windows, Ubuntu, and freebsd - and hope for the best.

The thing that usually brings me undone is that I tend to forget that some people like to run old, buggy stuff - be it perl itself, or a prerequisite perl module, or a prerequisite C library.
Such oversights generally go undetected until the "right" testing environment turns up.

Recently, I wrote a test script for Math::GMPz that required Math::MPFR.
Naturally, the availability of Math::MPFR was first specified .... but I failed to also specify that the version of Math::MPFR had to be less than 18 releases out of date. That is, current version is 4.25, 4.07 is minimum required :-(

Without cpantesters, my life wouldn't change much.
I'd still be simply sitting back, fixing any issues that are brought to my attention.
The main change that the demise of cpantesters would make for me is that there would be fewer such issues brought to my attention.

Cheers,
Rob