However I think that the fact that CPAN tests by default before it installs is one of the reasons that Perl's module system works so well - so I prefer to have maximal tests by default

I agree very much with this point. There are very few things out there like CPAN for any language. And without these tests (even if they are just smoke tests) I think CPAN would not have the reputation that it has for being a place to get good, reliable modules. Without running make test you would not know the module you installed was broken until you used it (and you would have little idea why it was broken too). Testing only increases the quality and relability of your code, anything that helps that is a good thing IMO.