"If this is for distribution to CPAN, then personally I would try and stay away from anything random - there are lots of different systems out there, and why risk literally random test failures?"
I wholeheartedly agree with this. Literal random failures.
The primary reason isn't because a user might run into random failures, because all they have to do is re-run the installer and it'll likely pass the next time, but I know for fact that the maintainers of some of the Tester platforms do spend time investigating why distributions fail, then spend more time contacting the author to inform them as to what's broken and oftentimes spend yet even more time sorting out and offering solutions on the fix.
I've been the recipient of dozens of said kind emails over the years. I would cringe to think I'd wasted their time after I uploaded tests that would be known to false-negative fail on me.