I think if your tests are inapplicable, the best approach is to bail out from Makefile.PL with an exit code of 1 and the message "BAIL OUT!" (or something like that) on STDERR. Maybe it's also enough to just not create a Makefile. Have a look around at other modules that are OS-specific. For Win32::Wlan, I use the following boilerplate code in the tests skip all the tests unless they are applicable:

#perl -w
use strict;
use Test::More;

BEGIN {
    if ($^O !~ /Win32/i) {
        plan skip_all => "Win32::Wlan only works on Win32";
    } else {
        plan tests => 1;
    }
};

...
[download]