We take some risks here either way. "perl" may not be the currently running perl (here, for example, "perl" is the stock perl that comes with the Linux distro I'm running - 5.8.0, while "perl5.8" is a symlink to the latest perl5.8 binary I've compiled - 5.8.5), while using $^X may also not be quite useful (since the currently executing code may be running in an embedded perl rather than a standalone perl executable - isn't that how mod_perl works?).
Personally, I'd just use require as others have pointed out. And if you don't want to use the extra memory, you can delete it from %INC afterwards - perl will then be able to re-use that memory. It does mean that you'll get a negative when the module exists and is found, but doesn't compile, but that's probably the same thing as not being there, really.