Your question is broader than Perl version: one also needs to choose what environments the code should work in. You may choose, for example, to target *nix and Windows only, so as to simplify the code (e.g. can use / as path separator without going to the bother of hauling in File::Spec).

My biggest beef is that many authors seem to not consciously, explicitly and clearly state their module's version and environment requirements in the module's documentation. In addition to the documentation, they should enforce it programmatically -- with require 5.005_03, for example.

Which Perl version and environment to target depends largely on the type of module. For example:

• At work, where I can precisely control the Perl environment, I generally target perl 5.8.7+, *nix and Windows only. That allows me to learn and leverage the latest Perl features and simplify the code.
• For CPAN modules, it's different. Modules that are expected to be needed in a broad variety of environments, such as Test::More, should target pre-historic Perl versions and all Perl environments. "Developer" modules, on the other hand, such as Devel::Cover, may reasonably choose to target only recent perl versions. You need to decide who the target users of the module are and whether it's reasonable to ask them to upgrade to a later perl version in order to use your module.

Update: As for testing for backwards compatibility, I install a standard perl 5.005, perl 5.6, perl 5.8 on my development system (with no extra modules). Then, before uploading my CPAN module, I run a little script to unpack the tarball and run "make test" against all these versions. More elaborate automated schemes are obviously possible, but this straightforward test should catch most backward compatibility (and dependency) blunders before you upload the module.