I'm a firm believer of testing every single change against the entire codebase. Here's what I'd recommend:
- immediately get your code into a version control system
- write some basic tests for all subs. Don't worry about edge cases initially, the thinking here is that in the interim, you'll just want to spot if a change breaks basic functionality
- after all subs are done, use Devel::Cover, and go through the parts that don't have tests (line-by-line, condition-by-condition etc is what I do), and try to hit every scenario
- now that you've got a baseline, go back and add more tests for your subs... make them very obscure (edge cases). Throw bad arguments, too many arguments, arguments out of order etc. Effectively, you literally want to try to break each sub
- now, you've got a very strong foundation, you can rest a lot easier knowing that you've got your bases covered for code changes. At this point, I'd go back to revisit the Devel::Cover results, and focus on changing the code where it is impossible to trigger certain items to make them reachable
- you now have your base test platform. Now using a VCS hook as others have mentioned, throw it at a test platform. Recently, I wrote RFC: Continuous Integration (CI) for your CPAN modules (for free!), which explains doing automated testing (Continuous Integration, aka CI) using tools online. Note however, that for private software, nominal fees can be paid and you can use the same procedure, and never have to worry about setting up a repository infrastructure or a testing infrastructure. It just works. *All* of your tests are run automatically on each push to the VCS, so you know immediately if something's broken
Now, get into the habit of writing new tests for new functionality (or functionality changes) before you write the new code. I do this so I can work out how the new functionality should behave at a basic level. Of course, all these tests will fail until you get the code written. You'll also end up changing/adding tests as you go, which is a good thing. At minimum, as soon as you write something new (or change something), immediately afterwards write tests (and documentation). Don't put it off, or it just won't get done.
I wrote a script a while back which I'm currently changing into a CPAN module that automagically sets up a full build/test platform using Perlbrew (and Berrybrew on Windows), using VCS hooks, where you can specify different perl versions, have the software select the versions randomly, or test on all perls. Each run builds the entire Perlbrew infrastructure from scratch, so each build is tested in a pristine environment. Later today, I'll at least share the script (I'll post it in Cool Uses for Perl and link to it in this thread). I don't expect the module to be done before next week (at minimum).