You might want to take a look at CGI::Test. It seems like it'll handle a lot of what you want.

However, I couldn't see how it'd be easy to setup a thorough tests for a really complex site. There are just too many ambiguities to think about. Spot-checking the results of a page seems a lot easier, especially when you're dealing w/ HTML templates, dynamic content, etc. If anyone's found a good way of managing this I'd like to know.