The consensus is anything that produces output compatible with Test::Harness::TAP. Usually, that means any testing module built on Test::Builder will work. Test::Class is a nice testing framework that's both Perl- and xUnit-ish. (It works nicely with Test::More and the other modules built on Test::Builder.)

Hm, yes, in the course of my digging around I came across that and it looked like it might be a good fit for what I want to do. I'll try playing around with it and see if it feels like the natural choice for me. Thanks for the tip.

I'll try playing around with it and see if it feels like the natural choice for me.

If it doesn't feel like a natural choice for whatever reason I'd love to know the details so I can consider improvements :-)

Is there any real consensus in the Perl community with respect to how unit testing ought to be done?

"Ought to be done" conflates two separate ideas: The first is the approach one takes to unit testing (e.g., TDD with or without mock objects, test-specific subclasses, etc.). Since the choice of approach is best dictated by the thing you're trying to build, and we're all building different things (albeit with some common themes), consensus here on a single approach is unlikely.

The second idea is about the specific testing tool or framework. A lot of unit testing in the non-Perl world uses some variant of xUnit, though the use of the tool may be less interesting than the choice of whether to mock, subclass, or whatever.

Which module, were I to use it, would cause the least scoffing from those in the know when it comes to unit testing Perl code?

I think you'll find that the scaffolding depends more on what you're trying to test than it does on your choice of testing framework. Test::More is a decent starting point. At work we use a custom subclass of Test which we're reasonably happy with. (And we have >12,000 unit tests.)

The fact that there is no well accepted standard is quite discouraging. Said lack makes it very difficult to do integration testing across modules, unless you could at least get everyone to agree to a common harness.

You're apparently seeing a problem here that I'm not. What do you mean by "integration across modules", and how does that vision depend on having a single test harness?

Edited to add: The part I find disappointing is that unit testing still isn't widely used.

Well, the kind of problem to which I was alluding is that to which mosquito refers to here.

And yes, it is extremely disappointing that so many people set about writing code without a formalized process for doing automated tests. I just finished graduate school and started a new job. On Friday I was introduced to an enormous blob of Perl code, entirely functional, completely undocumented, devoid of unit tests, and with data hard coded into it. I'm trying to figure out how best to suggest that I ought to be allowed to rewrite the thing. Perhaps I ought to suggest first that in order to learn how it works that I write some tests for it, and perhaps write some documentation as I figure out stuff, a Stone Soup strategy if you will. Oh, and did I mention that it's not in version control? The guy who developed it just takes occasional snapshots of the code.

So, basically I'm trying to figure out how to hijack this project and set it right, all without stepping on anyone's toes. I've got my work cut out for me... Or maybe I should just steer clear of it and work on something of my own from scratch. I seem to have a lot of flexibility in that regard, as developers in this group are basically just set free to find interesting problems to tackle. Maybe it'd be better to just lead by example than to try to beat a pre-existing project into good shape. I don't know... Entering a new group is a tricky and delicate process.

I'm starting to feel that it is my lot in life to be a Jonny Appleseed of the software development world.

So, basically I'm trying to figure out how to hijack this project and set it right, all without stepping on anyone's toes.

"Draining the swamp" problems are fun, especially when there are people still living in the swamp.

If you're starting from zero (tests), I'd suggest starting with strictly functional ones. If it's a web app, reach for WWW::Mechanize and Test::More (or Test::WWW::Mechanize, which I haven't used yet but am told is worth the effort), and figure out how to write functional tests that drive the application through various scenarios. If you're not dealing with a web app, the idea is the same.

Then, when you go on a cleanup binge, start by writing functional tests to cover what you're about to change, then see if you can get under the covers, isolate the code you're working with, and write some unit tests for it. The combination of functional and unit tests will give you some comfort as you restructure and refactor.

If the swamp dwellers don't want you writing unit tests, and won't be led gently in that direction, staying at the functional level is better than nothing.

On Friday I was introduced to an enormous blob of Perl code, entirely functional, completely undocumented, devoid of unit tests, and with data hard coded into it.

Then these nodes may be of interest:

Becoming familiar with a too-big codebase?
Analyzing large Perl code base.
What is the best way to add tests to existing code?

Friday I was introduced to an enormous blob of Perl code, entirely functional, completely undocumented, devoid of unit tests, and with data hard coded into it. I'm trying to figure out how best to suggest that I ought to be allowed to rewrite the thing.

If you don't already have a copy go get Perl Medic. Lots of useful tips for attacking this sort of evil task.

I don't have an answer for you (beyond offering one single data-point: I use Test::More).

But what you describe sounds to me like a special case of a general situation. I'm referring to the fact that typically users of CPAN have little to go by when confronted with multiple alternatives. (There is a rating system in place, but most distributions get too few votes for it to be informative.)

Or maybe you are going through a TMTOWTDI-induced existential crisis. :)

I often find myself grinding my teeth over TIMTOWTDI. I'm OK with it when the individual solutions are fairly orthogonal and tailored to a certain subset of the problem domain, and thus perhaps better suited to particular scenarios. However, for something as central to the software development process as unit testing, it strikes me as unfortunate to have a hodge podge of different solutions, many of them unofficial.

As a foil, I very much appreciate that Ruby has a single standard Test::Unit module that apparently everyone uses. It comes with the standard distribution, and it seems reasonably complete and well architected. It's sad that Perl does not have something similar. TIMTOWTDI is a double edged sword, the consequence of which is often that people end up speaking very different dialects of the same language, to the point that communication is stifled, or at least stilted.

As a foil, I very much appreciate that Ruby has a single standard Test::Unit module that apparently everyone uses. It comes with the standard distribution, and it seems reasonably complete and well architected. It's sad that Perl does not have something similar.

Perl does :-)

Perl's standard testing framework is Test::Harness, Test::Builder and Test::More. These have all been core (i.e. they come with the standard distribution) since 5.007003, long before Test::Unit was a core part of Ruby.

Indeed, Test::Unit is a fairly recent addition to Ruby, we were all using RubyUnit not too long ago.

Anybody who is writing test suites and not using Test::Builder based modules is (unless they are dealing with a legacy test suite) making a mistake.

IMHO anyway :-)

If you're doing testing work in Perl I strongly recommend that you join the perl-qa mailing list and take a gander at the material on http://qa.perl.org/.

TIMTOWTDI is a double edged sword, the consequence of which is often that people end up speaking very different dialects of the same language, to the point that communication is stifled, or at least stilted.

The bewildering array of testing modules on CPAN is actually less complicated than it looks.

The majority of testing modules on CPAN are built with Test::Builder and work together with the core Test::More. They're not competitors - they're specialists.

Modules like Test::ISBN, Test::Pod::Coverage and Test::Exception are not alternatives to the core testing framework - they're extensions. They just make testing (in this particular instance) ISBN numbers, POD coverage and exceptions easier than with Test::More.

The difference between Ruby and Perl testing frameworks is really down to more people using Perl for testing work over a longer period of time, rather than TIMTOWTDI. One of the few things I miss from Perl in the Ruby world is the excellent specialised testing tools that Perl gives me.

I think it's fair to say that by far the majority of people use either Test::More, something else that produces TAP-compatible output, or one of the many modules you can layer on top of Test::More itself.

TIMTOWTDI does have benefits that make up for the simplicity a single common module might provide, and that includes getting access to newer testing methodologies that are (arguably) better than xUnit in various cases.

For example, take a look at Test::Inline. This lets you write your tests inline in your modules, right next to the code that it tests, and then run these tests in the appropriate order.

Improvements like this may not have been possible were we limited to one specific package.

To summarise, TAP (Test Anything Protocol) is the standard. Test::More is the most common module used to produce TAP, but you are welcome and encouraged to use any of the dozens of other modules to produce TAP, as the situation dictates.

I agree wholeheartedly. I first learned Perl and love the core language but found TIMTOWTDI to be a real curse when weighing up CPAN's dizzying array of options. I think this is why Perl lost so much web development mindshare to PHP. Maybe it would have helped if the Perl community had put together something like Bundle::WebDevelopment without all the MVC madness and any dependence on mod_perl which many ISPs won't touch. I wince when having to write PHP, having learned Perl first, but there's often no alternative and I put this largely down to Perl's TIMTOWTDI philosophy.

As indicated here and here, Test::Unit appears to have been abandoned. Note that, unlike Perlish-mainstream Test:: modules, Test::Unit is not based on Test::Builder.

A common mistake I've seen from folks coming from a non-Perl xUnit background is to use Test::Unit in preference to the Perl-mainstream and much more widely used Test::Builder/TAP/Test::More/Test::Class set of modules ... only to regret that decision later. Have a read of the "Other Modules for xUnit Testing in Perl" section of the Test::Class documentation, which compares Test::Class with other xUnit-style Perl testing modules (including Test::Unit and Test::SimpleUnit).

Yes, I was gradually getting the impression that using Test::Unit might be a mistake, even though at first glance it seems like the natural choice if you are xUnit inclined. Does the fact that it neglects to base it itself on Test::Builder mean that it does not play nicely with the stadard testing phase of CPAN style module distributions? If Test::Unit seemed reasonably close to what I wanted, what is the closest thing that behaves in an appropriately Perlish fashion?

Does the fact that it neglects to base it itself on Test::Builder mean that it does not play nicely with the standard testing phase of CPAN style module distributions?

Not being based on Test::Builder makes it difficult to extend or integrate with other CPAN testing modules. See the excellent documentation that comes with Test::Class for more details.

If Test::Unit seemed reasonably close to what I wanted, what is the closest thing that behaves in an appropriately Perlish fashion?

Test::Class. However, don't forget plain ol' Test::More in cases where Test::Class may be overkill.

Does the fact that it neglects to base it itself on Test::Builder mean that it does not play nicely with the stadard testing phase of CPAN style module distributions?

Yes and no.

T::U can be made to output TAP (using Test::Unit::HarnessUnit) which means Test::Harness can understand it's output, so you can integrate it into the normal CPAN testing phase.

This disadvantages are that:

• It doesn't support all of TAP - so you cannot use things like SKIP and TODO tests if you want to
• Since it is not based on Test::Builder it doesn't integrate with all of the other Test::Builder based modules on CPAN - so you may end up reinventing some existing wheels
• Since it's not a core module (unlike Test::More) you have an extra dependency to install.

Well - that and the fact that its huge, overcomplicated and pretty much unmaintained :-)

If Test::Unit seemed reasonably close to what I wanted, what is the closest thing that behaves in an appropriately Perlish fashion?

Test::Class - but I would say that :-)

If Test::Unit does what you want, but sucks, then perhaps you could consider taking it over, upgrading it to use Test::More or some other TAP-based module, and improving it as you see fit?