|
in reply to And now write 100 times: "I'll automate our test-environment"
Automation can never hurt, but it is never a replacement for real human testing. And you can easily get into situations that are not cleanly testable -- scalability of large distributed apps, machine dependancy, kernel dependancy, timing issues only seen in real world apps, whitebox mentality of end users, firmware or driver involvement in some cases, etc, can result in test automation that requires a huge farm of available hardware. It depends on what kind of code you write, I guess, but in my case (at least), I see a lot of code (not just Perl) that does not fit cleanly into automated test frameworks. Moral of the story -- know when to use automated testing, as well as when automated testing is not the only solution. Obviously having an organization that doesn't agree to spend cycles in Q/A is a bad one in need of fixing!
|
Re: Re: And now write 100 times: "I'll automate our test-environment"
by Tomte (Priest) on May 13, 2004 at 09:15 UTC
|
Automation can never hurt, but it is never a replacement for real human testing.
As a general statement: you're of course right, but automatic testing is a real help in focusing your manual testing on broken things while fixing them. If we had a test-suite for the part I enhanced, I had noticed I broke it, and had a clear indicator as to where to look; we have automated builds in place, that inform you of errors/success in building a clean, freshly checked out version after you checked in new code (builds are done every ten minutes, if necessary); we should strive to do testing there too.
It depends on what kind of code you write, I guess, but in my case (at least), I see a lot of code (not just Perl) that does not fit cleanly into automated test frameworks.
This is the case here too with nearly all "legacy code"¹; you write different code, if you write with automated tests in mind; but I think this is a GoodThing(tm) -- and we should at least try to get to a point, where newly written code could be tested...
know when to use automated testing, as well as when automated testing is not the only solution. Obviously having an organization that doesn't agree to spend cycles in Q/A is a bad one in need of fixing!
Obvoulsy right :)
See my update to the OP; we will see if we can agree to something to assure more quality...
regards,
tomte
An intellectual is someone whose mind watches itself.
-- Albert Camus
| [reply] |
|
|
Perhaps you think this is a "legacy" issue, but it's not. Write some more systems-oriented code (not neccessarily simple Perl scripts or even Perl applications) and you will see what I mean. Anyhow, if you don't understand my caution, it doesn't apply to you (yet)... which is ok.
As for running builds as part of the test suite, umh, the developers had better run the builds themselves, else you have a serious issue of incompetance to deal with if they think they can check in uncompiled code! At least nightly builds are a very good thing, of course.
| [reply] |
|
|
Perhaps you think this is a "legacy" issue, but it's not. Write some more systems-oriented code (not neccessarily simple Perl scripts or even Perl applications) and you will see what I mean. Anyhow, if you don't understand my caution, it doesn't apply to you (yet)... which is ok.
Huh? Maybe I did again say things I didn't want to say (english is anything but my mother-tongue); but I didn't meant to contradict you....
As for running builds as part of the test suite, umh, the developers had better run the builds themselves, else you have a serious issue of incompetance to deal with if they think they can check in uncompiled code! At least nightly builds are a very good thing, of course.
So congrats for never forgetting to check in everything, and for never producing incompatibilities against code-changes a co-worker did make simultanious to yours, on different files... :-P
that a thing builds on your system doesn't mean it builds freshly checked out. And testing without a fresh build from a freshly updated <SourceControlSystemOfYourChoice>-snapshot wouldn't be proper testing IMO. If such a thing happens here, the developers in question get mails, including all error-messages and all changes with commit-comments done since the last build at most 15minutes after the checkin, very sensible and cool thing to do, really! (we're using CruiseControl, maybe there's a perl-solution out there I don't know of...)
regards,
tomte
An intellectual is someone whose mind watches itself.
-- Albert Camus
| [reply] |
|
|
|
|
