http://qs321.pair.com?node_id=353017


in reply to Re: And now write 100 times: "I'll automate our test-environment"
in thread And now write 100 times: "I'll automate our test-environment"

Automation can never hurt, but it is never a replacement for real human testing.
As a general statement: you're of course right, but automatic testing is a real help in focusing your manual testing on broken things while fixing them. If we had a test-suite for the part I enhanced, I had noticed I broke it, and had a clear indicator as to where to look; we have automated builds in place, that inform you of errors/success in building a clean, freshly checked out version after you checked in new code (builds are done every ten minutes, if necessary); we should strive to do testing there too.

It depends on what kind of code you write, I guess, but in my case (at least), I see a lot of code (not just Perl) that does not fit cleanly into automated test frameworks.
This is the case here too with nearly all "legacy code"¹; you write different code, if you write with automated tests in mind; but I think this is a GoodThing(tm) -- and we should at least try to get to a point, where newly written code could be tested...

know when to use automated testing, as well as when automated testing is not the only solution. Obviously having an organization that doesn't agree to spend cycles in Q/A is a bad one in need of fixing!
Obvoulsy right :)
See my update to the OP; we will see if we can agree to something to assure more quality...

regards,
tomte

An intellectual is someone whose mind watches itself.
-- Albert Camus

  • Comment on Re: Re: And now write 100 times: "I'll automate our test-environment"

Replies are listed 'Best First'.
Re: Re: Re: And now write 100 times: "I'll automate our test-environment"
by flyingmoose (Priest) on May 13, 2004 at 13:31 UTC
    Perhaps you think this is a "legacy" issue, but it's not. Write some more systems-oriented code (not neccessarily simple Perl scripts or even Perl applications) and you will see what I mean. Anyhow, if you don't understand my caution, it doesn't apply to you (yet)... which is ok.

    As for running builds as part of the test suite, umh, the developers had better run the builds themselves, else you have a serious issue of incompetance to deal with if they think they can check in uncompiled code! At least nightly builds are a very good thing, of course.

[reply]

      Perhaps you think this is a "legacy" issue, but it's not. Write some more systems-oriented code (not neccessarily simple Perl scripts or even Perl applications) and you will see what I mean. Anyhow, if you don't understand my caution, it doesn't apply to you (yet)... which is ok.
      Huh? Maybe I did again say things I didn't want to say (english is anything but my mother-tongue); but I didn't meant to contradict you....

      As for running builds as part of the test suite, umh, the developers had better run the builds themselves, else you have a serious issue of incompetance to deal with if they think they can check in uncompiled code! At least nightly builds are a very good thing, of course.
      So congrats for never forgetting to check in everything, and for never producing incompatibilities against code-changes a co-worker did make simultanious to yours, on different files... :-P

      that a thing builds on your system doesn't mean it builds freshly checked out. And testing without a fresh build from a freshly updated <SourceControlSystemOfYourChoice>-snapshot wouldn't be proper testing IMO. If such a thing happens here, the developers in question get mails, including all error-messages and all changes with commit-comments done since the last build at most 15minutes after the checkin, very sensible and cool thing to do, really! (we're using CruiseControl, maybe there's a perl-solution out there I don't know of...)

      regards,
      tomte

      An intellectual is someone whose mind watches itself.
      -- Albert Camus

[reply]
        So congrats for never forgetting to check in everything, and for never producing incompatibilities against code-changes a co-worker did make simultanious to yours, on different files... :-P

        Our builds do notify users of breakages, however running builds is not the same as testing, I don't know how this got mixed in here...it's not related.

        And again, extra testing by way of machine is nice, but it's just that -- extra testing. It's nice if you can ad it, but sometimes all you can get is 20% (or less) coverage. Here, we have a lot of hardware specific stuff (RAID cards and what have you) and we'd be lucky to get 1% coverage on the 'software' components.

        Anyhow, no harm done, no offense taken, etc, etc :)

[reply]

In Section Meditations