http://qs321.pair.com?node_id=401382

There's something that's always bothered me about unit tests. Basically, it's because I'm ridiculously pedantic, and it grates on my nerves when people say things like:

"If your refactored code passes all of its unit tests, then you haven't broken anything."

That's not true: if your refactored code passes all of its unit tests, then you haven't broken anything in the ways that you've thought to test for. Most of my code has to pass unit and integration tests (the bits that don't are difficult to test; real-time graphics stuff, mostly), but every time I change my code I get a little itch in the back of my brain that says, "What if you've introduced a bug that your original tests didn't check for?"

I don't think this is an unreasonable what-if. For instance, I might write a simple algorithm to deal with a data set that I don't fully understand. With more experience, I understand the data better and write a more complex algorithm to deal with it. Writing that more complex algorithm, I introduce new corner cases that I hadn't considered beforehand, and I probably introduce a bug at one of them. The code passes the unit tests fine -- those unit tests don't check the new corner cases -- and the first I see of the new bug is when my code segfaults in front of a visiting grant-agency representative.

Seems like the answer is to refactor (or at least reconsider) the tests every time you refactor the code. That sounds at first like a heavyweight solution, but it might just be my natural laziness coming to the fore. How do you folks deal with this issue?

--
Yours in pedantry,
F o x t r o t U n i f o r m

"Lines of code don't matter as long as I'm not writing them." -- merlyn