Where's The Problem?
Listen to this article
One of the biggest problems I see over and over again is the difficulty support and maintenance teams have diagnosing problems. It's unfortunate but developers have a knack of writing code for Happy Days scenarios - when the software works, it works well; when it fails, it can be a disaster.
A cow-orker and I were recently discussing the use of assertions in production code. He had previously been discussing the topic with one of his colleagues who had suggested that their existence was a smell; that it indicated a lack of testing.
Now, enough has been said on the topic of unit testing so suffice it to say that the great thing about unit testing is that it's easy to ensure your components work as advertised. You can even, and in many cases should, test what the behaviour would be given invalid input such as NULLs, etc. Then of course we move into integration, functional, acceptance, regression, etc. testing to prove that the application hangs together as a whole.
The problem is that tests don't necessarily prove that the software does what it's supposed to *GASP!*. Rather, tests prove that software works for the given scenarios and the assumptions made and the plain fact is that these do not always match reality. We may have the greatest, most comprehensive test suite in the known universe, but if it's testing the wrong things, it matters little. Sure, in a system over which you have complete control, high levels of functional/integration test coverage can compensate but even then, even 100% code coverage doesn't equate to 100% accuracy. In particular, at best, it is difficult to test for and therefore prevent clients of your code from passing invalid parameters. In fact if you've ever written and published a public API you'll know that, by definition, it's impossible.
The further into a system a problem propogates, the more difficult it becomes to diagnose and the greater the likelyhood of "damage". One of the major benefits of production code (as opposed to test code) assertions is that at run-time we can detect and prevent unexpected scenarios as early as possible, thereby preventing them from propogating. Maintenance developers and those familiar with the Fail Fast axiom will appreciate how important this is in a production environment.
Comments
One (okay, two) of the fallacies of 100% test coverage is that:
a) it doesn't test that the tests are right.
b) it doesn't test that the tests are complete.
TDD advocates need to remember this sometimes.
FailFast is also bloody useful. Worst part about diagnosing production errors is trying to work out if something is a symptom or a cause. NPEs are the ultimate example here ("Okay, so that's null. WHY was it a null?")
Posted by: Robert Watkins | November 19, 2004 05:15 PM
Your friend is a bigot and an idiot :-)
Both unit tests and asserts are important, and I certainly trust code that uses both much more than code that naively thinks unit testing is sufficient.
Posted by: Anonymous | November 20, 2004 01:09 AM
Personally I use assert statements all the time to catch situations that "couldn't possibly" arise, such as the default cases in switch statements. More than once the code has changed and the program has died with an assertion error which pin-pointed a bug which without the assertion statements would have been a real pain to find.
Posted by: gav | December 6, 2004 05:32 PM