Unit Tests As Complexity Sponge
Listen to this article
A number of people have variously commented that unit tests may in fact be more about design than actual testing. Many others (the links elude me at present) have also complained about the undue burdon imposed by a large number of unit tests and that because of this, and other very sound reasons, they prefer acceptance tests. If I was ever in any doubt about the importance of acceptance tests, I was certainly convinced after the last project where acceptance tests would fail where no unit test had, due no doubt in large part to the fact that the acceptance tests also acted as integration tests.
One thing I did notice however was that over and above their usefuleness as a design tool, unit tests seemed to act as yet another positive constraint helping reduce the overall complexity of the code. Because developers were forced to write unit tests, they were forced to produce relatively simple, testable code. Much simpler, I believe, than would have been the case otherwise. The down side to this testability was that in many cases, the corresponding unit tests were rather more complex than we would have liked. And, as noted previously, complex tests tend to be brittle and this has a knock on effect with respect to maintenance. But does this really matter?
Ultimately what is important is working software (for which you have acceptance tests) and clean, easy to understand code that is hopefully cheaper to maintain. You could choose to throw away all those "dirty" unit tests once you reach production and rely solely on your acceptance tests; or you might choose to buy new ones through refactoring/re-writing; or you may decide that the unit tests are worth the extra effort to maintain.
Whatever the course of action, it seems to me that, yet again, unit tests have benefits beyond simply (or not as the case may be) producing "correct" code.
Comments
My big problem with unit tests--or more specifically JUnit tests--is that you have to promote methods to package visibility in order to write them. This is fine if the class/method should be at least package visible to begin with, but that is often not the case. So unit tests force you to expose implementation details within the package that should be hidden.
Perhaps Tiger's annotation can be used to help solve this problem. TestNG would have to lead the way there, because the JUnit code base is stone cold dead, despite the bugs, quirks, and pathetic documentation. Perhaps Mustang's friendly packages concept will be designed with this problem in mind.
My point is that often the transparency of function imposed by unit tests makes the code worse from a user interface (API) viewpoint. This leads to misused APIs and brittle code.
Posted by: Anonymous | November 11, 2004 09:52 AM
I agree that it _can_ lead to worse APIs. I find it makes the APIs I write, better. Though I realise my argument is hardly scientific nor particularly objective ;-)
Rickard had an interesting discussion related to this: http://jroller.com/page/rickard/20040814
Posted by: Simon Harris | November 11, 2004 10:38 AM
What I find when using JUnit is that the desire to expose the internals of an object is a smell that another object is needed; in essence, the object under test has been given too much responsibility and needs to share the load around.
I don't always do it. I probably should do it more. When I do do it, it nearly always ends up in cleaner, better factored code.
Posted by: Robert Watkins | November 11, 2004 08:34 PM
Package protected is not _too_ bad, but yes, it may force you to expose things more then you otherwise would...
I had a look at a commercial tool called "agitator" which got around this trick by working on copies of the source code, and exposing its "privates" (snigger !) in its own copy of the source. This is as the tool was really designed to build coverage over existing code bases (that were not designed with the benefit of TDD)... an interesting approach (no I am not recommending the tool).
I think the problem that things like TestNG have is that they don't get popular. The fans/users of these fantastic tools and frameworks rave about them, but for some reason they don't become mainstream, and people persist with the flawed but workable mainstream stuff (like struts for instance). No surprise, it is the real world after all.
I find it interesting that so many people are leaping into the annotation features of J2SE 5, but the mainstream app servers don't support it yet - do they develop and unit test with 5 and compile for 1.4?
Posted by: Mic | November 11, 2004 08:47 PM
Sorry but i don't understand how unit tests make you promote methods to package level visibility. Also, how do they make you expose implementation details? As Robert suggests, exposing implementation details is a pretty stinky smell! I don't think we need TestNG or Java 1.5 to "work around" these problems. Again, as robert suggests, break down the responsibilities, maybe there is a collaborator or 2. Use mocks to define the interactions and follow the basic OO principle of Tell Don't Ask.
Posted by: Damian Guy | November 11, 2004 10:02 PM
I agree that you shouldn't need to test private methods, they should be counted as part of the method that calls them. However, I had a case recently where a particular public method on my class was quite difficult to test due to the use of some ugly underlying classes which I did mock but could not refactor. It required some repetitive calculations which I extracted locally in private methods and as the calling class didn't do much except call these calculations many times with different combinations, I wanted to test these calculations separately. Rather than expose them, I wrote a method in the tests superclass that changes the exposure of the method programmatically to protected and calls it via reflection. Not ideal, but refactoring this area is not an option and at least I tested the calculations fully.
One of the points I'm trying to get across to my team is that unit tests not only test code, provide a safety net, and help you design, they also force you to write simpler, more maintainable code with better use of OO - particularly if written first. (Ok, not so much for a decent developer but some of the code here is shocking, you wouldn't believe it if I told you). It's my way of trying to get them to stop coding as though Java is a scripting language (or at least, that's not what we're using it for!).
Posted by: smurfette | November 11, 2004 10:59 PM