This article mainly addresses the rest of us. If you’re not a TDD (test driven development) convert, yet your heart responds to the sirens of unit testing but you find yourself admiring the solution without cracking the nut, you may find it useful. Intermediate agile adopters may skim through as well. This isn’t a tutorial about how to write unit tests.
- If you’re not in the habit of writing tests, and you can’t get into it, the best time to write tests is when your code undergoes significant or overwhelming changes.
- Only write mocks or stubs if you find it easier than using actual classes. I know this may sound heretical or misleading. I’ll explain.
It won’t get us anywhere
Unless you start with generous faith and enthusiasm, getting started with unit testing (or worse, TDD) while hitting a new project won’t get you anywhere. My first experience with TDD and unit testing was starting with a weeny-tiny, critical yet project. Finally I put 3 or 4 classes under test (out of ~50). It was interesting but inefficient, and I also ended up doing 4 days overtime.
A unit test isn’t a footnote attached to a bug fix
The big plus with unit testing is that it forces us to design code in (usually) better ways. It doesn’t tell us how to do it. We’re left working it out, and this may turn out to be overly time consuming. Or it may piss you off and you’ll bury all the good stuff under a pile of curses.
In ‘legacy code literature’, unit tests are high on the list, along with suggestions to ‘write a test for every bug fix’. So I fix the bug and write a test and the critter won’t recur – job done. Right?
Uh-oh. The normal story is, as often as not this class isn’t even testable. It needs to be redesigned. That will affect dependencies with other classes (it is precisely because of these dependencies that the class can’t be tested), then you’ll be wanting to re-factor these other classes as well, and you’ll end up giving up, unless you actually call home and sign up for an overnight crusade. Sad but true, better designed code is less likely to break down, and easier to test.
The barbarian and the healer
The good news about badly written code is that it shouts for itself. You’ll be fed up, exhausted. You’ll be a dog that used their wise teeth gnawing a pile of skulls. And then…
You’ll be using the mace.
Maiming your code, changing, removing or invalidating methods, classes, sometimes entire packages. And then…
Well then, any good barbarian needs a healer. So after we made a mess for the good cause, we’ll be looking at the result bitterly, itching to sub-revert to whatever can-of-worms just needed a new expiration date hastily patched over it, and we’re ready for adding tests. Viz…
- We’ve already agreed to redesign our code for the better (better code) and the worse (overtime).
- We’ve entirely lost confidence into our codebase. Rewriting everything is a tempting (-ly dangerous) alternative and the prospect of running that code at all is simply frightening.
Am I drunk?
Some will claim that doubles (stubs, mocks, fakes… you name it) are inherently necessary when writing unit tests. I agree. Here’s why.
One foundational purpose of unit-testing is to ensure classes are well implemented.
Now let’s consider a class Acme depending on a class Foo. Suppose you write AcmeTest, carrying that dependency on Foo. Now say somebody changes Foo and AcmeTest breaks down. Well we’ve just proven that AcmeTest is testing Acme and Foo.
Among stories hailing the successes of unit testing, I’m sure you read something like ‘I made a minor change and a bunch of red lights fired up, signaling my change caused unexpected error in (apparently) totally unrelated classes’.
I’ll be surprised to hear that the guys witnessing ‘red lights in CC’ religiously used mocks or stubs. More likely, they had dependency chains ‘spoiling their tests’. These ‘half baked tests’, however, are still popular with some agile practitioners (read here. Yes, here!)
The bottom line is that well designed unit tests using mocks and stubs are testing less, less safely, than half-cooked unit tests depending on actual, production caliber collaborators:
- Tests using mocks and stubs are indeed asserting whether classes are correctly implemented. These only test interactions between classes to the extent that doubles are ‘locally faithful’ to the original. Foo can evolve independently from FooStub inasmuch as the original contract specified by Foo doesn’t change (mocks are even more flexible than stubs in this respect). Yea. Doubles are basically formal specs lying around. Like documentation, they can get out of date without anything breaking down.
- Tests using actual collaborators are non-local. We’re not testing just the ‘object under test’. We’re testing its actual collaborators (however indirectly) along with the net sum of interactions between the object under test and the underlying subsystem. This makes it potentially harder to tell what is breaking down. Yet…
Lazy’s Good. Less isn’t more.
Consider an ideal situation (Acme.com, IT division):
- We have enough time and discipline to maintain our tests. When a class implementation changes, we review stubs and mocks related to this class.
- We have a ‘full featured battery of tests’. Unit tests; component tests; acceptance tests.
In this situation, the advantages of mocks and stubs hold dearly true. Since we have component tests and acceptance tests, our unit tests need validate only class implementations, not their holistic sum. Since we have enough time and discipline, our stubs and mocks never get out of date.
Now consider a workable (if-ever stressful) situation (AgileKitten.com, $5,000 registered capital):
- We have 2% test coverage, courtesy Adjil Gik Jr (dude’s recently completely an internship with us, at least the test suite is up and running).
- We’re 8 weeks before release as we got this over-designed product that we just blew a year of angel funding on.
- The lead dev just decreed curfew (no leaving the office before 12pm) and the whole codebase needs to be ‘improved’.
In the workable, stressful situation, writing these stubs and mocks may just appear to be a waste of our precious time. We may end up writing mocks and/or stubs hastily and we don’t have component testing anyway, so how about writing a few of these half-cooked tests instead, since they double as integration tests?
Doing without mocks and stubs doesn’t just save time. It helps increase test coverage faster. Further, retrofitting a test using doubles likely will require more (re)design (just try with any class that instantiates its delegates). When coverage is low, we’re also likely to find collaborators are difficult to instantiate ‘in vitro’. The redesign effort involved in making it possible to instantiate collaborators without firing up the whole watchamacallit is the first step towards making these classes testable. Writing a stub won’t give you that.
Mock-ists often point out how tests written without doubles cause chain damage (lots of red lights) whenever anything breaks down. But if you’re only relying on unit tests, there are integration level errors that ‘true to form’ unit tests won’t catch. And by the way Mocks and stubs help masking dependency chains; here’s a design twist not encouraging us to modularize code.
The best time to write tests is when the shit hits the fan, and we lost confidence in our codebase. Mocks and stubs are neat, but we don’t usually need them to write tests. Half-cooked tests are easier and faster to write, providing more extensive coverage than mock-tests. True-to-form unit tests merely give the illusion of safety unless you have component testing and/or acceptance tests. By any means, write tests.
Acknowledgement and Disclaimer
This article has benefited explanations from an article written by Martin Fowler (Mocks aren’t Stubs, also quoted inline). Needless to say the views expressed in this article are mine. These views may appear radical, should be taken with a pinch of salt and merely track my slow progression up Agile Hill. Screw up your code at will, don’t blame me.