The title of this post makes it clear what we're going to be talking about: how much code coverage should we write? Writing tests is a good thing; all developers in their right minds will agree. In fact, it's a very good thing.
So that means we should write code coverage for everything, all the time . . . right?
Not quite.
The Dilemma
On the one hand, writing tests is a smart way to prevent bugs and keep code robust. On the other hand, we all have limited time. There's always some deadline we must meet and some bug that must be fixed. Where do we find time to write tests on top of everything else we must do? Plus if a project is at a very early stage, the team will likely discard the codebase and start fresh. So what's the point of writing tests for something that is likely to be thrown out?
So here's the question we're going to try to tackle: Given limited time and resources, just how much code coverage should we aim for so that we can look at ourselves in the mirror and call ourselves professional developers?
You're probably not going to like my answer to that question, because the answer is: "It depends." In this article, I am going to cover all the common factors that influence how much code coverage is needed. Hopefully, with all the factors listed out, you can make a better judgment of how much coverage you really need in your codebase.
Consider Your Psychology and Maturity as a Developer
There's a nice story about code coverage, which I will paraphrase briefly. If you are here looking for a simple answer, even if there are actually no simple answers to this question, then you should simply aim for 80% code coverage and stop reading any further. Sometimes we just want a number to aim at or to give to management as a simple answer to a question that is not simple. Giving you the 80% number is my way of preventing you from doing more mindless yak-shaving. Stop trying to find the perfect answer and do the real work of writing some tests.
If you are a fairly new developer or you have a codebase that has zero tests, then you should forget about test coverage and simply write some tests. I've been there before: fretting too much about hitting some perfect code coverage number when I should have simply written some tests. Because at the end of the day, some tests is better than zero tests. Fretting about hitting specific numbers simply delays you from taking the necessary action to get started.
Once you reach a certain amount of test coverage and becoming deeply familiar with the codebase, then you can make a better judgment call regarding other factors which the rest of this article will cover.
Test What's Easiest (and Most Important) to Test
Let's say your code is using some framework. Your obvious focus for unit tests would be on code that contains the main business logic and the parts that are most easily tested—ideally, the parts that fulfill both criteria. For example, if you are using the popular web frameworks that employ the MVC paradigm, you would have learned that such frameworks encourage the writing of business logic inside the model layer.
Try to get the classes in the model layer to reach as close to 100% as possible. If not, at least aim for 90%. Get code coverage settings to focus only on these parts. Feel free to exclude the calculation of code coverage from other parts, such as the view and controller layers. In other words, aim for close to 100% in the crucial code and 0% for the trivial stuff.
Examples of Code to Test and Code to Ignore
To be more explicit, I suggest ignoring:
1. External libraries and frameworks, which typically come with their own unit tests.
2. Trivial code like getters and setters.
3. Presentational logic, like code that works only with the UI such as the view layer.
4. Database and database access logic.
These, on the other hand, are examples of crucial code:
1. Code that contains business logic (as mentioned earlier).
2. Code that has high usage by many other modules.
3. High-risk code that is more likely to generate bugs (which I will explore later).
4. Code that your team repeatedly changes.
It goes without saying: aim for at least 90% or even 100% on crucial code. The more points on this list a piece of code meets, the higher coverage I would demand of it.
Sometimes, you'll have code that meets criteria on both the ignore and crucial lists, like presentation logic code (ignore list #3) that happens to change frequently (crucial code list #4). In this case, I will ignore it. The rule of thumb for such cases would be: so long as a piece of code falls on the ignore list, don't include it in your code coverage. Don't worry. You still have plenty of codebase for unit testing. Focusing on the most important code is key for making your codebase robust.
As mentioned at the start, you have limited time, so you should start with the ignore list first.
Categorize by Risks
Brian Marick of Testing.com wrote this paper in which he mentions segregating your codebase by risk levels.
Here are his three categories:
High risk code could cause severe damage (wipe out data, injure someone, give non-obviously wrong answers that might cost a user a lot of money), or has many users (so the cost of even minor bugs is multiplied), or seems likely to have many mistakes whose costs will add up (it was a tricky algorithm, or it talks to an ill- defined and poorly-understood interface, or I've already found an unusual number of problems).
Low risk code is unlikely to have bugs important enough to stop or delay a shipment, even when all the bugs are summed together. They would be annoyance bugs in inessential features, ones with simple and obvious workarounds.
Medium risk code is somewhere in between. Bugs here would not be individually critical, but having too many of them would cause a schedule slip. There's good reason to find and fix them as soon—and as cheaply—as possible. But there are diminishing returns here—time spent doing a more thorough job might better be spent on other tasks.
As common sense dictates, you can aim for low coverage or even outright ignore the low risk code. Your time is better spent focusing on hitting as close to 100% code coverage as possible for the high risk code. For the medium risk code, just pick a number your team is comfortable with and aim for that; I'd choose 70%, but there's no real rationale for this number.
Another thing to note is that your evaluation of what constitutes low, medium, and high risk will evolve. As you continue development, you'll gain a better of understanding of what's truly risky and what's safe to ignore. Then you can tweak your priorities accordingly.
Conclusion
In conclusion, if you have at least two years of experience writing software, you will have a visceral understanding of how brittle your code can be. You will also understand that having high code coverage is not the silver bullet some managers (and even developers) think. There's always some nuances, and I hope this article will help you develop your own nuances.
If you are just starting out in your career as a developer, or if you've a codebase with zero tests, don't worry too much about coverage. Just start writing your tests.
For high risk and highly important code, aim for 100% coverage. But remember: the key word here is aim. You may not always reach that goal, but your code will be stronger for trying.
This post was written by KimSia. Kim Sia writes under the nom de plume T.J. Simmons. He started his own developer firm five years ago, building solutions for professionals in telecoms and the finance industry who were overwhelmed by too many Excel spreadsheets. He’s now proficient with the automation of document generation and data extraction from varied sources.
Subscribe
