Test Driven Development is one of techniques which should be a significant part of a hypothetical book “All things you should know to be a great software developer”. I use it most of the time for over three years.

TDD helps me achieve two great outcomes. Obviously, after each step I have tests for my code. These tests make me feel safe and relaxed when I’m pushing code to the repository. More importantly, TDD helps to create, thanks to the refactoring phase and small incremental steps, good design and good quality code.

Nevertheless, there’s something about TDD what’s not emphasised enough. TDD is only a framework within our mind can operate. It’s not a magic wand which change the way you write code. You’re the one who write good tests, and you’re the one who create good design and clean code. So if it’s only a framework, do we need it at all? And even tough, should we follow TDD rules whenever we write code?

In this article, by presenting my journey on TDD, I’ll try to answer this questions.

Discovering TDD

TDD has been introduced to me by a colleague about three years ago. We’re working on a legacy project in the public communication domain. The project has been moved from one country to another. As an outsourcing company we were responsible for coding. The new owners knew nothing about the code. Business rules were different than in the country when the project was originally written. The technical debt was way to high. The code coverage was around 30% (or rather 10% when taking into account good tests).

We decided to use TDD for new features and bug fixing. Later on we started to refactor the ugly code in crucial parts of the system by covering existing functionality with end-to-end tests using Concordion as a BDD framework.

We strictly followed these three holy laws:

1. You are not allowed to write any production code unless it is to make a failing unit test pass.
2. You are not allowed to write any more of a unit test than is sufficient to fail; and compilation failures are failures.
3. You are not allowed to write any more production code than is sufficient to pass the one failing unit test.

Tasks have been even reopened because of the fact that someone had written production code first. We’re meeting on coding dojos to learn the TDD process. We also had a great TDD training led by pragmatists.

After about a year we’ve achieved something what can be considered as a total success. We refactored the main parts of our system. It was easier to understand and maintain the code. Nearly 90% of our code was covered by mostly good tests. Our small classes looked pretty and clean. We even convinced our customer to define requirements as combinations of givens, whens and thens.

It wasn’t only the result of applying TDD. We’ve also learnt a lot about good OO design. SOLID & GRASP principles, design patterns and a lot of so-called best practices. We still had an anemic domain model, but alongside with small business-focused classes and some DDD building blocks.

Nevertheless, does it prove that TDD should be used all the time we write code? Not necessary.

We did a lot wrong. We had end-to-end tests describing nearly every corner case for every feature, even if it was just returning the string DATABASE_ERROR in case of a database error. We spent a lot of time moving a-tiny-step-by-a-tiny-step. Finishing a small project generating HTML tables and diagrams for our performance tests results in Groovy took more than 2 weeks.

Moving from brown to green

Later on, I’ve joined another project. It was a greenfield project visioned as a scalable, modular, distributed digital assets management system. dropwizard, NoSQL, Resque, Amazon Cloud Services. No EJB, no Oracle, no Enterprise Service Bus! Basically, the project every developer could dream about.

Do we always have to TDD at the unit level?

My very first task in this project was about creating a custom authentication and fit it into the Spring authentication lifecycle. I was pairing with a colleague. We both hardly known the Spring Security framework. We’re working on something new (like most of the time in this project). We started writing unit tests, some code, gathering knowledge. Then we wrote integration tests. Another unit tests and code. Red, green, refactor all the time. Eventually, the task was completed.

The test-first approach gives us focus on the solution. But we ended up with duplicated tests. We had integration tests which were absolutely essential in this case and unit tests verifying exactly the same things, but on a different level. And what about design? We haven’t discovered anything different than the framework forced us to write. In this case integration tests written at the beginning and few unit ones only to catch possible future regression would have been definitely sufficient.

Could we test it well with the code-first approach? Maybe not. Did we need TDD to be focused and produce good quality code? No. Our minds would have been enough.

The goal of one of the next tasks was to integrate our application with Amazon S3. This time I wrote integration tests first, but then of course I wrote dozens of unit tests verifying that I’m using Amazon SDK classes properly.

What’s interesting, these unit tests failed not even once later on. But were extremely painful and time-consuming when we’re changing the API exposing storage services to the rest of the system. The only bug we discovered in this part of the system was related to uploading huge files. Fixed once without a single automatic test, never came back.

This two examples (and many others similar) allowed me to notice an emerging pattern: The classic TDD approach with many fine-grained unit tests may be not the best way to write code which integrate with external services (and code). Obviously, it does not apply to every case. It depends. In our case integration with S3 is that crucial to the application that testing it at integration level was just fine.

My next “integration” task was completed with very few unit tests (error handling, corner cases) and integration ones for so-called “happy path”. Less time, the same results.

There are also other situations when writing unit tests isn’t beneficial enough to do it. When I found code hard to test (static calls, methods I couldn’t mock or stub), I tended to wrap it and test interactions with a wrapper. The wrapper itself was tested using some fancy tool (like PowerMock). But is it possible that we do a mistake in the code like that?

1
2
3

public File createFile(String path) {
  return new File(path);
}

I also don’t see much value in unit testing the code which most likely won’t break. Extremely silly code, simple procedures are just few examples.

Do we always have to write tests first?

Especially, in greenfield development, there’s a need to write code to evaluate ideas which cannot be assessed based on experience. How often such code is eventually introduced to the system? In the ideal scenario never. But sometimes the prototype do what it should do, and with few refactoring steps turns out to be production ready. Should we delete everything and write it using TDD once again? No. In such a situation writing good tests last is fine.

Sometimes, we also have to see the big picture first. To create, nearly accidentally, something and then decide whether it solves a problem. It’s almost impossible to accomplish it being focused on small steps and details. TDD, thus, does not apply to spiking very well. At least for me.

Lastly, I’m going to say something for what some people could burn me on the stake. Occasionally, we need to see the big picture even if we know we what we’re trying to achieve. Then, writing a test last is just okay. Don’t blame yourself, if you do that. You won’t go to hell.

Do we always have to write automatic tests?

Why do we test code? Well, basically to make sure that it works as expected. But why do we write automatic tests?

I see it as applying the DRY principle to testing. You tested something. It’s fine even if you did it manually. But it’s fine only if you did it once. If you test something twice, there’s a high probability you will do it again. So writing automatic tests hopefully will save your time in the future. What’s even more important it guarantees you won’t forget how the code you’re changing in one place is related to tests cases. Though, the value from automatic testing is that it helps you to notice regression. Using different words it helps you to be sure that there are no new bugs and requirements are still met.

But when regression occurs? What is the root cause of it? The fact that you’ve changed the code and broke something. That simply tells that you don’t have to test code you never change. And how it that possible that code will never change? For me one example is the tool generating HTML reports for performance tests results I mentioned earlier. Without TDD it would have taken 2 days instead of 2 weeks. Could I foresee it’ll never change? I think yes. Still, in most cases you won’t know that upfront.

Getting back to spikes and prototypes. There’s no need to test the code written to evaluate an idea (if a test is not a part of evaluation). Testing in this case could take considerable amount of time which is most likely limited for spiking and prototyping.

TDD done right

So, let’s answer the questions introduced at the beginning. Do we need TDD at all? Yes, absolutely. It’s one of the easiest ways to write good quality, well-tested code.

Nevertheless, my point, and conclusion of this article, is that I do not agree with the statement that it is irresponsible for a developer to ship a line of code that he has not executed in a unit test. In my opinion it’s exactly the opposite. A responsible developer should put a strong importance to his thinking process and, based on that, evaluate every principle, best practice and technique he is learning. Doing that, most likely, he’ll discover that covering every line of code by a unit test is not the best idea. And TDD rules do not apply to every situation.

Recently, Rebecca Wirfs-Brock and Joseph Yoder started to talk (and teach) about Pragmatic TDD. I had a pleasure to see their talk in this topic at the JDD’12 conferencee. Frankly, the talk motivated me to write this article. I agree with almost everything they said. However, I completely do not like the new name. There’s no need to create any new label. I would prefer they call it just TDD. Alternatively, TDD done right, if there’s a need to distinguish from the classic approach.

Despite that, when you start learning TDD you should definitely do it as it’s described in good books. But all the time you should keep an eye on the value it gives you. And as you take further steps on the TDD skill acquisition path, you’ll probably see things different than at the beginning. And hopefully find that situations which TDD do not play well with.