Using acceptance tests to drive development

I have seen experiments with acceptance test driven development in many teams over the years but it has never quite worked out. The tests have never become the main guiding lights for development. Sooner or later they have become simply the tools for tester while developers pay little or no attention to them. In recent project we tried it a bit differently. Instead of use cases or user stories/tasks we use acceptance tests as our main backlog items.

At the beginning of this project we tried to use use case scenarios for iteration planning but this didn’t work well because in our case scenarios were just too big. We tried splitting them in different ways but didn’t find any good solution. The problem seems to stem from the format of use cases which is more strict than for user stories. On the one hand we liked use case as tool for doing analysis and finding out user’s needs but we didn’t like the ambiguity that resulted in half implemented scenarios. Not only did it mean that planning was hard but also it made it hard to give good overview to customer about our progress.

So we decided to change our working method. After having first version of use case description ready we write initial set of acceptance tests for it. In our case these tests are not written by customer since customer is just too busy. However, when we write them the focus is on giving customer good overview of main functionality of the system (not trying to cover every possible corner case). These acceptance tests are kind of like tasks only they are worded using the “given… when… then…” style. If such tests are not explicit items in the backlog then you have to think additionally how you will demo this or that work item. Whereas if using tests as backlog items you are always thinking how something can be demonstrated (tested).

In this project we usually end up with about 20-30 acceptance tests for each use case. Not all are written upfront but in general we try to write down main set of tests when starting with some new use case. This way we have better knowledge of our progress in implementing given use case. Of course we add more tests as we go. This number of acceptance tests may seem a bit daunting at first. However, if you think that this use case (main scenario plus around 4-6 alternative scenarios) is about the same weight as 7 user stories then having 3-4 tests per user story is not so heavy at all.

One thing we have noticed is that some tests are reappearing in many use cases. This is quite obvious as use cases are not meant for low level program logic decomposition. In most cases we then don’t repeat these tests. Only when it may not be clear that this functionality should work here exactly the same way as in some previously implemented use case do we duplicate the tests. Another case where we may want to duplicate tests is when we need to use some piece of functionality that has already been implemented but still needs some extra effort to be integrated into into slightly different context.

Of course it would be nice to automate all these tests. However, I think that one doesn’t necessarily have to be the precondition to the other.

Advertisement