Find bugs before they are born with Story Testing

Would you like to detect bugs even before they are written? It would be incredible to find bugs that are not “born” yet, while they still are in developer’s mind, right?  I’m going to explain you what Story Testing means for me, and how to apply this technique in the real world.

Well, what is Story Testing? You might heard of Acceptance Test-Driven Development (ATDD), Behaviour Driven Development (BDD), Story Test Driven Development (STDD), Specification By Example (SBE), Example Driven Development (EDD), … These terms are slightly different approaches that aim to the same goal: to implement what the customer asks at the first time. When we do Story Testing, what we do is to use any of those techniques. I prefer this term, since what we do, at the end of the day, is to make sure that the user story implementation does the right thing with few or no errors.

If you do Story Testing (regardless of the “flavour” you choose: ATDD, BDD, SBE, etc.), tests (or examples) are created before implementing the user story. But unlike in traditional testing, where the testers are the tests “owners”, testers, developers, and business stakeholders collaborate in this process.

It has an additional advantage too: tests will be examples of use, they will become an “executable” documentation of your system, so it will never become obsolete (or your tests will begin to fail…).Ok, let’s go for it:

• We start working from a user story that the team has analyzed and written (developers, testers, product owner,…). Real life fact: in some agile teams, user stories are written from the analyst work, and development team has  little interaction with business stakeholders or customers.
• We create the tests. These tests will be user story expected functionality examples. They will serve as a guide to implement the story. Keep in mind that using the terms example or test means the same, since they will always be executable test cases which will validate that the user story is properly implemented. I insist: example = test.
• You should begin with the most basic cases and then complicate it. I usually follow this order: happy-path (positive cases), negative cases and cases with errors. Real life fact: I work together with the user story developer,  and we ask business stakeholders if necessary. When you run these tests, they will fail.  Next step is writing the code that makes these tests pass.
• Write the code that implements the history, iterate with the previous step, until all examples are properly executed. This development is done following TDD technique: developer write unit tests that fail, the code that makes them pass and then refactor.
• You integrate development and tests and you start again with another user story.

You must take into account that:

1. You have to write the tests in a way that all people involved understand them, and that includes your business partners… The tool you use for testing must express the tests in a clear language. I will tell you how to do it in the next section 🙂
2. The examples must cover all the features expected in the user story and just these. Neither more nor less. Do not write more than one example or test for the same feature or include examples for features that are not expected in the user story.
3. If possible, it is recommended to run these tests on the business logic layer. If needed, you may run them as a user would do, at the user interface layer.
4. This technique does not replace TDD (Test Driven Development), you should continue writing your unit tests to make sure that your code is correct. The two techniques are complementary.

Tools

The examples must be written in a clear and understandable language by anyone without programming or testing knowledge. The traditional frameworks like JUnit testing are a bit “hard” for business people. They understand Word or Excel pretty well. The solution is using a testing tool that they can understand. Here are some:

I use this one:

• FitNesse is a wiki-like web server and an acceptance testing framework.  Tests are written in table format and they are very easy to understand for everyone. The code to “connect” tests with your software should not be very complex, but it can be tricky if your system is not ready. It has some plugins that will ease some test operations. It’s ATDD-oriented.

I’ve not use these, but they look good:

• Cucumber BDD-oriented framework. It supports a lot of programming languages.
• Concordion to automate with the SBE approach (Specification by Example). Supports Java and some features are also available for C#.
• Robot Framework is a framework to create automated acceptance tests.
• Spock framework of test and specification for Java and Groovy applications.

Some of these tools can be used together with Selenium, so you may find them useful if you need user interface tests.

If you want to recommend (or criticize) any of these tools, please send a comment.

A practical example

We had to develop a system that, under certain conditions, should perform a data exportation to an external system.

Roughly, user story said: as a Manager I want that when a new customer is created or if any of  his personal information is changed in the customer’s database, some data is transferred to the accounting database, so both databases are always on sync.

In the customer’s database we had these fields (I have simplified the example): CustomerNumber, fields with personal information, AccountingCustomerNumber and AccountingTransferDate. We began writting the basic test cases in FitNesse:

Quite simple, isn’t it?.

This was the conversation we had:

Tester:  What happens if there is a client with an assigned AccountingCustomerNumber but the AccountingTransferDate is null? Or the other way around?

Developer: That can not happen!

Tester: Mmmm… We should be sure of it.

A quick search in the database showed that there were a few thousand cases.

First bug found before development: it was assumed that this case can’t happen, but it was a false assumption. The system would have behaved in some erratic way.

So we completed the table:

Tester: Wait, why does an error occur in cases 4 and 5?

Developer: Mmm, in those cases some data is wrong so I will always log an error.

Tester: We should talk to the analyst and business and see together what to do…

After talking with the accounting expert he agreed that customer information should be transfered to the accounting database even if any of these fields were not consistent. These “dirty” fields would be updated after the transfer.

Second bug found before the development: accounting needed the system to behave differently than the development team thought.

A simple conversation avoided programming one thing that was not going to be useful. Finally,  the test table was:

From this experience we extracted two conclusions:

• The developer was assuming that if the fields combination was not consistent, he should log an error. But business had other needs. He would have introduced a bug which would have to be corrected later.
• By asking the right questions before writing the code, we really knew what our client needed and we didn’t wasted time rewriting code and fixing bugs.

Summary

This type of testing is part of the methodologies and techniques of Agile Testing, where software testing follow the same principles as agile development.  The agile methodologies do not see testing as a separate phase, but as an integral part of software development.

This technique will help you to properly implement what your customer has asked for. You will detect bugs that are still in team’s “mind”, either by misunderstandings or due incorrect ideas assumptions. Your test base will serve as a “living” documentation of your system and it will not become obsolete.

I hope this will be useful! Leave me a comment with your opinion 🙂