Tuesday, August 23, 2016

Acceptance Test Driven Development

The purpose of ATDD is to determine the acceptance criteria for a user story. The business and the development team work together to come to a shared understanding. This understanding is used to drive (guide) the design, development and testing effort of the team.

Some background

In an agile development environment, the user story (ideally represented by a 3” X 5” index card) is a lightweight reminder to have a conversation. The card does not contain requirements, but is a placeholder for a future conversation. Requirements are inventory, and excessive inventory is a form of waste. So, we have the conversation when a card has been selected for development, Just In Time.

Sometimes, the PO or BA has jotted a few notes on the back. In classic eXtreme Programming, the on-site customer would have expanded these notes into more detailed customer test cases before work began. However, with ATDD we now discuss and specify those tests collaboratively in a Three Amigos meeting.

Three Amigos (or cuatro or cinco)

Originally named for the three participants (Product Owner, Developer and Tester), this meeting of minds has outgrown its name. However, the purpose remains the same.  Bring the business and the development team together to collaboratively specify the value to be produced by a user story.  This meeting includes a UX/Designer, Product Owner, Developer, Tester, and perhaps a Business Analyst, or Technical Writer.

The primary output for this meeting is a set of acceptance tests for the story card. The most common way of specifying acceptance tests is to provide concrete examples of expected behavior. All parties are in agreement that those tests represent the definition of done.

Acceptance Tests

Many teams choose to express acceptance tests using Cucumber scenarios. It is certainly not the only way to do so, but there are many advantages.

Cucumber’s simple, expressive structure encourages the use of concrete examples for expected behavior. This improves the conversation and makes the desired behavior more clear. Here’s an example:

Instead of:
  User must provide Social Security number for loan application

You create this scenario:
  Given the applicant is using the online loan application
  When the applicant proceeds without supplying SSN
  Then the applicant is informed that SSN is required

Secondly, Cucumber can execute these scenarios against your software, or the System Under Test (SUT). The development team can write code in Ruby [or another inferior language ;) ] to execute each scenario and indicate if the SUT performed as specified in the “Then” step. Now, the tester can focus on more complex Exploratory Testing (ET) since the basic behavior is covered by automated acceptance tests (scenarios).

As the number of scenarios grows, they will begin to serve as the source of truth on the expected behavior of the system. Instead of reviewing documentation that may or may not reflect the current desired behavior, the scenarios are executing against the real system.

Reading those scenarios is a great way for a new team member to get acclimated to the domain and the product. For this reason, the team must take care in using consistent domain language in the scenarios. With a little discipline, a suite of scenarios is a great asset to any team.

Since the suite of scenarios is the source of truth, most teams run them regularly as part of a Continuous Integration(CI) build. In doing so, regressions can be detected by a failing scenario. Note that the goal is not to write a scenario for every possible input or edge case. In an environment with complex or nuanced rules, more scenarios with concrete examples will help document those rules. In other cases, just a couple of happy-path scenarios per story card will suffice. Each team finds the right blend.

Why wait for the Three Amigos meeting?

One question asked of this approach is, “Why wait?” Wouldn’t the development team get more done if they were handed completed cucumber scenarios? Well, maybe, but usually not.

What must be emphasized is that the Three Amigos is not a “transfer of information” meeting. While the primary output is a fairly concise set of acceptance tests (scenarios), the Three Amigos serves a higher purpose. It is a collaborative discussion to specify the acceptance criteria. In the true spirit of the Agile Manifesto’s “Customer collaboration over contract negotiation”, the business and the development team are discussing options, asking questions, seeking to understand the true value behind the requested feature. Don’t forget the input provided by UX, the low-fidelity screen drawn on the whiteboard during the meeting, the Tester’s notes on performing ET for the card, the other stories discovered (or split out) for the PO to consider, the clarification on validation rules, etc. In fact, the team often discovers a cheaper way to get the desired results.

Finally, remember that “The most efficient and effective method of conveying information to and within a development team is face-to-face conversation.” You are coming to a shared understanding, building relationships, and offering everyone a chance to participate in what is being built. You simply won’t get that by turning your talented team into order takers.

ATDD makes a difference

ATDD means pulling the team together to get a clear definition of done in a face to face meeting. Documenting it concisely (preferably with Cucumber scenarios) will mean less rework due to misunderstood requirements. Using ATDD, the whole team can now produce only what is necessary to meet the acceptance criteria. As a bonus, your software system gets a set of automated regression tests. This approach to communicating acceptance criteria is an effective addition to any software development team.