Specification-By-Example

with Gherkin
FROM STAKEHOLDER EXAMPLES TO LIVING DOCUMENTATION

Christian Hassa - ch@techtalk.at - @chr99ha

Scrum Gathering Barcelona, 2nd of October 2012
About me
Managing Partner at TechTalk:
agile consulting and delivery
Working in: Vienna, Budapest, Zurich

Vienna/Austria
 How do we specify
requirements?

3
Discovering what matters to customers
Why?
Impact Mapping
Impact, Goal
Isolated,
Capability Story Mapping formalized
example
Feature
Epic
User Story
Reminder
Acceptance for a Bug
Criterion discussion report

Specification-By-Example
Code
How?
Easier to define upfront Harder to define upfront
5
Collecting Acceptance Criteria
As a potential customer
I want to collect books in a shopping cart
So that I can order several books at once.
I would try to put a book into the
shopping cart
I would try to remove a book
from the shopping cart
Id check whether the shopping cart
is empty, when I enter the shop
I would try to add the same book
again to the shopping cart
6
Imagine this story is
already implemented:
how would you verify it?
Books can be placed into
shopping cart.
Books can be removed from
shopping cart.
Shopping cart should be empty
when entering the shop.
Adding same book again to
shopping cart should increase
quantity.
 Why
do we need
examples?

7
Examples for user stories

UI wire frames,
existing UI

rules, key examples

existing artifacts,
samples

8
 As a potential customer

Exercise I want to collect books in a shopping cart

So that I can order several books at once.

Work in groups of 2-3

Discuss examples of how you would try out
the shopping cart
Draw/write down necessary examples you
need during the discussion
(UI mockup(s), concrete books, )
Write down the list of
discussed acceptance criteria
DONT bother with formalizing examples yet
e.g. Given/When/Then
9
Specification-by-Example
Examples
make abstract descriptions
better understandable

However
examples are usually not formally Brian Marick
exchanged or documented

consist of
Examples Tests
describe verify
fulfillment of
Requirements

10
Discussion of acceptance criteria
We would like to encourage new users to
buy in our shop.
Therefore we offer 10% discount for their
first order.

public void TestInitialOrderDiscount() Register as bart_bookworm

{ Go to /catalog/search
Customer newCustomer = new Customer(); Enter ISBN-0955683610
Order newOrder = new Order(newCustomer); Click Search
newOrder.AddBook(
Catalog.Find(ISBN-0955683610)
Click Add to Cart
); Click View Cart
Assert.Equals(33.75, Verify Subtotal is $33.75
newOrder.Subtotal);
}

Original idea for the illustration: George Dinwiddie

http://blog.gdinwidiee.com
11
 illustrated with formalized examples

Given the user has not ordered yet

When the user adds a book into the shopping cart

Then a discount of 10% is applied to the total.

Original idea for the illustration: George Dinwiddie

http://blog.gdinwidiee.com
12
Discover hidden assumptions
Actually, this not quite right:
Books on sale should be excluded .

Original idea for the illustration: George Dinwiddie

http://blog.gdinwidiee.com
13
Collaboration: 3 amigos
Exceptions,
Happy Technical border cases
Path feasability

Original idea for the illustration: George Dinwiddie

http://blog.gdinwidiee.com
14
Abstract acceptance criteria

As a shop visitor
I want to collect books in my shopping basket
so that I can purchase multiple books at once.

Books can be added to the shopping basket

Books can be removed from the shopping basket

Shopping basket is initially empty
The same book can be added multiple times to the shopping
basket

15
Examples for acceptance criteria

As a shop visitor
I want to collect books in my shopping basket
so that I can purchase multiple books at once.

Books can be added to the shopping basket

Given my shopping basket is empty

When I add the book Harry Potter to my shopping basket

Then my shopping basket should contain 1 copy of Harry Potter

16
Examples for acceptance criteria

As a shop visitor
I want to collect books in my shopping basket
so that I can purchase multiple books at once.

Books can be added to the shopping basket

The same book can be added multiple times to the shopping basket
Given my shopping basket contains 1 copy of Harry Potter

When I add the book Harry Potter to my shopping basket

Then my shopping basket should contain 2 copies of Harry Potter

17
Structure of examples
Title: Describes intention/abstract acceptance criterion
Arrange: Context, initial state of the system Triple-A
Act: Execution of the feature constraint
Checks
Assert: Assertion of observable behaviour
The same book can be added multiple times to the shopping basket
Given my shopping basket contains 1 copy of Harry Potter

When I add the book Harry Potter to my shopping basket

Then my shopping basket should contain 2 copies of Harry Potter

And I should see the warning: Book already existed in basket

Chaining
up steps

18
Exercise
Work in same previous groups of 2-3
Pick an acceptance criteria from the
previously compiled list
Illustrate with examples formalized into
Given: Context, initial state
When: utilization of the feature
Then: assertions of observable behaviour
Validate that scenario title (acceptance
criterion) still describes intention of
formalized scenario
19
 How long do we
care
about the
examples?

20
Purpose of the examples
Shared understanding:
acceptance criteria

Documentation:
Look-up detail aspects
of the system

Regression-tests:
Understand what
assumptions have been violated
21
Continuous validation with automation
Step Definitions are binding individual steps
to an automatable interface of the application.
Automation does not necessarily have to bind to the UI.

Automatability of system is supported/evolving with development.

Given my shopping basket contains 1 copy of Harry Potter

When I add the book Harry Potter to my shopping basket

Then my shopping basket should contain 2 copies of Harry Potter

UI
System
Automation
Automatable
interface

22
Demo

http://www.specflow.org

Gherkin automation for .NET

Visual Studio plugin (VS-Gallery)
NuGet Package

23
 What does that
mean for

testing?

24
UI automation becomes expensive
when
trying to automate
structure
manual tests
making tests
unreadable when readability
automating them
automating after
completing point in time
development

25
Structure
Manual tests Automated Check

Asserts Multiple combined Single aspect of a

features single feature
Structure ACT-ASSERT- ARRANGE
ACT-ASSERT- ACT
ACT-ASSERT- ASSERT

Dependent on other Independent of
features other features
Long test path with Short test path with
high chance to break lower chance to break

26
Test automation pyramid
few

exploratory hard
testing

User

Automatability
journeys

Acceptance-
criteria

Units
easy
many
27
Source: Mike Cohn
Readability
// Go to web page 'http://localhost:40001/' using new browser instance
BrowserWindow localhostBrowser = BrowserWindow.Launch(
new System.Uri(this.RecordedMethod1Params.Url));

// Click 'Register found item' link

Mouse.Click(uIFundstckerfassenHyperlink, new Point(56, 9));

// Click 'Save' button

Mouse.Click(uISpeichernButton, new Point(44, 14));

int fundNr1 = int.Parse(uIFundNr127Pane.InnerText.Substring(9));

// Click 'Register found item' link

Mouse.Click(uIFundstckerfassenHyperlink, new Point(63, 7));

// Click 'Save' button

Mouse.Click(uISpeichernButton, new Point(34, 11));

int fundNr2 = int.Parse(uIFundNr128Pane.InnerText.Substring(9));

Assert.IsTrue(fundNr1 + 1 == fundNr2);

// Click 'Close' button

Mouse.Click(uICloseButton, new Point(26, 11));

28
A readable test case
Scenario: New found items should receive a
consecutive number for the current year

Given the previous found item of the

current year had the number 145

When I register a new found item

Then the last found item of the

current year should have the number 146
29
When to test (point in time)
business view

criticizing the product

defining the product

Acceptance criteria Exploratory tests

(ATDD, BDD) Workflow tests

Unit Tests Performance, Scalability,

(TDD) Usability,Security,

technical view
Agile Testing Quadrants: Brian Marick

New dimension: defining the product

Synergy: Specification of requirements and tests

30
 Cross-functional teams

Co-creation
Fast feedback
31
 Cross-functional work
Limit Collaboration for defining Extension of Collaboration for
WIP acceptance criteria Test Cases automation
US1
Plan
Implement & US4 US7

Exploratory Tests
autom. test US5 US8
Plan Plan
US6
Plan US9
Plan
Collaboration
US2 ImplementPlan& ImplementPlan&
in manual US3 autom. test &
Implement autom. test &
Implement
testing Plan autom. test &
Implement autom. test &
Implement
Plan autom. test
Implement & autom. test
Implement
autom. test &
autom. test Specification and test

Sprint 1 Sprint 2 Sprint 3

Preventing bugs
short iteration
instead of finding them!

32
 What does that
mean for
development?

33
Workflow - TDD

34
Extending TDD for business

35
 Transparency for stakeholders

In Progress

36
 Current sprint report: all sprint scenarios

37
 Starting with first scenario (AC)

38
 Finishing the first scenario (AC)

39
 Progressing scenario after scenario

40
 Progressing scenario after scenario

41
 Progressing scenario after scenario

42
 Implementing user stories in parallel

43
 First user story ready for testing

44
 Manual testing can start even earlier

45
 Already done work can break again

46
 Already done work can break again

47
 See what is temporarily not working

48
 Living
documentation

50
 Story Maps
Developed by Jeff Patton
Skeleton for fine-grained user stories
Built for a set of capabilities or outcome
that the system should support

51
 Building story maps
Book delivered desired effect
to my home or outcome

Find book Collect Complete Wait for Receive user

I want books order book book activities

time
browse receive receive
put into enter system
best delivery delivery
basket address features
necessity

sellers notificat. slip

search pay with inquiry
create
book by credit order
wish list
title card status

52
 Establish a walking skeleton
Book delivered
to my home

Find book Collect Complete Wait for Receive

I want books order book book

time
browse omitted receive
manual enter Walking
best workaround steps delivery
address skeleton
necessity

sellers slip
search pay with inquiry
create receive
book by putlist
into credit order
wish delivery
title basket card status
notificat.
53
Demo
Story Maps as Living Documentation

54
User Stories vs. Features
Product/Sprint Backlog Living Documentation

AccCrit 1 AccCrit 1
User Story 1
AccCrit 2 Done Feature 1
AccCrit 2
AccCrit 3 AccCrit 3
User Story 2
AccCrit 4 AccCrit 5
AccCrit 5 AccCrit 4
User Story n Feature n
AccCrit m AccCrit m

Future options of the system Current state of the system

Organized/refined according to Organized/refined for
priority, value, effort, risk, ... functional overview
Next possible increments of Versioned and maintained
the product (units of work) together with source code

55
Demo
Feature Tree as Living documentation

56
 Additional
thoughts

57
Level of automation
Out-of-process Browser
automation

In-process
Assert behaviour Trigger behaviour
View on model, db, .. through controller

Model

Controller

Setup pre-conditions
Business Layer through service
interfaces

Data Layer

58
Performance
Grouping tests
Current WIP
Completed work
Database
In-memory
Templates for setup
Parallel execution
Smart execution order

59
Internal vs. external DSL

Example Source:
Liz Keogh
https://github.com/lunivore/tictactoe-java/blob/master/
scenarios/com/lunivore/tictactoe/scenarios/
Three_in_a_row_wins.java

60
Non-functional acceptance criteria
Given there are 100,000 users registered on the system
When I create a new account
Then I should be taken to my dashboard within 5ms

Given 1000 users are hitting the homepage

simultaneously
Then each user should get a response within 2ms

Matt Wynne
http://blog.mattwynne.net/2012/03/13/using-cucumber-for-load-testing

61
 Tools

62
Gherkin automation tools

Ruby, Java, JavaScript, C++

www.cukes.info

.NET, Mono, Silverlight, WP7

www.specflow.org

PHP
www.behat.org

63
64
Books

Gojko Adzic Gojko Adzic Lisa Crispin

Bridiging the Specification by Janet Gregory
Communication Gap Example Agile Testing

65
66
Christian Hassa: ch@techtalk.at - @chr99ha