In the pursuit of continuous delivery, here is our reality: Unit testing is good, but just not enough. Every Project dedicate a big Functional Testing part at the end to deliver only the top of the testing pyramid. Then the questions are: - How can you deliver quickly if you need to retest your entire application manually for each release? - What automation tools can you use to ensure that your automated tests do not become a maintenance nightmare? - How can you write automated tests that testers can leverage to make their own manual testing more effective? The Agile Sydney Community is proud to receive John Ferguson Smart. Come and join us to challenge John on giving a live demonstration of automated acceptance criteria using JBehave, Selenium 2 and Thucydides. This will give us the opportunity to discuss a number of open source and commercial automated testing tools, with many examples and stories from real-world projects. John talks about a range of tools and approaches to automated functional testing, and discuss the place of functional testing in a modern agile project. He will look at topics such as: - Where does testing fit into an agile development life-cycle - Why it's too late to start testing at the end of the project or iteration - What tests should you automate, and how - What tools work best for different types of tests.

1. Automated
Functional
Testing

2. John Ferguson Smart
Consultant
Trainer
Mentor
Author
Speaker
Coder

3. Technical Agile Practices
Test Driven Development
Continuous Integration/Delivery
Automated
Acceptance Criteria
Refactoring
Pair
Programming
“Without
these
technical
prac1ces,
your
agile
adop1on
is
hollow”
-­‐
Ma6
Wynne

4. Technical Agile Practices
Test Driven Development
Continuous Integration/Delivery
Automated
Acceptance Criteria
Refactoring
Pair
Programming
“Without
these
technical
prac1ces,
your
agile
adop1on
is
hollow”
-­‐
Ma6
Wynne

5. A
good
func<onal
test
tells
a
story

6. Practices
Tools

7. QA loses its potential if only done
at the end of the project

8. The Three Amigos

9. Living Documentation - test results that serve everyone

10. So show me the tools!

11. The Record-Replay Scam

12. in theory
in practice
The Record-Replay Scam

13. What do ogres and good functional tests have in common?

14. (This test is not like an ogre)

15. Remember...good automated acceptance tests have layers

16. Start off with the business objectives
Business-­‐level
requirements

17. Describe the business flow
High-­‐level
steps
(business
flow)

18. Describe the page interactions
Page-­‐level
interac<ons

19. Page implementation details
HTML
details
go
here

20. Business
Rules
Business
Flow
Page/Component
interac<ons
Page/Component
details
Good automated acceptance tests have layers

21. Come on - can we see the tools now?

22. Unit
tes<ng
BDD
tools
(39%)
Given/When/Then
(40%)
Tabular
(13%)

23. WebDriver / Selenium 2
Mature
browser
automa<on
Wide
adop<on
Cross-­‐browser
support
Cross-­‐browser
support

24. ...or
use
Selenium
Hub

25. appium
Uses
the
WebDriver
API
Can
run
on
SauceLabs
(with
some
limita<ons)

26. Watir
Web Application Testing in Ruby
browser.goto 'http://bit.ly/watir-example'
browser.text_field(:name => 'entry.0.single').set 'Watir'
browser.checkbox(:value => 'Ruby').set
Ruby
DSL
for
web
tes<ng
WebDriver-­‐based
Fluent
and
readable
No
na<ve
support
for
Page
Objects?

27. Geb
Groovy Browser Automation
Browser.drive
{
go
"http://myapp.com/login"
assert
$("h1").text()
==
"Please
Login"
$("form.login").with
{
username
=
"admin"
password
=
"password"
login().click()
}
assert
$("h1").text()
==
"Admin
Section"
}
Groovy
DSL
for
web
tes<ng
WebDriver-­‐based
Fluent
and
readable
Supports
Page
Objects

28. 28
High level BDD
Many
tools
are
available

29. 29
Narrative:
In order to increase sales of advertised articles
As a seller
I want buyers to be able to easily find ads for articles they want
to buy
Scenario: Searching by keyword and location
Given Sally wants to buy a puppy for her son
When she looks for 'puppy' in the 'Pets and Animals' category
Then she should obtain a list of ads for puppies for sale.
search_by_keyword_and_location.story
Business-­‐friendly
specifica<ons
An
example
in
JBehave

30. 30
Scenario: Searching by keyword and location
Given Sally wants to buy a puppy for her son
When she looks for 'puppy' in the 'Pets and Animals' category
Then she should obtain a list of ads for puppies for sale.
search_by_keyword_and_location.story
Scenario: Searching by keyword and location
Given Sally wants to buy a <present> for her son
When she looks for '<present>' in the '<category>' category
Then she should obtain a list of ads for <expected> for sale.
Examples:
|present |category |expected|
|puppy |Pets & Animals | puppies|
|kitten |Pets & Animals | kittens|
|seiko |Jewellery & Watches| watch |
Tabular
examples

31. 31
Scenario: Searching by keyword and location
Given Sally wants to buy a puppy for her son
When she looks for 'puppy' in the 'Pets and Animals' category
Then she should obtain a list of puppy ads
search_by_keyword_and_location.story
public class SearchAdsSteps {
@Steps
BuyerSteps buyer;
@Given("Sally wants to buy a $present for her son")
public void buyingAPresent(String present) {
buyer.opens_home_page();
}
@When("she looks for $keyword in the $category category")
public void adSearchByCategoryAndKeyword(String category, String keyword) {
buyer.chooses_category_and_keywords(category, keyword);
buyer.performs_search();
}
@Then("she should obtain a list of $keyword ads")
public void shouldOnlySeeAdsContainingKeyword(String keyword) {
buyer.should_only_see_results_with_titles_containing(keyword);
}
}
Step
implementa<ons
in
your
favorite
language

32. 32
From web tests to living
documentation

33. 33
(Think “Two-CDs”)

34. 34
2 Automate your acceptance criteria
1 Discover your acceptance criteria
4 Execute your acceptance tests
3 Implement your acceptance criteria

35. 35
1 Discover your acceptance criteria
Feature: Browse Catalog
In order to find items that I would like to buy
As a customer
I want to be able to browse through the catalog
Story: Browse by category
In order to find items more easily
As a customer
I want to be able to browse through the product categories
Acceptance Criteria
See all the top-level categories
Browse through the category hierarchy
Should display the correct products for each category
Each category should have the correct sub-categories
Define
acceptance
criteria
for
each
story

36. 36
1 Discover your acceptance criteria
Acceptance Criteria
See all the top-level categories
Browse through the category hierarchy
Should display the correct products for each category
Each category should have the correct sub-categories
Scenario: See all top-level categories
Given I want to browse the catalog
When I am on the home page
Then I should see the following product categories: Clothing, Accessories, Shoes
Clarify
the
acceptance
criteria
with
examples

37. 37
2 Automate your acceptance criteria
Story: Browse by category
In order to find items more easily
As a customer
I want to be able to browse through the product categories
Acceptance Criteria
See all the top-level categories
Browse through the category hierarchy
Should display the correct products for each category
Each category should have the correct sub-categories
Scenario: See all top-level categories
Given I want to browse the catalog
When I am on the home page
Then I should see the following product categories: Clothing, Accessories, Shoes
Narrative:
In order to find items more easily
As a customer
I want to be able to see what product categories exist
Scenario: See all top-level categories
Given I want to browse the catalog
When I am on the home page
Then I should see the following product categories: Clothing, Accessories, Shoes
We
now
have
executable
specifica<ons

38. 38
2 Automate your acceptance criteria
...but
they
are
reported
as
‘pending’

39. 39
3 Implement your acceptance criteria
Narrative:
In order to find items more easily
As a customer
I want to be able to see what product categories exist
Scenario: See all top-level categories
Given I want to browse the catalog
When I am on the home page
Then I should see the following product categories: Clothing, Accessories, Shoes

40. 40
4 Execute your acceptance tests
What
tests
have
been
run

41. 41
What
requirements
have
been
tested?
4 Execute your acceptance tests

42. 42
4 Execute your acceptance tests
How
did
we
show
that
a
par<cular
requirement
worked?

43. 43
4 Execute your acceptance tests
How
did
we
show
that
a
par<cular
requirement
worked?

44. Demo
time!

45. hYp://wakaleo.com
hYp://thucydides.info
Want
to
learn
more?

46. John
Ferguson
Smart
Thank you