Innovations and adaptations in agile testing.

1. Innovations and Adaptations in Agile Testing
Subrahmaniam S R V ACP, PMP CSM
25th February, 2014
http://knowledgehut.com/blog/wp-content/uploads/2013/04/agilepicture.gif
Slide 1

2. What will we cover today?
Agile Testing – Role of testersr
Key Agile Testing practices
Lispin‘s Agile testing quadrants & challenges
Adaptations to handle these
Outcome assessment
Disclaimer: The views expressed are that of the presenter and do not necessarily represent the views of the company he works for.
Slide 2

3. Agile Testing - Role of testers
Test early
Reduce # of open
defects
Focus on buggy or
critical areas
Testers‘ objective is to enable timely release of
quality software. They find defects and verify the fixes
as part of this objective
Collaborate with
developers
(reproducing
defects, bug
verification etc.)
Automate
alongside during
Sprints
Sensitise
developers
towards planned
tests
Slide 3

4. Testing upfront moves the powers the project ahead
Slide 4

5. Continuous Integration and Automated test




Nightly builds followed by automated tests
Code to verification cycle will be less than a day.
Automated suite should grow with every Sprint.
Focus on Critical functional areas for automated System test
Image: http://www.agilenutshell.com/assets/continuous-integration/continuous-integration.png
Slide 5

6. Test throughout the Sprint duration, not just at the end
The user stories are to be tested throughout the Sprint, not just when they are
completed (mini-waterfall) – and prevent clogging at the end.
Image: Agile Testing by Elisabeth Hendrickson
Slide 6

7. Testing is everyone‘s responsibility!
Developer tests
Unit tests
Automated
tests
Tester‘s tests
QA practices (Pair
Programming, static
code analysis, CI etc.)
Need for dedicated testers is progressively reduced in mature agile teams
Slide 7

8. Co-location of test team members
 Developers, testers and product
management teams sit in the same
area.
 More face-to-face communication and
less reliance on emails.
 Quick resolution of queries and open
issues
http://www.environmentalevaluators.net/2011/07/key-concepts-of-complexity-in-science-feedback-loops/
Slide 8

9. Crispin’s Agile Testing Quadrants – Challenge in
executing these within Sprints
Especially in cases with High domain / technical complexity
http://img3.douban.com/view/page_note/large/public/p14837373-1.jpg
Slide 9

10. Typically, the right quadrant tests (that which critique the product)
are done only in dedicated test phases due to logistic issues
Code Freeze
Sprinting phase
Release
Dedicated test phases
 Full System tests
 Full Load and
Performance tests
 Other non-functional - ility
tests
 Exploratory tests
 Compliance to regulatory
standards and guidelines
Critical defects found during dedicated test phases can put the release in
quandry, because they can be hard to fix and found too late! Slide 10

11. Challenges emerging from the right quadrant tests!
Can these be scheduled during Sprints?
http://img3.douban.com/view/page_note/large/public/p14837373-1.jpg
Slide 11

12. Nature of exploratory and non-functional tests
May need long
duration test cycles
Defects hard to
uncover and hard
reproduce
Environment and tool
set-up likely to be
complex
Highly specialized set
of tests
Defects can potentially
affect release
schedules
Defects may have high
turn-around time
May require
architecture fixes
Slide 12

13. Considerations in scheduling Exploratory and Non Functional tests
Limited
feasibility
to do within
Sprint
Critical
tests need to be
performed
at earliest
Software
may not be
ready for
full-blown
tests
Sprint Test
Accomodating these factors in the Agile test framework vis-a-vis
what tests get carried out within Sprints is the critical decision
Slide 13

14. One possible recommendation – Test in Parallel, with a Sprint lag!
Functional tested package
Sprint 1
Sprint 2
Dedicated
test phases
Sprint 3
.....
Hardening Sprints
System &
NFR test
Defects
System &
NFR test
Or
Release Sprints
Product
Backlog
The possibility of any critical defects being found in dedicated test phases is reduced
Slide 14

15. Why this approach?
1
A stable build and undisturbed time window is provided for right
quadrant tests done along-side Sprints
2
Any bug found early will go towards avoiding late surprises in
dedicated test phases
3
Teams for Sprint tests and right quadrant tests can be groomed
for their skills accordingly
4
Duration of the dedicated test phases is optimized, by doing
critical tests early.
5
Intent is to ensure certainty of the release date.
6
Testers doing the right quadrant tests often need specialized
skills – be it in Load and Performance, domain or in NFRs
Slide 15

16. Evolving Definition of Done
1 Web application for internal use
3
System Complexity
Complex
2 Domain centric external facing
web application
Simple
3 Device centric multi-environment
2
Moderate
thick client
1
Production
ready
System tested
Functional
tested
Done Criteria
The gap between Sprint Done and being Production ready will be tested during the
hardening sprints.
It is the same as the gap between Potentially shippable and shippable
Slide 16

17. Manage Test environments
 Have parallel or shadow (to system test) test environments for testing
during Sprints
 These will also be of use for developers to test their fixes when long
test phases are ongoing during dedicated test phases
Primary ST environment
Shadow environments for use by Scrum Team
Slide 17

18. Re-skilling and competency building
Training areas
Test Automation
Tooling
Problem solving
- Automation has to
go hand in hand
during the Sprints.
- Automation
engineers need to
think beyond
standard tools and
look for custom
built test suites
working in tandem
with standard
tools
- Identify areas
where tools can
be applied
- Evaluation of new
tools for data
capture and
analysis
- Project planning
and scheduling
- Problem
identification and
evaluation of
alternatives
- Work scoping,
constraint and risk
management
Building technical Competence
Inter-personal
skills
- Communication
and Presentation
skills
- Team work
- Negotiation skills
Building self-organized teams
Slide 18

19. Items for Optimization
Test
scheduling
Environment,
Tools and
Test stubs
Increased use of
white box test
techniques and
custom built test
stubs
Automation &
Continuous
Team skilling
Integration
Slide 19

20. Status & Progress Indicators
+
the Contra indicators
the positives
• Key end-to-end and Non Functional
requirements are tested early
• Defects in Discussion / Clarification
/ Dispute mode are reduced
• Developers (along with testers)
doing informal testing in the test
environment
• Testers supply all the details need
for developers for bug fixing: Logs,
environment, data snapshot etc.,
-
• Critical defects found late –
impacting release date
• High number of open defects
• Increase in escape defects Testers getting influenced by
development team‘s thinking (?)
• Number of stories – Not done due
to test being incomplete
Slide 20

21. S R V Subrahmaniam,
SDLC professional and Agile Specialist
in.linkedin.com/in/subrahmaniamsrv/
Srv.subbu@yahoo.in
@srv_subbu