Smart Testing - Test Less, Cover More, Benefit Manifold

© Harbinger Systems | www.harbinger-systems.com
Smart Testing –
Test Less,
Cover More,
Benefit Manifold
Avinash Magdum
STeP-IN SUMMIT 2014

Agenda

Vicious Cycle of Quality Challenges
Shorter Release
Cycle
Inadequate Time
for Testing
Insufficient Test
Coverage
High Number of
Defect Leakages
Too Many
Maintenance
Releases
Adoption of Agile
Methodology Increasing
Testing Scope

Root Cause of Defect Leakages
Insufficient Test Cases
• No Traceability
• No clarity on requirements
• Misinterpretation of requirements
Poor Test Planning
• No Impact analysis
• Incorrect Prioritization
Inadequate Time to Execute
• Increasing OS – Browser support
• Variety of devices to be supported

The Solution…

Code Coverage Analysis
Definition: Its measure of the degree to which
the source code of an application is tested by
it’s test suite.
Categories
Functional Coverage
Conditional Coverage
Branch Coverage
Statement Coverage

Functional Coverage
Definition: It is the ratio of number of functions
(methods) covered by the test suite over
number of functions (methods) in the
application.
For example, if an application has 4500 methods,
and if 2700 of them get called after executing
entire test suite then,
Functional coverage = 2700 / 4500 = 60%

• Definition: It is the ratio of number of code
paths (branches) covered by the test suite over
number of code paths in the application.
Branch Coverage
In this code snippet, there are
5 code paths within switch
block. So for this block, if our
test cases can visit only 3
cases, then the branch
coverage for this block is 60%.
Same could be extended to
entire application

• Definition: It is the ratio of number of decisions
covered by the test suite over total number of
decisions in the application.
Conditional / Decision Coverage
In this code snippet, there are 4
decisions within the if block.
There are 4 decisions – both a and b
negative, only a negative, only b
negative and both a and b positive
If we test only for both negative and
both positive values, conditional
coverage is 50%

• Definition: It is the ratio of number of
statements executed by the test suite over
total number of statements in the application.
For example, if an application has 600,000 lines
of code, and if only 240,000 of them get
executed by the entire test suite then,
Statement coverage = 240,000 / 600,000 = 40%
Statement Coverage

Code Coverage Tools
Java
• Cobertura (open source)
• Jcover (commercial)
• EMMA (open source)
.NET
• dotCover (commercial)
• Ncover (commercial)
C / C++ / VC++
• BullsEye (commercial)
• Gcov (open source)

Objective 1:
Identifying Missing Test Cases

Execute entire test suite
Select / Unselect assemblies / modules for
coverage analysis
Verify the Coverage browser
Instrument the application
Select right code coverage tool
Identifying Missing Test Cases - I

Map uncovered methods (functional code) on
UI actions
Group Uncovered Methods
Dead Code Error Handling Code Functional Code
Identify Uncovered Methods
Analyze Coverage Report
Identifying Missing Test Cases - II

Identify Uncovered Branches
Analyze coverage report – Increased coverage
Execute new test cases
Create new test cases to include above UI
actions
Identifying Missing Test Cases - III

Analyze coverage report – Increased coverage
Execute new test cases
Create new test cases to include above UI
actions
Map Uncovered Branches on UI actions
Identifying Missing Test Cases - IV

Objective 2:
Identifying Redundant Test Cases

Identifying Redundant Test Cases - I
Compare every two successive Statement
Coverage counters
After every execution of a test case, check
Statement Coverage
Execute Test Cases sequentially, for a Feature
Setup Application For Code Coverage Analysis

Identifying Redundant Test Cases - II
Remove all the Redundant Test Cases from the
Test Suite
Repeat this Process for all the Features
Repeat above steps for all the Test Cases of the
Feature
If Statement Coverage Counters are identical,
mark the later test case as Redundant

Objective 3:
Identifying Optimal set of Test Cases (Test Plan)
for any Maintenance Releases

It’s a tool to identify optimal subset of Test Suite
for any maintenance release
It is created using Code Coverage Analysis
Gives list of test cases impacted by any code
change
Reduces testing efforts drastically
Arrests the bug leakages
Impact Analyzer

Creating Impact Analyzer - I
Execute Test Cases sequentially, one at a time
Reset all the Coverage counters
Create a database with 3 tables
Method Master Test Case Master Test Case Methods Map
Setup Application For Code Coverage Analysis

Creating Impact Analyzer - II
Create an UI to query modified Method and get
list of all associated Test Cases
Repeat the Process for all the Test Cases
Create a row in the Map table specifying Test Case
ID, and IDs of all the called Methods
List all the Method names called by the test case
Get Method Coverage after execution of test case

Creating Test Plan using I. A.
Add all the test cases impacted by each of the method
and include these in the test plan
Repeat the Process for all the Methods
For the method execute a query in Impact Analyzer to
get a list of test cases calling that method
Identify names of each method modified
Get a list of Defects fixed for the release

Takeaways
• Facilitates to identify and add missing test cases
– Reduces Defect Leakages drastically
• Facilitates to identify redundant test cases
– Reduces testing efforts
• Using Impact Analysis, provides optimum set of
test cases for maintenance releases
– Reduces testing efforts with highest coverage
Smart Testing – Less efforts, More
Coverage, Higher Quality, Less Cost

Q&A
Visit us at: www.harbinger-systems.com
Write to us at: hsplinfo@harbingergroup.com

Thank You!
Visit us at: www.harbinger-systems.com
Write to us at: hsplinfo@harbingergroup.com
Blog: blog.harbinger-systems.com
Twitter: twitter.com/HarbingerSys (@HarbingerSys)
Slideshare: slideshare.net/hsplmkting
Facebook: facebook.com/HarbingerSys
LinkedIn: linkedin.com/company/harbinger-systems
18

Smart Testing - Test Less, Cover More, Benefit Manifold

Recommandé

Recommandé

Contenu connexe

Tendances

Tendances (20)

Similaire à Smart Testing - Test Less, Cover More, Benefit Manifold

Similaire à Smart Testing - Test Less, Cover More, Benefit Manifold (20)

Plus de Harbinger Systems - HRTech Builder of Choice

Plus de Harbinger Systems - HRTech Builder of Choice (20)

Dernier

Dernier (20)

Smart Testing - Test Less, Cover More, Benefit Manifold