A comprehensive assessment of unforeseen risks in the project lifecycle can prevent costly breakdowns at the testing stage.

1. • Cognizant 20-20 Insights
Risk Mitigation: Fixing a Project
Before It Is Broken
A comprehensive assessment of unforeseen risks in the project
lifecycle can prevent costly breakdowns at the testing stage.
Executive Summary billion are wasted every year on badly managed
IT system deployments, only discovered during
Many IT projects have a reputation for exceeding
the testing stage.1
budgets, missing deadlines, failing to realize
expectations and/or delivering sub-par return on • Only about 16% of overall IT projects can be
investment. Surveys and reports on the accept- considered successful; even conservative
ability of new IT systems reinforce the same estimates put the costs of failure into the £20
problems and probable causes of failure; yet billion range across the European Union (EU),
businesses, large and small, continually make in IT software development projects.2
mistakes when attempting to improve informa-
tion systems, particularly investing in inappropri- As seen in Figure 1 (next page), the peak expen-
ate or unworkable changes without proper con- diture of both time and money on IT support and
sideration of the likely risks. development occurs at the testing stage. The
costs of failure are not limited to the narrow con-
Planning projects, establishing requirements siderations of budget and schedule. Companies
for change, selecting, implementing and testing do not pay adequate attention to conducting a
suitable systems are important components proactive risk assessment of all projects during
of a superior business management process. their entire lifecycle in an effort to mitigate and
Companies, however, often do not take adequate minimize the risk of failure at the testing stage.
precautions during the project lifecycle to
minimize failures at the testing stage. Consider Identifying and analyzing potential risks of failure
the following: during the project lifecycle is fundamental to
avoiding issues at the testing stage, as well as
• Our assessment of 134 publicly traded massive rework. To do this, it is vital to set up
companies reveals that over half admitted an early-stage diagnosis. This approach includes
to suffering a failed IT project in the past 12 analyzing projects throughout the entire lifecycle,
months due to issues encountered during from the requirements gathering stage through
the testing stage. Such failures come at an verification, when all potential variables and
extremely high cost. intermediate process steps are weighed in terms
of risk. Taking this approach can help IT organi-
• According to the Royal Academy of Engineer- zations understand and minimize the potential
ing and the British Computer Society, £3.5
impact on testing activities.
cognizant 20-20 insights | july 2012

2. Project Costs Increase at Testing Stage
Rate of Cost Investment During Projected Lifecycle
Late Defect
Discovery Results in
Defect Significant Rework
New Prevention
Requirements Design and Build Release Release
to Test to Field
Time
100x Increase in Cost of Removing Defects
Sources: Barry Boehm, “Software Engineering Economics,” Prentice Hall, Inc., 1981.
Basili Boehm, “Software Management,” IEEE Computer, January 2001.
Figure 1
In this whitepaper, we spell out our rigorous for each of these items. This means that an
process for identifying and avoiding project issues order function (risk rating = 5) could be impaired
that undermine many applications development because of a business partner failure (probability
projects. It also offers proven ways for organiza- of 60%) or could fail due to a system failure (prob-
tions to minimize development delays and costs, ability of 20%). The same function would have a
while building systems that deliver exceptional different criticality/probability score (see below),
business benefits. based on the fact that two different failures could
hit that business function.
Test Risk Assessment Process
Cognizant Business Consulting developed a The business criticality evaluation process is
structured, proprietary process called the Test strictly related to the risk management process
Risk Assessment for Project Improvement and is useful for any project stakeholder, at any
(TeRAPI), with the main objective of identifying project stage.
and minimizing risks that can occur at the testing
Measuring Business Criticality
stage across the entire product lifecycle.
To be successful, any test risk assessment has
The process consists of three main phases: to concentrate on the local identifiable issues
related to the business. During the test risk
1. Measure the business criticality. assessment process, risks to business functions
2. Profile the risk level of projects. will be identified and evaluated. The vulnerabil-
ity of the business to these risks will be rated.
3. Design a risk mitigation plan. Main actions to perform in a test risk assessment
include:
The purpose of measuring business criticality is
to determine the importance of each business • Alignment of testing with business
function. After all the business processes are requirements.
evaluated to determine their criticality to the
project, business functions can be prioritized to
• Test strategy and test environment
preparation.
determine which ones require a contingency plan
and which can be ignored and eliminated. • Test resourcing.
Each function, system, interface and third party
• Test environment.
can and should receive a risk rating. Probabili- • Test execution.
ties are assigned to each failure that can occur • Production readiness preparation.
cognizant 20-20 insights 2

3. Each of these business functions represents each assessed project in an effective way and
a dimension to be analyzed in setting up a risk in relation to each dimension and relative sub-
management process. dimensions (see Figure 2).
Profiling Risk Level of Projects The TeRAPI risk assessment identifies two types
After having defined main dimensions and sub- of risks for testing:
dimensions as a core part of the TeRAPI method-
ology, each project must be assessed to weigh the • High-level risks: Risks that are likely to impact
the schedule or the quality of the deliverable.
risk associated with each sub-dimension in each
The root causes for these risks need to be
stage of the project lifecycle.
identified and addressed through initiatives at
A business criticality score is determined based the higher level.
on the business purpose of the project. The result • Medium-level risks: Risks that can be
is a business risk index (BRI), which offers a addressed by the testing team with minimal
composite view and comparison of risks based on impact on schedule or quality. These risks can
the business criticality of the projects that require be addressed by project managers through
immediate management attention, as well as the tactical initiatives.
priorities of these projects.
Once the test risk assessment is completed, the
All dimensions are quantitatively determined, next step is to analyze and present the results so
from business requirements alignment, to test that executive management can get the most use
result analysis and mitigation plan design — and from the data. Data analysis will be the foundation
a qualitative index is assigned. Frequency of risk for planning actions to mitigate risk impact and
across each action step must be determined and provide recommendations.
graphed.
The recovery strategies that need to be developed
For each dimension and its specific sub-dimen- should be based on the findings of the risk
sions, a qualitative measure of risk importance assessment survey and interviews, as well as the
needs to be identified and quantified. This must be business impact analysis.
done across all projects where TeRAPI is applied.
Designing the Risk Mitigation Plan
This data is placed in a qualitative table that Each TeRAPI dimension must be developed and
summarizes all quantified risk-level results for risk causes determined and analyzed to create
TeRAPI: Project Test Risk Evaluation
Frequency of Risk for TeRAPI DIMENSIONS PROJECTS
Individual Projects Dimensions #1 #1 #2 #3 #4 #5 #6 #7 #8 #9 #10 #11 #12 #13 #14 #15
Sub-dimension #1 H M H H H M M M M
2456 15 4
Sub-dim ension #2 H H H M H M M
2620 11 6
Sub-dim ension #3 H H M H
2566 7 10
Sub-dimension #4 H M M M M M
2761 6 9
Sub-dimension #5 M M M M M M
2281 9 2
Sub-dimension #6 M M
2881 2 8
Projects
2681 1 8
4 5 Sub-dimension #1 4 5 High
2847
Medium
2673 2 6 Sub-dimension #2 3 3
2710 7
3 1 High (H)
Sub-dimension #3
2776c 1 5 Risks that are likely to impact the schedule or
2776b 1 4 Sub-dimension #4 1 5 quality of the deliverable
2815 4
Sub-dimension #5 6 Medium (M)
2912 2 1 Risks that can be addressed by the project manager
1 Sub-dimension #6 2 with minimal impact on schedule and quality
2817
0 5 10 15 20 0 2 4 6 8 10
10
Figure 2
cognizant 20-20 insights 3

4. a mitigation plan that addresses them. The The supervisor team, which includes the sponsor,
mitigation plan, or risk response plan, provides project manager, solution lead engineer, require-
options to reduce the likelihood that identified ments engineer and test manager, is responsi-
risks will occur. ble for gathering the requests coming from the
business and setting up an implementation plan
Options and actions are developed in a mitigation in order to get them deployed according to the
plan to enhance project opportunities and project plan. During the evaluation of the project
reduce threats to project objectives to “below an scope, the team sets up a risk plan to evaluate
acceptable threshold.” potential constraints during solution implemen-
tation that would potentially occur during the
Typically, every testing team maintains a mitigation
requirements definition, solution development
plan, which consists of an Excel file containing
and code testing stages.
different columns, including risk description,
date, category, impact areas, root cause, priority, For a tier one financial institution in Europe,
contingency, action plan, mitigation action plan, two run-the-bank projects are considered, one
risk status and risk close date. without a risk management application, and the
other with the TeRAPI application (see Figures 3
Each mitigation action is related to the corre-
and 4). The project scope is to implement several
sponding identified risk. Each action must be
change requests (CRs) for online private banking
planned and time-lined in order to track it and
services. Both projects have a similar normalized
update the probability that the corresponding
initial budget estimated at CHF 400,000.
risk will occur.
The implementation of a structured risk assess-
The mitigation plan is managed by a supervisor
ment process, based on TeRAPI principles, has
team belonging to the testing organization and
produced a cost savings of 19%, or around CHF
responsible for developing mitigation actions
76,000, and reduced identified defects from 17 to
that will address all identified risks. To facilitate
10 (about 60% less).
workload assignments, various workstreams can
be created and assigned to different execution
teams.
Before and After TeRAPI
Project 1 % Difference
If ranked risks are not mitigated properly, they
will occur at certain points in time during testing Initial budget CHF
activities. In that case, the risks become issues, 400,000
and the supervisor team is responsible for iden- Number of defects 17
tifying viable countermeasures and action plans at testing stage
to resolve them.
Additional budget CHF 55,000 13.8%
TeRAPI Process Implementation: allocation to fix
testing bugs
Business Case
A continuous risk management process is a Cost savings – CHF 55,000 -13.8%
necessary part of any approach to software
security. In particular, at the testing stage, the
primary objective is to improve the understand- Project 2 % Difference
ing of the major risks, in order to ensure stable Initial budget CHF
and reliable testing. The TeRAPI approach can 400,000
help test managers increase confidence in the
tests of security function behaviors. Number of defects 10 -58.8%
at testing stage
Risk assessment is widely applied in all market Additional budget CHF 0
industries and large organizations, but it is partic- allocation to fix
ularly important in banking and financial services. testing bugs
“Run-the-bank” projects implement a well-struc-
Cost savings CHF 20,000 5.0%
tured risk assessment process in order to manage
all approved change requests by minimizing the Net cost savings CHF 75,000 18.8%
risks during code testing. This process is set up at
the scope identification stage. Figure 3
cognizant 20-20 insights 4

5. TeRAPI Implementation: Project Advantages
80
60
40
20
Project 1
Project 2
0
Number of defects Additional budget Cost savings Net cost savings
at testing stage allocation
-20
-40
-60
Figure 4
Project 1 has managed about 12 CRs without a a proper due date and responsibility assigned to
risk management plan. It experienced 17 issues at members of the team.
the unit test stage, mainly related to not meeting
requirements in code development. This resulted Prior to unit testing, the number of unaddressed
in a large amount of rework and the need for risks was reduced to 10. During testing for the
additional resources to cope with unexpected first release of the year, the number of issues to
issues, increasing the project budget to CHF be addressed dropped by 60%, producing savings
55,000. of CHF 20,000 compared with Project 1. The net
savings obtained by Project 2 was about CHF
In Project 2, the sponsor decided to use TeRAPI 75,000.
principles in an effort to reduce project expendi-
tures, particularly during the testing stage. The All identified risks were properly weighted to
supervisor team was established, and risks were define risk relevance. The impact on the project
identified. At the beginning, the risk was very implementation strategy, schedule, functionality,
large, with 50 identified risks. For each risk, a costs and quality were weighed, and each value
set of mitigation actions were established, with concurred to define the risk probability.
TeRAPI: Risk Assessment Landscape
Top 15 Risks
100
Risk Type
75
Probability
20%
50  Project Risk
60%  IT Risk
25 20%
 Operational Risk
0
Low Middle High
Impact
Risk Type (Open Risks) Risk Category (Open Risks) Risk Category
Project Risk 3 Project Management 1  Project Management
IT Risk 1 Product Development 2 20% 20%  Product Development
Operational Risk 1 Product  Sponsorship and Budget
1
(procured, operational) 20%  External Risks
Sponsorship 40%
and Budget 1
External Risks 0
Figure 5
cognizant 20-20 insights 5

6. TeRAPI: Risk Assessment Statement
Keyword: (Release Aug’12) - Unit tests execution warning
Risk ID
Description: Due to the project’s dependencies on external parties, there’s a threat that release CRs’
testing could be delayed, which will result into overall timeline shift
Cause/Origin : Project organizational structure involving multiple external parties
49
Identification
Date: 07-Jan-12 Test Manager
Consequences: Overall delay on release timeline
Agreed functionalities during scope definition could not be delivered 100%
Impact Impact on Project
Probability on Program
Risk (%) Strategy Schedule Functionality Costs Quality
30 3 (1- 9) 7 (1- 9) 7 (1- 9) 5 (1- 9) 3 (1- 9)
Response Response Action Due Response
Cost Responsibility Type Date Effectiveness
Planning: 1 Close coordination and weekly tracking with testing team and
external support to verify and address identified test risks. 1'500 Test Manager Mitigation 05-Mar-12 G
Status: 2 Unit test execution planning reviewed daily with external resources responsible. 500 Project and Test Managers Mitigation 05-Mar-12 A
3 Regular followup with release management team and timely escalations. 1'000 Lead Engineer Mitigation 05-Mar-12 A
Open 4 Pre-release testing to increase test coverage. 1000 Project Manager Mitigation 05-Mar-12 A
Closed on:
Reason for
Closure
Risk Type: IT Risk Project
Risk Category: Product Development Area: Project Management
Risk Owner: Project Manager Costs: 10'000 3 Risk Grade
(Risk) Risk Grade
Last updated by: Project Manager 2.10 Program: 0.90
Project:
Last update on: 27-Mar-12 Risk Exposure: 3000.00
Figure 6
The higher ranked risks were shown in a context The final report will take the following into con-
that summarized all major risk properties, such sideration:
as the probability to occur, risk type and category.
• Previous disruption history.
Each risk was given a set of mitigation actions,
with a relative supervisor team member assigned
• Risks and vulnerabilities.
as risk owner and responsible for putting into • Preventative measures.
practice all identified mitigation actions and • Test risk assessment.
tracking risk addressing. A risk assessment
statement form (see Figure 6) collects all infor- • Mitigation plan.
mation concerning identified risk and is the kernel The risk assessment process is an essential
source for the final report. activity of continuity planning, in particular
during the testing phase. Catching errors prior to
TeRAPI Final Report testing the developed solution helps save time,
The risk landscape and mitigation plan will show money and effort and will result in cleaner, better
a clear overview of risks, root causes and action performing and more business-aligned code.
agenda to resolve all challenges. The findings
The TeRAPI process can enable agile and effective
from TeRAPI will form the basis for the final
IT systems to support a more effective business.
report. A risk assessment outcome is the starting
The purpose of the TeRAPI methodology during
point for building the TeRAPI final report. The risk
software testing is to identify high-risk appli-
landscape shows the status of major risks to be
cations that must be tested more thoroughly,
tracked and reviewed periodically, trends across
as well as identifying error-prone components
different reviews and the main categories to
within specific applications that must be tested
which the risks belong.
cognizant 20-20 insights 6

7. more rigorously. The results of the analysis can as the proper level of detail in prioritizing risks, at
be used to determine the testing objectives for each stage of the project lifecycle.
the application during test planning.
In order to implement this assessment process
Conclusion effectively, we estimate a team of three or four
Companies have always had to prioritize their experts is needed to implement the process and
testing efforts, but figuring out how to do it prepare it for practical application for all involved
properly — with the right approach and method- projects.
ology — has not always been clear. To compete,
Our TeRAPI methodology is an effective way of
enterprises must reduce testing costs as low as
minimizing risks during the testing stage, with
possible without sacrificing application quality or
the objective of minimizing delays and costs,
delaying critical application launch time.
and ensuring the highest business benefits. This
TeRAPI can help in this regard. It is backed by approach provides companies with a consistent,
statistical techniques and proven best practices business-based methodology for determining the
gleaned from many years of experience with level of risk present at the testing stage and how
testing code of enterprises worldwide. Our to mitigate it in order to deliver business value
approach ensures the appropriate amount of and the highest possible customer satisfaction.
input from business and technical experts, as well
About the Authors
Simon Erdmann is a Director at Cognizant Business Consulting (CBC) in Frankfurt, Germany. He has over
15 years of experience in operations, management and consulting for the retail, consumer goods and
transportation logistics industry, and is head of Germany, Austria and Switzerland for CBC Strategic
Services. Simon has studied at German Armed Forces University and VWA Essen. He can be reached at
Simon.Erdmann@cognizant.com.
Giuseppe L. Pannisco is a Consultant at Cognizant Business Consulting. He has six-plus years of
consulting experience with global clients on supply chain management, IT strategy and management
issues, working with companies in the automotive, aviation and aerospace, transportation and logistics
service provider fields. Giuseppe holds a master’s degree in aerospace engineering (aeronautical business
management specialization) from Universita’ degli Studi di Napoli “FEDERICO II.” He can be reached at
Giuseppe.Pannisco@cognizant.com.
cognizant 20-20 insights 7

8. About Cognizant
Cognizant (NASDAQ: CTSH) is a leading provider of information technology, consulting, and business process out-
sourcing services, dedicated to helping the world’s leading companies build stronger businesses. Headquartered in
Teaneck, New Jersey (U.S.), Cognizant combines a passion for client satisfaction, technology innovation, deep industry
and business process expertise, and a global, collaborative workforce that embodies the future of work. With over 50
delivery centers worldwide and approximately 140,500 employees as of March 31, 2012, Cognizant is a member of the
NASDAQ-100, the S&P 500, the Forbes Global 2000, and the Fortune 500 and is ranked among the top performing
and fastest growing companies in the world. Visit us online at www.cognizant.com or follow us on Twitter: Cognizant.
World Headquarters European Headquarters India Operations Headquarters
500 Frank W. Burr Blvd. 1 Kingdom Street #5/535, Old Mahabalipuram Road
Teaneck, NJ 07666 USA Paddington Central Okkiyam Pettai, Thoraipakkam
Phone: +1 201 801 0233 London W2 6BD Chennai, 600 096 India
Fax: +1 201 801 0243 Phone: +44 (0) 20 7297 7600 Phone: +91 (0) 44 4209 6000
Toll Free: +1 888 937 3277 Fax: +44 (0) 20 7121 0102 Fax: +91 (0) 44 4209 6060
Email: inquiry@cognizant.com Email: infouk@cognizant.com Email: inquiryindia@cognizant.com
©
­­ Copyright 2012, Cognizant. All rights reserved. No part of this document may be reproduced, stored in a retrieval system, transmitted in any form or by any
means, electronic, mechanical, photocopying, recording, or otherwise, without the express written permission from Cognizant. The information contained herein is
subject to change without notice. All other trademarks mentioned herein are the property of their respective owners.