Identifying and Managing Technical Debt

Identifying and Managing
Technical Debt

Dr. Nico Zazworka
Fraunhofer Center for
Experimental Software Engineering

Dr. Carolyn Seaman
University of Maryland
Baltimore County

http://www.technicaldebt.umbc.edu/

Outline
• Introduction to the Technical Debt metaphor
• Definition and examples of everyday debt
• Benefits of explicitly managing Technical Debt
• Technical Debt Framework
• Technical Debt properties: principal vs. interest
• Recording and communicating Technical Debt
• Identifying important Technical Debt
• Design debt: Code smells, Grime, ASA issues, Modularity
violations
• Other types of Technical Debt
• Management strategies to pay down Technical Debt 2
• Outlook

Software Maintenance
• Large inventory of operational systems that need to be
maintained
• Fixed
• Enhanced
• Adapted
• Such systems need constant modification in order to
remain useful
• Most such systems are too expensive to replace, so
considerable resources go into their maintenance
• However, maintenance, even more than development, is
characterized by tight budget and time constraints 3

Technical Debt
• Technical Debt is the gap between:

4

Technical Debt

• Making a change perfectly
• Preserving architectural design
• Employing good programming practices
and standards
• Updating the documentation
• Testing thoroughly
5

Technical Debt

• And making the change work
• As quickly as possible
• With as few resources as possible

6

Everyday Indicators of Technical Debt

7


“Don’t worry about the documentation for now.”

8



“The only one who can change this code is Carl”

9




“It’s ok for now but we’ll refactor it later!”

10




“ToDo/FixMe: this should be fixed before release”

11





“Let’s just copy and paste this part.”

12






“Does anybody know where we store the database access password?”

13







“I know if I touch that code everything else breaks!”

14








“Let’s finish the testing in the next release.” 15








“Let’s finish the testing in the next release.” 16
“The release is coming up, so just get it done!”

Technical Debt Metaphor

A metaphor, NOT a theory or a scientific concept

17

• Definition
• Incomplete, immature, or inadequate artifact
in the software development lifecycle
(Cunningham, 1992)
• Aspects of the software we know are wrong,
but don’t have time to fix now
• Tasks that were left undone, but that run a risk
of causing future problems if not completed

18

• Benefits
• Higher software productivity in the current release
• Lower cost of current release

19

• Benefits
• Costs
• “Interest” – increased maintenance costs
• Risk that the debt gets out of control

20

• Benefits
• Costs
• “Interest” – increased maintenance costs
• Risk that the debt gets out of control
• Little scientific research, but
• Discussions in blogs, forums, etc.
• Strongly related to Risk Management
21

How Technical Debt is Managed (implicitly)
Wow, this module
is really bad. It’s
going to be very
hard to make any
changes to it.

David Miriam 22
Developer Manager


Hey, Miriam, I think we
should take some time to
refactor this module in the
next release.

David Miriam 23
Developer Manager


Why would we do that?
That would take a lot of
time and effort.

David Miriam 24
Developer Manager


But if we don’t refactor it
soon, I have a gut feeling it’s
going to cause major
problems later.
David Miriam 25
Developer Manager


David is pretty
smart, and he’s
usually right
about these kinds
of things.

David Miriam 26
Developer Manager


David Miriam 27
Developer OK, I’ll put in the plan for Manager
the next release.


Another Example

28

Wow, this module
is really bad. It’s
going to be very Hey, Miriam, I think we
hard to make any should take some time to
changes to it. refactor this module in
the next release.

David Miriam 29
Developer Manager


Those developers
always try to make
their code perfect. I
need some
evidence that this is
worth it.

David Miriam 30
Developer Manager


What is the ROI of this
refactoring?

David Miriam 31
Developer Manager


RO…WHAT?!?
David Miriam 32
Developer Manager


David Miriam 33
Developer Let’s stick with Manager
implementing important
features.

Potential Payoffs of Explicitly
Managing TD
• Lowered maintenance costs
• Avoiding “interest payments”
• Avoiding unnecessary “perfecting” work
• Increased maintenance productivity
• Better prioritization of tasks in each release
• Maintenance always performed on code that is easier to work
with
• Avoiding surprises
• Fewer components that fail without warning
• Fewer unexpectedly large over-budget maintenance tasks
• Better estimation of the costs and risks of postponing
maintenance tasks 34

A Framework for Managing
Technical Debt

TD
Estimation

TD Decision
Identification Making

TD
List 35

Technical Debt List
 A list of TD Items
 Tasks that were left undone, but that run a risk of causing future
problems if not completed.
 Examples: Components/modules/classes that need refactoring, testing
that needs to be done, etc.
 Content of TD Item
 Description – what, where, why?
 Principal – how much will it cost to do the work?
 Interest – what happens if we don’t do this work?
 Amount – amount of extra work if this causes problems later
 Probability – probability that this will cause future problems
 TD List Update Policy
36
 The TD list should be reviewed after each release, when items
should be added as well as removed.

TD Item Example
ID 37

Date 3/31/2008 (Release 3.2)

Responsible Joe Developer

Type Design

Location Method calculateStateTax in Module TaxCalc

Description In the last release, Joe added method
calculateStateTax quickly and method is overly
complex and not documented.
Estimated principal Medium (medium level of effort to refactor and
clean code)

Estimated interest amount: High (it will be costly to make changes to the
method in future, especially by other developers)
37
Estimated interest probability High (it is very likely that this methods needs to be
changed with each future release)

Some definitions
• Principal
• The cost of eliminating a Technical Debt instance RIGHT NOW
• Interest
• The cost, over some period of time, of NOT eliminating a
Technical Debt instance
• Interest is where the risk lies
• Interest probability
• The probability, over a given period of time, that a TD instance will
increase the cost of some future activity
• Interest amount
• Assuming that a TD instance does in fact increase the cost of some
activity, the amount of the increase
• NOTE: Unlike financial debt, Technical Debt interest will change
over time 38

Be careful when using
approaches that quantify
principal only

39

Be careful when using
approaches that quantify
principal only

A statement such as:

“Your project has $1,000,000
Technical Debt.”
40
is only one side of the coin.

So….
The goal of managing TD is…

Eliminating all Technical Debt

41

So….

Eliminating all Technical Debt
NOT

42

So….

To determine when
• The amount of interestavoided justifies
• The cost to pay off the principal

43

Example Scenario
• Technical Debt: One of your code modules is in need of
refactoring
• TD Principal: Refactoring the entire module will cost $10,000
• From past data you have established that:
• This module is modified in 75% of all releases
• The cost of changing this module has gone up 10% each time it’s
been changed over its last 5 changes:
• 5 changes ago cost $10,000
• Last change cost almost $15,000

44

Example Scenario (cont.)
• Technical Debt Computation
• For the next release
• Principal for paying off debt: refactoring the module costs $10,000
• Interest:
• Cost of the next change to the module
• If refactored first: $10,000
• If not refactored first: $16,000
• Extra cost if not refactored: about $6000
• Interest avoided = interest amount * interest probability
• $6,000 * .75 = $4500

Principal Interest Decision:
$10,000 > $4500 Ignore
45

Identifying Important Technical Debt

46

Technical Debt Framework

TD
TD
Estimation
Estimation

TD
TD Decision
Decision
Identification
Making

TD
List 47

What are your goals for
managing Technical Debt?
• What are your pain points?
• Avoiding defects
• Better predictability
• Higher maintenance productivity
• Avoiding surprises
• Making developers happy
• Security, Portability, Efficiency ?
• Motivations are driven by:
• Business domain and market
• Past mistakes
48

Types of Technical Debt
• Design/code debt – can be identified by examining source
code and/or related documentation
• Happier developers and higher productivity
• Fewer defects
• Testing debt – planned tests that were not run, or known
deficiencies in the test suite (e.g. low code coverage)
• Fewer surprises and better predictability
• Documentation debt – missing or inadequate documentation
of any type
• Higher productivity
• Defect debt – known defects that are not fixed
• Happier developers and higher productivity 49

The Technical Debt Landscape
(under construction)
Pain Points Types of Debt Tools

Defect Code
Reliability
Debt Smells

Maintain- Design
ASA Tools
ability Debt

Documentation Modularity
Portability Debt Violations

Testing Manual
Security
Debt Inspection

… … …
50

Understanding your pain points will help to focus on the right types of Technical Debt.

Identifying Design Debt

• ASA issues
(line level)

• Code smells
(method and class level)

• Grime
(class interaction level)

• Modularity violations
(architecture level)

51

ASA Issues
• ASA: Automatic Static (Code) Analysis
• Identify problems on line level:
1 Person person = aMap.get("bob");
2 if (person != null) {
3 person.updateAccessTime(); Potential Null
4 } Pointer Exception
5 String name = person.getName();

• Inexpensive
• Point to the problem, suggest solution
• Gaining significant traction in practice:
• Used by Google to identify problems
• Google Fixit Event
52

Links: http://findbugs.sourceforge.net/

ASA Issue Types

Many (thousands) issues
identified:
Many are false positives:
• False warnings
• Violation, but interest is
negligible
53

ASA Issue Types

Configure tools towards your pain points.

Start with the Technical Debt that is linked to the high priority goals.

Many (thousands) issues
identified:
Many are false positives:
• False warnings
• Violation, but interest is
negligible
54

ASA Tools: Our Recommendation
Project
• Turn OFF all issue types in the ASA tool. Quality
Business
• Activate types “interest-driven”: Goals
• Decide what your quality and business
goals are.
• Prioritize them based on:
• Past experience (user stories)
• Measurable impact
• Research Results:
• Multithread correctness and Correctness
issues are located in classes with higher
defect-proneness
55

Code Smells
• Methods and classes that violate
the principles of good object
oriented design, e.g.:
• Clearly defined
single responsibility
• Encapsulation
• Information hiding
• Few and clear interfaces
• Proper use of inheritance

• Code Smells point to potential
problems:
• require investigation and final judgment by
developer 56
• Set of 20 more or less formally
defined Code Smells available

Code Smells Explained
• Automatic approaches have been proposed and implemented
to automatically detect Code Smells in object oriented code
• Based on Radu Marinescu’s Detection Strategies
• For Java: CodeVizard and Marple
• For C#: ReSharper, CodeRush, Gendarme, FxCop

• Two code smells important for TD:
• god class
• dispersed coupling

57

Further reading: http://www.codevizard.com

Code Smells: Our
Recommendation
• If avoiding defects and increasing maintenance productivity
are issues of concern, then…
• Start by detecting and examining God Classes and Dispersed
Coupling code smells
• Prioritize modules with these smells for refactoring efforts
• Research focus: God Classes (concept is easy to understand)
• God Classes are 5-7 times more change prone
• God Classes are 4-17 times more defect prone
• Baseline from our experience: most systems have 2%-8% God
Classes
• Dispersed Coupling code smell points to defect and 58
maintenance prone classes

Design Patterns and Grime
• Design patterns promise code to be more maintainable
and less defect prone
• Describe how multiple classes work together
• Design patterns can decay over time as systems evolve
• Grime: accumulation of non-pattern code in classes
following a design pattern
• Rot: changes that break the integrity of a design
pattern
• Early results show that grime has a noticeable effect on
testability
• As grime builds up, more test cases break
• In turn affects productivity during the testing phase 59
• Leads to testing debt

Modularity Violations
• Organization of software systems: inter-dependent modules
• Proper architecture leading to a clear structure of relationships
promotes reuse of modules and smaller ripple effects.
• Dependencies indicate how modules should change together:
• Example:
If the Model is changed, Controller A
Model
and Controller B might require
changes.
Controller Controller
• Modularity Violations: recurring A B
changes on classes within modules
that are not depending on each other: View 1 View 3
• Example: Classes in View 1 and View 3
changing together 60
View 2

Modularity Violations Research
• Studies have shown that modularity violations are an excellent
indicator of defect prone classes and change prone classes.
• Tool: CLIO (Drexel University)
• When applied, with other TD detection approaches, to an
open source system, the results for predicting defects were:

• Also, modularity violations were highly correlated with 61
modules that developers later chose to refactor
Further reading: http://www.slideshare.net/miryung/icse-2011-research-paper-on-modularity-violations


TD
TD
Estimation
Estimation

TD
TD Decision
Decision
Identification
Making

TD
List 62

Testing Debt
“There's no tests for buttons other
• Tests that were planned but: than <input type="submit"> yet. I'm
• not implemented pretty sure also <input
type="button"> works if other
• not executed <input>s work, but <button
disabled="disabled">Text</button>
• or they got lost should be tested separately.”
http://code.google.com/p/robotframework-seleniumlibrary/issues/detail?id=163

• Inadequate tests
• Test cases not updated for “While updating the package of
html5lib to 0.90 in Debian I
new/changed functionality realized that the unit tests are
gone. To ensure the keep the
• Low coverage package in a good working shape
while it transitions trough new
• Can be detected by: Python versions and new versions of
the modules it depends on, it would
• Comparing test results with test be *very* appreciated if the unit
plans tests would be shipped in the
zipfile again.”
• Code coverage tools http://code.google.com/p/html5lib/issues/detail?id=134&colspec=ID%2
0Type%20Status%20Priority%20Milestone%20Owner%20Summary%20P

• Comparing requirements ort
63
changes with test suite changes

Documentation Debt
• Documentation that is not “Except there is no such class or
field in the SDK. It is outdated
kept up-to-date, e.g. documentation that definitely needs
to be updated.”
• Installations and run http://code.google.com/p/android/issues/detail?id=8483

instructions
• Architecture “There is not much documentation
documentation available regarding the format
of .xclangspec files. As a starting
• Requirements and use case point, see for instance the
outdated documentation at:
documentation http://maxao.free.fr/xcode-plugin-
• API documentation interface/specifications.html”
http://code.google.com/p/go/source/browse/misc/xcode/go.xclangspec
?r=30b0c392132645259e053a2ba8904383a55bab03
• Can be detected by:
• Comparing code changes “This was apparently the old
behavior and it's changed
with documentation now, but the documentation
changes doesn't so say.”

• Comment density metrics
http://code.google.com/p/redis/issues/detail?id=514
64

Defect Debt
“There are a couple of latent
• Known defects that are not bugs in the linux TLS
implementation. I'm filing a
yet fixed single issue because they are
so small and easy to fix.”
• Low priority defects http://code.google.com/p/dynamorio/issues/detail?id=358

• Low severity defects
• Manifest rarely
• Workarounds present
• Can be detected by:
• Examining defect
repositories
• Test results

65

Bottom line
• Different techniques detect different instances and types of
technical debt
• No one approach is sufficient by itself
• No one approach is the right one for everyone
• The solution is a strategy based on
• Business and development goals
• Most painful types of debt
• A combination of approaches that focus on the most pain
• Don’t try to automate it all
• Some kinds of technical debt can only be detected by humans
• Most kinds of technical debt can only be interpreted by humans
• No substitute for talking about it 66

Technical Debt in Decision Making

67


TD
Estimation

TD Decision

TD
List 68

TD Attributes
 Three attributes of a TD item
 Principal
 Interest probability
 Interest amount
 Start with a rough estimate of the attribute values
 High, Medium, Low
 Defer more precise estimation until data is available
 Fault detection ability and defect density => testing debt
 Cost of fixing a defect pre-release & post-release => defect debt
 Time and effort for updating documentation => documentation debt

69

TD Attribute Estimation
• Principal => historical effort data
• Interest Probability => historical usage, change, and defect data
• Example questions
• How likely is it that a defect will occur in the untested part?
• How likely is it that code containing a known error will be exercised?
• A time element
• Interest amount
• Assume that the item has an effect on future work
• Example questions
• How much more it will cost to deal with defects later in the system’s lifetime than in
testing?
• These are all hard to estimate with any certainty
• Historical data will help
• Any estimation is better than the current method – “gut feeling”
70

Decision Making Scenario
• Question
• When and which technical debt items should be paid?
• Example
• Significant work is planned for component X in the next release, should
we pay down some debt on component X at the same time?
• Assumptions
• There is an up-to-date TD list that is sorted by component and has
high, medium, and low values for principal and interest estimates for
each item.
• Process

Select Re-evaluate Estimate Compare Add up 71

Other Decision Models
• The proposed TD management strategy is based on a simple
cost/benefit analysis
• But TD occurs in complicated development and business
scenarios
• TD items are inter-related
• Business factors are important, too
• Prediction is hard
• Other strategies for making decisions might be appropriate
• Portfolio model
• Options
• AHP
72

So where are we?

73


TD
Estimation

TD Decision

TD
List 74

“Avoid being a perfectionist in a
world of finite resources.”
Forrest Shull

Instead…

Manage Technical Debt to make the imperfections
• documented
• explicit
• not so scary

75

Summary
• Technical Debt is a metaphor that describes a very real
phenomenon
• Provides a way to talk and reason about the difficulties of
software maintenance
• Technical Debt comes in a variety of forms, all of which can be
detected in different ways
• Technical Debt can be documented and tracked in a TD list
• Management of Technical Debt must involve consideration of
interest, not just principal
• The types of Technical Debt that are relevant for a particular
situation depends on past experience, organizational culture,
and business environment.
• While the research is still early, it does provide some guidance 76
as to Technical Debt identification strategy.

What’s next…
• Identify your pain points
• Decide what types of Technical Debt are relevant for you
• Choose a small set of tools and indicators
• Start a TD list – can use our template - probably some
developers already have one
• Track the history of the TD items revealed by the tools to see if
they are detecting “real” debt
• Refine release planning process to incorporate TD
• Track your success in reducing the “pain”
• Add new detection strategies to fill the gaps
• Call us if you need help 77
• Tell us how it’s going!

Contact us…

Nico Zazworka
zazworka@gmail.com

Carolyn Seaman
cseaman@umbc.edu
78

Acknowledgments
• UMBC
• Yuepu Guo, Ph.D. candidate
• Rodrigo Spínola, Postdoctoral Researcher
• Fraunhofer Center
• Forrest Shull, Division Director
• Many past, current, and future interns
• Other academic collaborators
• Yuanfang Cai, Drexel University
• Clemente Izurieta, Montana State University
• Antonio Vetró, Ph.D. student, Politecnico Di Torino, Italy
• Sunny Wong, now at Siemens Healthcare
79

Acknowledgments (cont.)
• Industrial collaborators

• NSF Grant #0916699:
Measuring and Monitoring Technical Debt
80

References
Ayewah, N. et al. (2007). Evaluating static analysis defect warnings on production software. In Proceedings of
the 7th ACM SIGPLAN-SIGSOFT workshop on Program analysis for software tools and engineering - PASTE
’07. New York, New York, USA: ACM Press, pp. 1-8. Available at:
http://dl.acm.org/citation.cfm?id=1251535.1251536 .
Baldwin C. and Clark K. (2000). Design Rules, Vol. 1: The Power of Modularity. MIT Press.
Brown N., Cai Y., Guo Y., Kazman R., Kim M., Kruchten P., Lim E., MacCormack A., Nord R., Ozkaya I., Sangwan
R., Seaman C., Sullivan K., and Zazworka N. (2010). Managing technical debt in software-reliant systems.
FoSER 2010: 47-52.
Cunningham W. (1992). The WyCash Portfolio Management System. Addendum to the proceedings on
Object-oriented programming systems, languages, and applications, pp. 29-30, 1992.
Fowler M. (2003). Technical Debt. Available: http://www.martinfowler.com/bliki/TechnicalDebt.html
Guéhéneuc Y.G., and Albin-Amiot H. (2001). Using Design Patterns and Constraints to Automate the Detection
and Correction of Inter-Class Design Defects. Proc 39th International Conference and Exhibition
Technology of Object Oriented Languages and Systems, pp. 296-305, 2001.
Guo Y., Seaman C., Zazworka N., and Shull F. (2010). Domain-specific tailoring of code smells: an empirical
study. International Conference on Software Engineering, ERA Track Cape Town, South Africa, May.
Guo Y., and Seaman C. (2011). A Portfolio Approach to Technical Debt Management. Workshop on Managing
Technical Debt. Hawaii, USA, May.
81

References
Guo Y., Seaman, C., Gomes R., Cavalcanti A., Tonin G, Da Silva F.Q.B., Santos A.L.M., Siebra C. (2011). Tracking
Technical Debt – an exploratory case study. International Conference on Software Maintenance,
Williamsburg, VA, September.
Izurieta C., and Bieman J.M. (2008). Testing Consequences of Grime Buildup in Object Oriented Design
Patterns. 1st ACM-IEEE International Conference on Software Testing, Verification and Validation, ICST ’08,
Lillehamer, Norway, April 2008.
Kim S. and Ernst M. (2007). Prioritizing Warning Categories by Analyzing Software History. In Fourth
International Workshop on Mining Software Repositories (MSR’07:ICSE Workshops 2007). IEEE, pp. 27-27.
Available at: http://dl.acm.org/citation.cfm?id=1268983.1269041.
Marinescu R. (2004). Detection strategies: Metrics-based rules for detecting design flaws. IEEE International
Conference on Software Maintenance. Pp. 350–359.
Markowitz H. (1952). Portfolio Selection. the Journal of Finance. Vol. 7, No. 1, pp. 77-91.
Rothman J. (2006). An Incremental Technique to Pay Off Testing Technical Debt. Available:
http://www.stickyminds.com/sitewide.asp?Function=edetail&ObjectType=COL&ObjectId=11011&tth=DYN
&tt=siteemail&iDyn=2
Saaty T. L. (1982). Decision making for leaders: The analytical hierarchy process for decision in a complex
world. Belmot, California: Lifetime Learning Publications.
Schanz T., and Izurieta C. (2010). Object Oriented Design Pattern Decay: A Taxonomy. 4th ACM-IEEE
International Symposium on Empirical Software Engineering and Measurement, ESEM ’10, Bolzano-Bozen, 82
Italy, September 2010.

References
Seaman C., and Guo Y. (2011). Measuring and Monitoring Technical Debt.
ADVANCES IN COMPUTERS. Vol. 82, pp. 25–46.
Sortino F. A., and Price L. N. (1994). Performance Measurement in a Downside Risk
Framework. the Journal of Investing. Vol. 3, No. 3, pp. 59-64.
Sullivan K., Chalasani P., Jha S and Sazawal V. (1999). Software Design as an
Investment Activity: A Real Options Perspective in Real Options and Business
Strategy: Applications to Decision Making. Risk Books, November.
Vetró A., Torchiano M., and Morisio, M. (2010). Assessing the precision of FindBugs
by mining Java projects developed at a university. In Mining Software
Repositories (MSR), 7th IEEE Working Conference on. pp. 110-113.
Vetró A., Morisio M, Torchiano, M. (2011). An Empirical Validation of FindBugs
Issues Related to Defects. Evaluation and Assessment in Software Engineering
2011 (EASE 2011). Durham City, UK.
Wong S., Cai Y., Kim M., and Dalton M. (2011). Detecting software modularity
violations. Proc. 33th International Conference on Software Engineering. May
2011, pp. 411–420.
Zazworka N., Shaw M., Shull F., Seaman C. (2011). Investigating the Impact of
Design Debt on Software Quality. Workshop on Managing Technical
Debt, Hawaii, USA, May. 83

Tools
• Java (open source):
• FindBugs: http://findbugs.sourceforge.net/
• PMD: http://pmd.sourceforge.net/pmd-5.0.0/
• Checkstyle: http://checkstyle.sourceforge.net/
• C#/.NET (commercial)
• ReSharper: http://www.jetbrains.com/resharper/
• FxCop: http://msdn.microsoft.com/en-us/library/bb429476.aspx
• CodeRush: http://go.devexpress.com/CodeRushX.aspx
• Multi-language Framework (open source)
• Sonar: http://www.sonarsource.org/
• List of static analysis tools (over 100 tools for more than 10
languages):
• Wikipedia:
http://en.wikipedia.org/wiki/List_of_tools_for_static_code_analysis 84

Further Reading and Survey
• Further Reading:
• Our Research Group’s website:
http://www.technicaldebt.umbc.edu/
• OnTechnicalDebt
http://www.ontechnicaldebt.com/

• 5 minute online survey about common beliefs on TD

http://tinyurl.com/TechnicalDebt 85

Identifying and Managing Technical Debt

Recommandé

Recommandé

Contenu connexe

Tendances

Tendances (20)

Similaire à Identifying and Managing Technical Debt

Similaire à Identifying and Managing Technical Debt (20)

Dernier

Dernier (20)

Identifying and Managing Technical Debt