This document discusses agile metrics for measuring value, flow, quality, and culture. It presents common metrics used by Scrum teams like velocity, burnup/burndown, lead time, and defects. Flow-related metrics across the development lifecycle are explained, including the differences between cycle time and lead time. The document also discusses measuring value, happiness, culture, and frameworks for metrics like SAFe. Key takeaways are that outcomes rather than activity should be measured, and that culture, collaboration, and safety are important but difficult to measure metrics.

1. Agile Metrics:
Value, Flow, Quality, Culture
by Brad Appleton & Mahul Patel
27 October 2020
Delivery Frequency
Value Quality
Culture
(Social)
Flow
(Delivery)
Improve

2. About the Presenters
● Currently working as Agile/Scrum Transformation
Leader since 02/2020 at Discover
● Managed several technology initiatives as a IT
Management Leader for fortune 500 companies
● Lean Six Sigma leader experience with diverse
global groups to increase operational efficiencies
● Agile/DevOps Leader & Coach
● Leading Agile adoption & scaling for teams and
organizations since 2000.
● DevOps/ALM/SCM Solution Architect
● Publications
» Software Configuration Management Patterns
(Addison-Wesley, 2002)
» Agile CM Environments column & blog (AgileConnection
& CMCrossRoads, 2003-2014)
Suburban Agilists – 27 October 2020 2
Brad Appleton
Agile/DevOps Leader
<brad@bradapp.net>
Mahul Patel
Agile/Scrum Leader
<patelmahul@yahoo.com>

3. Outline & Contents
1. Introduction [5min]
▪ Metrics – Motivation & Goals
▪ Focus on Agile or Agility?
▪ Kinds of Metrics (Categories, Scope/Scale)
2. Poll – What Kinds of Metrics Have You Used? [10min]
3. What are the Metrics that Matter (Most) [25min]
▪ Value, Flow, Quality, Culture
▪ Collaboration/Culture, Happiness & Psychological Safety
▪ Examples: Flow Framework, SAFe, Agility Health Radars
4. Quiz [5min]
▪ General quiz on what we covered
5. Q&A and Concluding Thoughts [5-10min]
6. References & Additional Material
Suburban Agilists – 27 October 2020 3
Happy Halloween!

4. Metrics - Motivation & Goals
Suburban Agilists – 27 October 2020 4
New threats continue to put pressure on Business’ need to deliver quickly and accelerate change
We want to gauge how our ways of working & interacting affect business outcomes
● Do we have the insights to outpace our competitors and pursue the most impactful opportunities?
● Do we have strong feedback loops?
Various Transformation Initiatives are also “on the rise” (Digital, Lean/Agile, DevOps, Business Agility)
● Often using “frameworks” and stages/levels of maturity, mastery or “improvement targets”
● To gauge (even compare) “progress” over the course of our (continuous) improvement journeys
Improve
Product Health
Reduce Failure
Rates
Improve
Predictability
Reduce Time
to Market
Execute
Capability
Improvement
Plans
Measure
& Report
Outcomes
Review
Dashboards
& Insights

5. Measure Agile or Agility?
Suburban Agilists – 27 October 2020 5
Defined by
Values
Guided by
Principles
Manifested thru
many different
Practices (and Numerous
Frameworks)
Agile is a
Mindset
For Business Outcomes
Time to Market
Predictability
Customer Satisfaction
Employee Satisfaction
Reliability
Resilience
Enabling Agility
… the Capability to
Create & Adapt to Change
for Competitive Advantage
in a VUCA Environment
Being Agile Doing Agile

6. What Kind of Metrics Matter?
Measure/Improve …
● Product? | Project?
● Process? | People?
● Environment? | Evolution?
Scope/Scale …
● Unitary | Individual | Personal
● Team | Application | Service
● Organization | Product | Stream
● Program | Market | System
● Enterprise | Portfolio | Business
Suburban Agilists – 27 October 2020 6
Satisfy the Customer
Valuable Software
Early & Continuous
Welcome Changing Requirements
Frequently (Sooner)
Sustainable Pace
Servant-Leadership
Self-Organization
Face-to-Face
Motivated
Work Together Daily
Working
Software
Simplicity
Technical Excellence
Continuous
Improvement
(Inspect & Adapt)
Should They Align with Agile Values+Principles? Outcomes?
Better
Safer/Happier
Sooner

7. What are the Metrics that Matter?
Measure/Improve …
● Product? | Project?
● Process? | People?
● Environment? | Evolution?
Scope/Scale …
● Unitary | Individual | Personal
● Team | Application | Service
● Organization | Product | Stream
● Program | Market | System
● Enterprise | Portfolio | Business
Suburban Agilists – 27 October 2020 7
Culture (Social)
Value
Flow (Delivery)
Quality
Continuous
Improvement
(Inspect & Adapt)
Are we realizing
anticipated value?
(benefits, impact, revenue)
Are we building passionate,
engaged teams & learning
organizations with happy
stakeholders?
How are we improving our
capability to sense, respond
to, and deliver (valued)
changes to production?
Are we building quality in
(and “shifting left”) with
minimal waste/rework?
Should They Align with Agile Values+Principles? Outcomes?

8. Suburban Agilists – 27 October 2020 8
Interactive Poll:
What Metrics Have You Seen?
Value
(Customer) Valuable
Quality
Working Software
Simplicity
Technical Excellence
Culture (Social)
Sustainable Pace
Servant-Leadership
Self-Organization
Face-to-Face
Motivated
Work Together Daily
Safer/Happier
Flow (Delivery)
Early & Continuous
Welcome Changing Requirements
Frequently Sooner
Continuous
Improvement
Better
Delivery Frequency

9. What are the Metrics that Matter?
Suburban Agilists – 27 October 2020 9
Culture (Social)
Value
Flow (Delivery)
Quality
 Defects (by Severity)
 Defect Arrival/Closure/Repair Rate
 DefectVolume/Density
 Test Coverage / %Automation
 Passed/Failed/Blocked/Waived
 Technical Debt (SQALE)
 Change Failure Rate/Percentage
 Uptime/Down-time
 Collaboration (Cross Functional)
 Team Mood/Morale/Coherence
 Happiness / Pain
 Experimentation (Learning / Innovation)
 Psychological Safety
 Employee Engagement/eNPS
 Culture (Westrum | LaLoux)
 Lead-Time/Cycle-Time
 Burnup/Burndown
 Work-in-Process (WIP)
 Throughput/
 Commit/Build/Int/Deploy Frequency
 Mean-Time To Repair/Recover/Remediate
 Flow Efficiency
 Wait/Delay (w/Value-Stream Mapping)
 Cost (e.g., per service/unit/release/transaction)
 Cost of Delay, Downtime
 Customer Value
 Customer Satisfaction
 Net Promoter Score (NPS)
 ROI | NPV | CapEx | OpEx
 AARRR (Pirate Metrics)
– Acquisition, Activation, Revenue, Retention, Referral Transformation
 Mastery/Maturity
 To Scale/Scope

10. Most Common Agile & DevOps Metrics Used
Common Scrum Team Metrics:
● Velocity / Capacity
● Burnup / Burndown
● Cycle-Time / WIP
● Open Defects / Arrival-vs-Closure
● Test Coverage, %Passed/Failed/Blocked/Waived
Common DevOps Metrics:
● Deployment Frequency
● Lead-time for Change
● Change Failure Rate
● MTTR (Mean-Time To Restore/Recover/Repair)
Suburban Agilists – 27 October 2020 10
“Baker’s Dozen” Top Agile/DevOps Metrics
1. Velocity/Capacity
2. Burnup/Burndown
3. WIP | Cumulative Flow
4. Change Lead-Time
5. Delivery Frequency
6. Change Volume/Throughput
7. Delivery Success/Failure %
8. Mean-Time to Repair/Resolve/Recover/Restore
9. Availability/Uptime
10. Defect Density/Volume/Leakage
11. Downtime/Outages/MTBF
12. Culture/Collaboration/Experimentation
13. Customer/Employee Satisfaction

11. Flow & Quality: What Behaviors are we trying to Drive?
Suburban Agilists – 27 October 2020 11
 Focus on regular flow of features/changes through the delivery pipeline
 Shorten cycle-times & feedback-loops
 Reduce size of features/work-packages and the number in-progress
 Increase frequency of feedback on quality (and from stakeholders)
 Increase cross-lifecycle collaboration & breakdown silos
 Identify and remove dependencies, inconsistencies & delays
 Reduce likelihood of quality problems and failed deliveries/builds
 Improve resilience to change, and time to respond/repair/resolve

12. Flow-related Metrics across Development Lifecycle
Suburban Agilists – 27 October 2020 12
Application/Service Delivery Pipeline
Defects/FixesUser Stories Enhancements/
Change-RequestsFeatures
Test Stage ProdDeliverReady SCM
</>
Development
</>

Commit
Build
Demand
Package
. . .
Continuous
Deployment
Continuous
Integration
Agile/
Iterative
Continuous
Delivery
Feedback
Program/
Initiative
Response-Time
Development Cycle-Time (for an enhancement/story/change)
Wait-Time
Deployment
Time
Deployment Latency
Open Analysis Code &
Unit-Test
Integration
Build+Test Test Accept DeployReady
for Dev
. . .
Closed
Lead-time for Change (from code-commit to production-deployment)
(Feature/Story) Lead-time to Market/Value
Defect Resolution-Time

13. Lead-Time vs Cycle-Time (What’s the Difference?)
Suburban Agilists – 27 October 2020 13
http://www.full-stackagile.com/2016/04/03/improve-work-process-whats-cycle-time/
https://stefanroock.wordpress.com/2010/03/02/kanban-definition-of-lead-time-and-cycle-time/
Cycle Time
Order
Requested
Work
Started
Work
Completed End-Result Received
(in Production)
Prioritized
& Ready
Response Time
Wait Time
Lead Time
[a.k.a. Resolution Time]

14. “Continuous” Pipeline Metrics (by discipline)
Suburban Agilists – 27 October 2020 14
Change Lead-Time/Cycle-Time
Change Volume
Change Failure %
Defect Leakage
Defect Arrival/Closure Rate
Continuous
Design
Continuous
Integration
& Build
Continuous
Testing
& Quality
Continuous Delivery
& Deployment
Continuous
Operations &
Monitoring
Technical Debt
MTT Green-Bar
Commit-size
Code Coverage
Pull-requests
#Commits/Day
MTT Main-Trunk
Build-Time
Build Frequency
Build Failure %
MTTR
Test Cycle-Time
Test Coverage
#Tests Run Rate
Test Block/Fail %
MTT Detect/Correct
MTBF/Outages
MTTRS
Availability
#Incidents
Ticket Volume
Response-Time
Continuous
Planning
Continuous Provisioning & Infrastructure as CodeEvolutionary Architecture
Deliver/Deploy Cycle-Time
Deliver/Deploy Frequency
Deliver/Deploy Failure %
MTT Repair Failed D/D
Env Provision Cycle-Time
Env Provision Fail %
Env Availability
Time-to-Respond
Time-to-Decision
Capacity/Velocity
Burnup/Burndown
Sustainable Pace
-- by Brad Appleton, 2017

15. Flow Framework (Mik Kersten)
Suburban Agilists – 27 October 2020 15
FlowFramework.org ProjectToProduct.org

16. Value Metrics
Suburban Agilists – 27 October 2020 16
www.scrum.org/resources/evidence-based-management
www.slideshare.net/rsrivastava91/agile-metrics-v6
NPS

17. Happiness / Team Morale / Nico Nico
Suburban Agilists – 27 October 2020 17
Niki Niko Chart
Source: https://www.plays-in-business.com/happiness-index-how-to-measure-something-so-difficult-to-catch/

18. Culture, Psychological Safety
Suburban Agilists – 27 October 2020 18
Source: https://risk-engineering.org/concept/safety-culture
Schein’s “onion layer” model of organizational culture
Westrum Culture Survey
Respect

19. The Dangers of Velocity
DO NOT Equate Velocity with Productivity or Performance!
 And DO NOT compare velocity across teams or organizations
Suburban Agilists – 27 October 2020 19
Why Agile Velocity is the Most Dangerous Metric for Software Dev Teams: linearb.io/blog/why-agile-velocity-is-the-most-dangerous-metric-for-software-development-teams

20. Observations from Prior Metrics Collection Efforts
Suburban Agilists – 27 October 2020 20
• Shared responsibility in reporting (across development, release-management,
QA, and operations) vs. a single point-person had the most success quickly
determining how best to collect & report the right data
Collective
Ownership
• Siloed data across multiple tools/repositories (often from working in silos)
• Inconsistent naming & usage of workflow states and data across organizations
• Incorrect execution of transitions (skipped and/or late) create inaccurate data
Inconsistent
Usage
• Using a tool to track all work-items was more efficient & effective for reporting
• Using the tool to track backlog-items + defects + incidents made it easier to
report quality & flow than those using no/separate/loosely-integrated tools
• Teams that manage and perform their work as individual changes across the
entire AppDev Delivery pipeline, tend to report better cycle-times
• They are also more able to easily capture and report lead-time to production
Workflow
across the
Pipeline
Work-item
tracking

21. Iteration Performance Program Performance
EXAMPLE: SAFe Metrics
Suburban Agilists – 27 October 2020 21
Solution Train Performance Lean Portfolio Metrics
scaledagileframework.com/metrics/ scaledagileframework.com/measure-and-grow/

22. EXAMPLE: SAFe Charts
Suburban Agilists – 27 October 2020 22
www.scaledagileframework.com/metrics/

23. EXAMPLE: Agility Health Radars (agilityhealthradar.com/radars/ )
Suburban Agilists – 27 October 2020 23

24. Suburban Agilists – 27 October 2020 24
Interactive Quiz:
What Did We Learn?
Delivery Frequency
Value Quality
Working Software
Simplicity
Technical Excellence
Culture (Social)
Sustainable Pace
Servant-Leadership
Self-Organization
Face-to-Face
Motivated
Work Together Daily
Safer/Happier
Flow (Delivery)
Early & Continuous
Welcome Changing Requirements
Frequently Sooner
Continuous
Improvement
Better
Satisfy the Customer
Valuable Software

25. Enquiry: Is it ALL/EVERYTHING about Flow?
Suburban Agilists – 27 October 2020 25
Feelings?
Value
Form/Function?
Failure?
Flow of
Improvement
& Learning?
Flow of ???
Social/Emotional
Flow?
Flow of
Features/Changes?
Flow of “waste” &
waste removal?
References: Thermodynamics of Emotion [1], [2], [3], [4]

26. Dilbert Cartoons (Metrics Humor)
Suburban Agilists – 27 October 2020 26

27. Cautions, Caveats & Pitfalls
Comparing metrics across teams: Cross team comparisons inhibit safety & learning. (“Fear is
the mindkiller!”)
Focusing on lagging over leading Indicators: Too late for meaningful, impactful action!
Vanity Metrics: Good for feeling awesome, but bad for taking action.
Ignoring companion measures: Measure holistically, so you see the tradeoffs.
Hawthorne Effect: Individuals modify their behavior in response to their awareness of being
observed.
Friedman’s Thermostat: Correlation does not mean causation, but it sure can be a clue.
Campbell’s Law: The more any quantitative social indicator is used for social decision-making,
the more subject it will be to corruption pressures and the more apt it will be to distort and
corrupt the social processes it is intended to monitor.
Goodhart’s Law: Once a measure becomes a target, it ceases to be a good measure.
Suburban Agilists – 27 October 2020 27
Source: https://agilevelocity.com/article-metrics-in-agile-how-to-measure-transformation/

28. Summary / Conclusions / Key Take-aways
 Measure Outcomes / Systems (not activity/busy-ness of people)
 Steer Clear of Velocity / Productivity (prefer Flow instead)
 Culture, Collaboration & Safety are Important (and also harder to measure)
 Watch out for Survey Fatigue for soft/manual Metrics collection
 Tools/Dashboards/Automation are helpful (IF measure the right things / right way / Right Time)
 Measuring People / Teams can be very tricky (Easily Gamed, Campbell+Goodhart, Hawthorne)
 “People with targets and jobs dependent upon meeting them will probably meet the
targets – even if they have to destroy the enterprise to do it.” -- W. Edwards Deming
 “Tell me how you measure me, and I will tell you how I will behave!” --E.M. Goldratt
Suburban Agilists – 27 October 2020 28

29. Suburban Agilists – 27 October 2020 29
References & Resources
THANK YOU!
Culture
(Social)
Value
Flow
(Delivery)
Quality
Continuous
Improvement

30. Some Recommended Books [Each image is a hyperlink]
Suburban Agilists – 27 October 2020 30

31. More Agile Metrics Resources
● InfoQ.com/Metrics | AgileAlliance.org/?s=Agile+Metrics (List of Metrics-related articles)
● Appropriate Use of Metrics for Agile (Martin Fowler)
● Measuring Agile: A Disciplined Approach to Metrics (Scott Ambler)
● Understanding Agile Team Metrics and 30+ Metrics for Agile Teams (Andy Clef)
● Agile Metrics: The Ultimate Guide (Extreme Uncertainty) | Definitive List of Agile Metrics (HotPMO.com)
● Agile Metrics: The 15 That Actually Matter for Success (Plutora.com)
● 10 Powerful Agile Metrics – and 1 Missing Metric (Sealights.io)
● 17 Agile Metrics Your Team Should Care About, including “Vorticity” (LearningHub)
● Agile Metrics: The Good, The Bad, and the Ugly (Stefan Wolpers)
● Determining Agile Metrics (Globant) | Understanding Agile Metrics (Eliassen)
● Definitive Software Quality Metrics for Agile+DevOps (Forrester-Tricentis)
● Agile Measurement & Metrics for Accountability (ValTech)
● Aligning Agile Metrics to your Vision [PDF: How Good? How Much? How Fast? How Happy?]
Suburban Agilists – 27 October 2020 31

32. The Dangers of Velocity
References:
● Why Agile Velocity is the Most Dangerous Metric for Software Dev Teams
● Velocity: the Killer of Agile Teams, by Tomas Kejzlar (Skeptical Agile)
● Agile Metrics: Velocity is NOT the Goal (AgileAlliance)
● Why Not Use Velocity as a Metric? (Esther Derby) | Moving Beyond Velocity (DZone.com)
● Velocity is a rubbish metric. (Fight me.) (DevBridge) | The Problem with Velocity (SolutionsIQ)
● Scrum Velocity: 5 Things that can Go Wrong (SeaLights)
Suburban Agilists – 27 October 2020 32
blog.axosoft.com/measure-agile-metrics-that-work

33. Resources: Flow Metrics & Framework
FlowFramework.org | ProjectToProduct.org
● willys-cave.ghost.io/project-to-product-value-stream-metrics/
● uxdesign.cc/flow-framework-a-fad-or-here-to-stay-3a26ce99d48d
● medium.com/@Tasktop/moving-from-project-to-product-with-flow-metrics-what-are-they-and-why-should-you-care-
24346a9c1403
● www.tasktop.com/blog/how-the-flow-framework-maximizes-your-wins-from-safe/
● projecttoproduct.org/wp-content/uploads/2020/10/Getting-Insight-into-Flow-with-Tasktop-Flow-Metrics-1.pdf
Flow Thinking / Flow System
● https://www.infoq.com/articles/the-flow-system/
● The Flow System: The Evolution of Agile and Lean Thinking in an Age of Complexity
● https://flowguides.org/guide.php
Suburban Agilists – 27 October 2020 33

34. Resources: Happiness Metric/Index
● www.scrum.org/forum/scrum-forum/5997/methods-collect-happiness-index
● angryagilist.com/2018/08/30/agile-happiness-metric-why-do-we-care/
● www.andycleff.com/2020/05/agile-team-health-check-models/
● www.scruminc.com/happiness-metric-wave-of-future-2/
● agileleanlife.com/the-happiness-index-and-happiness-chart/
● medium.com/the-liberators/agile-teams-dont-use-happiness-metrics-measure-team-morale-3050b339d8af
● www.agilecockpit.com/blog/agile/team-morale-and-team-happiness-indicators/
● www.plays-in-business.com/happiness-index-how-to-measure-something-so-difficult-to-catch/
● nevadasmallbusiness.com/engaged-employees-happy-employees-yes/
● sites.google.com/a/scrumplop.org/published-patterns/retrospective-pattern-language/happiness-metric
Niko-Niko charts (pros & cons)
● www.agilealliance.org/glossary/nikoniko/
● www.tinypulse.com/blog/sk-niko-niko-calendar-workplace-morale
● medium.com/@mohsenahmadi/niko-niko-calendar-its-important-to-any-agile-team-62e4b2aa4410
● agiletrail.com/2011/09/12/how-to-track-the-teams-mood-with-a-niko-niko-calendar/
Suburban Agilists – 27 October 2020 34

35. Resources: Culture & Transformation
● Measuring Culture In Leading Companies (MIT Sloan Review) | The New Analytics of Culture (HBR)
● How To Measure Organizational Culture & Values (SurveyAnyPlace)
● 8 Ways to Measure Company Culture (Interact/Medium)
● How the Best Organizations Measure Culture (CultureAmp)
● How to Measure Organizational Culture Change? (MyHRFuture)
● The 10 Company Culture Metrics You Should Be Tracking Right Now (Entreprenuer.com)
● Measuring Agile Culture with Agile IQ (ZenExMachina)
● Westrum Typeology for Agile/DevOps Organizational Culture (Google, also Andy Kelk)
● LaLoux Culture Model (slides): BeyondTeal Readiness Assessment | Reinventing Organizations Map
Transformation
● How to Effectively Measure Your Agile Transformation Journey (AgileVelocity)
● 5 Agile KPIs for Measuring Success (NetMind)
● Agile Transformation Metrics (Dan Fuller & Chris Waggoner)
Suburban Agilists – 27 October 2020 35

36. Resources: Collaboration & Safety
Measuring Collaboration
● Guide to Measuring Impact with Agile Metrics (Excella)
● Review & Collaboration Metrics Overview (PluralSight)
● 16 metrics for tracking Collaborative Innovation performance (CustomerThink)
● Social-Coding Collaboration Strength Metrics & Analyses on GitHub (Batista & Brandão et.al.)
● Measuring Collaboration in Modern Organizations (Harvard Business School)
● Developing a Framework of Metrics to Assess Collaboration in Integrated Project Delivery (ASCIC2014)
Measuring Psychological Safety
● Measuring Psychological Safety (Richard McLean)
● The 4 Stages of Psychological Safety Behavioral Guide (LeaderFactor)
● How to Measure Psychological Safety at Your Company (PredictiveIndex)
● Understanding Team Effectiveness: Foster Psychological Safety (re:Work Guide)
● Chasing Psychological Safety (Duena Blomstrom)
Suburban Agilists – 27 October 2020 36

37. Other Resources
● Agile Performance Chains, Metrics, & Continuous Improvement (SolutionsIQ)
● Measuring Social Value and Measuring Social ROI [PDFs]
● Evidence Based Management (EBM) - Measuring value to enable improvement and agility | cebma.org
● AARRR Framework- Metrics That Let Your StartUp Sound Like A Pirate Ship | piratemetrics.com
● Innovation Measures : Pipeline ECV | Dev Cycle Time | R&D Growth Effectiveness | R&D Effectiveness Index
▪ Forecast Degradation | Resource Productivity | Innovation Vitality Index | Incremental Sales
SAFe Metrics:
● www.scaledagileframework.com/metrics/
● www.scaledagileframework.com/measure-and-grow/
● v46.scaledagileframework.com/metrics/
Suburban Agilists – 27 October 2020 37

38. 12 Rules for Metrics
Suburban Agilists – 27 October 2020 38
Source: https://stayrelevant.globant.com/en/determining-agile-metrics/

39. DevOps Metrics across Industry (2016-2018)
Suburban Agilists – 27 October 2020 39
Companies at all levels of mastery/maturity are working to advance
their capability to deliver products and services to market faster
Deployment Frequency
Lead time for changes
Mean time to recover (MTTR)
Change failure rate
Configuration
Management
28% 46%
Testing 35% 49%
Deployments 26% 43%
Change Approval
Process
48% 59%
200x more frequent
2,555x faster
46x more frequent
440x faster
96x faster
5x lower (1/5 as likely)3x lower (1/3 as likely)
24x faster
IT Metrics - Comparing High vs. Low Performers 2016 2017
% of Manual Work High Performers Low Performers
Source: 2017 State of DevOps Report | by Puppet +DORA

40. DevOps Usage Metrics across Industry (2016-2018)
Suburban Agilists – 27 October 2020 40
DevOps Metrics Sources
State of DevOps 2016 - Puppet Labs
DevOps Guys - May 2016
IBM Interconnect - Feb 2016
Datamation - Oct 2016 (ROI)
IntelliTect - DevOps 2015
XebiaLabs - July 2016
Pager Duty - June 2015
HPE - Aug 2015
DevOps Scorecard - Nov 2014
Datical - 2016
Nonlinear Digital - March 2015
Lowe - Thoughtworks
DOES Metrics Forum - 2015
O’Reilly DevOps Playbook – Oct 2016
Forrester - Aug 2016
Gartner - Sept 2015
CollabNet - Jan 2015
Gruver & Humble - Nov 2016
DevOps for Digital Leaders - Nov 2016
Top 10 DevOps Metrics
1. Change Lead-Time/Cycle-Time
2. Delivery Frequency
3. Change Volume/Throughput
4. Delivery Success/Failure %
5. Mean-Time to Repair/Resolve/Recover/Restore
6. Availability/Uptime
7. Defect Volume/Leakage
8. Downtime/Outages/MTBF
9. Culture/Collaboration/Experimentation
10. Customer/Employee Satisfaction
[NOTE: Source-links are 3+ years old]

41. DevOps Flow Metrics
Suburban Agilists – 27 October 2020 41
Velocity KPIs Quality KPIs
MLT (Mean Lead Time)
how long does it take for a bit of code to get built, tested and
deployed.
BFR (Build Failure Rate)
% of failed builds
DCR (Daily Change Rate)
number of changes getting committed to mainline and tested
per day.
DFR (Deployment Failure Rate)
% of failed deployments
MTTE (Mean Time To Environment)
how much time it takes developers/testers to bring up a testing
environment for verifying each delivered change.
IRFR (Infrastructure-Related Failure Rate)
% of build/deployment failures related to
infrastructure issues
MTTD (Mean Time to Detect)
how much time passes since the original commit of code until
the bug it introduces gets detected.
RWR (Rework Rate)
% of tickets being reopened
MTTR (Mean Time To Resolve)
how much time it takes to resolve an issue after it’s detected
ADR (Automated Detection Rate)
% of defects being detected by automated testing
cycles
MTTA (MeanTime To Approve)
how much time it takes to approve and verify a release.
(Measured from the moment all release content has been
delivered and until the release has passed all the defined test
and verification cycles)
UWR (Unplanned Work Rate)
% of unplanned issues
Source: http://devopsflowmetrics.org/

42. Gartner’s DevOps Metrics that Matter (2015)
Suburban Agilists – 27 October 2020 42
Business
Performance
Organizational
Effectiveness
Customer
Value
Operational
Efficiency
Service
Quality
Service
Velocity
Earnings
Cash Flow
Market Share
New Business Service Enablement
User Satisfaction
Response Time
Net Promoter Score
Mentoring
Sharing
Collaboration
MTTRS
Cycle Time
Deployment Frequency
Users/FTE
Servers/FTE
Lead Time
Epics Delivered
Releases per Month
Retention
Motivation/Morale
Responsiveness to Change
Production Support
Deployment Success Rate
Cost per Transaction
Cost of Change/Release
Gartner Inc., Data-Driven DevOps: User Metrics to Help Guide Your Journey, 9 Sept 2015 G00264319, Analyst(s): Cameron Haight, Tapati Bandopadhay

43. Comparison of Lifecycle-Models with Change-Workflows
Suburban Agilists – 27 October 2020 43
43
Waterfall
(phases)
DeployDefinition Analysis
Design,
Code &
Unit-Test
Integration
Build+Test
Test Accept
Iterative
(iterations)
Continuous
Delivery
Incremental
(integrations)
Agile
(sprints)
Continuous
Deployment
Hybrid
(Water-
Scrum-
Fall)