United Airlines Transforms Release Management with Data-Driven Approach

Case Study: United Airlines Transforms Release
Management for Its Modern Software Factory
Suzan Liu
DO3T013S
DEVOPS – CONTINUOUS DELIVERY
Principal Engineer eQRM Strategy & Planning
United Airlines
Rob Karlin
Sr. Principal Consultant
CA Technologies

2 COPYRIGHT © 2017 CA. ALL RIGHTS RESERVED#CAWORLD #NOBARRIERS
© 2017 CA. All rights reserved. All trademarks referenced herein belong to their respective companies.
The content provided in this CA World 2017 presentation is intended for informational purposes only and does not form any type
of warranty. The information provided by a CA partner and/or CA customer has not been reviewed for accuracy by CA.
For Informational Purposes Only
Terms of this Presentation

Abstract
Release management plays a critical function at United, but it was hindered by
manual processes and spreadsheets. As a part of an IT-wide reliability effort,
the company has transformed the role that IT release management plays in the
organization. To begin, United performed a governance model and maturity
assessment to formally define the service offering, standards and processes.
They launched a pilot to establish an onshore/offshore release model, shifting
the role of the release team in the process to ensure the right type of work was
fulfilled.
United moved from multiple spreadsheets to a centralized dashboard to track
and manage releases and engagement levels. As a part of the overall
transformation, many decisions were needed to ensure success and data was
heavily leveraged across all main decision points. In this session, you will learn
how transparency around metrics and a strong governance program is
transforming United’s IT release team. You will also see how CA Continuous
Delivery Director is helping United’s transformation.
Suzan Liu
United Airlines
Principal Engineer
– eQRM Strategy &
Planning
Rob Karlin
CA Sr. Principal
Consultant

Agenda
WHO WE ARE
RE-EVALUATE RELEASE GOVERNANCE
RELEASE TRANSFORMATION
LEVERAGING DATA
MOVING TO THE FUTURE: CA CONTINUOUS DELIVERY DIRECTOR
1
2
3
4
5

Who We Are

United Airlines At a Glance
We are a 88,000 member team
serving 143 Million passengers!
2016 AT A GLANCE
339 Destinations
(airports served)
216 Domestic destinations
(airports served)
122 International destinations
(airports served)
50 Countries Served
4,500 Daily Departures

eQRM at United
Internal IT service provider in IT
Combines Quality Assurance (300+
people) and Release Management
(23 people).
Formed two years ago and has
rapidly expanded, requiring
maximizing each role to keep up with
expansion pace.
Now supports releases from Dev,
Stage, QA, UAT, Prod Perf and Pre-
Prod to Production.
Release Management at United
eQRM Facts

Re-evaluate Governance

eQRM Governance Model
Reevaluation of eQRM
Service Offerings
 Added Release Planning
capability
 Broke out Release
Automation from
Environment Management
1
eQRM Operating Model
(Organization, Roles & Responsibilities & Governance)
eQRM Intake, Estimate and Revision2
Metrics & Reporting11
Release Management Quality Assurance
Release
Planning
Release
Coordi-
nation
Release
Execution
3 4 5
6 Release Automation
7 8
Test
Planning
Test
Execution
9
10
Environment Management
Test Data Management

Release Governance
Created The Release Management Capabilities And Self-assessed Maturity
Reference: How IT Operations Can Set up an Effective, Centralized Release Management Process. 03 June 2013
CURRENTMATURITY
Governance, Policies and Stakeholder Engagement
Mechanism for Managing Release Requests
Release Models
Risk Factor Checklist
Methods for Coordinating and Prioritizing
Release Schedule, Models and Teams
Methods for Building
Testing Strategies
Bundling and other Streamlining Methods
Method for Validating and Monitoring
Closure Process

Release Transformation

Defined Release Model
Created The Release Management Capabilities And Self-assessed Maturity
Reference: How IT Operations Can Set up an Effective, Centralized Release Management Process. 03 June 2013
RELEASE
PLANNING
RELEASE
COORDINATION
RELEASE
AUTOMATIONRELEASE
EXECUTION
STEP 1
Understand needs of
application, portfolio and
stakeholders. Ensure future
releases are set up for
success
STEP 3
Deploy the release
according to the plan
and manage escalation
scenarios when
releases do not go
according to plan
STEP 2
For each release, ensure
all deployment planning is
completed and all pre-
release activities and
identified and completed
STEP 4
Manage the release
pipeline from
development to
production leveraging
automation tools

Transformation Objectives
How It Affects People, Process And Technology
People
Process
Technology
Introduced offshore team
Ensure skillset alignment
Raise brand value
Alignment of process to release areas
 Portfolio Release Managers -> Release Planning
 Onshore RMs -> Release Coordination
 Offshore RMs -> Release Execution
Standardized toolset
Tool training
Created portfolio release manager role

Redefined Release Scope
Characteristics
 Changes over one hour duration
 Changes with outages
 Changes that spans across multiple teams and
departments
 Changes that requires multiple rounds of
communication and clear communication
 The introduction of new functionality and applications
 High-risk area (top priority incidents)
Other Guidelines
 Capital projects will always be
supported
 Core support work may be
supported depending on
scope, complexity, and risk
 Changes that do not allow for
strong planning are out of
scope (e.g., emergency or
incident mitigation)
 Changes that utilizes less
than 75% of release
management responsibilities
will not be supported
Support releases that are higher risk
and require strong planning.

Refined Release Management
Responsibilities
Release lifecycle oversight / governance
Demand management
Strategic bundling
Release coordination / communication
Release calendars and reporting
Test acceptance
Change management coordination
Release automation
Test environment management
Release execution
Post release reviews

Leveraging Data

The Role Of Data In The Transformation
STANDARDIZE APPS
AND HELP THEM
MOVE TO
AUTOMATION
 Track consistent metrics
for all the apps
 Know what a good
release looks like
 Help drive release
discipline
WORK ON THE RIGHT
THINGS, MANAGE
RESOURCES AND
CONTROL DEMAND
 RM involved in forecasting
 Ability to track on-shore/
off-shore RM hours
RAISE THE BRAND
OF RELEASE
MANAGEMENT AT
UNITED
 Measure release output
 Measure level of
engagement and reduce
low-value work

State Of Data
CHALLENGES
Data for release management was available in
multiple tools and owned by various
departments
 Change Management Reports
 PMO reports (resources)
 Internal team excel reports
 Intake reports
Cross-referencing data from multiple resources
was time-consuming and cumbersome and
therefore rarely done.
WHAT WORKED
High accuracy in reports – information in the
reports are trustworthy
Reports are automatically refreshed – latest
information is readily available
WHY SPOTFIRE?
United was already leveraging the platform and we had a
centralized group of Business Intelligence experts
Training to get started only took two days!
Rollout
 United is migrating from HP Service Manager to ServiceNow
so creating a reporting database to pull information out of is
out of scope for now.
 Dashboards are managed from standardized Excel reports
and manually refreshed daily
 Adoption of the dashboards was quick
 Dashboards were fine-tuned over time and end—users saw
new ways to use the dashboards
 Future-state: automatically feed data from source systems
directly into dashboard

Intake – Demand Forecast
Created centralized dashboard for the intake process, which included release
managers in estimation process
Estimates for release managers are broken down by onshore and offshore hours
Release
Estimates
CHALLENGE
Release estimations were often left out
of the process leading to weak release
resource forecasts
SOLUTION
Intake process includes release
estimations on all efforts, with more
detailed forecasting planned for future

Release Management Dashboard
Release Services Leveraged
TO ENSURE WE ARE
PROVIDING MEANINGFUL
SUPPORT TO OUR
ORGANIZATION, LEVEL
OF ENGAGEMENT IS
MEASURED
Opening RFC
Cutover Script
Comm.
Script Review
Go/no Go meeting
Running deployment
Low-value work has been cut out over time, freeing up resources for meaningful engagements

Overall Trends
Reduction in # of
releases after
narrowing release
scope
Month-by-month
total releases
Number of
releases by
release manager
Final release
status

Overall Trends
When releases occur by day of week
When releases occur by hour of day
How long releases last
How long outages last

Specific Trends
Expedites vs.
Normal releases
Risk level of
releases
If our QA team
was involved in
the testing
Final status of
changes
Outages
associated with
releases

Release Transformation
Next Steps
Move from manual RM process to
automated and orchestrated
platform that can be leveraged by
everyone
Enable automated, more
comprehensive visibility and
feedback on testing
Automate risk identification and
the ability to assess release health
for better go/no-go decision making
Leverage continuous delivery
pipeline data to improve/mature

Moving To The Future:
CA Continuous Delivery Director

Drive Continuous Delivery Orchestration
QA/TEST PRE-PROD PRODUCTIONDEV
AUTOMATED
DEPLOYMENT
Continuous Delivery DashboardOPTIMIZE
PIPELINE
CAContinuous
DeliveryDirector
PLAN & MANAGE
RELEASES
Release
Automation

Release
Execution
Process
Improvement
Conflict
Avoidance
Application
Tracking
Productivity
Metrics
Business
Value
Assess Application Release Risk
Automate delivery risk identification
based on scoring:
 Application release complexity
 Interdependencies between releases
 Code changes
CA CONTINUOUS DELIVERY DIRECTORImprove release governance with the
change management organization
Automate the visibility/feedback of
work items tested per release

Gain Release
Health Visibility

Improve Release Management Maturity
Provide a consistent release
platform for onshore and
offshore release teams
Implement automated release
management workflow and
approval
Mature processes between
release teams and QA

Release Governance Improvements with
CA Continuous Delivery Director
Release
Planning
Managing
Release
Requests
Building
Releases
Validating and
Monitoring
Closure
Process
Create integration
in CA Continuous
Delivery Director
Identify release
size, milestones,
and dependencies
and assign release
manager, manage
relationships with
CA Continuous
Delivery Director
Eliminating manual and scripted processes
Leverage CA
Continuous Delivery
Director for building
release templates
and sync with
ServiceNow
Automated
validation of
release success
through CA
Continuous
Delivery Director
Formalize closure
process for
automated releases:
feedback to quality
team when release
fails using CA
Continuous Delivery
Director

From Manual to
Automated
CA Continuous Delivery
Director plays a big role in
moving release management
from manual/scripted
processes to an automated,
orchestrated and measured
future.
Honest Self-
Assessment
We had to take our own
measure of release
management and release
maturity leveraging industry
standards before we could
know how to move forward.
Measured
Transformation
We leveraged data to help
track and standardize the
work, level set for
automation and raise the
release management brand.
Summary
A Few Words to Review

Must See Demos
Automate
Your DevOps
Ecosystem
End-to-End
Theater 3
308
Continuous
Delivery for
Developers
Theater 3
303
Modern
Software
Factory in
the Cloud
Theater 3
302

Questions?

Stay connected at communities.ca.com
Thank You.

Appendix – Release Management Tasks
Release Coordination Release ExecutionRelease Planning
High Level Portfolio and
Application Planning FOR EACH RELEASE
 Identify technical and business owners
and stakeholders
 Understand if release management
services is best applied to the portfolio
 Ensure stakeholder understanding of
release
 Understand frequency of releases and
drivers that affect release frequency
 Work with leadership of the team to
establish frequency of communication
and communication format
 Develop release plan and migration
schedule
 Know tier level of applications
 Identify critical release tasks and
activities
 Create Master SRPs if needed
 Understand bundling needs and
necessity
 Manage schedule of releases for each
application or portfolio
 Update centralize release management
documentation
 High level risk mitigation planning
 Dependencies on other RFCs
• Ensures all prep tasks are complete Communicate to
escalation manager release is ready to start (if applicable)
• Conducts deployment day/night activities according to the
developed cutover script
• Confirms and communicates release completion
• Track status of release participants
• Track application validation for all applications status with
deployment script
• Manages release failure or fallback situation
• Escalate releases that are deviating from the plan cutover
script
• Closes the RFC(s) and document issues within in the RFC
while closing out RFC
• Sends final release communication to all stakeholders
identified in the cutover script
• Supplies improvement suggestions to Enterprise Release
Manager
• Supplies KPI metrics updates for the Release to the Release
Management
• Communicates any deviation from the release plan to
Release Manager at the close of the release
• Work with Change and Problem for any P1/P2 incident review
• Hold Lessons Learned meetings
• Upload all archivable information to centralized release
repository
 Determine release scope and impact
 Track release status, risks and issues
 Develop deployment script for each SRP
 Prepare stakeholder and validator list
 Collaborate with stakeholders to coordinate release-
related activities
 Ensure all steps and tasks of the release are well
defined
 Understand and define dependencies between tasks to
deploy, go/no go tasks, fallback steps and instruction
 Identifying verification steps
 Identify project team members
 Identifying prep tasks
 Maintain and distribute status
 Verify agreed upon Q&A testing signoff
 Develop release-specific communication
 Add RFC to Change/Release Calendar
 Conduct Tier 0 &1 impact analysis
 Define all must-validate applications and application owners
 Add incident management plan to script
 Facilitate go/ no go meetings
 Follow up on approvals
 Attend CAB to represent release
 Conduct deployment-day pre-deployment activities

DevOps:
Continuous Delivery
For more information on DevOps: Continuous Delivery,
please visit: http://cainc.to/CAW17-CD

United Airlines Transforms Release Management with Data-Driven Approach

Recommandé

Recommandé

Contenu connexe

Tendances

Tendances (20)

Similaire à United Airlines Transforms Release Management with Data-Driven Approach

Similaire à United Airlines Transforms Release Management with Data-Driven Approach (20)

Plus de CA Technologies

Plus de CA Technologies (20)

Dernier

Dernier (20)

United Airlines Transforms Release Management with Data-Driven Approach