9. STARTOF OUR JOURNEY
August 2010
Sprint 1
Extensions
Nov 2015
Sprint 931ES
June 2014
Sprint 67
GVFS
June 2016
Sprint 102
Ranger community
adopted VSTS
April 2011
VSTS
April 2014
Sprint 64
over night
~destination!
continuous innovation!
14. But measure what’s important (KPI’s)
Customer Focused
• Original estimate
• Completed hours
• Lines of Code
• Team capacity
• Team burndown
• Team velocity
• # of bugs found
Things we don’t watch
• Acquisition
• Engagement
• Satisfaction
• Churn
• Feature Usage
Usage
• Time to Detect
• Time to Communicate
• Time to Mitigate
• Customer Impact
• Incident Prevention Items
• Aging Live Site Problems
• SLA per Customer
• Customer Support Metrics
Live Site Health
• Time to Build
• Time to Self Test
• Time to Deploy
• Time to Learn
Process Velocity
15. Maximize learning and influence value
Customer Focused
Validatedlearning
Deployment frequency
PROVEN DISPROVEN
VALIDATE
business
service
value
16. Feature Flags – fine tune user experience
Customer Focused
if ( flag )
else
20. Alerting is the key to fast detection
Production First Mindset
Before
• Redundant alerts for same the issue
• Needed to set right thresholds and
tune often
• Stateless alerts contributed to
further noise
After
• Every alert must be actionable and
represent a real issue with the
system.
• Alerts should create a sense of
urgency – false alerts dilutes that
• Use alerts to auto-dial the DRIs.
24. Security Mindset
Production First Mindset
Double blind test
Full disclosure at or near end
vs.
Share tactics & lessons learned
Continued evolution
Assume Breach - Use War Games to the learn attacks and practice response
26. Agile at Scale with Aligned Autonomy
Team Autonomy & Enterprise Alignment
Alignment
within the
business
Team
autonomy
“Let’s try to give our teams three things….
Autonomy, Mastery, Purpose”
Anarchy
31. Yes, there are other roles…
Team Autonomy & Enterprise Alignment
Program Management Engineering Service DeliveryUX UE
Service Delivery is integrated
directly into our organization.
32. Teams
Team Autonomy & Enterprise Alignment
Program Management is responsible for:
WHAT we’re building, and
WHY we’re building it
Engineering is responsible for
HOW we’re building it, and that
we’re building it with QUALITY
33. Teams
Physical team rooms
Cross discipline
10-12 people
Self managing
Clear charter and goals
Intact for 12-18 months
Own features in production
Own deployment of features
35. We strive for Vertical
Team Autonomy & Enterprise Alignment
UI
API
Data
36. Employee choice, not
manager driven
Typically <20%
change, but 100% get
to make a choice
Cross-pollinate talent
and micro-culture
Sticky Note Exercise - Self Forming Teams
56. • Tests that anyone can run anywhere (inc production)
• Shifted to unit tests from automated functional tests
• Core tests run before pull request
• Fast and 100% reliable build and test is critical
• Rolling tests run after commit
Test Portfolio - Shift Left from Integration to Unit
59. Five habits we’ve learned so far
Customer Focused
Production First Mindset
Team Autonomy + Enterprise Alignment
Shift Left Quality
Infrastructure as Flexible Resource
61. Multiple Data Centers with incremental roll out
Shared Platform Services (SPS)
North Central
TFS SU1
North Central
AT
AT
AT
JA
JA
JA
Blob
TFS SU7
Australia
TFS SU0
West Central
Containerized Services
62. Architecture Modernization
•
•
•
•
•
• If you are starting out today and cloud native, consider
PaaS, Service Fabric + Azure Functions
• We need to ship same code to on-prem & cloud
64. Our DevOps Transformation – the story so far
After
• 3-week sprints
• Vertical teams
• Team rooms
• Continual Planning & Learning
• PM & Engineering
• Continual customer engagement
• Everyone in master
• 8-12 person teams
• Publicly shared roadmap
• Zero debt
• Mockups in PPT
• Inner source
• Flattened organization hierarchy
• User satisfaction determines success
• Features shipped every sprint
Before
• 4-6 month milestones
• Horizontal teams
• Personal offices
• Long planning cycles
• PM, Dev, Test
• Yearly customer engagement
• Feature branches
• 20+ person teams
• Secret roadmap
• Bug debt
• 100 page spec documents
• Private repositories
• Deep organizational hierarchy
• Success is a measure of install numbers
• Features shipped once a year
65. Lastly the DevOps Ranger Transformation
DevOps even works for a part-time open source community
After
• 2-5 person teams
• 0.25 program managers
• Automated CI
• Automated CD
• 3-5 sprints cadence
• 3-week sprints
• Proactive telemetry
• Minutes to days to resolve issues
• Minutes to build
• Minutes to release
Before
• 10-15 person teams
• 2 program managers
• Manual and error prone builds
• Manual and error probe releases
• 6-12 sprint cadence
• 1 month sprints
• Issues detected by users
• Days to weeks to resolve issues
• Hours to build
• Days to release
![Willy[-Peter] Schaub
Program Manager | Developer Ops Canada
willys@microsoft.com
@wpschaub](data:image/gif;base64,R0lGODlhAQABAIAAAAAAAP///yH5BAEAAAAALAAAAAABAAEAAAIBRAA7)