The almost annual State of the Stack, version 4, an end-to-end view of OpenStack. This edition focuses on what the challenges are within the community and how they can be addressed. v1 of SOTS has over 90,000 views and is one of the highest viewed OpenStack presentations ever.

1. CCA - NoDerivs 3.0 Unported License - Usage OK, no modifications, full attribution*
* All unlicensed or borrowed works retain their original licenses
SOTS v4
State of the Stack
May 20th, 2015
OpenStack Summit, Spring 2015
@randybias
With significant help from many Cloudscalers and EMCers. Thank you!

2. The Randy Bias
• Built big clouds; production clouds
• An OpenStack Original
• part of launch in 2010, on Foundation Board since formation
• built some of the largest and earliest OpenStack clouds
• Top <insert number here> cloud/twitter/pioneer/visionary
• you pick…
2

3. What Winning Looks Like
3

4. Fastest Growing Open Src Community
4
COMPANIES
TOTAL DEVELOPERS AVERAGE MONTHLY
CONTRIBUTORS
TOTAL CODE CONTRIBUTIONS
3,654 >600 125,000+
502 TOP 10 COUNTRIES
27,398
INDIVIDUAL MEMBERS
COUNTRIES
140+
United States, India, China, United
Kingdom, France, Russia, Canada,
Germany, Japan, Australia

5. Now That’s Something
5
OpenStack
CloudStack
Eucalyptus
VMware vSphere
Source: trends.google.com

6. Active Contributors in The Community
6
Qingye Jiang (John) - Open Source IaaS Community Analysis CY15 - Q1
http://www.qyjohn.net/?p=3801

7. Monthly Commit Volume
7
Qingye Jiang (John) - Open Source IaaS Community Analysis CY15 - Q1
http://www.qyjohn.net/?p=3801

8. Accumulated Developers
8
Qingye Jiang (John) - Open Source IaaS Community Analysis CY15 - Q1
http://www.qyjohn.net/?p=3801

9. 9
Source: LinkedIn Groups
via RedMonk analysis
AAWS Community Network Azure OpenStack

10. Why Did We Win?
10
Explosive Growth

11. What’s Also Very Dangerous?
11
Explosive Growth

12. Let’s Review
12

13. 13
Infrastructure as a Service
Compute
Nova
Ironic
Magnum
Network
Neutron
(LBaaS)
(VPNaaS)
(FWaaS)
Storage
Swift
Cinder
Manila
Cloud Management
Telemetry
Ceilometer
Deployment
Triple O
Orchestration
Heat
Test Suites
Tempest
Rally
Advanced Services (Consume IaaS)
Image Management: Glance
Data
Processing
Sahara
Key
Management
Barbican
DNS
Management
Designate
Database
Management
Trove
Message
Queue
Zaqar
Service
Catalog
Murano
Workflow
Management
Mistral
Policy
Management
Congress
Common/Shared: Identity: Keystone Common Libraries: Oslo
User/Admin
UI API CLIKilo

14. OpenStack Interdependence
14

15. – Me / Ako / Moi / Yo *
“OpenStack is at risk of
collapsing under its own weight.”
15
* http://www.cloudscaling.com/blog/openstack/the-future-of-openstack-is-now-2015/

16. Lots of Improvements
• Product WG formed
• Create an aggregation point for longer term planning, bring user
feedback into process, prioritization of blueprints, lobbying TC and
PTLs for work queues, “funding” of key blueprints, etc.
• Integrated release & 6-month cycle reformed to “Big Tent”
approach
• No more forced 6-month integration, more project autonomy,
encouragement of 3rd party integration testing of drivers, tagging for
release, etc.
16

17. Product Working Group
17
User Committee
N+3 members: 3 selected by the board, the TC and an additional nominated representative. An additional N
members elected by the user community.
Enterprise
Focused teams to gather user requirements from
segments and represent them
Telco / OPNFV
Application Ecosystem
Large Deployments
API Working Group
Working Groups to address a particular requirement set.
These WGs should have a target set of deliverables and
conclude when those are met. Maintenance should be a
function of the regular workflows.
Logging
Ops Tools
Monitoring
HPC
Product Working Group
Gather requirements from both sets of WGs (Segment and Requirement Oriented) above in the form of user
stories, work with cross-project team to populate blueprints from user stories across projects, work to identify
developers to help complete blueprints, communicate with project PTLs and core team to collect feedback on
future directions, and compile this data into a multi-release roadmap that is publicly available. In summary,
facilitate a feedback loop between projects, user community, and working groups.
Multi-Release Roadmap
New

18. “Big Tent” Release Cycle Reform
18
Solving for “How do we allow for the additional projects in the future without breaking down?”
(Current) Tag Categories:
Release Team
Tag Description
integrated-release Frozen tag, not given to new projects. Identifies projects that were integrated prior to Kilo.
release: indepdent Projects with this tag “release as needed” and don’t have to coordinate with other projects.
release: at-6mo-cycle-end
Projects that commit to being a part of a coordinated release every 6 months. They can still have intermediate
releases independent of the 6 month cycle “final” release.
release: has-stable-branches Projects that have stable branches (from the last release in the cycle)
release: managed Projects that agree to follow the processes/timelines outlined by the OpenStack Release Management Team
team: diverse-affiliation
This tag shows that the developer team for the project is from a diverse set of organizations (1 < 50% and 2 < 80%).
This is tested every 6 months.
Details at http://governance.openstack.org/reference/tags
(Current) Tags:

19. It’s not enough …
19

20. Complexity Kills
20

21. Enterprise Cloud Maturity
21
Most enterprises in the “thick middle” of maturity cycle

22. The “Thick Middle” Struggling
22

23. Docker is Simpler
.. And Adopted
Faster:
3M downloads in H1’2014
100M downloads at end of 2014
23

24. Reality Check Time
24

25. Technology Adoption Curve 101
25

26. Technology Adoption Curve 101
25
We are roughly here!

27. Technology Adoption Curve 101
25
We are roughly here!
… and headed for here!

28. 26

29. How Do We Know?
• Growing skepticism from analysts, reporters, and pundits
• Growing dissatisfaction with certain aspects of OpenStack
• Lots of failures in the field, enough to be worrisome
• Peak OpenStack?
• 6K+ attendees, early signs of slow down in adoption?
• Decide for yourself; I could be calling it early
27
1
2
3

30. The Growing Skepticism
28
Linthicum believes that despite the fact that OpenStack has "the only game in
town" for open source, the implementation hasn't met up to all of the hoopla since
its release.
http://searchcloudcomputing.techtarget.com/podcast/OpenStack-talk-of-open-source-town-but-is-it-hype
OpenStack can run a fine private cloud, if you have lots of people to throw at the
project and are willing to do lots of coding, according to Alan Waite, a research
director at Gartner.
OpenStack has the following drawbacks as a platform on which to build a private cloud*:
1 Difficulty of implementation
2 Shortage of skills available in the market
3 Conflicting or uncoordinated project governance
4 Weak spots in some projects
5 Integration with existing infrastructure *Recent Q1’2015 Gartner Report

31. OpenStack Self-Improvement Survey
• Intention:
• determine if and where project dissatisfaction exists
• report back to provide perspective on where we need to change
• After 10 days:
• 65+ respondents w/ 30 months average time with OpenStack
• Survey: http://tinyurl.com/improve-openstack [ TAKE ME! ]
29

32. How Would Your Characterize Your Participation in
OpenStack Land?
30
19%
16%
30%
35%
OpenStack Developer
OpenStack Operator/Administrator
OpenStack End-User (MIA)
Pundit, Analyst, Reporter, OpenStack Evangelist, or Groupie
Other
Average Time Working With OpenStack:
32 months

33. What is Your MOST favorite OpenStack Project?
31
Nova
Swift
Heat
Keystone
Neutron
Ironic
Cinder
Designate
Ceilometer
Trove
Barbican
Glance
Horizon
Manila
Oslo
Sahara
TripleO
Zaqar
Other
0 4 8 12 16
Responses

34. What is Your LEAST favorite OpenStack Project?
32
Ceilometer
Neutron
TripleO
Cinder
Horizon
Oslo
Glance
Keystone
Nova
Heat
Ironic
Sahara
Swift
Trove
Zaqar
Barbican
Designate
Manila
Other
0 4 8 12 16
Responses

35. User Survey Feedback
• Neutron:
• “Neutron is a lot more complex and harder to provide real HA” – Survey
Respondent
• “Complexity, availability and scalability remain some of the concerns [ of
the operators during the Operator Meetup in March ]” – User Survey
Team
• Ceilometer:
• “adoption has not been rising as quickly as expected … dozens of
comments related to stability and reliability, particularly at scale.” – User
Survey Team
33

36. User Survey: Neutron
34

37. Well run technology organizations will often throw away or re-architect
v1 and even v2 products. Do you think this is a good practice?
35
19%
81%
Yes No

38. Why can’t we fix these?
It’s been years now…
36

39. OpenStack Threats
• Explosive growth drives complexity
• Continued complexity slows adoption
• Can’t simplify, kill, or re-architect “broken” projects
• Rigid technical governance model (still too centralized)
• Long term vision & product strategy doesn’t emerge (in process)
37

40. Towards Glory
38

41. Technology Adoption Curve 101
39

42. Technology Adoption Curve 101
39
Need to get here!!

43. Path to the Plateau of Productivity
40
Plan Item Objective
#1) Streamlining Governance Model
empower projects, scale TC, focus Product WG, focus
Board and Foundation on marketing and interoperability
#2) Allow Competition
force poor projects to evolve or die, allow other projects,
particularly non-Python to come under our “big tent”
#3) Conform to Well Known APIs
don’t create new APIs in places where they exist (e.g.
OAuth 2.0)
#4) Testable Reference Architectures
allow for vertical and horizontal-specific OpenStack
reference implementations and separate infrastructure from
platform
#5) Ruthless Simplification
downloadable “OpenStack Basic IaaS” should be 1-click
download and install to run a POC/trial on a simple stack (1
switch, 10 servers)

44. 41
18 Categories (including retired), 252 Projects
The ASF Scales
Source: http://apache.org/foundation/governance/orgchart
To Manage This… You Need This.

45. Allow Competing Projects & Multiple Languages
• Competition is good; pretending our shit doesn’t stink is bad
• Poor projects must die; survival of the fittest works
• There is already leeway for this:
• “Where it makes sense, the project cooperates with existing projects rather than
gratuitously competing or reinventing the wheel.”*
• i.e. Competitive projects are OK as long as they have good reason
• Python isn’t good for everything
• A bigger tent means allowing non-Python projects
• Swift is already experimenting with re-writing pieces in Go Language (golang)
42
Source: http://governance.openstack.org/reference/new-projects-requirements.html

46. –Thierry Carrez, Chairman of the TC, Foundation Release Manager
“OpenStack is about community, common values, and a common
governance model.”
43

47. Keystone API
• Seriously … WTF?
• There are dozens of well known, documented, scalable, tested, standard APIs for
authN/authZ
• OAuth1/2, SAML, Kerberos
• There is no excuse for creating something from whole cloth
• Google is secure as hell and they use OAuth 2.0
• You aren’t better at security than the Google team; sorry
• We don’t apply this standard to our community (completely new Nova API anyone?)
44

48. Example Reference Architectures
45
OpenStack Interop Standard RA “Key” Components
RA Optional
Components
Basic IaaS
1+ of Nova/Magnum/Ironic
OAuth 2.0 Server (Keystone or other)
Glance, Horizon
Advanced IaaS
OpenStack Basic IaaS
Cinder, Swift
Neutron (or an alternative?)
OAuth 2.0 Server (Keystone or other)
Glance, Horizon
OpenStack App
Services
Zaqar, Trove, Designate, OAuth2.0 Horizon
OpenStack App
Management
Heat, Murano, Mistral, Horizon, OAuth2.0 Horizon
OpenStack for NFV
Basic IaaS
Pluggable SDN Controller w/ Neutron APIs
OpenStack Public Cloud
Advanced IaaS + OpenStack App Svcs +
OpenStack App Mgmt
ec2api, gce-api, etc.

49. P2
tests
How it Might Work
46
Reference Architecture
Default
Config Opts
Key +
Optional
Projects {
Project 1
Project 2
Interoperability Test Suite
Defined “Capabilities”
(previously “DefCore”)
RefStack
P1
tests
Capabilities
Tests
API Code
Owner(s):
Infrastructure Team &
Working Groups
TC, Board, Vertical/Horizontal
Working Groups, Community, &
Foundation
PTLs & key committers/reviewers
(more like Apache PMC??)
Unit Tests

50. We Are t3h Borg. You Will Be Assimilated.
47
“Maybe I’m an idiot, but I have no idea what anyone is talking about.
What is it? It’s complete gibberish. It’s insane. When is this idiocy
going to stop?”
Larry Ellison on Cloud
ComputerWorld, July 2000
"This 'telephone' has too many shortcomings to be
seriously considered as a means of
communication. The device is inherently of no value
to us.”, Western Union internal memo, 1876.
Decca Records rejected the Beatles, saying
"guitar groups are on the way out" and "The
Beatles have no future in show business,"

51. We Can Do It!
• Interrelated, but not interdependent projects
• Testable reference architectures that are interoperable
• Streamline governance
• Survival of the fittest project and programming language
• OpenStack is not specific code or APIs, it’s:
• Community, common values, and common governance
48