Slide deck from Texas Linux Fest 2013

1. How to (Almost) Kill a Successful Project and Bring
It Back to Life Again
Russell Pavlicek
Xen Project Evangelist
Citrix Systems
Russell.Pavlicek@XenProject.org
Lessons Learned from the Xen Project

2. A guy with a lot of experience and a
really big mouth
So Who is the Old, Fat Geek Up Front?

3. • Linux user since 1995; Linux desktop since 1997
• Linux advocate before I ever saw the software
• Early advocate in DEC, Compaq
• Former columnist for Infoworld, Processor
• Former panelist on The Linux Show
• Wrote book, Embracing Insanity: Open Source Software
Development
• Speaker at 40+ conferences
• Currently Xen Project Evangelist employed by Citrix
About the Speaker...

4. • We will spend a couple minutes reviewing the
project
• We will spend a few minutes considering its
history
• But we will spend the bulk of the time
considering lessons to take away
• We are not here for the project's history; we are
here for your future
About This Talk

5. • The premier Open Source Hypervisor
• Powering some of the biggest Clouds in the
industry (Amazon, Rackspace, Terremark)
• Celebrating its 10th Anniversary
• Now a Linux Foundation Collaborative Project
– Sponsoring organizations include: Google, AMD,
Intel, Samsung, Cisco, Oracle, Amazon, Verizon and
more
What is the Xen Project?

6. • The Xen Hypervisor, including ARM servers
– Type 1 (Bare Metal)
• Xen Cloud Platform & XAPI
– Cloud readiness out of the box
• Other Projects
– Mirage
– ARM Hypervisor for mobile devices
What Does the Xen Project Produce?

7. • It was the first industrial-strength Open Source
Hypervisor
• It enjoyed a very high rate of adoption
• It employed excellent technology
• It had a FOSS-friendly corporation behind it
• And, yet, 2 years ago, it ran the risk of being
abandoned by the FOSS community before its
10th birthday
The Xen Project Story (30 second
version)

8. • The project, though viable, developed an inward
focus
– Reach out to the rest of the Open Source community
was limited
– Reach out to its users was minimal
– Code development continued, but the community
became insulated and stagnated
– No one stepped up to contradict the rumor that Xen
was dying technology, overcome by competitors
How Did This Happen?

9. • The project forgot the importance of working
with its ecosystem
– Upstream projects (Linux Kernel, QEMU) were
branched rather than engaged with patches
– The project decided that others in the ecosystem
(i.e., the distributions) would have to carry the
burden of maintaining and supporting those
differences
– This went on too long, and the ecosystem got fed up
How Did This Happen? (continued)

10. • The corporation backing it (XenSource) was sold
to a company with a long closed source
software history (Citrix)
• The new corporation was interested in the
technology, but had no particular interest in the
project itself
How Did This Happen? (continued)

11. • It was not about malice
• It was not about fear
• It was about disconnection
– The project became disconnected from the FOSS
community
– The project became disconnected from the users
– The new company became disconnected from the
needs of the project, because, in part, the project
never really explained what it needed from the
company
Why Did This Happen?

12. • The prognosis was not good
• Xen Hypervisor had been overtaken by a
commercial offering in IT mindshare
• Xen Project had been overshadowed by another
Open Source Hypervisor in the community
• Distributions stopped facilitating Xen
• The FOSS Community began to forget Xen
About Two Years Ago: Battling for a
Future

13. • About 2 years ago, a new direction was plotted
• Citrix decided it wanted to understand FOSS and
reinvigorate the Xen community
• The company began to hire FOSS-savvy people
to reconnect with the community and with
users
• Brought about efforts to birth Apache
CloudStack, OpenDaylight, and to move the Xen
Project under the Linux Foundation
A Conscious Reversal in Direction

14. • Common themes heard at FOSS events:
– What is Xen?
– Xen is dead, right?
– Isn't Xen closed source?
– No one uses Xen anymore
Reality Two Years Ago: Xen Who?

15. • Linux kernel 3.0 contains all that Xen needs to
exist by default
• Most Linux distributions are Xen-enabled
– CentOS has a project to give RHEL6 users a Xen
option
• Xen Project now part of Linux Foundation
• Launch of a new user-friendlier website
(XenProject.org)
Reality Today: Xen is Back!

16. So What Did We Learn?

17. • It is possible to die while you are winning
– Being first is not enough
– Great technology is not enough
– Having a FOSS-friendly corporation behind you is not
enough
• A project must stay vibrant as an Open Source
organism or it will perish
Lesson 1

18. • Disconnection from users can make you a “Dead
Project Walking”
– You can be adding functionality, issuing new
releases, and still be dying
– The connection between project and users is
essential
– Focusing on software alone is not enough
• If you are not interacting with your users, you
are at serious risk
Lesson 2

19. • Connecting with your developers != connecting
with your users
– You need information sources for both users and
developers
– If users have to dig through technical websites,
wikis, etc. to answer simple questions, you are in
trouble
– Even Linux kernel development – arguably one of
the most insular projects – cannot thrive in a
vacuum
Lesson 2 (continued)

20. • Never ignore your project's Open Source root
structure
– Cut flowers are beautiful – until they die
– Living plants need their roots
– FOSS community is the root structure, and it must
spread wide
– The project team cannot stand alone
• Pay attention to your partners in FOSS: libraries,
kernel, packaging
Lesson 3

21. • Never ignore your support structure
– Xen needed cooperation from Distributions to be
properly supported
– The relationship with the distributions was allowed
to stagnate; it was not continuously cultivated
– When one distribution invested in another Open
Source virtualization solution, other distributions
were swayed
• Your distribution route can be critical to success
Lesson 4

22. • Having corporate backing is not enough
– The corporation has its own set of goals, and they
rarely align exactly with the project's goals
– When the project and the company don't mesh,
friction can occur
– This isn’t about good versus evil; business and
projects are just two separate animals with different
needs
Lesson 5

23. • Having a FOSS company backing you is no
guarantee
– Even FOSS-centric companies can be sold
– Sometimes they are sold to other FOSS companies
(e.g., JBoss, Gluster)
– Sometimes they are sold to closed source
companies (e.g., MySQL, Xen, Cassatt)
• If a project won’t survive without FOSS company
backing, consider options (e.g., Linux
Foundation)
Lesson 6

24. • In FOSS, there is no such thing as autopilot
– Intent is critical
– If you are not planning to succeed, you are planning
to fail
– Great software is not enough; you can have the best
technical solution, but if a “big dog” starts throwing
its weight around, you need to be able to respond
– If you're not looking at your whole ecosystem, you
are inviting failure
Lesson 7

25. • If it ain't growing, it's dying
– If your project team is seeing no new blood over
time, be concerned
– Open Source organisms must move and grow
– New folks are needed from time to time to add new
ideas and keep focus on what users need
Lesson 8

26. • Know where your project could fit in the world
and make a plan to get there
– Competition means you probably won't be best fit
for every situation
– It may not be possible to have every feature your
competitors have (especially if they have much
corporate backing)
– Figure out who your users are, what they need, and
what you need for them to use your code
Lesson 9

27. • Competition Increases Innovation
– Lack of competition can cause stagnation (consider
Unix CDE)
– Competing technologies keep the ball moving
forward continuously
– Xen's competition with KVM and VMware insured
that new virtualization capabilities would keep
flowing
– A competing project has to stay on top of its game
or it won't make it
Lesson 10

28. • Major new features can keep your mindshare
alive in the community
– Large advances (e.g., ARM and Mirage) generate
attention from the FOSS ecosystem and the
userbase
– If you aren’t making headway, your root and support
structures may stop working to give you what you
need
– Periodic advances keeps the project growing
Lesson 11

29. • Sometimes, perception really is reality
– You can have the best code in the world, but if no
one cares, it’s useless
– If the rumor arises that you are dying, outmoded, or
outdone by some other project, you must fight that
perception
– The unchallenged lie will become fact for many
people
Lesson 12

30. • In contrast, KVM managed perceptions well
– It could have looked like a purely Red Hat/IBM
business play when Red Hat purchased Qumranet
– The relationship between Red Hat and the KVM
project was well-defined & appropriate; no
disconnect occurred
– FOSS community embraced KVM project
– Clearly, Red Hat and IBM are still focusing major
business initiatives on KVM, but the community
accepts that because it was done correctly
Lesson 12 (continued)

31. • Go to local FOSS events
– Submit talks
• Get used to rejection and learn from it
• Talk to the track chairman
– “I don’t like speaking” – get over it; calculus was
way harder than this
– Get an ORG booth for cheap
• Stock it with flyers, CDs, business cards,
stickers
• Shoot your mouth off
– Blogs
– A usable website
– Podcasts (TLLTS)
– Social Media
• Twitter, Facebook, Google+, LinkedIn
• More mouthing off
– YouTube demos and tutorials
– Write for Linux.com LXer, LWN.net
• Get a “kick-*aas” mascot!
– But our buddy the Xen panda is taken!
• Shout out and live, or shut up and die!
– Passion is your ally
– Let it leak over everyone
– Don’t imitate the suits; do what fits you
Crash Course in Perception
Management

32. • There's a new reality for FOSS projects: the
corporate connection
– Projects used to be primarily volunteers working
nights and weekends
– Today, corporations play a big role in development
• You need to have a good grip on what your
corporate sponsors want from you, and what
you need from them; disconnection can be fatal
Lesson 13

33. • Manage the relationship between business and
project
– Prevent the loss of the project’s identity
– If the project appears “owned” by a business, the
FOSS community might become suspicious and back
away
– In this case, perception is as dangerous as reality
– If you forget what the project is, every else will too
– Project ecosystem will wither away; only the
business remains
Lesson 13 (continued)

34. • Establish a symbiotic relationship
– Business provides user feedback, resources
– Project provides overall focus, goal, direction, labor
– Both sides need to color in the lines
– Otherwise, you get “fake Open Source:” the code is
open, but there is no community, no support, no
ecosystem
Lesson 13 (continued)

35. • Make sure your project addresses its entire
ecosystem:
– Is the code good?
– Are you reaching out to your users?
– Is your development community active, engaged,
and growing?
– Are reaching out to the FOSS community?
– Have you insured you have proper support (libraries,
distros, kernels, etc.)?
Final Thoughts

36. • If you are in a relationship with a corporation, is
that relationship healthy?
– Do you have freedom to do what you need to do?
– Are you getting user feedback to seed new growth in
the project?
– Is your project's identity getting lost in the corporate
identity? (if so, consider a foundation route)
• Whatever else, don't give up!
Final Thoughts (continued)

37. Russell.Pavlicek@XenProject.org
XenProject.org
Questions?