Recommandé
Recommandé
Contenu connexe
Tendances
Tendances (20)
En vedette
En vedette (20)
Similaire à Managing experimentation
Similaire à Managing experimentation (20)
Dernier
Dernier (20)
Managing experimentation
- 6. 25 Million Members 900,000 Shops `
- 7. 25 Million Members 900,000 Shops 1.4B Page Views / Month
- 11. 1. Experiments 2. Rampups 3. Communications
- 15. Geocities-ize
- 16. Initial Idea & Planning Design & Development
- 18. Initial Idea & Planning Deploy & Launch! Design & Development
- 19. Pain?
- 20. Testing?
- 21. Communication?
- 22. How can we do better?
- 23. Living in a Continuous Deployment World
- 24. Total Deploys: 4,040 Total Commits: 45,447 Total Unique Authors: 220 2013 ... So far
- 25. Total Deploys: 4,040 Total Commits: 45,447 Total Unique Authors: 220 Average Deploys per day: 35.75 Average Authors per deploy: 7.01 Average Commits per deploy: 11.25 2013 ... So far
- 26. That’s a lot of people
- 27. We are only halfway through the year!
- 28. CHAOS
- 29. Config System $server_config['my_awesome_feature'] = [ 'enabled' => 10, 'admin' => 'on', 'groups' => [1234], 'users' => ['wstuckey', 'dmckinley'], ];
- 30. Branching in Code if (Feature::isEnabled('my_awesome_feature')) { // display my_awesome_feature... } else { // continue on }
- 33. Initial Idea & Planning Design & Development Launch!
- 34. Initial Idea & Planning Design & Development Launch! Internal Admin Launch
- 35. Initial Idea & Planning Design & Development Launch! Internal Admin Launch Public Prototype
- 36. Initial Idea & Planning Design & Development Launch! Internal Admin Launch Public Prototype 50% Experiment
- 37. • Small iterative changes
- 38. • Small iterative changes • Product exists in prod as soon as possible
- 39. • Small iterative changes • Product exists in prod as soon as possible • Can be available for small groups for testing and feedback.
- 40. • Small iterative changes • Product exists in prod as soon as possible • Can be available for small groups for testing and feedback. • Easy to communicate milestones because they happen as part of the development process.
- 41. Because of the constant change you have to constantly communicate.
- 42. Start Simple
- 43. Wiki + Email
- 44. • Works great in the beginning Wiki
- 45. • Works great in the beginning • Quickly becomes overwhelming as you increase your concurrent launches. Wiki
- 46. • Works great in the beginning • Quickly becomes overwhelming as you increase your concurrent launches. • Requires manual labor to keep stakeholders informed Wiki
- 47. FRUSTRATION
- 48. We can do better!
- 50. Launch Calendar
- 51. Launch Calendar • Simple web app • Collect structured meta-data about the launch • Central location for upcoming, current and past launch information
- 52. Launch Calendar Collect launch specific meta-data
- 53. Launch Calendar • Owner • Launch Date • Internal Launch Date • Team • Expected Reaction Collect launch specific meta-data
- 54. Launch Calendar ✉ Make your email work for you
- 55. ✉ Launch Calendar • Automated emails to product owners around deadlines • Daily emails to product groups containing upcoming and recently launched launches Make your email work for you
- 56. Launch Calendar Make your email work for you ✉
- 57. Launch Calendar • Highly utilized across the organization • During the year+ that it was running we recorded and tracked over 275 launches How did it go?
- 58. “This beats the crap out of the wiki page in every conceivable way.” - real feedback
- 59. What could we do better?
- 60. What would a more integrated product look like?
- 61. Can we automate more?
- 62. We can do better!
- 64. Catapult A unified launch management tool.
- 66. “I believe that by changing the ‘add to cart’ button to blue we will increase conversion rate.”
- 67. Catapult
- 70. $server_config['my_awesome_button'] = [ 'enabled' => 50, 'admin' => 'on', ]; Remember this?
- 74. Hack, Deploy, Hack, Hack Deploy!
- 83. Start Simple.
- 84. Build process that enable you to Ship.
- 86. Thank You!