Optimizely Product Vision: The Future of Experimentation
•Télécharger en tant que PPTX, PDF•
2 j'aime•862 vues
Join Optimizely's product team to hear features and releases planned for 2019 and beyond.
Recommandé
Recommandé
Contenu connexe
Tendances
Tendances (20)
Similaire à Optimizely Product Vision: The Future of Experimentation
Similaire à Optimizely Product Vision: The Future of Experimentation (20)
Plus de Optimizely
Plus de Optimizely (20)
Dernier
Dernier (20)
Optimizely Product Vision: The Future of Experimentation
- 1. 1 Optimizely’s Product Vision The Future of Experimentation Claire Vo SVP Product Jon Noronha Sr. Director, Product Byron Jones Director, Product Giannis Psaroudakis Director, Product
- 2. 2 Giannis Psaroudakis Director, Product Management Claire Vo Senior Vice President, Product Management Byron Jones Director, Product Management Jon Noronha Senior Director, Product Management Introductions
- 3. 3 Agenda 1. Introductions 2. Optimizely’s Vision & Priorities 3. Creating Universal Experiences 4. Embedding Trusted Results 5. Leveraging Enterprise Scale 6. Q&A
- 4. 4 Housekeeping ● We are recording this webinar ● You’ll receive the recording and slides ● We’ll answer questions at the end
- 5. 5 “Our success is a function of how many experiments we do per year, per month, per week, per day.” - Jeff Bezos “Our aim is to create the best product for our customers, and we do that through constant innovation and testing.” - Gillan Tans, CEO “Our company culture encourages experimentation and free flow of ideas.” - Larry Page “We use experimentation and testing to inform as much of the business as we possibly can.” - Gregory Peters, CPO Today’s Digital Leaders Win By Using Experimentation At-Scale
- 6. Spent with Digital Media 19hr week 2018 Digital-Centric Customer Spent with Digital Media 41 $396m Mobile commerce Mobile commerce $1 8t. 2008 Digital-Convenient Customer hr week Your customer has fundamentally changed how they engage
- 7. The digital-centric customer journey: end-to- end, multi-device engagement Marketing Product & Engineering Support Views ad at Checkout Signs up for 7 day trial Tracks packages Manages account Converts to subscriber Downloads App
- 8. 8 How does Optimizely build products?
- 9. 9 Device coverage Complete experience coverage Open platform Trusted results quickly Consumer-grade usability Security & compliance Program Management Strategic & tactical partnership Build Digital Products Market & Sell Service & Retain What you need OTT Voice/IOT Mobile Web / SPA Build Digital Products Market & Sell Service & Retain DEVICE COVERAGE
- 10. 10 Device coverage Complete experience coverage Open platform Trusted results quickly Consumer-grade usability Security & compliance Program Management Strategic & tactical partnership Build Digital Products Market & Sell Service & Retain What you need OTT Voice/IOT Mobile Web / SPA Build Digital Products Market & Sell Service & Retain Frontend and Server-side Experimentation & Personalization EXPERIENCE COVERAGEDEVICE COVERAGE
- 11. 11 Device coverage Complete experience coverage Open platform Trusted results quickly Consumer-grade usability Security & compliance Program Management Strategic & tactical partnership Build Digital Products Market & Sell Service & Retain What you need Build Digital Products Market & Sell Service & Retain TRUSTED RESULTS Open Platform Data & Integrations Security & Compliance OTT Voice/IOT Mobile Web / SPA Frontend and Server-side Experimentation & Personalization EXPERIENCE COVERAGEDEVICE COVERAGE
- 12. 12 Device coverage Complete experience coverage Open platform Trusted results quickly Consumer-grade usability Security & compliance Program Management Strategic & tactical partnership Build Digital Products Market & Sell Service & Retain What you need Build Digital Products Market & Sell Service & Retain ENTERPRISE SCALE Consumer grade usability for adoption & velocity Program Management Open Platform Data & Integrations Security & Compliance OTT Voice/IOT Mobile Web / SPA Frontend and Server-side Experimentation & Personalization TRUSTED RESULTSEXPERIENCE COVERAGEDEVICE COVERAGE
- 13. 13 Device coverage Complete experience coverage Open platform Trusted results quickly Consumer-grade usability Security & compliance Program Management Strategic & tactical partnership Build Digital Products Market & Sell Service & Retain What you need Build Digital Products Market & Sell Service & Retain Consumer grade usability for adoption & velocity Program Management Open Platform Data & Integrations Security & Compliance OTT Voice/IOT Mobile Web / SPA Frontend and Server-side Experimentation & Personalization ENTERPRISE SCALETRUSTED RESULTSEXPERIENCE COVERAGEDEVICE COVERAGE COLLECTDATA UNIVERSAL EXPERIENCES ANALYTICS ENTERPRISE SCALE
- 14. UNIVERSAL EXPERIENCES Test and personalize every digital moment across all channels. ANALYTICS ENTERPRISE SCALE
- 15. DEVICE & EXPERIENCE COVERAGE Optimizely Web & Full Stack FULL STACK. Test server-side logic & data to deploy confidently WEB. Experiment on & deliver a winning front-end experience Copy Images & Colors Layout i.e. booking flow, search bar location & visual treatment CTA Navigation Pricing or discounts Personalized content based on user Segment offers based on persona Search algorithms New feature rollouts
- 16. DEVICE & EXPERIENCE COVERAGE Universal Experiences Cross-Platform Metrics Share conversions across frontend and backend with universal attribution Web Performance Build experiments in a visual editor, deliver them with lower latency Consistent Features Use Environments in Web projects. Track Change History in Full Stack. Full Stack Usability Do more out-of-the-box with Full Stack. Easier SDKs running in more places.
- 17. WEB + FULL STACK Cross-Platform Metrics ● Reuse client-side event tracking with server-side tests ● Measure how web experiments impact downstream backend metrics ● Share event tracking across multiple Full Stack projects Share conversion events across Web and Full Stack experiments Materials presented are for demonstration purposes only and may not be developed and / or delivered. Optimizely provides no guarantees and has no liability for information contained herein. Coming Soon
- 18. CROSS-PLATFORM METRICS Native Mobile Experiment: Homes vs. Experiences
- 19. CROSS-PLATFORM METRICS Web Conversion: Booked Room Attribution Web conversion counts back Toward mobile experiment
- 20. OPTIMIZELY WEB Client-Side Experimentation Client Web Snippet Visual Editor Default Treatment Server Experimental Treatment Experimental Treatment Client Server Full Stack SDK Feature Flags in Code User User OPTIMIZELY FULL STACK Server-Side Experimentation Experimental Treatment
- 21. Experimental Treatment Client Server User WEB VELOCITY + FULL STACK PERFORMANCE Client + Server-side Experimentation Snippet Visual Editor SDK Feature Flags in Code Smooth handoff from Full Stack to Web
- 22. WEB Performance Innovation ● Optimizing key platform components for faster desktop & mobile execution ● Integrated new CDN delivery technology for lower latency connections, using Full Stack technology ● New implementation options to give you full control over your site’s performance We’re making Optimizely faster than ever for the modern web Materials presented are for demonstration purposes only and may not be developed and / or delivered. Optimizely provides no guarantees and has no liability for information contained herein. Coming Soon
- 23. ● Build an experiment once, and push through specialized environments ● QA experiments in dedicated environments before launching to production ● Streamline developer work by setting up environments in Optimizely that mirror your own development workflows Easily integrate experimentation into your existing development & QA workflows FULL STACK → WEB Web Environments Materials presented are for demonstration purposes only and may not be developed and / or delivered. Optimizely provides no guarantees and has no liability for information contained herein. DEV TEST STAGE PROD Coming Soon
- 24. WEB → FULL STACK Change History ● Comprehensive records: see exactly who made a change, what they edited, and when ● Detailed audit log lets you investigate what changed at a specific time ● Improved search and filtering Track all changes made to experiments and feature flags in your account Materials presented are for demonstration purposes only and may not be developed and / or delivered. Optimizely provides no guarantees and has no liability for information contained herein. Coming Soon
- 25. 25 The right experience is not always the same for everyone The right experience depends on context The right experience requires continuous iteration Experimentation Personalization Sweet Spot Why You Need to Experiment to Personalize Effectively
- 26. 26 Different experiences resonate with different individuals, and personalization enables you to ensure you find an experience that resonates with your customers ✓x x Audience Students Page Homepage Everyone Banner B -2% Students Banner B +19% Students Banner C +19% Variation 1 Students Banner D +25% Variation 2 Students Banner E +10% Variation 3 Audience Everyone Page Homepage Everyone Banner A +5% Everyone Banner B -2% The right experience is not always the same for everyone
- 27. ● Match users attributes with new number, string, and boolean comparisons ● Easily adjust targeting conditions without re-deploying new code ● Compose multiple audiences into new segments Target experiments and feature rollouts using flexible rules and composable segments WEB → FULL STACK Flexible Targeting Released Materials presented are for demonstration purposes only and may not be developed and / or delivered. Optimizely provides no guarantees and has no liability for information contained herein.
- 28. ● Target users based on their predicted intent, using natural language keywords ● Adapt to changing user behavior and expectations automatically, without needing to manually tweak rules ● Experiment your way to the best personalization strategy - test larger or smaller audiences, and multiple variations Build predictive segments based on online behavior, powered by machine learning WEB Adaptive Audiences Materials presented are for demonstration purposes only and may not be developed and / or delivered. Optimizely provides no guarantees and has no liability for information contained herein. Beta
- 29. BUILD LEARN MEASURE WEB + FULL STACK Product Experimentation Think hypothetically ● Painted doors ● Validating MVPs ● Qualitative research ● Exploratory A/B testing Quantify product impact ● Feature validation ● Multivariate tests ● Iterative adjustments ● Measuring business impact Mitigate development risk ● Frequent, small releases ● Feature flagging ● Staged rollouts ● Safe rollbacks
- 30. FULL STACK Optimizely Rollouts Materials presented are for demonstration purposes only and may not be developed and / or delivered. Optimizely provides no guarantees and has no liability for information contained herein. Early Access ● Reduce risk with staged feature rollouts and safe rollbacks ● Powered by the same SDKs as Full Stack. Grow into full experimentation. ● Generally available next month. Contact your CSM if you’re interested in early access! www.optimizely.com/rollouts Free feature flagging for product development teams
- 31. FULL STACK Developer Efficiency Materials presented are for demonstration purposes only and may not be developed and / or delivered. Optimizely provides no guarantees and has no liability for information contained herein. Coming Soon ● New languages: Swift, Go ● Datafile Management: automatically sync configuration between Optimizely and your application ● Event Dispatching: simplify event tracking at enterprise scale ● User Profiles: manage user state and variation history across devices ● New Core: easy access to your configuration data, improved hooks for QA and testing, more detailed log messages Do more, faster. Our redesigned SDKs are more powerful and easier to use
- 32. API Gateway Go SDK Ruby SDK JS SDK React SDK Rest API Rest API Adapter 1 Rest API Rest API Rest API Rest APIJava SDK Adapter 2 Adapter 3
- 33. OUR VISION Universal Experiences DYNAMIC PRICING, ALGORITHMS ON OUR WAY Your order has just shipped! Track it now. NOTIFICATIONS, CONVERSATIONAL INTERFACES ADVERTISING Bringing experimentation and personalization to the full spectrum of digital experiences
- 34. UNIVERSAL EXPERIENCES ANALYTICS Make trusted decisions at scale with your experiment data ENTERPRISE SCALE
- 35. 35 MISSION Help the world’s innovators make trusted decisions at scale Experimentation Hero 10’s experiments / year Experimentation Program 100’s experiments / year Culture of Experimentation 1000’s experiments / year
- 36. ANALYTICS Optimizely Analytics Materials presented are for demonstration purposes only and may not be developed and / or delivered. Optimizely provides no guarantees and has no liability for information contained herein. Results & Stats Engine 1. Optimize your experiences and drive business value. 2. Learn how users respond to your experiment. 3. Control the risk of launched features. Machine Learning 1. Accelerate learnings and impact with ML-powered Stats Accelerator. 2. Optimize rewards for a period of time. 3. Identify statistically significant variants faster. Integrations 1. Integrate with best-of-breed technology platforms. 2. Discover powerful ways to target content. 3. Measure performance in the tools you already use. TODAY Standard Experiment Experiment with Stats Accelerator Reach statistical significance up to 300% faster
- 37. ANALYTICS Easy Event Tracking Track and optimize every element of the customer journey with Optimizely. Materials presented are for demonstration purposes only and may not be developed and / or delivered. Optimizely provides no guarantees and has no liability for information contained herein. Released E E E Website Sign ups Conversions …. Payments Revenue and Refunds BI Lifetime Value ● Attribute visitor actions from any system to your experiments. ● Optimize metrics across the customer journey. ● Simplify data integrations with other applications in your technology stack.
- 38. ANALYTICS Easy Event Tracking | Customer Use Cases Materials presented are for demonstration purposes only and may not be developed and / or delivered. Optimizely provides no guarantees and has no liability for information contained herein. Released E E E E cross-channel actions offline conversions computed metric data external events Easy Event Attribution (purchased in-store) (e-mail opened) (lifetime value) (notification clicked)
- 39. ANALYTICS Results Export Access, analyze and visualize experiment performance with enhanced flexibility. ● Analyze experiment results with any SQL or Data Science tool. ● Enrich your analysis with data from your Data Warehouse. ● Monitor experiment performance using your own dashboards. Beta Materials presented are for demonstration purposes only and may not be developed and / or delivered. Optimizely provides no guarantees and has no liability for information contained herein. Reporting Data Science Data Warehouse
- 40. ANALYTICS Results Export | Customer Use Cases Materials presented are for demonstration purposes only and may not be developed and / or delivered. Optimizely provides no guarantees and has no liability for information contained herein. Beta Custom Analysis (SQL, Data Science/Mining...) Reporting Dashboards
- 41. ANALYTICS Event Debugger Inspect and debug your experiment events faster. ● Verify in real-time that Optimizely is receiving your events. ● Debug issues with your event setup to accelerate your experimentation velocity. Materials presented are for demonstration purposes only and may not be developed and / or delivered. Optimizely provides no guarantees and has no liability for information contained herein. Coming Soon
- 42. ANALYTICS Extensible Analytics Platform Materials presented are for demonstration purposes only and may not be developed and / or delivered. Optimizely provides no guarantees and has no liability for information contained herein. Query Data Compute Metrics Statistical Analysis Display Results Metrics Object Results Object Data Object Experiment Analysis Model External Data Custom Models Certified Reports An Experiment Analysis platform designed to be extended and certified by Data Scientists... … so that it’s used by everyone in the organization to make trusted decisions at scale Future EVENTS METRICS BUILDER STATS ENGINE RESULTS EXTENSIBLE INTERFACES
- 43. UNIVERSAL EXPERIENCES ANALYTICS ENTERPRISE SCALE Govern, orchestrate, and report on your experience optimization program.
- 44. 10X YOUR PROGRAM Enterprise Scale AUTOMATED ORCHESTRATION Use automation to accelerate teams into building high-velocity experimentation programs. CUSTOMIZED REPORTS Easier socialization of an experimentation through custom reporting & dashboards. ENTERPRISE GOVERNANCE Roles, controls, and workflows that give large teams the ability to scale with confidence. Govern, orchestrate, and report on your experience optimization program.
- 45. ENTERPRISE SCALE Jira Integration ● Track development in Jira and experiments from Optimizely with a unified view across systems ● See the status of Optimizely rollouts in Jira issues to reduce context switching and friction in development ● See key information about experiments and ideas right in Jira Streamline your workflow by linking Optimizely ideas, experiments and feature flags to Jira issues Released Materials presented are for demonstration purposes only and may not be developed and / or delivered. Optimizely provides no guarantees and has no liability for information contained herein.
- 46. ENTERPRISE SCALE Slack App ● Integrate with Slack to ensure real- time updates on progress of ideas and experiments ● Users stay informed and can engaged with experiments from outside of Optimizely ● Improve experimentation communication and collaboration and increase testing velocity Integrate Program Management with Slack, enabling more effective and efficient collaboration Released Materials presented are for demonstration purposes only and may not be developed and / or delivered. Optimizely provides no guarantees and has no liability for information contained herein.
- 48. Coming Soon ENTERPRISE SCALE Experiment Reports ● Easily and quickly report on experiment results ● Acts as a system of record and cohesive view of an experiment ● Helps improve prioritization and decision making over time Easily create a report summarizing key details, results, and assets for your experiments Materials presented are for demonstration purposes only and may not be developed and / or delivered. Optimizely provides no guarantees and has no liability for information contained herein.
- 49. 49 Program Reporting Core operational metrics like win rate and velocity across your teams. Experiment Reports Shareable summary of your experiments ROI Measure your team’s revenue impact on your business. TODAY FUTURE Make it easier to share results and derive your return on investment in experimentation ENTERPRISE SCALE Reporting
- 50. 50 2019 Optimizely ROADMAP WEB & ✔ Support for Dynamic Websites Adaptive Audiences PERSONALIZATION ✔ Enhanced Multivariate Testing Faster Performance Environments FULL STACK ✔ Feature Management ✔ Flexible Targeting ✔ Environments Change History ✔ Multi-language Experiments ✔ Fast Config Sync ✔ Custom Analytics Integrations PLATFORM ✔ Stats Accelerator Enhanced Export ✔ GDPR ✔ Cross Product Events ✔ Easy Event Tracking Event Debugger ✔ Flexible Segmentation PROGRAM ✔ Teams Experiment Reports MANAGEMENT ✔ Program Reporting ✔ Slack App ✔ Jira Integration ROI Reporting BETA BETA Talk to your CSM to learn more Coming Soon 2019 Delivered 2018
- 51. 51 Q&A
- 52. 52 Thank you!
- 53. 53 Archive
- 54. 58 Easy Event Tracking (Data-In) 1. Track events from any system into Optimizely. 2. Optimize metrics across the customer journey. 3. Simplify event integrations with other applications in your stack. Results Export (Data-Out) 1. Analyze results with any SQL or data science tool. 2. Enrich your analysis with data from your DW. 3. Build dashboards using tools you are familiar with. THIS YEAR This Year | Analytics Released Beta
- 55. 59 Easy Event Tracking (Data-In) 1. Track events from any source. 2. Optimize metrics across the customer journey. 3. Simplify event integrations with other applications in your stack. Results Export (Data-Out) 1. Analyze results with any SQL tool. 2. Enrich your analysis with external data from your data warehouse. 3. Build dashboards using tools you are familiar with. Event Debugger 1. Verify that Optimizely is receiving your events in real- time. 2. Debug issues with your events faster to accelerate your experimentation velocity. THIS YEAR Coming SoonBetaLaunched This Year | Analytics
Notes de l'éditeur
- My name is Adam…. I’m excited to be joined by Olivia... Olivia leads the digital testing for Wall Street Journal.com and Barrons.com. She’s helped inject testing into all aspects of the customer lifecycle, including acquisition, engagement, product, and retention. Her team has run over 500 experiments over the past 3 years and is continuing to find new ways drive the product roadmap and expand acquisition strategies.
- A few housekeeping items for today: We are recording this webinar You’ll receive the recording and slides in a follow-up email and we want today’s session to be interactive. So please ask questions. You can do so by entering them in the chat box of Go to webinar console.
- Many of the world’s greatest companies have already broken the code on how to win in the new world order of digital. The answer for them is experimentation. Companies like Amazon, Google, Facebook and others are all saying exactly the same thing. They have figured out the experimentation is the key to winning in an increasingly digital world. Jeff Bezos at Amazon is widely quoted about how their success is directly tied to the number of experiments they run per day. And as you can see here, very similar comments from Larry Page, and Mark Zuckerberg And if that’s not enough, even the Harvard Business Review is weighing in. In a recent article called the ”Surprising Power of Online Experiments” the authors stated that “if you want to achieve a competitive advantage, you need to master the practice of online experiments.” So, yes, we do believe experimentation is the next big thing in business management and is rapidly becoming a mainstream business best practice.
- Let’s talk about why this is important now more than ever. These stats are to drive home that consumer behavior has fundamentally changed. Ten years ago we bought physical CDs and DVDs, shopped in brick-mortar-stores, and if we needed to file an insurance claim, we called into a call center. Now all of that is digital. Digital has shifted from being a convenient way for consumers to engage brands to being the central way.
- And this shift is not something just affecting marketing. Every single touchpoint - across web, mobile, voice - along the entire customer journey is an opportunity to win or lose your customer. We see everyday more and more companies adopting digital experience optimization across all business functions because it is key to acquiring more users, driving more and deeper product engagement, and retaining more customers.
- Complete coverage of digital channels so you can optimize everywhere your users are The ability to optimize and personalize the entire experience from the front-end UI to server-side business logic and data Trusted results so you can make sound decisions quickly Consumer-grade usability to scale experimentation enterprise-wide Protection of your user data through enterprise grade security & compliance Efficient ways to govern and manage your optimization programs at scale - the more you test and learn, the faster you grow A strategic partner with the hard-won best practices, knowledge and services to build with you into a world-class optimization program.
- Complete coverage of digital channels so you can optimize everywhere your users are The ability to optimize and personalize the entire experience from the front-end UI to server-side business logic and data Trusted results so you can make sound decisions quickly Consumer-grade usability to scale experimentation enterprise-wide Protection of your user data through enterprise grade security & compliance Efficient ways to govern and manage your optimization programs at scale - the more you test and learn, the faster you grow A strategic partner with the hard-won best practices, knowledge and services to build with you into a world-class optimization program.
- Complete coverage of digital channels so you can optimize everywhere your users are The ability to optimize and personalize the entire experience from the front-end UI to server-side business logic and data Trusted results so you can make sound decisions quickly Consumer-grade usability to scale experimentation enterprise-wide Protection of your user data through enterprise grade security & compliance Efficient ways to govern and manage your optimization programs at scale - the more you test and learn, the faster you grow A strategic partner with the hard-won best practices, knowledge and services to build with you into a world-class optimization program.
- Complete coverage of digital channels so you can optimize everywhere your users are The ability to optimize and personalize the entire experience from the front-end UI to server-side business logic and data Trusted results so you can make sound decisions quickly Consumer-grade usability to scale experimentation enterprise-wide Protection of your user data through enterprise grade security & compliance Efficient ways to govern and manage your optimization programs at scale - the more you test and learn, the faster you grow A strategic partner with the hard-won best practices, knowledge and services to build with you into a world-class optimization program.
- Complete coverage of digital channels so you can optimize everywhere your users are The ability to optimize and personalize the entire experience from the front-end UI to server-side business logic and data Trusted results so you can make sound decisions quickly Consumer-grade usability to scale experimentation enterprise-wide Protection of your user data through enterprise grade security & compliance Efficient ways to govern and manage your optimization programs at scale - the more you test and learn, the faster you grow A strategic partner with the hard-won best practices, knowledge and services to build with you into a world-class optimization program.
- Optimizely has been the site experience laboratory for the better part of a decade. After examining thousands of tests and working with thousands of customers, providing a relevant experience based on previous behaviors and profile data tends to lead to a higher desired conversion rate. We have learned that the winning experience is not the same for everyone. The right experience really is dependent on context and that the right experience requires continuous iteration. Those that have really excelled at this often have not gotten that conversion rate on their first try and the way that they have gotten this is by experimenting every step of the way, different audience definitions, different experiences/variations and ultimately treating every decision as a test
- Let’s look a bit at what we mean when we say that the right experience is not always the same for everyone. We see that different experiences resonate with different individuals, personalization enables you to ensure you are delivering experiences that resonate with your customers. Let’s walk through an example +++Click Here we start out with an A/B test that everyone sees. We can see that Banner B does not do well in this test. +++Click However, when we dig into this and look at segments, we can see that while Banner B didn’t resonate with everyone, it does resonate better than the alternative with students. +++Click From this information we then decide to experiment and apply an experimentation mindset to this banner, optimising for the student segment, and actually find that banner d is the winner (+++Click) This is an example of how we can see that by applying an experimentation mindset, we are able to effectively prsonalize.
- Trend : product experimentation
- As we talk with companies like yours we find that they are seeking ways to accelerate product innovation to meet rising customer expectations, and reach the market faster in order to see immediate impact to revenue. However, we also know that adding velocity greatly increases the associated risk. That’s why leading companies adopt processes like agile and devops—to mitigate risk while moving quickly. And finally, these companies also experiment on everything, so they can know the impact of new features on key metrics before rolling out to everyone.
- So what does the infrastructure that backs this process look like when implemented at scale? It starts with feature flags, simple toggles that enable you to separate deploying code from enabling features for customers. Then these teams do staged rollout of new features, to a small percent of traffic to ensure things work as expected, and can safely roll back if they don’t. Features are also A/B tested to quantify impact on key business metrics, not just performance. As these companies iterate, they perform many more A/B/n or multivariate tests to try out different variations on a feature or experience and tune to perfection. Every feature goes through this process before launch. And it’s deceptively simple. There’s a lot under the hood that these companies have built, including: Robust data infrastructure – statistical analysis, targeting Tools to democratize testing across the product org – a usable interface for business users, support for many languages, documentation for onboarding more teams Enterprise ready capabilities—is the system secure, compliant, and available when they need it? And integrations to other systems that power the customer experience. When you look at the multitude of discrete components that make up a robust experimentation stack, it's a daunting task to undertake.
- As we talk with companies like yours we find that they are seeking ways to accelerate product innovation to meet rising customer expectations, and reach the market faster in order to see immediate impact to revenue. However, we also know that adding velocity greatly increases the associated risk. That’s why leading companies adopt processes like agile and devops—to mitigate risk while moving quickly. And finally, these companies also experiment on everything, so they can know the impact of new features on key metrics before rolling out to everyone.