Beta testing can provide an immense value to your company, but it can be hard to pinpoint the exact ROI. In this webinar, we take a closer look at the impact beta testing has on your company's bottom line so you can showcase what your betas are bringing to the table. Click this link to access your on-demand webinar: https://www.centercode.com/webinar/2016/october/

1. 1
The ROI of Beta Testing

2. 2
Meet the Speaker
Emily Hossellman
Director of Marketing @ Centercode
emily@centercode.com
@ehossellman

3. Agenda
1. Where beta impacts your product
2. The tangible vs. intangible effects of beta
3. Measuring the impact of beta
a. The framework
b. The inputs
c. The results
4. Presenting ROI to stakeholders & executives
5. Q&A
3

4. 4
Who is Centercode?
Services Fully managed beta tests and programs
Software Complete beta program management platform
Testers Access to 150k+ deeply profiled global candidates
Content Free tools and best practices
We've been running beta tests since 2001
Beta test management solutions are our core competency

5. 5
Beta Testing =
Customer Validation
Targeted customers using your product in real environments
Increases quality, improves user experience, validates performance
Many names: CAT, UAT, FUT, Customer Evaluation, Field Trials, Pre-release

6. 6
What Beta Brings to the Table
1. Introduces you to your target audience
2. Facilitates a real meaningful conversation
3. Gives you the opportunity to really listen

7. 7
What Makes Beta ROI Challenging
1. Hard to define
2. Hard to measure

8. Beta Test ROI Calculator
8

9. 9
The Impact of Beta

10. 10
What does releasing a better product get you?
The Impact of Beta

11. 11
The Impact of Beta
Development
Product Management
Development/QA
Usability
Product Management
Support
Marketing
Sales
Beta impacts every part of your product:

12. Development/QA
12
What you can achieve with beta:
● Identify bugs to improve quality (with context and frequency)
● Evaluate real-world impact of known issues
● Regression testing on solved issues
● Analyze and improve real-world performance
● Test real-world compatibility
Leading to:
● More effective development and quality assurance teams
● Less time spent post-release on fixing issues

13. Usability
13
What you can achieve with beta:
● Study and improve the user experience
● Assess the new user experience
Leading to:
● Higher customer retention
● Lower support costs
● Fewer returns
● Better reviews

14. Product Management
14
What you can achieve with beta:
● Evaluate the total customer experience
● Assess customer acceptance or product readiness
● Solicit feature requests/suggestions for roadmap validation
● Perform competitive analysis
Leading to:
● Better product release
● Higher customer retention
● Improved product planning
● Decreased time to market

15. Support
15
What you can achieve with beta:
● Test documentation and support materials
● Test support processes
Leading to:
● Lower support costs
● Higher customer retention
● Increased customer satisfaction

16. Marketing/Sales
16
What you can achieve with beta:
● Collect mock reviews and NPS scores
● Collect testimonials, references, or case studies
● Acquire early adopters/evangelists
● Generate product/launch awareness
Leading to:
● Better product reviews
● Increased buzz and goodwill
● Increased sales
● Fewer returns
● Increased customer satisfaction

17. 17
Visualizing the ROI of Beta

18. 18
Tangible vs. Intangible

19. 19
● Product Defects
○ Bugs
○ Interoperability
○ Usability
○ Alignment
Tangible Benefits of Beta Testing

20. 20
● Testimonials
● User-Generated Content
● Press and Analyst Access
● Pre-Launch Customer Facetime
● Documentation Review
● Product and Brand Strength
Intangible Benefits of Beta Testing
● Infrastructure Testing
● Product Review Simulation
● Marketing Messaging Refinement
● Customer Goodwill
● Tester Community
● Feature Requests (Maybe)

21. 21
Measuring ROI:
The Framework

22. 22
Two definitions of quality:
1. A product’s features and characteristics that influence
customer satisfaction
2. Freedom from defects
Cost of Quality Framework

23. 23
Cost of quality includes the costs invested in attaining a
given level of quality, as well as the consequences of
failing to meet expectations.
Cost of Quality Framework

24. ROI Calculation
24
Gains
Costs of Beta + Costs of Changes
X 100

25. ROI Calculation
25
Gains
Costs of Beta + Costs of Changes
X 100
Increased Sales
Decreased Returns
Decreased Post-Release
Development/QA Cost
Decreased Support Costs
Increased Customer
Retention
Beta Management Time
Shipping/Logistics
Beta Units
Incentives
Tools
Increased Pre-Release
Development/QA Cost

26. 26
Measuring ROI:
The Inputs

27. 27
● Beta Management Time
● Beta Units
● Shipping/Logistics
● Incentives
● Tools
● Additional Dev/QA Time
Costs (Beta + Additional Dev/QA)

28. 28
● Decreased Post-Release Dev/QA Cost
● Decreased Support Costs
● Decreased Returns
● Decreased Warranty Reserves
● Increased Customer Retention
● Increased Sales
Gains

29. 29
Measuring ROI:
The Results

30. 30
Product: Wi-fi Router, $100, 500,000 units sold, $50 million in sales
Beta Test: 100 testers, 6 weeks
● Labor: $5,500
● Shipping: $3,000
● Units: $22,000
● Incentives: $5,000
● Tools: $500
● Dev/QA Cost: $7,000
● Total: $43,000
Example ROI Calculation
Gains
● Dev/QA Cost: decreased $8,500
● Support Costs: decreased $35,000
● Returns: save $77,000
● Warranty Reserves: save $173,000
● Sales Profit: increased $218,000
● Total: $511,500
1.023%
ROI of 1,090%

31. Optimizing Your Beta Test ROI
31

32. Optimizing Your Beta Test ROI
32
1. Beta Management Time
2. Better Quality Feedback
3. Feedback Workflow
4. Beta Participation and Total Testers
5. Length of Beta Phase
6. Beta Management Tools

33. Optimizing Your Beta Test ROI
33

34. 34
Presenting ROI

35. 35
The ROI is high - be prepared to defend it
Tie it back to company initiatives (customer success, increasing sales, new
market disruption)
Show predictive nature of tests
- Compare NPS/star ratings in beta to released product ratings
Don't forget about the intangibles
- Quotes from testers to illustrate points (bugs)
- Future gains from product roadmap validation
Presenting ROI to Stakeholders

36. 36
Q&A
Emily Hossellman
Director of Marketing @ Centercode
emily@centercode.com
@ehossellman
Sign up for our newsletter and our future webinars
For more resources, visit our library

37. 37
Thank You!