In today's competitive landscape, your product must exceed expectations and create a compelling customer experience. Here, we discuss how to leverage real customers in real environments to evaluate the stability, satisfaction, and adoption of your product before release and achieve Customer Validation.

2. 2
What You'll Learn Today
Whatexactly is Customer Validation?
Whyis it critical to the success of your product?
Whodoes it apply to in the industry?
Wheredoes it fit within your organization?
Whendoes it take place within the product life-cycle?
Howdoes it actually work (high level)?

3. the speaker and Centercode
is Customer Validation (CV)?
is CV critical to the success of your product?
does CV apply to?
does CV belong within your organization?
does CV occur in PLM?
does CV actually work?
and next steps
About
What
Why
Who
Where
When
How
Q&A

4. 4
About Me
Luke Freiler
CEO + Co-founder
Centercode
Engineering & UX Background
Previously with Samsung & Ericsson
Hardware & Software • Consumer & Business
luke@centercode.com
linkedin.com/in/freiler/

5. 5
"Technology is a word that describes
something that doesn't work yet"
- Douglas Adams
Centercode Mission
We help companies leverage their
customers to build better products.

6. ASSETS
Centercode Overview
Customer Validation (CV) is our core competency • Founded in 2001 • Industry leader
Tester Panel 180k+ deeply profiled global test candidates
Framework Scalable and adaptable Customer Validation process
Platform End-to-end cross-platform Customer Validation platform
OFFERINGS
Managed CV
Programs1
CV Platform
Subscription2
Customer
Research3

7. Industry Leading Customers
WE SERVICE AMAZING HIGH-GROWTH TECHNOLOGY AND MODERN ENTERPRISE COMPANIES

8. the speaker and Centercode
is Customer Validation (CV)?
is CV critical to the success of your product?
does CV apply to?
does CV belong within your organization?
does CV occur in PLM?
does CV actually work?
and next steps
About
What
Why
Who
Where
When
How
Q&A

9. Customer Validation?
What is
Lower Case
Customers testing real
unreleased products in real
environments.
AKA Beta testing
Upper Case
A distinct discipline for gathering and synthesizing representative customer
feedback to produce a more successful product.
Specific proven research and testing methodologies utilized to capture unique
actionable insight about the customer experience of actual products
throughout their development, from targeted users in their true environments.

10. Beta Testing
Charter Program
Client Validation
Controlled Release
Customer Acceptance Testing (CAT)
Customer Development Program
Customer Experience Trials
Customer Feedback Sessions
Customer Integration
Customer Satisfaction
Customer Tech Preview
Customer Technology Preview
Customer Validation
Design Validation Test
Early Acceptance
Early Adopter Program (EAP)
Early Preview
Early Release
Early Visibility
End User Validation
Field Test
Friendly User Trials (FUT)
Labs
Let the Customer Bang on It
Limited Availability
Limited Release
Market Readiness Testing (MRT)
Pilot Test
Play Test
POC
Pre-Release
RA Release
Ramp Up
Soft Launch
Soft Release
Trusted Testers
User Acceptance Testing (UAT)
User Test
User Validation
Voice of the Customer
The Need for "Upper Case" Customer Validation

11. Defining Attributes of CV
11
Targeted Users • recruited based on demographic and technical
characteristics that are representative of your true audience
Actual Products • tangible pre-production products beyond the
prototype stage of development
Real Environments • the actual homes, workplaces, vehicles, and
lives of the representative customers
Over Time • extended study of actual real-world product usage
by over weeks or months

12. Customer Validation Test Stages
12
Field
Evaluate Adoption
Targeted customers focused on the real-world customer experience
Beta
Evaluate Satisfaction
Targeted customers guided through a comprehensive product tour
Alpha
Evaluate Stability
Technical users identifying quality issues in real environments

13. Customer Validation Stage Details
ALPHA TEST BETA TEST FIELD TEST
Evaluate Stability Evaluate Satisfaction Evaluate AdoptionTest Objective
Does it work? Do they like it? How will they use it?Topic Question
Bug Hunt Guided Evaluation Natural UsageStrategy
Test Ready
(60-80%)
Near/Feature Complete
(80-90%)
Release Candidate
(90%+)
Product State
2 Weeks
(per cycle)
3-6 Weeks
(1-2 cycles)
6+ Weeks
(1 cycle)
Typical Test Length
25-50 25-200 100+Typical Testers

14. the speaker and Centercode
is Customer Validation (CV)?
is CV critical to the success of your product?
does CV apply to?
does CV belong within your organization?
does CV occur in PLM?
does CV actually work?
and next steps
About
What
Why
Who
Where
When
How
Q&A

15. Relevant Technology Product Trends
15
Products have become far more complex1
2
3
The audience has changed; higher expectations
Fewer channels severely increase risk

16. 16
IoT ("Internet of Things") products are a model example of this:
1. They're deeply complex (SW + HW + Services)
2. They're highly interconnected
3. They never stop evolving (OTA)
4. They're hyper competitive
Example: IoT Products

17. PROFIT
Customer Validation Impact and ROI
QUALITY COST
DEV TIME
INT. BIAS
TIME TO MARKET
RETURNS
SUPPORT COST
QUALITY
UX
REVIEWS
CUST SAT
PLANNING
SALES
17

18. the speaker and Centercode
is Customer Validation (CV)?
is CV critical to the success of your product?
does CV apply to?
does CV belong within your organization?
does CV occur in PLM?
does CV actually work?
and next steps
About
What
Why
Who
Where
When
How
Q&A

19. Product Type Product Audience
19
What Products Benefit from Customer Validation
Short Answer: Any product with a customer
Hardware
Software
Services
Consumer
Business
Enterprise

20. the speaker and Centercode
is Customer Validation (CV)?
is CV critical to the success of your product?
does CV apply to?
does CV belong within your organization?
does CV occur in PLM?
does CV actually work?
and next steps
About
What
Why
Who
Where
When
How
Q&A

21. 21
Customer Validation Within an Organization
Quality UX/CX
Market
Research
Customer
Validation

22. 22
Customer Validation Within an Organization
Quality UX/CX
Market
Research
Customer
Validation
Technology .
.
. Customer
Business
Product
Management

23. the speaker and Centercode
is Customer Validation (CV)?
is CV critical to the success of your product?
does CV apply to?
does CV belong within your organization?
does CV occur in PLM?
does CV actually work?
and next steps
About
What
Why
Who
Where
When
How
Q&A

24. 24
When
does CV
occur?

25. 25
CV within Product Life-cycle
Quality Alpha Beta Field Release
Core Customer Validation
Continuous
Customer
Validation
(next time)

26. the speaker and Centercode
is Customer Validation (CV)?
is CV critical to the success of your product?
does CV apply to?
does CV belong within your organization?
does CV occur in PLM?
does CV actually work?
and next steps
About
What
Why
Who
Where
When
How
Q&A

27. 27
Typical Legacy Process (aka "beta testing")
1. Beg engineering for a beta release
2. Scavenge for people willing to be testers
3. Send out the beta product (with minimal context)
4. Beg testers for feedback (via email)
5. Shotgun distribution of results
6. Pray nothing fell through the cracks

28. 28
The Source of the Problem
PROCESS
PEOPLE
TECHNOLOGY

29. Key Customer Validation Challenges
29
Project design and scoping
Allocating qualified resources
Recruiting ideal testers
Persistent user engagement
Collecting useful feedback
Deriving actionable recommendations

30. 30
Legacy vs. Customer Validation Objectives
Data
e.g. Exports
Information
e.g. Charts
Insight
e.g. Facts
Recommendation
e.g Actions
CV Effort PO Effort
Platform CVF
Problems Solutions

31. 31
Ongoing
Feedback
Directed
Research
Analysis and Recommendation
Customer Validation Process Overview
Test Scoping and Product MappingPlanning
Recruitment and SelectionPrep
Weekly
Tester
Support
Engagement
Management
Collection and Wrap-upClosure

32. Test Scoping and Product Topic Mapping
Topics represent the individual features, functions, and experiences you want tested.
Product
Setup
The process of connecting the product
for first time use.
Find a spot to plug the device into both
the network and wall.
1.8 M
Account
Creation
The process of creating a personal
account to customize the product.
Log into www.xyz.com and create
a new user account.
1.2 S
Topic
Out of
the Box
Onboarding
Description
The initial experience opening the
product package.
Activity
Receive and unbox the product.
Impact
1.2
Size
S

33. 33
Recruitment and Selection (Beta Example)
US Resident, Own 2+ Smart Home ProductsCore Requirements
Android iOS .Technical Segmentations
Smart Home Interest + ExperienceAdditional Traits
Demographic Segmentations Age 18-25 Age 26-35 Age 36-50 . Targeted
Enthusiastic
Strangers
Profiled Panel +

34. 34
Ongoing Feedback Management
All Feedback must be Filtered, Categorized, and Prioritized based on your objectives (Topics and concerns)
Bug Reports Test quality, interoperability, and real-world performance
Measure acceptance, prioritize backlog, generate new ideasSuggestions
Gather user insight and sentiment; study user experiencePrivate Journals
Emulate real-world discussion; drive focused collaborationOpen Discussions

35. Assign
Topic
Activities
Directed Research Management
Centercode
Analysis
Weekly Presentation including Recommendation Scorecards (FIPs)
35
Engage
and Support
Testers
Customer
Acceptance
Survey

36. Results Overview(Beta Sample)
4.0
out of 5
Recommendations to FIX 3
TOPIC: UNBOXING
Customer Acceptance Score
This score is AVERAGE.
Recommendations to IMPROVE 5
Recommendations to PROMOTE 1
RecommendationsUnboxing Task
TASK
Review the packaging of the product
DESCRIPTION
Review the product packaging and
information without opening the box.
Get to know the product as if you were
considering it prior to purchase.
36

37. Fix Recommendation Sample
Tester Quotes
RELATED FEEDBACK
BUG-0015
Antenna broke pulling it out
of the box
SUG-0013
Better packaging for the
antenna
Reinforce the Antenna
in Packaging
Some testers reported frustration with
the antenna breaking while attempting
to remove it from the packaging.
A broken antenna can lead to a poor
perception of product quality as well
as performance issues.
“While trying to remove the product from the box
the antenna snapped off. I did not even apply that
much pressure.”
“I did not realize that the antenna was broken off
until I heard it banging around in the box. Very
cheaply made. Plus the packaging was so hard to
remove, it is no wonder it broke off."
15%
RESPONDENTS
P1
PRIORITY
Major
SEVERITY
FIX
RECOMMENDATION
37

38. Improve Recommendations Sample
Tester Quotes
RELATED FEEDBACK
BUG-0041
Twist Ties Were Broken
SUG-0013
Needs Better Twist Ties or
Restraints
“Everything was going great as I pulled stuff out of the
box, except the twist ties that held the ethernet cord
together were already broken.”
“I enjoyed pulling everything out of the box. Things
were neat and easily accessible. But, the twist ties
around the cords looked like twist ties on bread and
took the feel of quality out of most the experience.”
20%
RESPONDENTS
P4
PRIORITY
Minor
SEVERITY
IMPROVE
RECOMMENDATION
Provide Higher
Quality Twist Ties
Testers feel that the twist ties are
very cheap compared to the rest of
the contents in the packaging. Some
testers reported that twist ties were
broken upon first inspection.
This gives testers the mindset of a
product of lesser quality and can
result in a poor display.
38

39. Promote Recommendation Sample
Tester Quotes
RELATED FEEDBACK
DIS-0100
Everything is nicely stored
“WOW! This thing is clean, the
packaging felt very orderly making it
simple to unbox. I wish all the products
I purchased came like this."
30%
RESPONDENTS
P3
PRIORITY
PROMOTE
RECOMMENDATION
Packaging Contents
are Neat and Orderly
Testers felt like, overall, everything
was very well organized within the
box. The hub and the light were very
nicely stored and made it easy to pull
them out.
This should be promoted to your
project teams or other members that
can benefit from positive feedback.
"Very impressed by the packaging.
I'm going to end up holding onto it
just in case I move and need to store
all of the components again.
Everything fit into its nice little
compartment, for some reason I
even felt compelled to take a picture
of it. Got me in the mood to organize
some stuff at home."
“Packaging rocks. The box feels
awesome and the contents are neatly
stored. I was amazed with how neatly
things were stored."
39

40. the speaker and Centercode
is Customer Validation (CV)?
is CV critical to the success of your product?
does CV apply to?
does CV belong within your organization?
does CV occur in PLM?
does CV actually work?
and next steps
About
What
Why
Who
Where
When
How
Q&A

41. 41
Open Q&A
Luke Freiler
CEO @ Centercode
Thank You!

42. Thank You