3. INTRODUCTION
• Sheen Yap
• UX expert, design leader and lean evangelist
• 17-years’ UX experience across web, mobile, iTV
• Consulting to startups and corporates to help
building better products through Lean and UX
• Co-creator circula.co
4. WORKSHOP OBJECTIVES
• Early stage startups - new idea, pivoting or existing
• Go through process of validating ideas
• Walk through and apply at each step
• Understand and willing to perform user testing
• Have fun, make mistakes, learn!
10. PROBLEM-SOLUTION FIT - NOW
• Product idea is about problem-solution fit
• Need to solve Real Problems for Real People
• Avoid eventual failure
• Avoid waste in time, effort and money
• Avoid pain in dismantling
• Missed opportunity to pivot to a better idea
11. ..HOW TO CREATE PRODUCTS THAT PEOPLE WILL USE
HOW TO AVOID CREATING PRODUCTS NO ONE WILL USE
TEST WITH USERS, & TEST EARLY!
13. 1. Create
Model
2. List
Assumptions
3. Generate
Hypothesis
4. Design
Experiment
6. Run
experiment
7. Analyse
Results
8. Decide
Next Steps
5.
Experiment
(interview)
Captured
Data
14. 1. Create
Model
2. List
Assumptions
3. Generate
Hypothesis
4. Design
Experiment
6. Run
experiment
7. Analyse
Results
8. Decide
Next Steps
5.
Experiment
(interview)
Captured
Data
15. 1. Create
Model
2. List
Assumptions
3. Generate
Hypothesis
4. Design
Experiment
6. Run
experiment
7. Analyse
Results
8. Decide
Next Steps
5.
Experiment
(interview)
Captured
Data
CREATE
MODEL
16. “IDEAS”
• Ideas normally come as a “solution” i.e.
A Thing that has A, B and C Features.
• We are seduced by its perceived tangibility
• No way for anyone else to evaluate its true value to
the world
• Can’t tell if it fulfils a genuine need
• Restricts the flexibility for a better solution
21. EXERCISE
• Who are your customer segments?
• What are their problems/pain points/issues?
• Who are your customers?
• What are their problems/pain points/issues?
• Who could be your early adopter?
22. 1. Create
Model
2. List
Assumptions
3. Generate
Hypothesis
4. Design
Experiment
6. Run
experiment
7. Analyse
Results
8. Decide
Next Steps
5.
Experiment
(interview)
Captured
Data
LIST
ASSUMPTIONS
23. ASSUMPTIONS
• “Theory behind the statements”
• Unvalidated, each statement is a figment of our
imagination: just because we believe it, does not
mean it’s true
• Everything on the canvas are assumptions - we
need to methodically test each of them
24. ASSUMPTION STATEMENT
• We believe that: [user segment] want to do [expected
behaviour] because [pain point].
• We believe this [user segment] exist and we can find
them at the following [channels]
• We believe that [user segment] will pay [amount] to
use our service.
• We believe that [our UVP] will solve the [user
segment]’s [problem].
25. EXAMPLE - FB’04
• Online presence lacking:
We believe that lots of students find their current
online presence unsatisfactory
• Hard to stay in contact:
We believe that many students worry about losing
contact with others
26. EXERCISE
• Pick one of your problems
• Rewrite into an assumption:
We believe that [user segment] has this [pain
point/issue/need].
27. 1. Create
Model
2. List
Assumptions
3. Generate
Hypothesis
4. Design
Experiment
6. Run
experiment
7. Analyse
Results
8. Decide
Next Steps
5.
Experiment
(interview)
Captured
Data
GENERATE
HYPOTHESIS
28. FALSIFIABLE HYPOTHESIS
• Assumptions need to be testable for validation
• Creating a Falsifiable Hypothesis: i.e. can be
proved true or false by empirically testing
• Format:
If I were to [perform activity] on [sample subject],
then I would be able to [observe an outcome]
29. EXAMPLE AND EXERCISE
• Assumption:
We believe that lots of students find their current online
presence unsatisfactory
• Hypothesis:
If I were to investigate a number of Harvard students, I would
find that 90% of them would be unhappy with their online
presence
• Create a hypothesis for your assumption:
If I were to [perform activity] on [sample subject], then I would
be able to [observe an outcome]
30. 1. Create
Model
2. List
Assumptions
3. Generate
Hypothesis
4. Design
Experiment
6.
Run
experiment
(perform
7. Analyse
Results
8. Decide
Next Steps
5.
Experiment
(interview)
Captured
Data
DESIGN
EXPERIMENT
31. WHY EXPERIMENTS
• Typical statement:
“Built something, put it out there, learnt a lot.”
• Question: What did you REALLY learn?
• Stuff that could be semi-relevant, might be useful
later. Hopefully.
• Wasted cycle: still no progress on your most critical
problems; worse, could be distracted onto a different
direction
32. AN EXPERIMENT
• Has a learning goal
• Explicit stated hypothesis
• Anticipates observable outcome
• Repeatable
• Avoid biases (try our best to!)
33. EXPERIMENT TEMPLATE
Goal: To validate your stated assumption
Hypothesis: Put your hypothesis here
Procedure: will talk about this next…
Expected Results: come back to this later…
Actual Results: input results after running the experiment…
34. 1. Create
Model
2. List
Assumptions
3. Generate
Hypothesis
4. Design
Experiment
6. Run
experiment
7. Analyse
Results
8. Decide
Next Steps
5.
Experiment
(interview)
Captured
Data
EXPERIMENT
36. INTERVIEW OBJECTIVES
1. Validate Problem
Do customers have the problem you are solving?
2. Capture Existing Alternatives
How do customers solve these problems today?
3. Validate Customer Segments
Are these customers the ones with the pain?
37. INTERVIEWING TIPS
• Set context first
Statement or closed question.
• Then ask open questions
Use What/How/Why?
• Follow up questions
Why? Tell me more?
• Let them talk; expect messy outcome
38. INTERVIEW QUESTIONS
• Hypothesis: If I were to investigate a number of Harvard
students, I would find that 90% of them would be unhappy with
their online presence
• Context setting:
I’m trying to find out more about people’s online presence at the
moment. Can you tell me if you have an online presence?
• Open questions:
(If yes) What do you have? How are you finding it?
(If no) Why is that?
• Now write your own question(s) for your hypothesis
39. EXPERIMENT TEMPLATE
Goal: To validate your stated assumption
Hypothesis: Put your hypothesis here
Procedure: Your interview question(s) to applied to ? people
Expected Results: State your expected outcome
Actual Results: input results after running the experiment…
40. TEMPLATE EXAMPLE
Goal: To validate our belief that lots of students find their
current online presence unsatisfactory
Hypothesis: If I were to investigate a number of Harvard
students, I would find that 90% of them would be unhappy with
their online presence
Procedure:
Interview question(s) to applied to 5 people
Expected Results: At least 4 people will register some
complaint about their current online presence
Actual Results:
41. TEMPLATE EXERCISE
Goal: To validate your stated assumption
Hypothesis: Put your hypothesis here
Procedure: Your interview question(s) to applied to ? people
Expected Results: State your expected outcome
Actual Results:
42. 1. Create
Model
2. List
Assumptions
3. Generate
Hypothesis
4. Design
Experiment
6. Run
experiment
7. Analyse
Results
8. Decide
Next Steps
5.
Experiment
(interview)
Captured
Data
RUN
EXPERIMENT
43. INTERVIEW PROCESS
• Perform interview with the person next to you
• Interviewee pretend to be target audience
• Try noting/recording in verbatim
• Take turns, 5 minutes each
44. INTERVIEW EXAMPLE
• Interviewing Harvard Student, circa 2004
• I’m trying to find out more about people’s online presence at the
moment. Can you tell me if you have an online presence?
“Yes I do.”
• What do you have? How are you finding it?
“I have the college directory profile but it’s very limiting in terms of
the information it shows, doesn’t tell people much about the real me.
I also have a myspace page but it’s kinda kept between my friends
and not shown to others in college as stuff there isn’t appropriate. It’s
fun but I am spending far too much time maintaining it.”
45. INTERVIEW EXERCISE
• Perform interview with the person next to you
• Interviewee to pretend to be target audience
• Try noting/recording in verbatim
• Take turns, 5 minutes each
46. 1. Create
Model
2. List
Assumptions
3. Generate
Hypothesis
4. Design
Experiment
6.
Run
experiment
(perform
7. Analyse
Results
8. Decide
Next Steps
5.
Experiment
(interview)
Captured
Data
ANALYSE
RESULTS
47. ANALYSIS GUIDELINES
• Complete ALL interviews first
• Based on what you have recorded:
1. Has it met your expected results, not met them
or inconclusive? What does it mean to your
hypothesis?
2. What new information have you heard?
3. Any new ideas/insights?
48. ANALYSIS EXAMPLE
• I have the college directory profile but it’s very
limiting in terms of the information it shows,
doesn’t tell people much about the real me. I also
have a myspace page but it’s kinda kept between
my friends and not shown to others in college as
stuff there isn’t appropriate. It’s fun but I am
spending far too much time maintaining it.
Result:
complains
registered
Hypothesis:
validated New info:
college
directory
limiting
New info:
myspace is
inappropriate
for everyone
New info:
Both don't
present the
“real me”
Insight/Idea:
Showing Real
Identify is
important
49. ANALYSIS EXERCISE
• Based on what you have recorded:
1. Has it met your expected results, not met
them or inconclusive? What does it mean to
your hypothesis?
2. What new information have you heard?
3. Any new ideas/insights?
50. 1. Create
Model
2. List
Assumptions
3. Generate
Hypothesis
4. Design
Experiment
6.
Run
experiment
(perform
7. Analyse
Results
8. Decide
Next Steps
5.
Experiment
(interview)
Captured
Data
DECIDE NEXT
STEPS
56. PICK THE RIGHT TOOLS
Usability
Testing
Ethno-
graphy
Interviews
Partici-
patory
Design
Focus
Groups
Eye
Tracking
Compe-
titive
Research
Diary
Study
Bench-
marking
Forum
Mining
Blogpost
Test
Landing
Page
Tests
PPC Tests
A/B Tests
Customer
Feedback
Card
Sorting
Wizard of
Oz
Neuro
Imaging
Data
Mining
Data
Analysis
Ad Hoc
Question-
naire
Intercept
Surveys
Email
Surveys
Product
Stubs
57. IT IS ABOUT PEOPLE…
…try walking in their shoes!