4. LUXR.CO JUNE 2012
• Learn the difference between an experiment and a solution test.
• Discover the seven steps to crafting an experiment with a clear
result.
• Practice identifying assumptions and defining testable
hypotheses.
• Understand the anatomy of an experiment.
• Design an experiment to test a real-life hypothesis.
• "Get out of the classroom" to run a real-world experiment on
your fellow conference-goers.
• Define a result and make a go-forward decision.
• Refine the experiment based on cost, time, and manpower.
• Identify the four characteristics of successful experiments.
26. LUXR.CO JUNE 2012
In foolproof
experiments, the
results should be self-
evident.
27. LUXR.CO JUNE 2012
1. Choose the problem to tackle. Clarify what’s
uncertain, what you need to learn.
2. Understand the possible answers.
3. Brainstorm “what you could do to prove it”.
4. Choose the most broadly differentiated set of
indicators.
5. Run the experiment.
6. Write down the results...on a sheet of paper.
7. Project the results into your future.
7 steps to foolproof experiments
29. LUXR.CO JUNE 2012
Hypothesis
We believe that modern families would benefit
from having better ways to store their
belongings, and are willing to pay for a service
that will pick up and store boxes, then deliver
them back home on demand.
Today’s Challenge
30. LUXR.CO JUNE 2012
Box Me Up
Provides you with plastic storage boxes
Transports your stuff to storage
Pay to get your stuff back
Per-box per year
2800¥ per box per year
outside of the city (less expensive place)
box size: 30 gallon box
service area is in Tokyo
insurance is 20,000 yen per box
temperature and humidity are normally conditioned
no limit for storage perioe
24 hours access to the box
privately storage
payment method: paypal, square, credit card, no cash
delivery will be by car
nothing illegal, perishable, waste will be stored
3 employees in the company
main customer will be families
got 70,000,000 yen for initial investment
31. LUXR.CO JUNE 2012
Experiment Framework
For Example:
We believe people like [customer type] have a need for
(or problem doing) [need/action/behavior].
The smallest thing we can do to prove that need is
[experiment].
We will know we have succeeded when [quantitative/
measurable outcome] or [qualitative/observable outcome].
Every experiment has three parts:
1. Hypothesis that is provable/disprovable
2. The experiment itself; the thing you build
3. An indicator of result
37. LUXR.CO JUNE 2012
{Activity}
Design an experiment
to learn if this is true.
we believe that ________________
_________________________________
_________________________________
_________________________________
_________________________________
_________________________________
_________________________________
__________________________
Briefly describe it.
State how you will know if the
hypothesis is valid or invalid.
This can be quantitative
evidence or qualitative.
How much time/money/effort
will it take?
38. LUXR.CO JUNE 2012
{Activity}
Discuss, then pick the
experiment to run.
we believe that ________________
__________________________________
__________________________________
__________________________________
__________________________________
__________________________________
__________________________________
____________ in 2 days.
Decide
we believe that ________________
__________________________________
__________________________________
__________________________________
__________________________________
__________________________________
__________________________________
____________ in 2 weeks.
we believe that ________________
_________________________________
________________________________
________________________________
________________________________
_______________________________
_______________________________
____________ in 2 months.
✔
39. LUXR.CO JUNE 2012
Experiments that were designed
assumption experiment
measurable/observable
outcome
busy people need a service
to help them get
menial stuff done
fb ad, neighborhood targeting
presents an offer, landing page.
Phone number or tweet. No out
of pocket cost.
10% ctr on ad, 10% ctr
on landing page. 2 days
busy people need a service
to help them get
menial stuff done
find busy people (in offices). Set up a
“shop”. Sit in their reception and see if
we can get jobs. Prevail upon friend
who runs 300-person company
5% of employees will make
a hire. 1 repeat booking
2 wks
busy people need a service
to help them get
menial stuff done
ad in 10 different offices,
diff types of companies
2 weeks.
43. LUXR.CO JUNE 2012
Techniques Checklist
• Brainstorm quietly
• Use Sticky Notes for ideas
• Self-Edit
• Go to the wall for team sharing
• Stack-ranking
• Roman Voting
• Right-Sizing the work