Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Testing Your Baby
“How do I get Janice and the
luxr mafia off my back?”

                  Rick Boardman
                  ...
Cast
                     Product/Biz              UX


                                              User
               ...
Think                                                                                                                     ...
http://www.all-tribes.com/eng/catalog/rtfm-vol2-girls-mastic-p-51.html
How do I do Unix?




      http://www.all-tribes.com/eng/catalog/rtfm-vol2-girls-mastic-p-51.html
RTFM!
How do I do Unix?




Read The F****** Manual
      http://www.all-tribes.com/eng/catalog/rtfm-vol2-girls-mastic-p-5...
Pic: http://www.flickr.com/photos/victoriapeckham/164175205
How do we
              test our
            hypotheses?




Pic: http://www.flickr.com/photos/victoriapeckham/164175205
How do we                           LTYFU!
              test our
            hypotheses?




   Listen To Your F****** Us...
Head in sand        Guess
  Ask Janice
                                                Ask Jason
        Which method?
Dem...
Head in sand      Guess
  Ask Dave
                                         Ask Enrique
        Which method?
Demo at
 SXS...
Head in sand               Guess
       Ask Dave
                                                                 Ask Enri...
Head in sand               Guess
       Ask Dave
                                                                 Ask Enri...
WTF? ... thats not very
     helpful ...
Sol  ution:
  Trian  gulate
Mult iple M ethods
               Analytics




Usability
              Offer Janice beer
A nalytic s suck


*Analytics are useful but have strengths and weaknesses
Why Analytics Suck
1. Tell you <what> not <why>
2. Hard to isolate specific effects
3. Which numbers do I pay
  attention t...
Oooo
AAARRRR, got a
  dashboard?




       Link: Metrics
       for Pirates
Example Conversion Dashboard
                 (note: *not* actuals… your mileage may vary)


  Stage                    Co...
Re mote (  autom  ated)
  Usa bility S ucks*

*Remote usability is useful but has strengths and weaknesses
Why Remote
     (Automated)
    Usability Sucks
1. Participant quality
2. Hard to dig deep
3. Need high-fidelity prototype
...
Why Remote
      (Automated)
    Usability Sucks
1. Participant quality
     Useful tool but
2. Hard to dig deep
     dang...
Live U sability
            S ucks*

*Live usability is useful but has strengths and weaknesses
Why Live Usability
      Sucks
1. Pricey? ($$$ and time)
2. Hard to cover large sample
3. You have to talk to people
Usability testing
• Watch your product in action
  with a target user
• Follow some tricks to make as
  objective as possi...
Usability Howto
1. Identify research goal -> Scenario
2. Recruit
3. Prepare interview outline, pilot test!
4. Test!

  -  ...
Golden Rule of
   Usability
Golden Rules of
    Usability
Participant should do 90% of the talking



                  Shut up
Golden Rules of
    Usability
Participant should do 90% of the talking



     An Sh
       d ITE RuTEup
              At !
Things to look for

• Obvious usability issues
• Bugs
• Opportunities
• Don’t forget Delights!
Lean UX questions

1. What gear do I need?
2. How many users?
3.Which users?
observation
     test room                          room

                 1-way mirror
eyetracking
                      ...
Observer /
                        notetaker /
                        prototype
                        debugger /
Modera...
Lean UX questions

1. What gear do I need?
2. How many users?
3.Which users?
Lean UX questions

  1. What gear do I need?
  2. How many users? >=0*
  3.Which users?
* 2: Ideally, small n and iterate....
Lean UX questions

   1. What gear do I need?
   2. How many users? >=0*
   3.Which users? Any*
* 2: Ideally, small n and ...
Awesome
2 hour read
              http://www.sensible.com/rocketsurgery/index.html
Interview Practice
Interview Practice

        I’m super
      nervous ... am I
       being tested?
Interview Practice
 What do I do
   next?

                Should I sign up?
                  Is this OK?
Interview Practice

         I HATE IT.
      THIS IS &^**(}ED
         UP, MAN.
Interview Practice

      I LOVE THIS. ITS THE
        BEST THING EVER
Interview Practice

           <click>
      <click> <done>
         <silence>
Interview Practice
      Well it looks like you can
    use this menu to browse and
     you can use that button to
      ...
Interview Practice
      Weather wafflewaffle
     waffle Superbowl waffle
     waffle waffle waffle waffle
     waffle waffle waffle...
Interview Practice

           ?
Interview Practice

    Let me show you the
    product! What we’re
     trying to do here ...
Interview Practice

    This is my baby. What
    do you love about it?
Interview Practice

    This is my baby. What
    annoys you about it?
Interview Practice
      Weather wafflewaffle
     waffle Superbowl waffle
     waffle waffle waffle waffle
     waffle waffle waffle...
Interview Practice

    Lets do a focus group
Interview Practice

         ?
GO FORTH & LTYFU!



  Q & A!
You’ve finished this document.
Download and read it offline.
Upcoming SlideShare
Acw presentatie meten is weten do 14.06.2012
Next
Upcoming SlideShare
Acw presentatie meten is weten do 14.06.2012
Next
Download to read offline and view in fullscreen.

1

Share

The RTFM of Usability at LUXR June 2011

Download to read offline

Related Books

Free with a 30 day trial from Scribd

See all

Related Audiobooks

Free with a 30 day trial from Scribd

See all

The RTFM of Usability at LUXR June 2011

  1. 1. Testing Your Baby “How do I get Janice and the luxr mafia off my back?” Rick Boardman @uxrick luxr Jun/17/2011
  2. 2. Cast Product/Biz UX User Eng Thanks to Braden Kowitz for smiley graphics
  3. 3. Think Usability? Intro to Intro to (Speaker/ How to Lean UX Task generative Personas Intersect What are methods? Analysis? Lecture) interviewing Distinguish & POVs Brainstorm you building btw needs w/ Biz & Analytics Distro in next week & insights Agile? & next 3 mo? User Customer/ Solution Needs Uses Features Prototype Story/ Test Empathy Problem Ideation Release How do we test our Definition Make hypotheses? Make persona (Method/ Practice with of target user(s) Framing Wireframes Mock Activity) partner Brainstorm dashboard POV madlib Practice Ecosystem & Storyboard Sketches synthesizing Distribution Flows empathy map map How to test conversion Refine survey Compare personas Prioritize How to events? questions created by diff based on tech/ test? Test with real team members biz/distro users & Check Product Design Guild What are How to (Method/ riskiest How to test? Activity) Show partner test? hypotheses? map Interview real users? Thurs Fri Sat
  4. 4. http://www.all-tribes.com/eng/catalog/rtfm-vol2-girls-mastic-p-51.html
  5. 5. How do I do Unix? http://www.all-tribes.com/eng/catalog/rtfm-vol2-girls-mastic-p-51.html
  6. 6. RTFM! How do I do Unix? Read The F****** Manual http://www.all-tribes.com/eng/catalog/rtfm-vol2-girls-mastic-p-51.html
  7. 7. Pic: http://www.flickr.com/photos/victoriapeckham/164175205
  8. 8. How do we test our hypotheses? Pic: http://www.flickr.com/photos/victoriapeckham/164175205
  9. 9. How do we LTYFU! test our hypotheses? Listen To Your F****** Users Pic: http://www.flickr.com/photos/victoriapeckham/164175205
  10. 10. Head in sand Guess Ask Janice Ask Jason Which method? Demo at “Launch and cognitive walkthrough SXSW see” Analytics expert review Usability testing Friends n’ Family A/B testing Dogfooding Funnel analysis remote testing Contextual interviews Beta click/5sec tests Focus groups Surveys Show to David Pogue
  11. 11. Head in sand Guess Ask Dave Ask Enrique Which method? Demo at SXSW H ere’s a dirty “Launch and cognitive walkthrough see” secUsability expert review ... ret .testing Friends n’ Family Analytics A/B testing s shhhh Funnel analysis sssDogfooding remote testing Contextual interviews Beta click/5sec tests Focus groups Surveys Show to David Pogue
  12. 12. Head in sand Guess Ask Dave Ask Enrique Which method? Demo at H ere’s a dirty “Launch and cognitive walkthrough TheeUsability.testingsFriends kFamily yeta.l.l. ucn’ * SXSW see” Analytics s cr expert review A/B testing s shhhh Funnel analysis sssDogfooding remote testing Contextual interviews Beta click/5sec tests Focus groups Surveys Show to David Pogue *More diplomatically: they have strengths and weaknesses
  13. 13. Head in sand Guess Ask Dave Ask Enrique Which method? Demo at H ere’s a dirty “Launch and cognitive walkthrough TheeUsability.testingsFriends kFamily yeta.l.l. ucn’ * SXSW see” Analytics s cr expert review A/B testing s shhhh Funnel analysis sssDogfooding remote testing Contextual interviews Beta Focus groups Surveys Especiallytests one click/5sec this Show to David Pogue *More diplomatically: they have strengths and weaknesses
  14. 14. WTF? ... thats not very helpful ...
  15. 15. Sol ution: Trian gulate Mult iple M ethods Analytics Usability Offer Janice beer
  16. 16. A nalytic s suck *Analytics are useful but have strengths and weaknesses
  17. 17. Why Analytics Suck 1. Tell you <what> not <why> 2. Hard to isolate specific effects 3. Which numbers do I pay attention to?
  18. 18. Oooo AAARRRR, got a dashboard? Link: Metrics for Pirates
  19. 19. Example Conversion Dashboard (note: *not* actuals… your mileage may vary) Stage Conversion Status Conv. Est. Value % (*not* cost) Acquisition Visitors -> Site/Widget/Landing Page 60% $.05 (2+ pages, 10+ sec, 1+ clicks = don’t abandon) Activation “Happy” 1st Visit; Usage/Signup 15% $.25 (clicks/time/pages, email/profile reg, feature usage) Retention Users Come Back; Multiple Visits 5% $1 (1-3x visits/mo; email/feed open rate / CTR) Referral Users Refer Others 1% $5 (cust sat >=8; viral K factor > 1; ) Revenue Users Pay / Generate $$$ 2% $50 (first txn, break-even, target profitability) Link: Metrics for Pirates
  20. 20. Re mote ( autom ated) Usa bility S ucks* *Remote usability is useful but has strengths and weaknesses
  21. 21. Why Remote (Automated) Usability Sucks 1. Participant quality 2. Hard to dig deep 3. Need high-fidelity prototype 4. Don’t get to meet fun people (i.e. miss out on useful context)
  22. 22. Why Remote (Automated) Usability Sucks 1. Participant quality Useful tool but 2. Hard to dig deep dangerous too 3. Need high-fidelity prototype 4. Don’t get to meet fun people (i.e. miss out on useful context)
  23. 23. Live U sability S ucks* *Live usability is useful but has strengths and weaknesses
  24. 24. Why Live Usability Sucks 1. Pricey? ($$$ and time) 2. Hard to cover large sample 3. You have to talk to people
  25. 25. Usability testing • Watch your product in action with a target user • Follow some tricks to make as objective as possible erview bonus : opportunity Live int for m ore generative research!
  26. 26. Usability Howto 1. Identify research goal -> Scenario 2. Recruit 3. Prepare interview outline, pilot test! 4. Test! - Intro (welcome/confidential/reassure/thinkaloud) - Record / take notes - Debrief 5. Review, GET CODING, and iterate
  27. 27. Golden Rule of Usability
  28. 28. Golden Rules of Usability Participant should do 90% of the talking Shut up
  29. 29. Golden Rules of Usability Participant should do 90% of the talking An Sh d ITE RuTEup At !
  30. 30. Things to look for • Obvious usability issues • Bugs • Opportunities • Don’t forget Delights!
  31. 31. Lean UX questions 1. What gear do I need? 2. How many users? 3.Which users?
  32. 32. observation test room room 1-way mirror eyetracking UI usb mic Eng Moderator User PM 32
  33. 33. Observer / notetaker / prototype debugger / Moderator User coffee maker (optional but nice) 33
  34. 34. Lean UX questions 1. What gear do I need? 2. How many users? 3.Which users?
  35. 35. Lean UX questions 1. What gear do I need? 2. How many users? >=0* 3.Which users? * 2: Ideally, small n and iterate. If no one available, do a walkthrough!
  36. 36. Lean UX questions 1. What gear do I need? 2. How many users? >=0* 3.Which users? Any* * 2: Ideally, small n and iterate. If no one available, do a walkthrough! Focus on core interaction flow. * 3: Start with your neighbour or Janice, then focus.
  37. 37. Awesome 2 hour read http://www.sensible.com/rocketsurgery/index.html
  38. 38. Interview Practice
  39. 39. Interview Practice I’m super nervous ... am I being tested?
  40. 40. Interview Practice What do I do next? Should I sign up? Is this OK?
  41. 41. Interview Practice I HATE IT. THIS IS &^**(}ED UP, MAN.
  42. 42. Interview Practice I LOVE THIS. ITS THE BEST THING EVER
  43. 43. Interview Practice <click> <click> <done> <silence>
  44. 44. Interview Practice Well it looks like you can use this menu to browse and you can use that button to signup <i.e. here’s a guided tour>
  45. 45. Interview Practice Weather wafflewaffle waffle Superbowl waffle waffle waffle waffle waffle waffle waffle waffle waffle waffle waffle waffle Superbowl ...
  46. 46. Interview Practice ?
  47. 47. Interview Practice Let me show you the product! What we’re trying to do here ...
  48. 48. Interview Practice This is my baby. What do you love about it?
  49. 49. Interview Practice This is my baby. What annoys you about it?
  50. 50. Interview Practice Weather wafflewaffle waffle Superbowl waffle waffle waffle waffle waffle waffle waffle waffle waffle waffle waffle waffle Superbowl ...
  51. 51. Interview Practice Lets do a focus group
  52. 52. Interview Practice ?
  53. 53. GO FORTH & LTYFU! Q & A!
  • theheurist

    Jun. 11, 2013

Views

Total views

1,463

On Slideshare

0

From embeds

0

Number of embeds

5

Actions

Downloads

8

Shares

0

Comments

0

Likes

1

×