Adam Babajee-Pycroft's presentation at UX Bristol July 2016. Validating Ideas Fast with Remote User Research Involving users in the design process is critical to UX but how many times have you been told the following? We don’t have time We can’t afford it We’ll do it after the site is built As our industry moves inexorably towards Lean/Agile methodologies, there’s no reason that user research should suffer. The solution? Remote user research. In this workshop Adam will share what he’s learnt having executed hundreds of hours of remote research on a variety of projects including: How to convince people to let you try it in the first place How to write tasks for self-moderated testing? Finding the right participants Common mistakes and issues The advantages and disadvantages of certain tools You’ll leave with a knowledge of techniques, tools and hopefully the enthusiasm to try it for yourself.

1. Validating Ideas
Fast with Remote
User Research

2. I’m Adam Babajee-Pycroft
UX Director at Natural Interaction
@adam_ux on Twitter
2 : Hello
Hello
@adam_ux #UXBristol

3. 1. Talk - 15 Minutes
○ Introduction
○ Tools
○ How to do it
2. Exercise - 25 minutes
3. Showcase - 10 minutes
4. Conclusion - 5 mins
3 : Overview > The plan
The plan
@adam_ux #UXBristol

4. Introduction

5. Remote user research can include
anything from self-moderated or
remotely moderated usability testing,
to feature validation, to card sorts, to
depth interviews to tree-tests.
5 : Overview > Definition
Definition
@adam_ux #UXBristol

6. Simply put, it means conducting the
same research we always have, but
doing it remotely.
6 : Overview > Definition
Definition
@adam_ux #UXBristol

7. 7 : Overview > I used to be a skeptic
I used to be a skeptic
@adam_ux #UXBristol

8. Speed
● Quicker to arrange
● Multiple tests can happen simultaneously
● You don’t have to sit there during the tests
8 : Overview > Advantages
Advantages
@adam_ux #UXBristol

9. Cost
● Self-moderated sessions don’t consume your time
● Participants are cheaper to recruit as they don’t
have to travel
9 : Overview > Advantages
Advantages
@adam_ux #UXBristol

10. Participant location
● Removes the temptation to intervene
● You can recruit participants from anywhere
10 : Overview > Advantages
Advantages
@adam_ux #UXBristol

11. See users in their natural habitat
11 : Overview > Advantages
Advantages
@adam_ux #UXBristol

12. Harder to build relationships
● This specifically applies to self moderated
Not impossible:
● Remotely moderated or Skype calls, Hangouts
etc could help bridge the gap
12 : Overview > Disadvantages
Disadvantages
@adam_ux #UXBristol

13. You can’t see their wider context
13 : Overview > Disadvantages
Disadvantages
@adam_ux #UXBristol

14. Connection speeds vary
14 : Overview > Disadvantages
Disadvantages
@adam_ux #UXBristol

15. Choosing tools

16. 16 : Choosing Tools : Surveys
Surveys
@adam_ux #UXBristol

17. 17 : Choosing Tools : Interviews
Interview Tools
@adam_ux #UXBristol

18. 18 : Choosing Tools : Quantitative Research
Quantitative Tools
@adam_ux #UXBristol
● Card Sorting
● Tree Testing
● Click Testing
● Card Sorting
● Tree Testing
● More...
● Click tests
● Annotation
● Memory
● Mood

19. 19 : Choosing Tools : Qualitative Tools@adam_ux #UXBristol
Qualitative Tools
Plenty of choice…
(Giant table imminent)

20. 20 : Choosing Tools : Qualitative Tools@adam_ux #UXBristol
Speed Recruitment Options What do users install?
2 hours
(simple screener)
Panel (Few frequency
limits)
Self Moderated macOS / Windows App
Native iOS / Android App
2 days desktop
5 days mobile
(simple screener)
Panel
(1 test / user / month)
Bring your own
Self Moderated macOS / Windows App
Native iOS / Android App
1-2 hours
(simple screener)
5 days
(complex screener)
Social Media Bespoke
Bring your own
Self Moderated
Remotely
Moderated
Chrome Plugin
Native iOS / Android App
UserZoom Same day
(simple screener)
Multiple panels
(No limits)
Bring your own
Self Moderated macOS / Windows App
Native iOS / Android App
Loop11 1-2 hours (simple
screener)
Multiple panels
(No limits)
Bring your own
Self Moderated
Quantitative
macOS / Windows App
Native iOS / Android App

21. How to do it

22. First decide what you’re testing
22 : How to do it > Writing Tasks
Writing tasks
@adam_ux #UXBristol

23. Be succinct
23 : How to do it > Writing Tasks
Writing tasks
@adam_ux #UXBristol

24. Avoid ambiguity,
you won’t be there to explain it
24 : How to do it > Writing Tasks
Writing tasks
@adam_ux #UXBristol

25. Don’t use leading questions
25 : How to do it > Writing Tasks
Writing tasks
@adam_ux #UXBristol

26. Establish a rhythm
26 : How to do it > Writing Tasks
Writing tasks
@adam_ux #UXBristol

27. Poor screeners
27 : How to do it > Common mistakes
Common mistakes
@adam_ux #UXBristol

28. Participants too familiar with the process
28 : How to do it > Common mistakes
Common mistakes
@adam_ux #UXBristol

29. Not testing the test yourself
29 : How to do it > Common mistakes
Common mistakes
@adam_ux #UXBristol

30. Screeners: Missing the “Country” filter
30 : How to do it > Common mistakes
Common mistakes
@adam_ux #UXBristol

31. Exercise 1
Prepare Tasks

32. Together, we’re going to write some
self-moderated usability tasks.
● On your table, there is an envelope containing
the website you’ll be testing.
● As a table, please choose desktop or mobile.
32 : Exercise 1 > Prepare Tasks
Exercise 1
Prepare tasks
@adam_ux #UXBristol

33. Individually write tasks on the post-it
notes provided for your assigned
website.
5 minutes
This first task is to be done individually, you’ll get to
work with the rest of your table shortly.
Plan on the basis that your test will take up to 5 mins.
33 : Exercise 1 > Prepare Tasks
Exercise 1
Prepare tasks
@adam_ux #UXBristol
Please show us how
you would contact
this company.

34. Discuss your tasks with the rest of the
table, prepare a unified task list
combining your best ideas.
5 minutes
Plan on the basis that your test will take up to 5 mins.
34 : Exercise 1 > Prepare Tasks
Exercise 1
Prepare tasks
@adam_ux #UXBristol
Please show us how
you would contact
this company.

35. Exercise 2
Test your tasks

36. Swap one person between each table.
Observe your guest trying your written
tasks, without intervening.
5 minutes
36 : Exercise 2 > Test Your Tasks
Exercise 2
Test your tasks
@adam_ux #UXBristol

37. Discuss how your guest responded to
your tasks.
Refine your tasks ready for the next
participant.
5 minutes
37 : Exercise 2 > Test Your Tasks
Exercise 2
Test your tasks
@adam_ux #UXBristol

38. Replace your participant with another.
Observe your guest trying your written
tasks, without intervening.
5 minutes
38 : Exercise 2 > Test Your Tasks
Exercise 2
Test your tasks
@adam_ux #UXBristol

39. Showcase

40. We’ll now try one of the sets of tasks out
with a willing volunteer connected to
the projector.
5 minutes
40 : Showcase
Showcase
@adam_ux #UXBristol

41. We’ll now try one of the sets of tasks out
with another willing volunteer
connected to the projector.
5 minutes
41 : Showcase
Showcase
@adam_ux #UXBristol

42. Conclusion

43. ● Use free trials
● Demonstrate the benefit
● Seek forgiveness not permission
43 : Conclusion > Getting permission
Getting permission
@adam_ux #UXBristol

44. Too much to fit into this session
● Hold yourself to the same standards of proof
● Show your working out (including video evidence)
● Remember your research objectives
44 : Conclusion > Analysis
Conducting analysis
@adam_ux #UXBristol

45. 1. Try it at least once, overcome preconceptions
2. Use the same budgets to do more rounds of
research, don’t use this as an excuse to spend less
3. Ensure instructions are easily understood
4. Use the right tool for the job
5. Be rigorous in your analysis
45 : Conclusion > Takeaways
Takeaways
@adam_ux #UXBristol

46. Any questions?

47. Adam Babajee-Pycroft
adam@naturalinteraction.co.uk
@adam_ux
Thank you for your time