Slides from half-day Collaborative Research workshop presented in Malmö on August 21, 2014.

1. Collaborative
Research
with Erika Hall (@mulegirl)
The Conference | Malmö

2. Hello!

4. I have a
question...

5. Do you enjoy being
right?

6. You are correct!

7. YESSS!

8. p0wned!

9. No.

10. ?

11. ? > !

12. Dogma

13. Flickr/Chris Voll

14. Ego!

16. Why?

21. People!

22. Collaboration!

23. 9–10:30
AgeTexnt da
10:30–10:40
10:40–12:00
Introduction.
The research process
Questions and activities
Understanding your organization
Break
User research
Analysis
Models and reports
Finale

24. Barriers

29. Overcoming Objections

30. Laziness
Fear
Lunch
Following
Losing Control
Sharing

31. Shared
Understanding

32. We don’t have
the time.

33. We don’t have
the money.

34. We don’t have the
expertise.

35. We’re already A/B
testing

36. Everyone wants better products,
faster.

37. No one wants to read a report.

41. What is your
experience?

42. Research +
Collaboration

43. A design project is
a series of decisions.

44. Data doesn’t
change
minds.

45. What
is
What ought
to be

47. Design-Led
Research-Led
Expert
Mindset
participatory mindset
Participatory
Mindset
Design-led
with
expert mindset
Design-led
with
Users seen as subjects Users seen as partners
Research-led
with expert mindset
Research-led
with participatory
mindset
Dubberly Design Office

48. Goal Driven
Increase chance of success
Reduce risk
Skeptical Mindset
Willing to question the value of any approach

49. Team + Goal + Reality = Good

51. One Simple
Process

52. Form
Questions
Analyze
Data
Gather
Data

53. Form
Questions
Analyze
Data
Think
Critically

54. Form
Questions
Analyze
Data
Observe

55. Form
Questions
Analyze
Data
Interview

56. Form
Questions
Analyze
Data
Read

57. Form
Questions
Analyze
Data
Think
Observe
Interview
Experiment
Read

58. Personal
View
Personal
View
Personal
View

59. Shared
Reality

60. Research
is a Craft

62. Form
Questions
Analyze
Data
Gather
Data

63. Questions determine results.

64. Questions give research meaning.

65. Research high-priority questions.

66. Good Questions
Specific
Actionable
Practical

67. A Bad Question
“What do people
think about pets?”

68. A Better Question
“How do single
urban adults
choose and
acquire a pet?”

69. A Bad Question
“What do people
do around here all
day?”

70. A Better Question
“How do editors
and designers work
together?”

71. The Best Question
The unknown with
the most risk.

73. Bias

74. Bias: Something that causes an
influence or prejudice

75. Confirmation Bias:
You selectively weight the
information that confirms
what you already believe.

76. Sampling Bias:
Your sample of research
subjects isn’t sufficiently
representative.

77. Interviewer Bias:
You insert your opinion into
interviews.

78. Social Desirability Bias
People don’t say the true
things that they worry will
make them look bad.

79. Ease
Related
Clear Display
Primed Idea
Good Mood
Feels Familiar
Feels True
Feels Good
Feels Effortless
Daniel Kahneman

80. Feeling confident?
It’s not a good sign.

81. You might have a bad case of
Dunning-Kruger.

82. Critical
Thinking

83. Critical Thinking
Disciplined
Self-correcting
Clear
Logical

84. Uncritical Thinking
“I hate yellow, so a
yellow website
won’t succeed.”

85. Critical Thinking
“I hate yellow, but
based on the
evidence, it might
work for our
audience.”

86. Critical Thinking
“I don’t know.”

87. Activities!

88. Form
Questions
Analyze
Data
Gather
Data

91. Users
Interviews Interviews
Descriptive
Descriptive
Questions
About
Evaluative
Org Product
Evaluative
Competition
Usability
Testing
A/B
Testing
Contextual
Inquiry
Literature
Review
SWOT
Analysis
Analytic
Brand
Audit
Usability
Testing
Competitive
Analysis
Heuristic
Analysis
Evaluative
Analytic
Generative

92. Research
Activity
Topic
Purpose
Time
Money

93. Phone Interviews
What do we
need to know
about?
What kind of
decision will it
inform?
How long do we
have?
What is our
budget?
In-Person Interviews
Contextual Inquiry
Usability Testing
Competitive
Analysis

94. Why not just
make a
prototype?

96. If we only test bottle openers, we
may never realize customers prefer
screw-top bottles.
– Victor Lombardi, Why We Fail
20

97. Topics

98. Organizational
Research

100. Stakeholders

101. Stakeholders

102. Executives

103. Sales People

104. Customer Service

105. Editors

106. Production Team

107. Organizational research
helps you with:
Requirements
Politics
Workflow
Capabilities
Goodwill

108. Requirements
What are the top
business priorities
for this project/
product?

109. Politics
What does
success mean to
the individual
stakeholders?

110. Workflow
Do we have to
change how
people are
working together
to be successful?

111. Workflow
How do we have
to change how
people are
working together
to be successful?

112. Workflow
How can we
possibly change
how people are
working together?

113. Capabilities
What are the
strengths and
weaknesses of our
team?

114. Capabilities
Where is the
internal expertise?

115. Goodwill
How can this
project make your
job easier (or
harder)?

118. Get them alone

119. Basic Stakeholder Questions
What is your title? How long have you been in this role?
What are your essential duties and responsibilities?
What does a typical day look like?
Who are the people you work most closely with? How is that going?
What does success mean from your perspective, what will have changed
for the better once this project is complete?
Do you have any concerns about this project?
What do you think the greatest challenges to success are? Internal and
external?

120. For each stakeholder, note the following:
What’s their general attitude toward this project?
What’s the goal as they describe it?
To what extent are this person’s incentives aligned with the project’s
success?
How much and what type of influence do they have?
Who else do they communicate with on a regular basis?
To what extent does this stakeholder need to participate
throughout the project, and in which role?
Is what you heard in harmony or in conflict with what you’ve
heard from others throughout the organization?

121. Stakeholder power moves
“Why are you asking me this?”
“I don’t understand that question. It doesn’t make any sense.”
“I don’t feel comfortable talking to you about that.”
“No one pays attention to anything I have to say, so I don’t know why I
should bother talking to you.”
“How much more time is this going to take?”

122. Practice!

123. 10 minutes practice. Find a partner. Take turns.
What is your title? How long have you had this job?
What are your essential duties and responsibilities?
What is a typical day like?
Who are the people you work most closely with? How is that going?
What do you think the greatest challenges to your success are?
Internal and external?

124. Empathy

125. Break!

126. To Review
Team + Goal + Shared Reality = Good
Research is a simple process you can apply to however you work. You
shouldn’t be dogmatic.
Even though this sounds obvious, some people will resist this because
questions can feel threatening.
Facts will not change the minds of people who are threatened.
You need to appeal to what you know is important to them, and fit
your facts into their story.
So, understanding what is important to your stakeholders is necessary
for design and research to succeed.

127. User
Research

128. Photo: Flickr/theloushe
Ethnography

129. How to do bad user research:
Ask people what they want.

130. How to do bad user research:
Ask people what they like.

131. Never ask users what
they want or like.

132. The Four Ds of Design
Ethnography

133. Deep Dive
Daily Life
Data Analysis
Drama

134. “...true ethnography reveals not just
what people say they do, but what they
actually do.”
–PARC

136. Photo: Flickr/lintmachine

137. The Art of
The Interview

138. Interviewing is not talking.

139. Interviewing is listening.

140. Good Interviewers:
Know Your Question
Warm Up
Shut Up

141. Interview Structure:
Introduction
Body
Conclusion

142. Introduction:
Smile
Express gratitude
Describe the process
Ask to record
Warm up questions

143. Body:
Ask open-ended questions
Probe for more
Allow silence
Use questions as checklist

144. Conclusion:
Transition to wrap-up
Ask if there is anything else
Thank for time

145. You are the host
You are the student

146. Out of your comfort zone,
and into theirs.

147. Interview Checklist
Create a welcoming atmosphere to make participants feel at ease.
Always listen more than you speak.
Take responsibility to accurately convey the thoughts and
behaviors of the people you are studying.
Start each interview with a general description of the goal,
but be careful of focusing responses too narrowly.
Avoid leading questions and closed yes/no questions. Ask
follow-up questions.
Prepare an outline of your interview questions in advance, but don’t be
afraid to stray from it.
Also note the exact phrases and vocabulary that participants use.

148. Look for
Goals
Priorities
Tasks
Motivators
Barriers
Habits
Relationships
Tools
Environment

149. Roles
Interviewer
Notetaker
Observer

150. Practice!

151. Interview Scenario
You want to develop a new service
to help people give gifts.
The goal of the research is to
identify unmet needs people might
have with regard to giving gifts.

152. Interview Practice
Break into groups of 3-4 people
1 interviewee, 1 interviewer , 1 notetaker, 1
observer (optional),
Switch in 15 minutes
2 rounds

153. Look for
Goals
Priorities
Tasks
Motivators
Barriers
Habits
Relationships
Tools
Environment

155. Switch!

156. Get
ready to
finish.

157. Time’s
Up!

158. How did that go?

159. How about a
focus group?

161. “Even when the subjects are well
selected, focus groups are supposed to
be merely the source of ideas that need
to be researched.”
14
–Robert K. Merton, Sociologist,
invented focus groups

162. Everybody Lies

163. Competitive
Research

164. How else might your
target customer solve
the same problem?

165. Competitive Review
What do they say they offer?
Who is their customer? How is this the same or different from your
target audience or users?
What are the key differentiators—the factors that make them uniquely
valuable to their target market, if any?
How do the user needs or wants they’re serving overlap or differ from
those that you’re serving or desire to serve?
What do you notice that they’re doing particularly well or badly?
Based on this assessment, where do you see emerging or established
conventions in how they do things, opportunities to offer something
clearly superior, or good practices you’ll need to adopt or take into
consideration to compete with them?

166. Your target customers
have to love you
more than they
hate change.

167. (Usability)
Testing

168. A good research activity:
•Answers a key question
•Addresses identified
assumptions
•Informs specific decisions
•Involves your team
•Fits your level of expertise
•Fits your schedule and budget

169. •Fundamentally research is a simple process
•There are many activities and definitions
•No pressure!
•Select the methods that inform decisions
•Begin by understanding your organization
•Never ask what people like
•People are lazy, forgetful creatures of habit
•Keep each other honest
•Practice and learn

170. Research and Collaboration
Working together across disciplines and making decisions based on
evidence shouldn’t be hard, but they can be.
Done right, research and working collaboratively reinforce each other
through a shared understanding of reality.
Start with your goal in mind, not with any process or buzzword.
Asking questions and cutting across traditional roles can both be
threatening to the established order.
Commit to clear communication and critical thinking.
Research questions follow from goals, assumptions, and risk.
Always have a framework and a plan.

171. Creating Meaning
From Data

172. 1. Compile data
2. Analyze
3. Identify Insights
4. Create Model

174. Basic Analysis
Closely review the notes.
Look for interesting behaviors, emotions, actions, and verbatim quotes.
Write what you observed on a sticky note (coded to the
source, the actual user, so you can trace it back).
Group the notes.
Watch the patterns emerge.
Rearrange the notes as you continue to assess the patterns.

175. Observation
Observation
Observation
Observation
Observation
Observation

176. Collaborates
on purchases
Uses several
devices
Needs
affirmation
Observation
Observation
Observation
Observation
Observation
Observation
Observation

177. Ground rules
Acknowledge that the goal of this exercise is to better understand the
context and needs of the user. Focus solely on that goal.
Respect the structure of the session. Refrain from identifying larger
patterns before you’ve gone through the data.
Clearly differentiate observations from interpretations (what happened
versus what it means).
No specific solutions until after you’ve gone through insights and
principles. Solutions come next.

178. Practice!

180. 15 minutes analysis.
Break into groups of 6-8 people
Each group work together to fill out one diagram with the strongest
patterns.
Negotiate and advocate for your perspective.

181. Look for
Goals
Priorities
Tasks
Motivators
Barriers
Habits
Relationships
Tools
Environment

183. 15 minutes analysis.
Break into groups of 3-4 people
Each group work together to fill out one diagram with the strongest
patterns.
Negotiate and advocate for your perspective.

184. Get
ready to
finish.

185. Time’s
Up!

186. How did that go?

187. Models

194. Reporting

196. You are collaborating with
your future selves.

197. Study Title Research Report
Date Completed
Research Goal
Related Decisions
Activities
Key Insights
Supporting Observations
Recommended Actions
Questions for Further Study

198. A useful report supports
Clear goals
Shared values
Access to information
Clear decision-making

199. You decide if it’s important for the report to be
Informing?
Inspiring?
Focusing?
Remembering?
Recording?
Deciding?

200. Finale

201. In summary
Research creates a shared understanding of reality.
Asking questions is uncomfortable. Embrace that feeling.
A truly collaborative approach and environment is necessary for research to be
effective, and it also makes it more fun.
Clear goals and good questions are required.
Choose only the research activities that answer real questions and inform your top
priority design and development decisions.
Practice! Observe and listen every day.
Document! Report! Share! It’s easy to lose what you learn.

202. Any questions?

203. Additional sources:
Designing Together by Dan A. Brown
http://www.designingtogetherbook.com/
LeanUX by Jeff Gothelf
http://www.leanuxbook.com/
Remote Research by Nate Bolt & Tony Tony Tulathimutte
http://rosenfeldmedia.com/books/remote-research/
Interviewing Users by Steve Portigal
http://rosenfeldmedia.com/books/interviewing-users/
Google Ventures Library | Design
http://www.gv.com/library/design/
Pacific Standard Magazine
http://www.psmag.com/
Helsinki Design Lab (closed, but excellent publications still available)
http://www.helsinkidesignlab.org/pages/publications

204. Brief books for people who make websites No. 9
Erika Hall
JUST ENOUGH
RESEARCH
You might
enjoy the
book.
www.abookapart.com
For 15% off, use code: HALL-THE14
(Good through November 2014)