The long, textual written report is dead, isn’t it? So how do you deliver your findings to your clients? Is it PowerPoint? An e-mail? A spreadsheet? Post-it notes? And what do you include? Positive findings? Screenshots with callouts? Just issues? Or recommendations as well? Are they prioritized? If you ask our panelists, some of us have developed templates that we use and modify for each research activity, and others change the deliverable based on the activity and client. Jen McGinn, Principal Usability Engineer, Oracle Eva Kaniasty, Founding Principal, RedPill UX Dharmesh Mistry, Usability Specialist, Acquia Kyle Soucy, Founding Principal, Usable Interface Carolyn Snyder, Founding Principal, Snyder Consulting

1. New Hampshire Usability Professional’s Association www.nhupa.org
January 2012 Meeting
Delivering Results: How Do You Report User
Research Findings?
Jen McGinn
Eva Kaniasty
Dharmesh Mistry
Kyle Soucy
Carolyn Snyder

2. New Hampshire Usability Professional’s Association www.nhupa.org
Introduction
About NH UPA
We want to make New Hampshire a good place for user experience
professionals to work and live
Join Us: www.nhupa.org or @nhupa or NH UPA on Facebook
Help Us: Volunteer or give us suggestions
Announcements
• NH UPA Board of Directors:
Debra Arneson, Megan Fields, Rick La Vache, Bob Thomas
• Thanks to Mad*Pow
• Jobs

3. New Hampshire Usability Professional’s Association www.nhupa.org
Upcoming NH UPA Meetings
Tentative Date: Wednesday, February 15
Tom Clancy, “Using Development Frameworks to Rapidly Prototype
Applications”
• Have an idea for an upcoming meeting?
• Want to practice an upcoming presentation or panel you’re giving?
• Want to give a 10-minute talk?
Contact Us: robertl.thomas@libertymutual.com or debra.arneson@tds.net

4. New Hampshire Usability Professional’s Association
Delivering Results: How Do You Report User
Research Findings?
The long, textual written report is dead, isn’t it? So how do you deliver
your findings to your clients? Is it PowerPoint? An e-mail? A spreadsheet?
Post-it notes? And what do you include? Positive findings? Screenshots with
callouts? Just issues? Or recommendations as well? Are they prioritized?

5. New Hampshire Usability Professional’s Association
Panelists
If you ask our panelists, some of us have developed templates that we use
and modify for each research activity, and others change the deliverable
based on the activity and client. Each panelist will spend 5 minutes
showing you their typical deliverables, and then we’ll open the floor for
audience Q&A.
•Jen McGinn, Principal Usability Engineer, Oracle
•Eva Kaniasty, Founding Principal, RedPill UX
•Dharmesh Mistry, Usability Specialist, Acquia
•Kyle Soucy, Founding Principal, Usable Interface
•Carolyn Snyder, Founding Principal, Snyder Consulting

6. New Hampshire Usability Professional’s Association
Jen McGinn
Principal Usability Engineer, Oracle

7. New Hampshire Usability Professional’s Association
Overview
• I present my research results via slides in 60-minute
meeting (generally remote, via web conference)
• I’m going to spend 3-5 minutes walking you
through the structure of one of my PowerPoint
presentations
• Then I’ll summarize the take-aways 

8. New Hampshire Usability Professional’s Association
[Product ][Method]
My name, title, and date
2/6/2012

9. New Hampshire Usability Professional’s Association
Executive Summary
• In [When?], the [what product?] was tested by
[number and type of participants] in [method type]
to evaluate the ease of use of several features
including [features or use cases].
• High level findings included [usually a total of 3 to 4
bullets]:
• [ 1 - 2 biggest positive findings]
• [ 1 - 2 biggest positive findings]
• [ 2 or 3 biggest usability issues]
• [ 2 or 3 biggest usability issues]
• This presentation covers all of the findings and
subsequent recommendations.
2/6/2012

10. New Hampshire Usability Professional’s Association
Agenda
• Goals
• Tasks
• Participants
• Findings
• Recommendations
• Next Steps
2/6/2012

11. New Hampshire Usability Professional’s Association
Goals
Evaluate the usability of the following features of the
U-Haul.com website:
– Are users confused about how to price a rental? A storage unit?
– How do users react to the insurance options? Do they
understand the coverage?
– How do users feel about the presentation of items for purchase
or for rent?
– How effective is the shopping cart content? Are users confused
by when they need to pay for items?
– Do users value the star ratings? U-Haul brand?
– How do users feel about the targeted FAQ and search result
pages?
– Does our online documentation help prevent calls to the service
center? Can they determine how to reach out to the U-Haul
vendor nearest them?
2/6/2012

12. New Hampshire Usability Professional’s Association
Tasks
1. Get the price of a 1-way move across country
2. Find a specific piece of information in the FAQ
3. Determine the size and cost of a storage unit needed to
hold specific items
4. Find the phone number of a U-Haul location
5. Book the truck (and insurance), adding rental items and
purchased items
6. Determine insurance coverage
7. Find the U-Haul location nearest you
2/6/2012

13. New Hampshire Usability Professional’s Association
Participants
Participant Gender Age Occupation Web-
ID savvy
U1 Male 24 Missionary Average
U2 Male 52 Small business manager Average
U3 Female 62 62 Retired. Formerly Average
television news producer,
then licensed paralegal.
U4 Female 36 Housewife Average
U5 Male 31 Sales and marketing Average
2/6/2012

14. New Hampshire Usability Professional’s Association
Findings
2/6/2012

15. New Hampshire Usability Professional’s Association
Choosing a Truck
2 participants had this issue and did
„x‟ to work around it
Another issue
One participant
suggested this fix
Jen McGinn
5/25/2011

16. New Hampshire Usability Professional’s Association
Goals and Questions Re-visited
• [All the same as before] Are users confused about how
to price a rental? A storage unit?
• How do users react to the insurance options? Do they
understand the coverage?
• How do users feel about the presentation of items for
purchase or for rent?
• How effective is the shopping cart content? Are users
confused by when they need to pay for items?
• Do users value the star ratings? U-Haul brand?
• How do users feel about the targeted FAQ and search
result pages?
• Does our online documentation help prevent calls to
the service center? Can they determine how to reach
out to the U-Haul vendor nearest them?
2/6/2012

17. New Hampshire Usability Professional’s Association
Positive Findings [these always come first]
• All participants easily found the links to the FAQs and had no trouble
finding the answer to the license question under FAQs
• All participants made use of the maps when comparing options.
• All participants did scroll down to compare prices, locations and
reviews
• 4 participants valued the presence of the [higher] star ratings
• 2 participants valued U-Haul location more than the off-brand
vendors
• 2 participants were pleased that the truck rental page "retained her
information" -- the addresses and dates
• 2 participants appreciated the visuals of the items inside the storage
units and the graphic of the person shown in the small unit icon
• 2 participants easily added the dolly, blankets and boxes during the
truck rental task flow
2/6/2012

18. New Hampshire Usability Professional’s Association
Recommendations
Priority Description Recommendation Location
Participants don't understand what Re-format coverage and
High the purchased insurance actually exclusions into bulleted Damage coverage
covers lists; Don't use legal jargon
Participants have a very hard time
High Provide more user Self Storage location
estimating the storage unit size that
assistance details page
would meet their needs
Put the purchased items
Up-sell process for items to rent or into another page in the Additional rental items,
Medium
purchase is confusing flow, and make it clearer Shopping cart
that users can opt out.
Participants are concerned that the Add a link to display the
Select your preferred
Medium site is incorrectly calculating the map, so they can check it in
pickup location
mileage and therefore overcharging place
Participants were not sure what Display the distance "from"
Select your preferred
location the giant thumbtack/pin was the specified location, like
Low pickup location,
(address or zip code) or how far the Self-storage results
Location
away the locations were page
2/6/2012

19. New Hampshire Usability Professional’s Association
Next Steps
• Work with [which stakeholders or teams] to prioritize
changes
• Work with [stakeholders or teams] to design
alternatives
• Validate that the new designs address the issues
with users
2/6/2012

20. New Hampshire Usability Professional’s Association
Summary
• Tell them what you’re going to tell them
– Executive summary
– Agenda
– Goals/Questions
• Tell them
– Tasks & participants (sometimes methodology)
– Animated slides for progressive disclosure
– Screen shots annotated with findings
• Tell them what you told them
– Review goals of the research and the questions it was
intended to answer
– Positive findings (go slowly here)
– Prioritized opportunities for improvement

21. New Hampshire Usability Professional’s Association
Eva Kaniasty
Founding Principal, RedPill UX

22. New Hampshire Usability Professional’s Association
Report Formats
PPT: visually engaging but real-estate constrained
(and will force you to be brief). Formatting can be
time-consuming.
MS Word/Narrative: more room for context; quick, but
can appear dry and boring.
3rd Option: No report.

23. New Hampshire Usability Professional’s Association
Deciding Factors
• Time/Budget
• (Mode of) Presentation of Results
• Company Culture / Industry
• Stakeholder Involvement
• Deliverable Shelf Life

24. New Hampshire Usability Professional’s Association
Reporting Findings (1)

25. New Hampshire Usability Professional’s Association
Reporting Findings (2)

26. New Hampshire Usability Professional’s Association
Dharmesh Mistry
Usability Specialist, Acquia
Content Management System Products built on Drupal
Open Source Software Open Source/ Proprietary
Community Start-up

27. New Hampshire Usability Professional’s Association
Stakeholder
behavior
Supporting information Main Report Detailed Information
Transparency
http://drupal.org/node/1289476 http://drupal.org/node/1399056 http://drupal.org/node/1399258

28. New Hampshire Usability Professional’s Association
Recommendations
Tracking
Stakeholders
comments
Main
Report
http://www.drupalusability.org/ http://drupal.org/node/1175694

29. New Hampshire Usability Professional’s Association
Credibility
Turn around
time

30. New Hampshire Usability Professional’s Association
Not Cool?
Cool

31. New Hampshire Usability Professional’s Association
Stakeholders Thousands of Stakeholders 3-5
Turn around time Weeks-Months Hours-day-week
Tracking High, Extensive Moderate-Very High
Twitter, Conferences, Front Email, Conference meetings
Presenting
page on Drupal.org
Provide
No, never! Sometimes
recommendations

32. New Hampshire Usability Professional’s Association
Kyle Soucy
Founding Principal, Usable Interface

33. New Hampshire Usability Professional’s Association
Usable Interface
Formal Usability Testing/Research Report

34. New Hampshire Usability Professional’s Association
Executive Summary

35. New Hampshire Usability Professional’s Association
Findings: Severity Ratings

36. New Hampshire Usability Professional’s Association
Findings
Major Usability
Problem

37. New Hampshire Usability Professional’s Association
Findings: Recommendations

38. New Hampshire Usability Professional’s Association
Presenting Findings…
Finding: All of the participants were disoriented after clicking the “View in 3D”
button on the quote.
Recommendations:
• The navigation tabs should not disappear in the 3D view.
• If the page needs to be reloaded than the orientation should remain the same
and not take the user back to the top of the screen.
• If possible, the “View in 3D” button should download the ActiveX plug-in.

39. New Hampshire Usability Professional’s Association
Highlight Video

40. New Hampshire Usability Professional’s Association
Observer Debrief Notes

41. New Hampshire Usability Professional’s Association
Carolyn Snyder
Founding Principal, Snyder Consulting
• There is no one “best” format
• Do what works for the client, culture, circumstances
• Steal good ideas, drop losers

42. New Hampshire Usability Professional’s Association
Text Report: “I’m not dead yet!”
Finding
Severity rating
Explanation of
issue
Supporting
observations from
notes
Recommendations

43. New Hampshire Usability Professional’s Association
PowerPoint with “report” in Notes Field

44. New Hampshire Usability Professional’s Association
PowerPoint, Screen Shots with Callouts
Interest in these
Interest in these
links Ambiguous
Some people saw this as a bar
graph, but others did not. One
Important sentence person read only [first 4 words]
buried in paragraph
Why just show Most people read this text;
[scenario]? Some everyone drilled into [noun]
misinterpreted it as
People liked
“worst case scenario.”
suggestions, but
wanted concrete,
prioritized advice.
Order can imply
priority.
Amount isn‟t
explicit. The user
Can‟t explore must do the
[action]. People math.
knew it was
important.
People seemed to
People understood the understand the
purpose stacked bar graphs,
Not clear why it showed [2
variations of graph] 44

45. New Hampshire Usability Professional’s Association
Sometimes the best report is…

46. New Hampshire Usability Professional’s Association
Questions
1. Do you change your delivery of usability results depending on your role as a
internal/external consultant or as a company employee?
2. How important are positive vs. negative findings?
3. How have your reports changed over the years? Is there anything you do
differently than when you first started writing them?
4. How do you categorize the findings in your reports? For example, do you
categorize them by the page/screen, by the step in a certain process (e.g.
checkout process), or by the task?
5. Lean UX is a trending topic. Have you had experience with Lean UX or Agile
methods, and had to change the way you conduct research and deliver results?
6. What guidelines do you follow when writing recommendations or proposed
solutions to problems?
7. Do you decide ahead of time how long a report should be and make an effort to
keep it that length? If so, what dictates the length?
8. If you think a report is too long and needs to be trimmed down, how do you
decide what to cut out?
9. What part of a report is the hardest for you to write?