On one hand, Net Promoter Scores (NPS) are great for gauging how customers feel – but they don’t provide any specific insights. On the other hand, unstructured comments from support tickets and similar sources are rich with detail – but with 15,000 comments per week coming in, it was impossible to categorize and understand what that information actually meant. We needed a better way. Join the head of our Voice of the Customer team, Sean Cramer, to learn about the innovative system Atlassian has built: RUF (Reliability, Usability, Functionality). RUF categorizes, measures, and prioritizes issues gleaned from not just Net Promoter feedback, but from all our sources of customer feedback. Come learn how your organization can take Atlassian’s learnings, and use them to build a similar system for your organization.

1. SEAN CRAMER • HEAD OF VOICE OF THE CUSTOMER • ATLASSIAN •
@COSMOCRAMER
RUF’ing it out with Customer
Feedback: Knowing the “Why”

2. • Big cool statistic
• 2,56
9
• Add-Ons in Marketplace

3. • Big cool statistic
• 2,56
9
• Add-Ons in Marketplace

4. • Big cool statistic
• 2,56
9
• Add-Ons in Marketplace

5. • Big cool statistic
• 2,56
9
• Add-Ons in Marketplace

6. THE SOLUTION
THE FEEDBACK
THE LEARNINGS
Agenda
DIY

7. THE SOLUTION
THE FEEDBACK
THE LEARNINGS
DIY

8. 1 2 3 4 5 6 7 8 9 100
DETRACTORS PASSIVES PROMOTERS
% Promoters - % Detractors Net Promoter Score (NPS)

10. NPS Support JAC Social Media

11. NPS Support JAC Social Media

12. NPS Support JAC Social Media

13. 15,000… a week

14. • Big cool statistic
• 2,56
9
• Add-Ons in Marketplace

15. THE SOLUTION
THE FEEDBACK
THE LEARNINGS
DIY

16. “There is a great
difference between
knowing and
understanding: you
can know a lot about
something and not
really understand it” -
Charles F. Kettering

17. Reliability
=

18. • Performance
• Up-time
• Quality
• Security

19. FunctionalityReliability
=

20. • Missing
Features
• Would-like
• Might-help

21. UsabilityFunctionalityReliability
=

22. • Complexity
• Ease-of-use
• Discoverability
of features

23. UsabilityFunctionality Reliability
F U R

24. Usability FunctionalityReliability
FUR

25. FUR

26. THE SOLUTION
THE FEEDBACK
THE LEARNINGS
DIY

28. 40,000

29. • Big cool statistic
• 2,56
9
• Add-Ons in Marketplace

30. RUF’ing it out…yo!!
0 %
17,5 %
35 %
52,5 %
70 %
14 %
63 %
23 %
Reliability Usability Functionality

31. Usability - 63%
Reliability - 23%
Functionality - 14%
Drill-down
•Performance - 20%
•Bugs - 3%
•Editing - 14%
•Complexity - 22%
•Content- 32%
•Navigation - 9%

33. Customer Insight Opportunity

34. • Big cool statistic
• 2,56
9
• Add-Ons in Marketplace

35. • Big cool statistic
• 2,56
9
• Add-Ons in Marketplace

36. • Big cool statistic
• 2,56
9
• Add-Ons in Marketplace

37. Lots of features and easy to use. Main
detractor was performance though it seems
fast today, maybe a recent upgrade? If the
performance stays fast I would be more
likely (to recommend it).
DEVELOPER, CONFLUENCE
“
”

38. Page loading speed improved very
much!!
PRODUCT MANAGER, CONFLUENCE
“ ”

40. • Big cool statistic
• 2,56
9
• Add-Ons in Marketplace

41. THE SOLUTION
THE FEEDBACK
THE LEARNINGS
DIY

42. • Big cool statistic
• 2,56
9
• Add-Ons in Marketplace

43. Categorize content
Understand your feedback funnel
along with the traffic and frequency
Measure the feedback
Who is it from? What area is it
effecting? How many said it?
Find the sources
Group into RUF and sub themes that
help give clarity into insights
Categorize and
Measure

44. Establish baselines
Too many comments in one area can
cause problems
Get commitment
Insight without action is useless.
Make sure your company is willing to
listen to feedback. If not, call me!!
Determine redlines
You can’t determine if things aren’t
changing if you don’t know where you
are. (NPS, RUF, CSAT)
Impact of
insights

45. Communicate internal
Focus on 3-5 opportunities per
product
Monitor improvements
Scorecard it out! Monthly or quarterly.
So the impact that change has made
Create your insights
Communicate that impact to the biz
by constantly sharing user feedback
Build the system

46. Closing the
RUF loop
Email and blog
Don’t let feedback fall on empty and
deaf ears. Follow up with you users
Insights, no promises
Comments come in themes. If you
are going to make changes, share it
24-48 hours
Be proud of the changes you make.
Blog about them externally and let
your upset customers know

47. • Big cool statistic
• 2,56
9
• Add-Ons in Marketplace
Directions are instructions given to
explain how. Direction is a vision
offered to explain why.
- Simon Sinek

48. • Big cool statistic
• 2,56
9
• Add-Ons in Marketplace

49. Join the discussion
Help grow the measurement
<insert>

50. Thank you!
SEAN CRAMER • HEAD OF VOICE OF THE CUSTOMER • ATLASSIAN •
@COSMOCRAMER