My UX London workshop; short version of the full-day workshop I'm teaching this year in San Francisco, Atlanta, and Chicago: http://bit.ly/gL7HaH

1. Design to Refine
Developing a tunable information architecture
Lou Rosenfeld • Rosenfeld Media • rosenfeldmedia.com
London • 14 April 2011

2. Hello, my name is Lou

3. Agenda
1. The quick intro
2. Prioritizing and tuning top-down navigation
3. Demo: content modeling
4. Prioritizing and tuning contextual navigation
5. Group exercise: site search analytics
6. Prioritizing and tuning search
7. Changing your work and your organization

4. I’ve already dissed redesign
See the slides here:
http://www.slideshare.net/lrosenfeld/

5. The alternatives to redesign
1. Prioritize: Identify the important problems
regularly
2. Tune: Address those problems regularly
3. Be opportunistic: Look for low-hanging
fruit

6. Prioritize because
a little goes a long way

7. A handful of your queries/ways to navigate/documents meet
A little data goes a long way
the needs of the few audiences that use your site most

8. A handful of your queries/ways to navigate/documents meet
A little data goes a long way
the needs of the few audiences that use your site most
LOVE IT

9. A handful of your queries/ways to navigate/documents meet
A little data goes a long way
the needs of the few audiences that use your site most
LOVE IT
LEAVE IT

10. Zipf in text

11. Report card for essential wants and needs

12. Be an incrementalist:
tune because things change

13. From projects to processes:
a regular regimen of design
Example: the rolling content inventory

14. Impact of change on design
(queries)

15. Be an opportunist:
look for the low-hanging fruit
1. Top-down navigation:
Anticipates interests/questions at arrival
2. Bottom-up (contextual) navigation:
Enables answers to emerge
3. Search:
Handles specific information needs

16. Life by a thousand cuts
50% of users are search dominant
x 5% of all queries are typos, fixed by spell checking.
2.5% improvement to the UX
50% of all users are search dominant
x 30% (best bet results for top 100 queries)
15% improvement to the UX
Ditto for improving content, search results design,
navigation design…

17. Summary
You can refine
1. Prioritize the problems that are most important
to your users
2. Regularly address these problems
3. Identify opportunities to make small
improvements that go a long way

18. Prioritizing and Tuning
Top-Down Navigation

19. The data-driven main page:
Who wants what and when?

20. Who wants what?
US English speakers

21. Who wants what?
German speakers

22. When do they want it?

23. Commerce sites get it

24. The IRS gets it

25. But really, who cares
about the main page?

26. But really, who cares
about the main page?

27. The risk of main page fixation
From Tony Dunn’s Tales from Redesignland
(http://redesignland.blogspot.com/)

28. Focusing on main page =
taking Zipf too far
...plus lots of competition (Google, ads/landing pages)

29. The tail that wags the dog:
site map drives
improved site hierarchy

30. Site map by tool, unit, and format

31. Site map by tool, unit, and format

32. Site map by tool, unit, and format

33. User-centered site map
User-centered site map...

35. Asking the possible
from your site index

36. Specialized site indices

37. Specialized site indices

38. Specialized site indices
Cisco’s site indices are
specialized by content
type (products, services)

39. Best bet-based site indices
MSU’s site index is built on
popular information needs
(based on best bet search results)

40. Going broad and deep with
guides (AKA microsites)

41. Vanguard’s main page loves
guides

42. Vanguard’s main page loves
guides

43. The Tax Center is a guide

44. One more example: IRS

45. One more example: IRS

46. ...e-filing is presented as
sequential steps

47. Summary: Top-down navigation
Prioritize main page content and layout
1. Confuse as necessary by diverting attention
2. Counter politics with data; e.g., use seasonality to drive design
Tune and prioritize site-wide navigation
3. Use the site map as a skunkworks for site-wide hierarchy
4. Base site indices on specialized content or popular
information needs (e.g., best bets)
5. Use guides (micro-sites) as narrow/deep complement to
broad/shallow navigation schemes

48. Agenda
1. The quick intro
2. Prioritizing and tuning top-down navigation
3. Demo: content modeling
4. Prioritizing and tuning contextual navigation
5. Group exercise: site search analytics
6. Prioritizing and tuning search
7. Changing your work and your organization

49. concert calendar
album pages artist descriptions
TV listings
Demonstration:
Content Modeling
album reviews discography artist bios

50. What are the common content objects in your site?
album pages artist bios
artist descriptions
album reviews
53

51. How do they fit together? concert calendar
album pages artist descriptions
TV listings
album reviews discography artist bios

52. What content objects are missing? concert calendar
And how do they fit?
album pages artist descriptions
TV listings
album reviews discography artist bios

53. Where do you start? concert calendar
album pages artist descriptions
TV listings
album reviews discography artist bios

54. How will you connect those objects?

55. Use content models
for content that’s...
Homogeneous
High-volume
High importance
What’s the most important deep content in
your site?

56. Use content models
when you need to...
Incorporate user research into your
deep content
Improve contextual navigation
Identify missing content
Prioritize metadata choices
Really benefit from your CMS

57. Steps for developing
content models
1. Determine key audiences (who’s using it?)
2. Select important tasks to test (what are they
using it for?)
3. Determine important content areas (what do
they want?)
4. Determine content types (what are they using?)
5. Determine metadata attributes (how will we
connect the objects?)
6. Determine contextual linking rules (where should
the objects lead us to next?)

58. Agenda
1. The quick intro
2. Prioritizing and tuning top-down navigation
3. Demo: content modeling
4. Prioritizing and tuning contextual navigation
5. Group exercise: site search analytics
6. Prioritizing and tuning search
7. Changing your work and your organization

59. Prioritizing and Tuning
Contextual Navigation

60. Establishing Desire Lines
Use
Content modeling
• Site search analytics

61. Where do searches begin?
Not just the main
page, according to a
User Interface
Engineering study
(http://is.gd/j1NHeS)

62. Using site search analytics
to identify desire lines

63. Choose a
common content
type (e.g., events)
!
Where should !
users go from here?
!

64. !
!
!
!
! !
Analyze frequent queries generated from each content sample

65. !
!
!
Can you type these queries to improve your
content model?
Link events to:
• the site’s articles on the event’s topic
• info on locales for each event

66. What content types
should we be connecting?

67. Important content types emerge
from content modeling concert calendar
album pages artist descriptions
TV listings
album reviews discography artist bios

68. Using SSA to prioritize content
types

69. Getting content types out of
site search analytics
Take an hour to...
• Analyze top 50 queries (20% of all search activity)
• Ask and iterate: “what kind of content would users be looking for when
they searched these terms?”
• Add cumulative percentages
Result: prioritized list of potential content types
#1) application: 11.77%
#2) reference: 10.5%
#3) instructions: 8.6%
#4) main/navigation pages: 5.91%
#5) contact info: 5.79%
#6) news/announcements: 4.27%

70. What should we use to
connect content types?

71. Which metadata attributes will your
content model depend upon?

72. More on prioritizing metadata attributes

73. Prioritizing semantic relationships

74. How do we
prioritize content?

75. Some content value variables
I

76. Some content value variables
I
Usability
Popularity
Credibility

77. Some content value variables
Currency
Freshness
Authority
Follows guidelines
(e.g., titling,
I metadata)
Usability
Popularity
Credibility

78. Some content value variables
Currency
Freshness
Authority
Follows guidelines
(e.g., titling,
I metadata)
Usability
Popularity
Credibility
Strategic value
Addresses compliance
issues (e.g., Sarbanes/Oxley)
Content owners are good
partners

79. Subjectively “grade” your content’s value
1.Choose appropriate
value criteria for each
content area
2.Weight criteria (total
= 100%)
3.Subjectively grade for
each criterion
4.weight x grade =
score
5.Add scores for
overall score

80. Subjectively “grade” your content’s value
Subjective
assessment
1.Choose appropriate
value criteria for each
content area
2.Weight criteria (total
= 100%)
3.Subjectively grade for
each criterion
4.weight x grade =
score
5.Add scores for
overall score

81. Put the grades together for a more
objective “report card”
Helps prioritize content migrations, refreshes, ...

82. Put the grades together for a more
objective “report card”
Objectifies subjective
assessments
Helps prioritize content migrations, refreshes, ...

83. Summary:
contextual navigation
Use content modeling and site search analytics to
1. Identify and prioritize content types
2. Identify desire lines
3. Improve contextual navigation between content
types
4. Identify and prioritize metadata attributes
Prioritize content areas/subsites by establishing
balanced value criteria

84. Agenda
1. The quick intro
2. Prioritizing and tuning top-down navigation
3. Demo: content modeling
4. Prioritizing and tuning contextual navigation
5. Group exercise: site search analytics
6. Prioritizing and tuning search
7. Changing your work and your organization

85. Group exercise:
Site search analytics

86. Agenda
1. The quick intro
2. Prioritizing and tuning top-down navigation
3. Demo: content modeling
4. Prioritizing and tuning contextual navigation
5. Group exercise: site search analytics
6. Prioritizing and tuning search
7. Changing your work and your organization

87. Prioritizing and Tuning Search

88. Make “the Box” accommodate
most searchers’ queries

89. How long are our queries?
Top 500 queries
(37% of all traffic)

90. Mean = 10.6 characters
Median = 10 characters

91. Mean = 10.6 characters
Median = 10 characters
Long tail queries likely longer

92. Mean = 10.6 characters
Median = 10 characters
Long tail queries likely longer
Top queries often in low 20s
!

93. Mean = 10.6 characters
Median = 10 characters
Long tail queries likely longer
Top queries often in low 20s
Desired: @30 characters;
Can you get that many?
!

94. Mean = 10.6 characters
Median = 10 characters
Long tail queries likely longer
Top queries often in low 20s
Desired: @30 characters;
Can you get that many?
!
Safe: @15-20 characters

95. We’ve seen this before:
auto-completing queries

96. Auto-completing from a
known, common items (e.g.,

97. Auto-completing from a
known, common items (e.g.,
Uses known terms:
e.g., movie titles and
actor/director names

98. Auto-completing from queries

99. Uses common queries
Auto-completing from queries

100. Auto-completing from best
bets

101. Auto-completing from best
bets
Uses best bets

102. Making change easy:
supporting query refinement

103. The absolute
meaninglessness
of
advanced search

104. The absolute
meaninglessness
of
advanced search
!
At University of Alaska-Fairbanks,
advanced = expanded search

105. The absolute
meaninglessness
of
advanced search
!
At University of Alaska-Fairbanks,
advanced = expanded search
At the IRS,
advanced =
narrowed search
!

106. Contextualizing “advanced” features

107. Look to session data for
progression and context

108. Look to session data for
progression and context
search session patterns
1. solar energy
2. how solar energy works

109. Look to session data for
progression and context
search session patterns
1. solar energy
2. how solar energy works
search session patterns
1. solar energy
2. energy

110. Look to session data for
progression and context
search session patterns
search session patterns 1. solar energy
1. solar energy 2. solar energy charts
2. how solar energy works
search session patterns
1. solar energy
2. energy

111. Look to session data for
progression and context
search session patterns
search session patterns 1. solar energy
1. solar energy 2. solar energy charts
2. how solar energy works
search session patterns
search session patterns 1. solar energy
1. solar energy 2. explain solar energy
2. energy

112. Look to session data for
progression and context
search session patterns
search session patterns 1. solar energy
1. solar energy 2. solar energy charts
2. how solar energy works
search session patterns
search session patterns 1. solar energy
1. solar energy 2. explain solar energy
2. energy
search session patterns
1. solar energy
2. solar energy news

113. Improving performance for
specialized queries

114. Recognizing proper nouns,
dates, and unique ID#s

115. Surfacing
specialized content types
in search results

116. Tuning Search Results:
Handling specialized answers

117. Tuning Search Results:
Handling specialized answers

118. Tuning Search Results:
Handling specialized answers

119. Tuning Search Results:
Handling specialized answers
“Product quick links” come directly from product content model
These results are a strong counterbalance to raw results

120. When raw isn’t good enough:
best bet search results

122. best bet #1

123. best bet #1
best bet #2

124. best bet #1
best bet #2
even more
best bets

125. best bet #1
best bet #2
even more
best bets
raw results

126. best bet #1
best bet #2
even more
best bets
raw results

127. best bet #1
best bet #2
even more
best bets
competition
raw results

128. best bet #1
best bet #2
even more
best bets
competition
danger?
raw results

129. best bet #1
best bet #2
even more
best bets
competition
danger?
data
raw results

130. The 0 search results page:
search’s equivalent of the 404

131. Tuning Search Results:
0 results pages
Not helpful

132. Tuning Search Results:
0 results pages
Not helpful
Much better:
“Did you
mean?” and
Popular
Searches

133. Summary: Search systems
Tune query entry
1. Make “The Box” wide enough
2. Support query auto-completion to focus queries
3. Surface the right features to support query refinement
4. Recognize and take advantage of specialized queries
Tune search results design
5. Surface specialized content types as results for specialized
queries
6. Complement raw results with best bets
7. Enable recovery from finding 0 search results

134. Agenda
1. The quick intro
2. Prioritizing and tuning top-down navigation
3. Demo: content modeling
4. Prioritizing and tuning contextual navigation
5. Group exercise: site search analytics
6. Prioritizing and tuning search
7. Changing your work and your organization

135. Changing your work
and your organization

136. Doing your work differently
1. Processes, not projects
2. Rebalancing your research and design

137. From time-boxed projects
to ongoing processes
Example: the rolling content inventory

138. What else can roll?
Most everything
Each week, for example...
• Content scouting and sampling (rather than inventory)
• Analyze analytics to identify spikes, new trends
Each month...
• Identify new tasks, run new task analysis studies
• Develop new best bet search results
Each quarter...
• Field study
• Review and tune personas

139. Build a practice that’s
balanced and data-driven

140. User Research Landscape
from Christian Rohrer: http://is.gd/95HSQ2

141. User Research Landscape
Ongoing coverage
of each of these
4 quadrants
from Christian Rohrer: http://is.gd/95HSQ2

142. Lou’s TABLE OF
OVERGENERALIZED Web Analytics User Experience
DICHOTOMIES
Users' intentions and
What they Users' behaviors (what's
motives (why those things
analyze happening)
happen)
Qualitative methods for
What methods Quantitative methods to
explaining why things
they employ determine what's happening
happen
Helps users achieve goals
What they're Helps the organization meet
(expressed as tasks or
trying to achieve goals (expressed as KPI) topics of interest)
Uncover patterns and
How they use Measure performance (goal-
surprises (emergent
data driven analysis)
analysis)
Statistical data ("real" data Descriptive data (in small
What kind of data
in large volumes, full of volumes, generated in lab
they use errors) environment, full of errors)

143. Getting your organization
to support your work
1. Making friends and allies
2. Changing your leaders’ minds

144. Making friends and allies

145. Showing content owners
how their content performs

146. Showing content owners
how their content performs

147. Helping marketing
develop better messaging
Jargon vs. Plain Language at Washtenaw Community College
• Online courses were marketed using terms
“College on Demand” (“COD”) and “FlexEd”; signup rates
were poor
• Compare jargon with “online”
(used in 213 other queries)
• Content was retitled rather
than re-marketed

148. Helping IT say “no” with authority
Reduce pressure to solve problems with technologies by
making what we have work
Minimize radical changes to platforms
• Enterprise search
• Content management systems
• Analytics applications
• ...

149. Changing leaders’ minds

150. Talking points
for refining, against redesigning
1. Solve the problem(s)
2. Save money
3. Reduce/end radical organizational changes

151. Solving the problem(s)
• Forcing the issue: ban the term “redesign”
from discussions
• Data-driven definition / prioritization /
tuning / opportunism
• Creating anchors to keep project from
spinning out of control: elevator pitch /
mission / vision / goals / KPI

152. This can be very, very helpful
Gamestorming
by Dave Gray,
Sunni Brown, and
James Macanufo
(O’Reilly, 2010)

153. Saving money
• Life by a thousand cuts: small changes have
huge impacts (see: Zipf)
• Reuse and retain technology investments
• Retain institutional knowledge
• Get more from your (empowered) team and
make it pay for itself
• Spend less on external support and fire your
agency

154. Reduce/end radical
organizational changes
• End the pendulum swing from centralized
to decentralized approaches
• Reorganize information, not people
• Build self-sustaining, steady in-house
capabilities to prioritize and tune

155. Being prepared to fail

156. Sometimes your leaders
are in a hurry

157. Sometimes your leaders
are not very smart

158. Sometimes your organization
is immature

159. Nurit Peres’ Company UX Maturity Model
(http://is.gd/x1dOuP)

160. Renato Feijó’s UX Maturity Model
(http://is.gd/dul2t2)

161. Always be ready to go
under the radar

162. Summary: changing your work
and your organization
Do your work differently
1. Move from time-based projects to ongoing processes
2. Build a balanced, data-driven practice
Get your organization to support your work
3. Make friends and allies
4. Change leaders’ minds by
• Solving problems
• Saving money
• Reducing radical change
Be prepared to fail

163. Agenda
1. The quick intro
2. Prioritizing and tuning top-down navigation
3. Demo: content modeling
4. Prioritizing and tuning contextual navigation
5. Group exercise: site search analytics
6. Prioritizing and tuning search
7. Changing your work and your organization

164. Say hello
Lou Rosenfeld
lou@louisrosenfeld.com
Rosenfeld Media
www.louisrosenfeld.com | @louisrosenfeld
www.rosenfeldmedia.com | @rosenfeldmedia