Full deck from Conversion Elite 2017 with additional slides and further on slide notes added. Covers the concept of understanding where you are in the user relationship and which additional metrics to consider when planning tests as well as those which you are trying to optimise in an MVT or AB test

1. Test with Intelligence
Using Data for Decisive Testing

3. Convert Intent

4. Into Action

5. Who is this intended for?
The Conversion Elite

6. Not General Marketers

7. Conversion Specialists like me

8. Conversion Specialists like me

9. Not Data Scientists

10. Know Analytics & KPIs

11. But this is not a “How To Set Up Analytics”

12. This will not teach you Query Folding and Data Modelling

13. Credit you with the intelligence to investigate or delegate

14. If that is not you

15. What is in this presentation?
Thinking about the data & detail needed to plan for better results

16. What is in this presentation?
Not just measuring Traffic & Sales
Considering WHAT is a positive outcome

17. What is in this presentation?
Leveraging micro conversion data
Understand different parts of the research and buy user journeys

18. What will you get as a result?

19. Understand how to turn user
behaviour data into value

20. Think through
factors that may
impact test results

21. Understand causes of
bottlenecks on key templates

22. Why is that important?
Improved understanding of data aids performance
Test uplift can still fail to meet business KPIs
Understand business data
Investigate User data

23. Makes on-target delivery easier

24. Growth is limited
If Optimisation strategy doesn’t fit

25. Why is that YOUR problem?

26. If business growth targets are missed
If Revenue targets fall short
Optimisation costs come under more scrutiny

27. The business problem IS your problem

28. It quickly becomes everyone’s problem

29. You can do
something about it

30. You can do
something about it
Because you solve the customers problems

31. How will you feel at the end?

32. Inspired

33. Excited

34. Exhausted

35. Data can be sexy

36. Disclaimers
The stunts you are about to see have been performed by a trained
professional using the appropriate safety equipment
It may contain
• Bad puns, jokes and visual gags
• Deliberately silly and inappropriate stock imagery
• Some NSFW references and images
• Tortured analogies and stretched metaphors

37. You must be this Tall
to join the ride

38. Do not try this at home

39. Do not try this at home
But do try this at work

40. Tools & Stacks

42. Google Analytics API Script
or paid plugins like

43. Google Analytics API to Excel, csv or database
Also API for Facebook, Twitter, AdWords, Search Console & more
Allows scheduled and batch data pulls

44. Google Analytics API to Excel, csv or database
Also API for Facebook, Twitter, AdWords, Search Console & more
Allows scheduled and batch data pulls
Also has save to Drive or Cloud database options

45. Combine Business & Web Data. Deeper Analysis and visualisation
PowerBI also has API link to Google Analytics API etc.
PowerQuery and PowerBI have JSON API (make your own API query)
Also have csv folder extract for batch processing

46. Bigger datasets, more powerful statistical analysis
PowerBI cloud has real language querying
“What is the best performing product for revenue this Quarter?”

47. Geo lookup or CRM for customer addresses (or combine both)
Visualise on drill down map or overlay ArcGIS data
e.g. Conversions > Avg from Areas with Household income > average

48. Tableau, R or PowerBI to look at and query distribution patterns
Average alone can mask opportunity & risks
Skewed distribution or Revenue to consider for Test “winners”
Best for your lowest value, least loyal customers, worst for your big spenders
Often not a business win, even if it reports as a Test win

49. User & Session
Recording

50. Also, you know. Maybe talk to them
Ask Customer Services
Look at Feedback form data
Jump on a Live Chat

51. Hit level – Measurement Protocol

52. Gets a little spendy at the top end but Free Option is good to try out

53. Traffic is vanity

54. Do you Agree?

55. Conversion Rates are vanity

56. Do you Agree?

57. Profits are Sanity

58. Do you Agree?

59. Relationships develop Value

60. Do you Agree?

61. Shopping Analogy
Google Analytics in Real Life Playlist
http://bit.ly/GA-irl

62. Shopping
Friction – UX choices
Mechanical – Basic Functionality and Accessibility
Look for those, minimise Friction
Yes, make sure the Shop actually works
(for the user)

63. Dating not Shopping
If talking profit and business growth
Its about customer relationships
Particularly if we are talking lifetime value from ecommerce
For Lead Gen, Media Owner, SaaS …
Thinking Relationship works better to understand what to measure
What Measurement matters, depends on what Relationship

64. What type of relationship are we trying to model?
What are the metrics that will show it developing?
What sort of customers do we attract?
What sort of customers do we want to attract?
What Type of Conversion?

65. Tantric
Lifestyle
Identity
Affirming

66. Relaxed Casual FWB

67. Er.. Fun?

68. Seeking Longer Commitments

69. Finding Longer Commitments

71. Quick Fling

72. Time to Convert?

73. Forgetting something?

74. Times have changed

75. Foreplay

76. Metrics and Measurement

77. Micro Conversions
Next page
Clicked to View Larger
Clicked Reviews Tab
Checked Stock
Showed interest by playing with your drop downs
If they aren’t Adding or Checking out
What are they doing?
• Which of them relate to positive outcomes?
• Which indicate they are unsure on the relationship on offer?

78. Yes?
Little confirmations build up
How do those first touches feel?
Do they still seem a little unsure?
Did your change not increase Add to Basket
But increased use of Shipping estimator?
Tempted them but not convinced yet
Measuring other outcomes provides insight

79. Yes!
You don’t sell the final decision first
You sell the next page, the not-Bounce, the not-Exit
They start to warm up to your offers

80. Yes!!
Encourage them to use their fingers a little more
To spend a Click to explore deeper
Deliver on what you promised from each Click until…

81. YES!!!
You build lots of little endorphin hits…
into one moment of ….completion

82. Don’t disappoint
with an over eager CTA
Oh they didn’t bounce?
They must love my offer
Buy me NOW. BUY… oh…Bye
Oops, sorry, this never usually happens…

83. Don’t blindly “Test CTA Copy”
Think about what action you are calling to happen
• Will you really deliver Buy next?
• Are they Ready to Buy?
• Does the rest of the page fit that message?
• Has their journey primed them to want to take that Action?
When is the right time?
When is right for this type of relationship?
What extra data will help to understand this?
Add that.

84. Keep momentum
Measure the appropriate metric

86. Quantify Value of Changes
IF fixed THEN [this] is the range of Potential Value or Risk Avoided
Not a “Quick Fix” or uplift on a “Test Metric”
Identifying optimal relationship value is a longer term process
You get a first date, you get the next date
You update your Facebook status to Its complicated
Maybe you get lucky first time; it can and does happen
But to get lucky again you need to put in the work

88. Flirt

89. Intent and
Attraction
Where they are
from is not just a
Marketing
problem
Traffic patterns
and motivation
can impact test
planning and
reporting

90. Venue &
Context
Where you met
How you interacted
What you promised
All factor into the journey to conversion
Will change what you measure to progress further

91. First Impressions
Do Count
What can you measure as the baseline?
Flirt (Marketing)
or First Date (Landing)
General Rules:

92. 1. Be Attractive

94. 2. Don’t be Unattractive

96. Attraction is Subjective
Are you attractive to YOUR target users?
Don’t feel it should work because You like it
Measure if THEY like it
Users have varied ideas of attractive
And their reaction counts most

98. Sometimes very er… Specific

99. Long Tail Traffic?

100. Landing Pages
The S of your Advertising AIDAS
Attention, Interest, Desire, Action … on click do they land on Satisfaction
• Does this deliver what you promised with your Flirt?
• Do they start the journey satisfied? It is easier if they do
• Have they had their first little Yes?
Every page is a Landing Page (almost)
Think of it as the Venue for your First Date
Context and Audience appropriate

101. Bounce Traffic
“I came, I puked, I left”
Avinash Kaushik
Stood up for your date
Expectations mismatched
Profile pic vs Reality
Total Loss?
Maybe not, but that’s another area to investigate
Because the user, usually has an alternative

102. Plenty more fish

103. Non Bounce Traffic
Actual interest?
Yes this looks like something I want?
For now this is: Failed to Lose
They did something you can actually measure
But in the early stages that’s all you can say for sure
They didn’t bounce. Or your analytics setup is faulty
If it was a positive reaction – a positive forward action - then non-Bounce
Non-Bounce, interpreted as positive forward Action = Turned up to your date.

104. Your initial offer got a bite
You have them hooked (for now)

105. Homepage

106. Homepages are Noisy
The absolute worst page to optimise
Unless you have a single clear offering
Homepage is trying to pull in a nightclub
All flashing lights, noise and moving images

107. Desperate Much?
Desperate is never a good look
What is a positive step here?
Clicking ANYTHING?
Only interested if you are easy?
More valuable if they click
• Sale Banner?
• Category?
• Brand?
Didn’t Bounce - will not tell you much

109. Mobile Homepage
Eek!
Half Price sale: Best at Forward Clicks?
Or because it’s the only one they see?

110. Trying to please all the people
• Returning Regulars
• New to Vaping
Hardware, Juices, Coils, Info
Some want a quick restock
Some are completely lost
Who does this page serve best?
How do Clicks Forward split?
New | Returning?
Desired Outcome?

111. Mobile
Less Noise
But not Better
Worse
Especially as Lost New Vaper
Is mostly on Mobile

112. Homepage 3 x3 grid
Every decision is friction
Each distraction slows the first Yes
Simplify the concept: 3 x 3 equal grid
Most important things on site?
• For Business
• For User (and which ones)
At a Functional level: What is NEEDED by the USER on homepage
Set aside design & promotion bias – 9 equal slots.
What helps the user most? What is currently less obvious but clicked lots?

113. Weighted Grid
Not all priorities are equal
Now 9 must-have items: but weight them
Top products may still work in not-Top positions
Lagging Products may benefit from emphasis
Negative space can draw attention to key areas
Signpost the areas most users need first
Net effect of more areas appealing = more clicks
Lose on some, gain on others. Net Gain on page
Test to improve your Not-Bounce Positive Forward total

114. Business Focus
Every business priority above fold
“Which Device” as key focus
Hardware = Most Profit
3 panels plus the main Navigation
Subscriptions = New Potential
Does this help those Lost New Vapers?
Or assume knowledge of where to go?

115. User Intent Focus
Who am I? Which path for me?
Left vs Right split
Starters | Pro Users
Eliquids | Hardware
• Equally valid for both
Eliquid click & purchase frequency MUCH higher
• Middle of page
• Whitespace for clarity
• Supporting Text “Help to choose?”
Hardware demoted and again split Starter | Latest
Measure clicks per section by Intended audience

116. What won?
Indecision
Impatience

117. I told you Homepages
were shit to optimise
What won?

118. Lesson?
Let the Wookiee win

119. First Date

120. Product List Page
Weird
There’s a sale on
They should have mentioned that
PLP is the Get to Know You stage
Some people spend a lot time here
How much time? Why?
What tempts them to take it further?

121. Venue &
Context
It depends on the relationship they plan to have
What your proposition is
Do they need to browse?
Is that part of the appeal and courtship?
Or are they trying to get to your goodies ASAP?

122. Intent By Type
Again consider how you first met
New vs Returning
Channel
Devices
Product List Page Behaviour varies a LOT
Segment out interaction metrics to look for patterns
On a varied or non-Normal distribution catalogue
Split out by Content Groups or Categories

123. Things escalate

124. Interest rises

125. Interest rises!
Are they really showing they are ready for …
Brown chicken, brown cow
(say it fast)

126. Or…Considering Options
Maybe Lots of PLP activity is not escalation
• Browsing
• Comparing
• Sizing up your proposition
Is there lots of filter use?
• Which Filters?
Is there a Compare function?
• Do they use it?

127. Or… are they lost?
Too much choice
Poor signposting means they are hunting not finding
Too little help for where they are in the relationship

128. Can’t see what they want?

129. Is it in an odd place?
Taxonomy
IA issues

130. Weak Signal
Not seeing Product
Only seeing part of Proposition
Not reaching first Decision point

131. Your User Journey
Not theirs
Does this show issues with Information Architecture?
You know your way from A to B – it’s your site
Is poor signposting making the User work too hard?
Are you looking to optimise a path they simply don’t take

132. Are you measuring Lost?
Drop off points – particular Categories or # PLP loads
Lots of Pathfinding
• Frequent ‘Reset’ to Homepage or Category Level
• High Filter/Sort Clicks but Low Product Clicks

134. Product List Page
• Behaviour shows Intent
• Audience provides Context
Lots of Category & Top Nav resets
Very few New users Add from PLP
Return Users more likely to Add here
Mobile users almost never (either)

135. Second Date but no kiss

136. Common to New and Returning
Common to both – Drop off to Product Detail View
• Basket Add at similar levels
• But they Add in different places
• Are you tracking WHERE your Adds to Cart happen or just Total?

137. How much effort for that first kiss?
That is some funky bad breath
And impossible on Mobile – our New & Lost Vapers
[animated version shows three mandatory drop downs before Add enables]
[CTA buttons only show on mouse over]

138. Multi Adds are rarer but do happen
Multi Adds most common on Return
• Some Outliers
• 2 users > 130 items
• These 2 users would be in 2nd place for Items Added
“Average” Adds per user is misleading
Do you know your distribution? Largest Bin by volume/value?
Mix of:
• New Browsers/Lost: 1 or 2 adds each
• Return Restockers: 2 to 130 (3-5 being the Median)

139. Product List Page
User Survey Feedback
Lots of extra Detail Clicks just to see what flavours in
Liquid because it is not clear on PLP
“Pro” Restock-Users switch to List view where Add to
Cart is visible without hover: Click, Click, Click, Checkout
Users often reset Categories hunting for similar flavours
that are in different Categories e.g. Lemon in Fruits,
Lemon Cheesecake in Bakery
Text to explain flavours on PLP?
Or Flavour Note Icons on PLP
• Can we make them filters?
Can we get rid of stupid mouseover?

140. Product List Page
Wireframe New Layout
• Bring Results to top of page
• Move filter nav to left
• Compact listings
• More above fold
Quick Add or Read more visible on load
• No mouse over needed
Flavour Notes text
• Click and it will filter to similar
• Brings results from all Categories

141. PLP New Version
Most ideas tested well and implemented
• PLP total clicks reduced – less hunting
• Category Resets reduced
• PLP Adds increased +34%
Users get what they want, more quickly
Items Added Per User increased
• New Users closer to 2
• Return Restockers 4 – 6
AOV and Revenue impact… big

142. Product Detail
Consider this heavy flirting
More than just superficial stuff
They don’t want to just sort by Most Popular
They want to know your Full Specifications baby
What previous romantic interests thought about your goods & delivery

143. Product Detail Page
We’ve barely started
Literally 5 seconds after load
But you want feedback?!
Feedback:
• Don’t stop me doing what I want
• Now is not appropriate
If you do this
1. Track how fast people close it
2. Track how many people leave
3. Don’t do this (so early)

144. Product Detail Page
Major Drop out point – some was users from PLP looking for flavour detail
The Rest? If they don’t click Add, what ARE they clicking?

146. Same painful 3 drop down selection as was on PLP
Minimum 7 clicks to Add to Cart was the big heat concentration
But Check Available Stock was nearly as highly used
Why?
Because site logic meant user only told Out of Stock after they confirmed options
Still want to Add? Then you need to Check Stock to see what combinations ARE in stock

147. Little investigation into when this happened most, to which type of users and how it impacted Revenue.

148. This required being able to split out by Product SKU and Category to work out where it happened most
This also meant it could be filtered In or Out of tests where Add to Cart metrics were being used
But this required a little more work than just using the Analytics UI – example Data Model to split this type of detail out

149. This can then be crunched – in this case in Tableau – and Category & Product Performance compared through the steps
Some obviously Popular products get lots of Views, lots of Clicks, much lower Purchases
Most often out of Stock – revenue (and conversion rate) impact can be quantified. Plus: Most Popular with New First Daters
Eek

150. Track Failure not just Success

151. New Stock
Less No, No, Damn
More Yes Yes YES
Sometimes fixing conversions is
about fixing the business
processes not the website
Measure errors, test to reduce
frustration & understand why
users don’t get to Complete

152. PDP - New
• Thumb friendly No Drop Downs
• Stock Key shown on load
• Flavour Notes
Plus change in production process
• Data means Popularity better predicted
• Range & Options drastically simplified
• Stock levels more consistent
• Out of Stock now only end of line or
sell-out promotions (i.e. deliberate)

153. Time to Convert

154. Practice Safe Conversion

155. Reassure, don’t frighten

156. Login | Register | Guest
A common debate
“Guest is best Practice”
Depends on many factors
There is no one Right answer

157. But there are some horrifically bad standard templates

158. Approach 1: Default to Guest
Make other options clear
Minimise CRM data loss
Still ask for basic opt-in

159. I’d Like to Register
Even though there is NO
incentive or benefit explained

160. Welcome Back?
Nice to see you again?
Your loyalty discount is ready?

161. Approach 2: Default to Guest
Defer Account Create until later
(because the ONLY difference is password in terms of the effort required)
Or Login
(but we won’t say Hi or show Guest users what they are missing)

162. Approach 3: Default to Login/Returning
Brave move? It depends
How many of each user type do you have?
What is the acquisition & growth strategy?
New Customer OR Guest is treated as one
(because again, only 1 field of extra effort)
And again
No benefits shown (Login to see your discount points?)
No Welcome to Return Users
In relationship terms:
If your FWB blanked you next time
Or you saw them blanking a previous conquest
How attracted does that make you feel?
They can still walk away at this point. Many do.

163. Approach 4: Guest Only as Default
Straight to business
Give me all that field filling action right now
(and I won’t touch your dirty PO Box)
Slight hint of Benefit of an Account
If you had an account, this isn’t needed
Not that compelling though, is it?
4 different approaches
4 different “best practices”
But which of these looked at User Types?
Which of these thought about feel of the message
When it is time to convert, don’t stop the flirt

164. Which is the best approach? What wins tests?
Usually whichever users you have most of
New or Returning? Quick one-off Guests, occasional FWB, Loyal Lovers?
Do you plan sample volumes, design, metrics & segment for that?
What benefit is there in having an account?
Do you tell people? Don’t Assume. Tell. Show. Make it look GOOD
Make Guest users want to be in Team Return next time?
Are you One Night Stand or Lifetime Value focus? Are they?
At the door to the bedroom… willing to Convert
….And the message you send is … this > >
One Night Stands only
I may remember you next time
But I won’t treat you any differently
In fact I might put you behind the next Guest
“Best” practice? SRSLY?

165. Conversion

166. Mutually fulfilling

167. The deed is done
You got your Transaction
You generated that Lead hard
Next meeting you can brag about how you Returned that Investment
Well done you. Slow clap
One more question:
How does your partner in this dance feel?
Also happy?
Transaction Regret?
Buyers Remorse?
Already adding your email to their spam folder?

168. Transaction Satisfaction
Your Next Flirt. Your Next opportunity to Convert
Starts NOW.
It will depend on the memory of a low-friction good time. Delivered on time and in nice packaging
The Hug, the warm afterglow
You got what you want, you made sure they got what they wanted too. Didn’t you?
You’ve promised to follow up in in 3 – 5 working days “I’ll call you, real soon babes”
(24 hours if you convert before 5pm and pay for next day service, except in the Outer Hebrides and the Channel Islands. Terms & Conditions may apply)
Did you?
Your next Cost Per Acquisition will be lower if you have delivered
If you did, you won’t have to handle as many objections
If you are a known quantity for satisfaction, even if they see better offers, CPA will be lower

169. Or do they feel a little railroaded?
Worried about what happens next?

170. Mutually fulfilling
One final YES
• Your Order is Safe
• Thank You
• Now Social Me
• Call me if you need me
• Email Thanks
• Reassure on delivery

171. Retention
Avoid complacency
Keep looking sharp
Treat them better than your new side piece
Become that reliable partner
Even if they do stray, they’ll be thinking of you
And you can win them back

173. Revenue is not the only Goal
You can also measure and test
User Guides, Customer Service & FAQ
This is quite often more for your benefit than theirs
This is how you weed out the unsuitable suitors
Reduce customer service costs
Guide your potential partners to know how to handle you
the way you need to be for best results
This saves you time, customer service costs, returns costs