This document summarizes Andy Young's presentation on analytics tools and tips for marketers in 2016. The presentation covers why analytics are important for businesses, common analytics failures to avoid, key metrics to track at different stages of business and product growth, techniques for early-stage and growth-stage businesses, and considerations for selecting analytics tools and platforms. The goal is to provide marketers with guidance on using data-driven approaches to optimize growth.
1. 📈 🚀
Andy Young, 500 Startups / Freelance
Turing Festival 2016
Analytics tools and tips for marketers in 2016
Data-driven Growth
2. Andy Young - @andyy - http://bit.ly/analytics-turing
Hi, I’m Andy!
@andyy
#fb
Photo by Dan Taylor - dan@heisenbergmedia.com
3. Andy Young - @andyy - http://bit.ly/analytics-turing
Agenda
Why analytics?
Deconstructing the modern analytics stack
Lies, damn lies, and tech metrics..
- analytics in the real world
4. Andy Young - @andyy - http://bit.ly/analytics-turing 🤔
Why analytics?
5. Andy Young - @andyy - http://bit.ly/analytics-turing
Don’t start with the data
6. Andy Young - @andyy - http://bit.ly/analytics-turing
Common Analytics FAILS
• Drowning in data
• Not identifying
key questions to answer
• Not starting with
clear hypotheses
7. Andy Young - @andyy - http://bit.ly/analytics-turing
Common Analytics FAILS
• Drowning in data
• Not identifying
key questions to answer
• Not starting with
clear hypotheses
8. Andy Young - @andyy - http://bit.ly/analytics-turing
We need to know how we’re doing
9. Andy Young - @andyy - http://bit.ly/analytics-turing
• Overall business/product
• Individual experiments, campaigns, channels, customer segments
We need to know how we’re doing
10. Andy Young - @andyy - http://bit.ly/analytics-turing
• Overall business/product
• Individual experiments, campaigns, channels, customer segments
We need to know how we’re doing
• What is working?
• What is not?
• Where to focus for improvement?
11. Andy Young - @andyy - http://bit.ly/analytics-turing
Analytics Pros..
• Start with a hypothesis
• Identify, collect + analyse necessary relevant data
• Conclude & action
• Iterate, revise
12. Andy Young - @andyy - http://bit.ly/analytics-turing
Analytics for Growth
📈
13. Andy Young - @andyy - http://bit.ly/analytics-turing
Growth?
•More users
•More revenue
14. Andy Young - @andyy - http://bit.ly/analytics-turing
Analytics techniques
• Early stage:
- innovation accounting (Lean Startup)
- cohort analysis
16. Andy Young - @andyy - http://bit.ly/analytics-turing
Early stage questions
17. Andy Young - @andyy - http://bit.ly/analytics-turing
Are we getting traction?
18. Andy Young - @andyy - http://bit.ly/analytics-turing
“Traction is the
rate
at which
monetizable value
is extracted
from customers”
19. Andy Young - @andyy - http://bit.ly/analytics-turing
Innovation
accounting?
20. Andy Young - @andyy - http://bit.ly/analytics-turing
Cohort analysis?
21. Andy Young - @andyy - http://bit.ly/analytics-turing
22. Andy Young - @andyy - http://bit.ly/analytics-turing
23. Andy Young - @andyy - http://bit.ly/analytics-turing
Growth stage questions
24. Andy Young - @andyy - http://bit.ly/analytics-turing
Are we growing?
25. Andy Young - @andyy - http://bit.ly/analytics-turing
Growth Accounting
26. Andy Young - @andyy - http://bit.ly/analytics-turing
Are we growing?
27. Andy Young - @andyy - http://bit.ly/analytics-turing
Growth Accounting
28. Andy Young - @andyy - http://bit.ly/analytics-turing
Growth Accounting
👌👉 http://bit.ly/social-capital-growth-accounting
29. Andy Young - @andyy - http://bit.ly/analytics-turing
Funnel performance?
30. Andy Young - @andyy - http://bit.ly/analytics-turing
As marketers
we need to understand
the full funnel
31. Andy Young - @andyy - http://bit.ly/analytics-turing
Growth stage
Which channels bring
the most customers?
Which channels bring
the best customers?
32. Andy Young - @andyy - http://bit.ly/analytics-turing
How much do we want to
pay per click?
33. Andy Young - @andyy - http://bit.ly/analytics-turing
How much do we want to
pay per click?
..as much as possible?!
Outspend competitors
Activate new channels
34. Andy Young - @andyy - http://bit.ly/analytics-turing
Customer segmentation
& channel performance
For different groups of customers:
• Where do they come from?
• How do they behave?
35. Andy Young - @andyy - http://bit.ly/analytics-turing
Customer segmentation
Users acquired via different channels
will have different behaviours
Different cohorts will have
different experiences of your product
Different users will have been exposed to
different A/B tests
36. Andy Young - @andyy - http://bit.ly/analytics-turing
These are all properties of your users
UTM tags: source, medium, campaign, terms
Landing page
Signup time
A/B test buckets
Referrer
Viral source
Customer segmentation
37. Andy Young - @andyy - http://bit.ly/analytics-turing
Demystifying
direct traffic?
38. Andy Young - @andyy - http://bit.ly/analytics-turing
Demystifying
direct traffic?
39. Andy Young - @andyy - http://bit.ly/analytics-turing
Tag all the things!
Aim: map all Direct + Referral traffic
to a specific source/medium
Do this for Email, Social,
Offline(!)..
Tagging guide:
- bit.ly/tagging-guide
- bit.ly/utm-reference
40. Andy Young - @andyy - http://bit.ly/analytics-turing
Landing
page:
often a big
clue to source
41. Andy Young - @andyy - http://bit.ly/analytics-turing
Cross-device?
Offline channels?
42. Andy Young - @andyy - http://bit.ly/analytics-turing
Cross-device?
• Common pattern:
Research on mobile/tablet,
purchase on tablet/desktop
• Challenge: attribution on mobile
• We need: unique identifying information
E.g. email, mobile/cell #
43. Andy Young - @andyy - http://bit.ly/analytics-turing
Cross-device attribution?
https://www.twilio.com/blog/2012/11/increase-app-downloads-with-mobile-app-distribution-powered-by-twilio.html
44. Andy Young - @andyy - http://bit.ly/analytics-turing
Filling the
attribution
blanks..
https://twitter.com/cannonjw/status/432586695490670592
45. Andy Young - @andyy - http://bit.ly/analytics-turing
Filling the
attribution
blanks..
https://twitter.com/cannonjw/status/432586695490670592
46. Andy Young - @andyy - http://bit.ly/analytics-turing
Geographic A/B tests
https://econsultancy.com/blog/64204-12-inspiring-marketing-campaigns-from-google/
47. Andy Young - @andyy - http://bit.ly/analytics-turing
Brand vs. non-brand terms
for organic + paid search channels
48. Andy Young - @andyy - http://bit.ly/analytics-turing
Channel Groupings
49. Andy Young - @andyy - http://bit.ly/analytics-turing
Attribution models?
• First touch? Last touch? Multi-touch?
50. Andy Young - @andyy - http://bit.ly/analytics-turing
Attribution models?
• First touch? Last touch? Multi-touch?
• First touch:
grow our audience reach
• Subsequent touches:
grow our conversion rate %
51. Andy Young - @andyy - http://bit.ly/analytics-turing
Most campaigns FAIL
52. Andy Young - @andyy - http://bit.ly/analytics-turing
Most campaigns fail
..but most every failed campaign will contain
partial successes
• Plan in advance: how will we know
at which point in the user journey
a campaign failed?
• Do things that don’t scale
53. Andy Young - @andyy - http://bit.ly/analytics-turing
Pulling this all together
Why analytics?
Deconstructing the modern analytics stack
Lies, damn lies, and tech metrics..
- analytics in the real world
54. Andy Young - @andyy - http://bit.ly/analytics-turing
What to track?
• Discovery / Acquisition
• Activation
• Engagement
• Conversion / Purchase
• Retention
• Referral
55. Andy Young - @andyy - http://bit.ly/analytics-turing
So many platforms
56. Andy Young - @andyy - http://bit.ly/analytics-turing
So many platforms
¯_( )_/¯
57. Andy Young - @andyy - http://bit.ly/analytics-turing
What functionality
do you need?
58. Andy Young - @andyy - http://bit.ly/analytics-turing
What functionality do you need?
• Session/Pageview Analytics
Google Analytics, Chartbeat
59. Andy Young - @andyy - http://bit.ly/analytics-turing
What functionality do you need?
• Session/Pageview Analytics
Google Analytics, Chartbeat
• User/Event-based analytics
Mixpanel/Amplitude, Heap, Kissmetrics, Localytics
60. Andy Young - @andyy - http://bit.ly/analytics-turing
What functionality do you need?
• Session/Pageview Analytics
Google Analytics, Chartbeat
• User/Event-based analytics
Mixpanel/Amplitude, Heap, Kissmetrics, Localytics
• Mobile deep linking and attribution
Branch, Adjust, AppsFlyer, Tune, Kochava
61. Andy Young - @andyy - http://bit.ly/analytics-turing
What functionality do you need?
• Session/Pageview Analytics
Google Analytics, Chartbeat
• User/Event-based analytics
Mixpanel/Amplitude, Heap, Kissmetrics, Localytics
• Mobile deep linking and attribution
Branch, Adjust, AppsFlyer, Tune, Kochava
• Mobile-specific
Flurry, Swrve, Leanplum
62. Andy Young - @andyy - http://bit.ly/analytics-turing
What functionality do you need?
• Session/Pageview Analytics
Google Analytics, Chartbeat
• User/Event-based analytics
Mixpanel/Amplitude, Heap, Kissmetrics, Localytics
• Mobile deep linking and attribution
Branch, Adjust, AppsFlyer, Tune, Kochava
• Mobile-specific
Flurry, Swrve, Leanplum
• A/B testing
Optimizely, WVO, Google Content Experiments, StoreMaven
63. Andy Young - @andyy - http://bit.ly/analytics-turing
What functionality do you need?
• Session/Pageview Analytics
Google Analytics, Chartbeat
• User/Event-based analytics
Mixpanel/Amplitude, Heap, Kissmetrics, Localytics
• Mobile deep linking and attribution
Branch, Adjust, AppsFlyer, Tune, Kochava
• Mobile-specific
Flurry, Swrve, Leanplum
• A/B testing
Optimizely, WVO, Google Content Experiments, StoreMaven
• Querying & Charting
Mode, Periscope, Tableau, RJMetrics, Statsbot
64. Andy Young - @andyy - http://bit.ly/analytics-turing
What functionality do you need?
• Session/Pageview Analytics
Google Analytics, Chartbeat
• User/Event-based analytics
Mixpanel/Amplitude, Heap, Kissmetrics, Localytics
• Mobile deep linking and attribution
Branch, Adjust, AppsFlyer, Tune, Kochava
• Mobile-specific
Flurry, Swrve, Leanplum
• A/B testing
Optimizely, WVO, Google Content Experiments, StoreMaven
• Querying & Charting
Mode, Periscope, Tableau, RJMetrics, Statsbot
• Dashboards
Geckoboard, Klipfolio
65. Andy Young - @andyy - http://bit.ly/analytics-turing
What functionality do you need?
• Session/Pageview Analytics
Google Analytics, Chartbeat
• User/Event-based analytics
Mixpanel/Amplitude, Heap, Kissmetrics, Localytics
• Mobile deep linking and attribution
Branch, Adjust, AppsFlyer, Tune, Kochava
• Mobile-specific
Flurry, Swrve, Leanplum
• A/B testing
Optimizely, WVO, Google Content Experiments, StoreMaven
• Querying & Charting
Mode, Periscope, Tableau, RJMetrics, Statsbot
• Dashboards
Geckoboard, Klipfolio
• Audience demographics, interests & rankings
Quantcast, Comscore, Alexa, SimilarWeb
66. Andy Young - @andyy - http://bit.ly/analytics-turing
What functionality do you need?
• Session/Pageview Analytics
Google Analytics, Chartbeat
• User/Event-based analytics
Mixpanel/Amplitude, Heap, Kissmetrics, Localytics
• Mobile deep linking and attribution
Branch, Adjust, AppsFlyer, Tune, Kochava
• Mobile-specific
Flurry, Swrve, Leanplum
• A/B testing
Optimizely, WVO, Google Content Experiments, StoreMaven
• Querying & Charting
Mode, Periscope, Tableau, RJMetrics, Statsbot
• Dashboards
Geckoboard, Klipfolio
• Audience demographics, interests & rankings
Quantcast, Comscore, Alexa, SimilarWeb
• Marketing Automation, CRM, Email & Push Notifications
Marketo, Hubspot, AppBoy, Kahuna
67. Andy Young - @andyy - http://bit.ly/analytics-turing
What functionality do you need?
• Session/Pageview Analytics
Google Analytics, Chartbeat
• User/Event-based analytics
Mixpanel/Amplitude, Heap, Kissmetrics, Localytics
• Mobile deep linking and attribution
Branch, Adjust, AppsFlyer, Tune, Kochava
• Mobile-specific
Flurry, Swrve, Leanplum
• A/B testing
Optimizely, WVO, Google Content Experiments, StoreMaven
• Querying & Charting
Mode, Periscope, Tableau, RJMetrics, Statsbot
• Dashboards
Geckoboard, Klipfolio
• Audience demographics, interests & rankings
Quantcast, Comscore, Alexa, SimilarWeb
• Marketing Automation, CRM, Email & Push Notifications
Marketo, Hubspot, AppBoy, Kahuna
• Platforms
Google Firebase, AWS Mobile Analytics
68. Andy Young - @andyy - http://bit.ly/analytics-turing
What functionality do you need?
• Session/Pageview Analytics
Google Analytics, Chartbeat
• User/Event-based analytics
Mixpanel/Amplitude, Heap, Kissmetrics, Localytics
• Mobile deep linking and attribution
Branch, Adjust, AppsFlyer, Tune, Kochava
• Mobile-specific
Flurry, Swrve, Leanplum
• A/B testing
Optimizely, WVO, Google Content Experiments, StoreMaven
• Querying & Charting
Mode, Periscope, Tableau, RJMetrics, Statsbot
• Dashboards
Geckoboard, Klipfolio
• Audience demographics, interests & rankings
Quantcast, Comscore, Alexa, SimilarWeb
• Marketing Automation, CRM, Email & Push Notifications
Marketo, Hubspot, AppBoy, Kahuna
• Platforms
Google Firebase, AWS Mobile Analytics
😱
69. Andy Young - @andyy - http://bit.ly/analytics-turing
How to pick?
• What functionality do you need?
• Who will be using it? Devs, data scientists, product folks, marketing people?
• How do you want to use it? Analysis, reporting, dashboards..
• Which platforms? Web, mobile, server?
• Which other tools/data do you need to integrate with?
• What data volumes? Data points, events per second, data points per month.
• Budget?
70. Andy Young - @andyy - http://bit.ly/analytics-turing
How to pick?
Things to be wary of:
• Data lock-in, future portability
• Building your own
71. Andy Young - @andyy - http://bit.ly/analytics-turing
Tools are not a panacea
72. Andy Young - @andyy - http://bit.ly/analytics-turing
Biggest single factor?
73. Andy Young - @andyy - http://bit.ly/analytics-turing
Biggest single factor?
Quality of your
tracking implementation
- Coverage/depth
- Accuracy/lack of bugs
74. Andy Young - @andyy - andy@apexa.co.uk
Use a tag manager
75. Andy Young - @andyy - andy@apexa.co.uk
Data collection: Segment.com
76. Andy Young - @andyy - http://bit.ly/analytics-turing
👉 http://sitehound.co
Free code!
77. Andy Young - @andyy - http://bit.ly/analytics-turing
Dashboards?
78. Andy Young - @andyy - http://bit.ly/analytics-turing
What makes a
good dashboard?
79. Andy Young - @andyy - http://bit.ly/analytics-turing
👉 http://bit.ly/kpi-sheet
80. Andy Young - @andyy - http://bit.ly/analytics-turing
Growth model spreadsheet
• Review weekly - zoom out vs. zoom in
• Share with whole team
• Use to predict and prioritise
• What to look at daily vs. weekly vs. monthly
👉 http://bit.ly/kpi-sheet
81. Andy Young - @andyy - http://bit.ly/analytics-turing
Metrics are people too
Every metric should be measured in
unique people
or % conversion
82. Andy Young - @andyy - http://bit.ly/analytics-turing
Joining the dots
Plugging all these tools and platforms together
83. Andy Young - @andyy - http://bit.ly/analytics-turing
Initially, manual is OK!
84. Andy Young - @andyy - http://bit.ly/analytics-turing
Initially, manual is OK!
• Forcing function to ensure we:
• learn what works
• understand the data
• need the data
• Paste weekly into a Google Sheet/Excel
85. Andy Young - @andyy - http://bit.ly/analytics-turing
Automate over time
to reduce effort
86. Andy Young - @andyy - http://bit.ly/analytics-turing
Plumbing for automation
• Zapier, Tray.io
• Supermetrics, Blockspring for Google Sheets/Excel
• Segment.com
• Own database/data warehousing - AWS RDS/Redshift
• Custom queries - Tableau, Periscope etc
87. Andy Young - @andyy - http://bit.ly/analytics-turing
Why analytics?
Deconstructing the modern analytics stack
Lies, damn lies, and tech metrics..
- analytics in the real world
88. Andy Young - @andyy - http://bit.ly/analytics-turing
The numbers
never add up
89. Andy Young - @andyy - http://bit.ly/analytics-turing
The 5 stages of
(Analytics)
Grief
90. Andy Young - @andyy - http://bit.ly/analytics-turing
The 5 stages of Analytics Grief
• Denial
91. Andy Young - @andyy - http://bit.ly/analytics-turing
FB ad clicks
vs.
GA visits
92. Andy Young - @andyy - http://bit.ly/analytics-turing
The 5 stages of Analytics Grief
• Denial
• Anger
93. Andy Young - @andyy - http://bit.ly/analytics-turing
Mixpanel vs. own database
94. Andy Young - @andyy - http://bit.ly/analytics-turing
The 5 stages of Analytics Grief
• Denial
• Anger
• Bargaining
95. Andy Young - @andyy - andy@apexa.co.uk
Measure
Twice:
Compare
data across
different tools
https://jeffpeachey.com/2010/12/02/measure-twice-cut-once-woodcut-for-sale/
96. Andy Young - @andyy - http://bit.ly/analytics-turing
The 5 stages of Analytics Grief
• Denial
• Anger
• Bargaining
• Depression
97. Andy Young - @andyy - http://bit.ly/analytics-turing
Where did
it all go
wrong?
98. Andy Young - @andyy - http://bit.ly/analytics-turing
Where did
it all go
wrong?
• The data is bad
• Our definitions are wrong
99. Andy Young - @andyy - http://bit.ly/analytics-turing
The numbers never add up
• Absolute truths
• Signups, Transactional data, UGC, data from our
own DB
• Lossy/noisy measurements
• Client-side tracking: GA, Mixpanel/Amplitude etc
• Nuanced definitions
• Uniques vs. totals, funnel conversion rates
100. Andy Young - @andyy - http://bit.ly/analytics-turing
Even if the numbers do add up,
they are still often misleading
101. Andy Young - @andyy - http://bit.ly/analytics-turing
102. Andy Young - @andyy - http://bit.ly/analytics-turing
Data discrepancies
Google Analytics vs. Mixpanel
• Definition of unique user
- cookie rules
• Ad/tag blockers
103. Andy Young - @andyy - http://bit.ly/analytics-turing
Data discrepancies
• Page did not finish loading
- e.g. tag manager, or user gave up & hit
back button
• Cross-browser JS bugs
104. Andy Young - @andyy - http://bit.ly/analytics-turing
The 5 stages of Analytics Grief
• Denial
• Anger
• Bargaining
• Depression
• Acceptance
105. Andy Young - @andyy - http://bit.ly/analytics-turing
3-5% discrepancies between
platforms?
Roll with it.
106. Andy Young - @andyy - http://bit.ly/analytics-turing
Large (>5%) difference?
Debug/analyse to figure out why
107. Andy Young - @andyy - http://bit.ly/analytics-turing
Takeaways?
108. Andy Young - @andyy - http://bit.ly/analytics-turing
Secrets to successful analytics
• Start with your questions / hypotheses
• Be precise about what to measure, and why
• Plan in advance, simplify
• Automate required work for weekly/monthly
calculations
• Document all analysis with simple bullet-points
109. Andy Young - @andyy - http://bit.ly/analytics-turing
The numbers tell us What
They don’t tell us Why
110. Thanks, & good luck!
@andyy // andy@apexa.co.uk
Slides: bit.ly/analytics-turing
111. Andy Young - @andyy - http://bit.ly/analytics-turing
References & further reading
• My more in-depth analytics presentations: Slideshare, YouTube
• KPI google sheet
• SiteHound website tracking javascript library
• UTM Tagging Guide
• Lean Analytics
• Social Capital blog posts: - Accounting for User Growth