Presented at Strata San Jose 2018. Shares how Netflix enables business teams to perform cohort analysis on very large, high dimensional data by using Big Data and web application technologies such as Spark, Druid, Node, React, and D3
8. Partners are companies that make it easier
for people to sign up and engage with our
service and help us retain members.
BLAKE IRVINE | STRATA SAN JOSE 2018
15. ● Are partners helping us acquire members?
● Through which channels?
● How does a partner impact the regional market?
● Do members use our service differently on partner devices?
● How do partners compare to each other?
Evaluating Partners
BLAKE IRVINE | STRATA SAN JOSE 2018
16. Cohorts are the collections of members
associated with a partner that are relevant to
the business question.
BLAKE IRVINE | STRATA SAN JOSE 2018
17. ● Our trip example showed how one member can be associated
with many partners.
● Business teams want to explore the nuance of cohorts...
Cohorts can be complex
BLAKE IRVINE | STRATA SAN JOSE 2018
19. ● 100+ million members
● Dozens of partners
● Many combinations of members and partners
● Leads to…
○ High dimensionality, high cardinality datasets
○ Very large datasets of member-level time-series activity
Evaluating Cohorts
BLAKE IRVINE | STRATA SAN JOSE 2018
23. Data construction: Data Model
member
device
partner
playback
events
isp
billing
events
billing
processor
BLAKE IRVINE | STRATA SAN JOSE 2018
24. Data construction: Cohort Dataset
signup_events
cohort
playback_events
billing_events
x_events
BLAKE IRVINE | STRATA SAN JOSE 2018
25. Data construction: Flat Tables
playback_f cohort_playback_s
device_d
isp_d
geo_d
partner_d
cohort_d
BLAKE IRVINE | STRATA SAN JOSE 2018
26. Data for consumption: Flat Table
key memb
er_id
device
_id
device_name device_categor
y
partner_name country region data_payload
1 1213 674 Amazon Fire
TV
Set Top Box Amazon US Americas [{"id":5025945823792539,"sequen
ce":41,"time":1491962092955},
{"id":5025947899236389,"sequen
ce":95,"time":1491962104824}]
2 7623 1172 Chromecast Streaming
Stick
Google DE EMEA …
3 4291 129 PS3 Game
Console
Sony ES EMEA …
4 9013 447 iPad 4 Tablet Apple CA Americas …
BLAKE IRVINE | STRATA SAN JOSE 2018
28. ● Goals
○ Serve dozens of users
○ Provide interactive / low-latency tool
○ Provide many different perspectives
● Challenges
○ Manage high dimensionality
○ Very large time-series datasets
Data Product for Cohort Analysis
BLAKE IRVINE | STRATA SAN JOSE 2018
29. Analytic Tool Choices
Choice 1 Choice 2 Choice 3
Analytic Tool Tool 1 Tool 2 Tool 3
Data Engine MPP Cloud In memory
Data Size 1B rows 10B rows 100M rows
Performance
(SWAG)
Up to many
minutes
Many
minutes
Several
Seconds
BLAKE IRVINE | STRATA SAN JOSE 2018
30. ● Data stored in Druid
● Custom app built with Javascript
Choice 4...
BLAKE IRVINE | STRATA SAN JOSE 2018
31. ● An open source data store for analytic applications
● Distributed, column-oriented, indexed architecture
● Well suited to serve our “flat” tables
Druid white paper: http://static.druid.io/docs/druid.pdf
BLAKE IRVINE | STRATA SAN JOSE 2018
32. ● Built with Express, React, Redux, D3
● Custom UX / UI to manage views and dimensionality
● Enabled access to data served by Druid
● Enabled management of query execution and caching
BLAKE IRVINE | STRATA SAN JOSE 2018
33. ● Video demo with simulated data
BLAKE IRVINE | STRATA SAN JOSE 2018
36. ● Dimensions (aka slice-n-dice)
○ More is always better
○ Changes require restatement
● “Typical” use cases must be met also
○ Not a solution for every data question
○ Analysts and other tools are still needed
Challenges
BLAKE IRVINE | STRATA SAN JOSE 2018
37. ● Data volume always increases…
○ More members, more partners, more devices, more metrics
● Custom app development time is longer, and ongoing
○ But for the right use cases, worthwhile
Challenges
BLAKE IRVINE | STRATA SAN JOSE 2018
38. Partners help Netflix grow.
We measure partner value through cohorts.
Big data tools enable efficient analysis.
BLAKE IRVINE | STRATA SAN JOSE 2018
Thank you!
39. Blake Irvine - Growth Data Products
birvine@netflix.com
@blakeirvine
linkedin.com/in/blakeirvine/
Editor's Notes
In 2016, Netflix became available globally in almost all countries!