SlideShare a Scribd company logo
1 of 49
DON’T SNOOZE YET!
5 min.. 10 min.. 15 min.. You find yourself still scrolling but don’t hit play
Time’s Up!
We’ll talk about 5 things.
Understanding Netflix
Understanding user pain points
Proposing a solution
Measures of success & Launch
Way forward
Understanding Netflix
Mission
“Entertaining everyone & making the world smile.”
Understanding Netflix User Pain Points Proposing A Solution Measuring success Way forward
Company Shareholders care about?
Understanding Netflix User Pain Points Proposing A Solution Measuring success Way forward
Subscribers!
MORE SUBSCRIBERS!
Huge Competitive
Threat
Loyal & Growing
Customer Base
Reed Hastings cares about?
Understanding Netflix User Pain Points Proposing A Solution Measuring success Way forward
Content for everyone Accurate Recommendations &
Personalised Experience
Guiding Metrics
Discovery Time
Viewing Time
Drop-off Rate
Understanding Netflix User Pain Points Proposing A Solution Measuring success Way forward
Netflix
10%
Not
Netflix
90%
TV Viewership
Annual TV viewership: 1B Hours*
Number of TV households: 120 M*
Average TV viewing hours/day per HH: 8.33
Netflix users (2018): 66 M
Average Netflix viewing hours per
household per day: 1.65
In the US
Understanding Netflix User Pain Points Proposing A Solution Measuring success Way forward
Interesting Facts To Note
Source: Netflix ’18 Q4 Earnings Call
We can estimate
*Source: Neilson Research
Interesting Facts To Note
6000+ titles on Netflix (in the US alone)
In 2018 alone Netflix added 700+ titles
It could take upto 3.88 years to watch everything on Netflix
(if watched 24/7 without breaks)
Understanding Netflix User Pain Points Proposing A Solution Measuring success Way forward
THERE IS STILL MORE THAN 6 HOURS WORTH OF
OPPORTUNITY PER HOUSEHOLD!
Understanding Netflix User Pain Points Proposing A Solution Measuring success Way forward
Who are the users?
On the Dimension of Time Spent Watching Netflix
The Daily The Hardly Evers
Understanding Netflix User Pain Points Proposing A Solution Measuring success Way forward
The Weekenders
Understanding User Pain
Points
Let’s talk to some users.
Survey
Sample Set includes:
100 Respondents
US based
Age group: 21 - 40
Equal gender representation
Understanding Netflix User Pain Points Proposing A Solution Measuring success Way forward
Aim: Validating if users experience fatigue on Netflix.
In person
interviews
Observing
users
Finding #1
User Behaviour:
Average Discovery Time (DT) for most users = 15 min+
0 0.15 0.3 0.45 0.6 0.75
Less than 15 min
10-15 min
30 min
More than 45+ min
Understanding Netflix User Pain Points Proposing A Solution Measuring success Way forward
Note: Total avg. viewing hours on Netflix per user: 1.65 hours
User Behaviour (when they don't find anything to watch):
Most Switch to another platform / Don’t watch anything
Finding #2
0 0.1 0.2 0.3 0.4
Quit to another platform
Quit and don't watch anything
Message a friend for recommendations
Watch something from NF recommendations
]
Understanding Netflix User Pain Points Proposing A Solution Measuring success Way forward
Finding #3
User Behaviour Frequency (when they don't find anything to watch):
Most users said they switch/quit frequently
Never
3%
Quite
Frequently
80%
Sometimes
12%
Almost
Always
5%
Note: Quite Frequently defined as: 2 / 5 or 40% of the times
Understanding Netflix User Pain Points Proposing A Solution Measuring success Way forward
Finding #4
0 0.15 0.3 0.45 0.6
Friends
Recommendations
Netflix
Recommendations
Own Curated
Lists
Social Media, Blogs,
Online Reviews
Users Trust:
Recommendations from friends > Recommendations from Netflix
]
Understanding Netflix User Pain Points Proposing A Solution Measuring success Way forward
0 0.1 0.2 0.3 0.4
Quit to another platform
Quit and don't watch anything
Message a friend for recommendations
Watch something from NF recommendations
Finding #5
User behaviour (when they don't find anything to watch):
Most don’t text a friend in the moment
Understanding Netflix User Pain Points Proposing A Solution Measuring success Way forward
]
Summary of Company & User Research
More Titles +
Imperfect
Recommendations
Increase in
Discovery Time
User Fatigue
Drop
Off/Switch
• Netflix content is growing at a very fast pace.
• Users spend significant amount of total avg. viewing time on discovery.
• Netflix recommendations haven’t been perfect.
• Friends are better trusted source for recommendations.
Summary of the User’s Problem
Understanding Netflix User Pain Points Proposing A Solution Measuring success Way forward
My Conclusion
Since machine based recommendations are
great but not perfect, yet,
user fatigue is real.
Understanding Netflix User Pain Points Proposing A Solution Measuring success Way forward
Proposed Solution
Identifying the user fatigue point is crucial.
Adding trusted human aspect - at the points of fatigue
- to machine based recommendations can increase
viewership and decrease drop offs.
Understanding Netflix User Pain Points Proposing A Solution Measuring success Way forward
Meet Neil - The Explorer
Software Engineer, 32
Lives in Mountain View, CA
A Netflix Daily User
Loves watching & finding out about new shows &
movies
Also subscribes to Amazon Prime, Hulu, HBO
On weekends, he enjoys hanging out with friends
Pain Point:
“I watch anything and everything. But my Netflix feed has too much content
similar to what I watched but didn't like. My friends & I discuss interesting new
content all the time that I don’t see on my feed. A fresh set of unbiased
recommendations would be helpful at times when I’m just tired of scrolling.”
Understanding Netflix User Pain Points Proposing A Solution Measuring success Way forward
Introducing
Understanding Netflix User Pain Points Proposing A Solution Measuring success Way forward
Don’t Snooze Yet – A Netflix Original
Understanding Netflix User Pain Points Proposing A Solution Measuring success Way forward
How will it work?
2 Key Aspects:
• Identifying unique fatigue point for every user.
• Leverages on ‘The Social Effect’ at fatigue point.
(Facebook is used as an example here)
Understanding Netflix User Pain Points Proposing A Solution Measuring success Way forward
Unique Fatigue Point
Estimated per user based on machine learning algorithms
built on data models that aggregates and captures signals*
as:
• Drop off time: Amount of time a user spends looking for shows before abandoning
the platform without playing anything.
• Viewing time per session:  How long did the user watch the show, and did he
interrupt to go back to browsing.
• Discovery time per session :  Amount of time user spends to browse shows before
watching something.
• Content Watched : What did the user end up watching, something we
recommended, something random and new, or shows he’s watched already but
repeats frequently.
*Based on an assumption that Netflix already captures the above usage logs
Understanding Netflix User Pain Points Proposing A Solution Measuring success Way forward
Link to Feauture Demo
The Social Effect
Measuring Success
Measuring success
Discovery Time
Viewing Time
Drop-off Rate
Understanding Netflix User Pain Points Proposing A Solution Measuring success & Launch Way forward
At Product Level
Fatigue Detection
Don’t Snooze Yet Screen
Social Media Plugin
Measuring success
At Feature Level
Activation Retention
My List
How many times does Don’t
Snooze Yet screen show up
for Neil per viewing hour?
Did Neil drop off before Don’t
Snooze Yet screen showed up?
How many times does Neil play
from Don’t Snooze Yet screen?
How many times does Neil play
from what a followed friend
watched?
Has Neil integrated his
social media to Netflix?
Does he add friends’ shows
to his own list?
Is he adding more
shows/movies to his lists? Is
adding categories?
Is Neil interacting with Don’t
Snooze Yet Screen?
Understanding Netflix User Pain Points Proposing A Solution Measuring success & Launch Way forward
Launch
Understanding Netflix User Pain Points Proposing A Solution Measuring success & Launch Way forward
Beta Test MVP
Phase 1: Internal Test with Netflix employees
Phase 2:
Users who have agreed to Participate in Tests.
US based
100k users (standard Netflix testing sample size)
Don’t Snooze Yet Screen on TV only
Social Plugin enabled across all devices
Share your Netflix
stories with your friends now!
Expect lots of
memes!
• Social Media
• Push Notifications
• In-app / Web notifications
• Blog
• Email
Go To Market
Understanding Netflix User Pain Points Proposing A Solution Measuring success & Launch Way forward
Across all devices
Don’t Snooze Yet!
Share your Netflix stories now!
Understanding Netflix User Pain Points Proposing A Solution Measuring success & Launch Way forward
3 of your friends are watching Narcos Mexico
• Social Media
• Push Notifications
• In-app / Web notifications
• Blog
• Email
Go To Market
See what your friends are watching now.
Add their shows to your list. And Watch Better.
See what your friends are watching now.
Create your own recommendations. Watch
better.
Understanding Netflix User Pain Points Proposing A Solution Measuring success & Launch Way forward
• Social Media
• Push Notifications
• In-app / Web notifications
• Blog
• Email
Go To Market
Don’t Snooze Yet!
Share your Netflix stories now!
Adding recommendations from trusted friends to the Netflix
experience.
Understanding Netflix User Pain Points Proposing A Solution Measuring success & Launch Way forward
• Social Media
• Push Notifications
• In-app / Web notifications
• Blog
• Email
Go To Market
Don’t Snooze Yet!
Share your Netflix stories now!
3 of your Friends Have Been Watching Narcos Mexico, Neil!
Understanding Netflix User Pain Points Proposing A Solution Measuring success & Launch Way forward
• Social Media
• Push Notifications
• In-app / Web notifications
• Blog
• Email
Go To Market
Estimated Roadmap
Phase 1 (MVP): Don’t Snooze Yet
Product
Design/UX
Engineering
Sprint 1
Wireframe
Prototype:
1. DSY Screen
2. User Profiles, Social Plugin
3. My List Update
Sprint 2
Fatigue
Identification / Data
Model
Sprint 3
Back End/Server-side
Implementation
UI Development for all screens
Integration
Sprint 4
QA
Launch
User Testing; User Feedback
Coordination with
Marketing to prep for
announcement
Firefighting Across Teams
Test!
Understanding Netflix User Pain Points Proposing A Solution Measuring success & Launch Way forward
Design Validation
Beta Testing
Way Forward
Phase 2
Main Feed
Further down the road: Short Term
Understanding Netflix User Pain Points Proposing A Solution Measuring success & Launch Way forward
• Making fatigue point identification 100% accurate.
• Training data for better machine learning based
recommendations model.
• Capturing data on kind of content users prefer / don’t prefer to
watch.
Other Benefits
Understanding Netflix User Pain Points Proposing A Solution Measuring success & Launch Way forward
Further down the road: Long Term
Phase 1 Phase 2 Phase 3
Don't Snooze Yet
Main Feed
Pattern Based
Recommendations
Neil likes watching Netflix when he is getting dinner. He usually
picks something out from his Dinner Time shows. Recommend
something from there to him on his main feed.
Right recommendations to the
right user at the right time.
Understanding Netflix User Pain Points Proposing A Solution Measuring success & Launch Way forward
"I think about it really as us winning time away,
entertainment time, from other activities. Instead of doing
Xbox or Fortnite or YouTube or HBO or a long list, we want
to win and provide a better experience.”
- Reed Hastings
Thank You!
By: Sneha Sultania
Hi, I am Sneha
Die-Hard Netflix Fan.
A Product Enthusiast
About me
sul.sneha@gmail.com

More Related Content

Similar to Don't Snooze Yet - A Netflix Original

Building innovative products
Building innovative productsBuilding innovative products
Building innovative productsShishir Choudhary
 
Building innovative products
Building innovative productsBuilding innovative products
Building innovative productsShishir Choudhary
 
How to quickly get more views on youtube
How to quickly get more views on youtubeHow to quickly get more views on youtube
How to quickly get more views on youtubeJibranAzhar
 
UX Field Research Toolkit - A Workshop at Big Design - 2017
UX Field Research Toolkit - A Workshop at Big Design - 2017UX Field Research Toolkit - A Workshop at Big Design - 2017
UX Field Research Toolkit - A Workshop at Big Design - 2017Kelly Moran
 
DON WILCOX - BEST PRACTICES - DIGITAL PLATFORMS TO ENGAGE NEW AUDIENCES
DON WILCOX - BEST PRACTICES - DIGITAL PLATFORMS TO ENGAGE NEW AUDIENCESDON WILCOX - BEST PRACTICES - DIGITAL PLATFORMS TO ENGAGE NEW AUDIENCES
DON WILCOX - BEST PRACTICES - DIGITAL PLATFORMS TO ENGAGE NEW AUDIENCESHilary Ip
 
یوتیوب ابزار مارکتینگ
یوتیوب ابزار مارکتینگیوتیوب ابزار مارکتینگ
یوتیوب ابزار مارکتینگfoad ansari
 
Four Steps to Sure-Fire Live Streaming Success
Four Steps to Sure-Fire Live Streaming SuccessFour Steps to Sure-Fire Live Streaming Success
Four Steps to Sure-Fire Live Streaming SuccessLynn Teatro
 
9010 group the co-creation consultancy
9010 group the co-creation consultancy 9010 group the co-creation consultancy
9010 group the co-creation consultancy 90:10 Group - Italy
 
At Your Service: What Netflix and Assessments Have In Common | SoGoSurvey
At Your Service: What Netflix and Assessments Have In Common | SoGoSurveyAt Your Service: What Netflix and Assessments Have In Common | SoGoSurvey
At Your Service: What Netflix and Assessments Have In Common | SoGoSurveySogolytics
 
DeepakRawat_APMF.pdf
DeepakRawat_APMF.pdfDeepakRawat_APMF.pdf
DeepakRawat_APMF.pdfDeepak Rawat
 
Cause We Care - Design Thinking Project Final Report
Cause We Care - Design Thinking Project Final ReportCause We Care - Design Thinking Project Final Report
Cause We Care - Design Thinking Project Final ReportMatthew Stuckings
 
Google Help Outs: Your New Best Friend
Google Help Outs: Your New Best FriendGoogle Help Outs: Your New Best Friend
Google Help Outs: Your New Best FriendFLBlogCon
 
10 tips for Successful Crowdsourcing
10 tips for Successful Crowdsourcing10 tips for Successful Crowdsourcing
10 tips for Successful CrowdsourcingJW Alphenaar
 
The Only Metric That Matters by a Partner at Greylock Partners
The Only Metric That Matters by a Partner at Greylock PartnersThe Only Metric That Matters by a Partner at Greylock Partners
The Only Metric That Matters by a Partner at Greylock PartnersProduct School
 
Customer Development - Notes from the Field
Customer Development - Notes from the FieldCustomer Development - Notes from the Field
Customer Development - Notes from the FieldChristian Gammill
 

Similar to Don't Snooze Yet - A Netflix Original (20)

Data analysis and brain
Data analysis and brainData analysis and brain
Data analysis and brain
 
Building innovative products
Building innovative productsBuilding innovative products
Building innovative products
 
Building innovative products
Building innovative productsBuilding innovative products
Building innovative products
 
Omnyscope e245 march 2014 final
Omnyscope e245 march 2014 finalOmnyscope e245 march 2014 final
Omnyscope e245 march 2014 final
 
Recommender Systems
Recommender SystemsRecommender Systems
Recommender Systems
 
How to quickly get more views on youtube
How to quickly get more views on youtubeHow to quickly get more views on youtube
How to quickly get more views on youtube
 
UX Field Research Toolkit - A Workshop at Big Design - 2017
UX Field Research Toolkit - A Workshop at Big Design - 2017UX Field Research Toolkit - A Workshop at Big Design - 2017
UX Field Research Toolkit - A Workshop at Big Design - 2017
 
DON WILCOX - BEST PRACTICES - DIGITAL PLATFORMS TO ENGAGE NEW AUDIENCES
DON WILCOX - BEST PRACTICES - DIGITAL PLATFORMS TO ENGAGE NEW AUDIENCESDON WILCOX - BEST PRACTICES - DIGITAL PLATFORMS TO ENGAGE NEW AUDIENCES
DON WILCOX - BEST PRACTICES - DIGITAL PLATFORMS TO ENGAGE NEW AUDIENCES
 
یوتیوب ابزار مارکتینگ
یوتیوب ابزار مارکتینگیوتیوب ابزار مارکتینگ
یوتیوب ابزار مارکتینگ
 
Four Steps to Sure-Fire Live Streaming Success
Four Steps to Sure-Fire Live Streaming SuccessFour Steps to Sure-Fire Live Streaming Success
Four Steps to Sure-Fire Live Streaming Success
 
9010 group the co-creation consultancy
9010 group the co-creation consultancy 9010 group the co-creation consultancy
9010 group the co-creation consultancy
 
At Your Service: What Netflix and Assessments Have In Common | SoGoSurvey
At Your Service: What Netflix and Assessments Have In Common | SoGoSurveyAt Your Service: What Netflix and Assessments Have In Common | SoGoSurvey
At Your Service: What Netflix and Assessments Have In Common | SoGoSurvey
 
DeepakRawat_APMF.pdf
DeepakRawat_APMF.pdfDeepakRawat_APMF.pdf
DeepakRawat_APMF.pdf
 
Methodology - Design Sprint
Methodology - Design SprintMethodology - Design Sprint
Methodology - Design Sprint
 
Cause We Care - Design Thinking Project Final Report
Cause We Care - Design Thinking Project Final ReportCause We Care - Design Thinking Project Final Report
Cause We Care - Design Thinking Project Final Report
 
How to Do Better Market Research
How to Do Better Market ResearchHow to Do Better Market Research
How to Do Better Market Research
 
Google Help Outs: Your New Best Friend
Google Help Outs: Your New Best FriendGoogle Help Outs: Your New Best Friend
Google Help Outs: Your New Best Friend
 
10 tips for Successful Crowdsourcing
10 tips for Successful Crowdsourcing10 tips for Successful Crowdsourcing
10 tips for Successful Crowdsourcing
 
The Only Metric That Matters by a Partner at Greylock Partners
The Only Metric That Matters by a Partner at Greylock PartnersThe Only Metric That Matters by a Partner at Greylock Partners
The Only Metric That Matters by a Partner at Greylock Partners
 
Customer Development - Notes from the Field
Customer Development - Notes from the FieldCustomer Development - Notes from the Field
Customer Development - Notes from the Field
 

Recently uploaded

Unleash Your Potential - Namagunga Girls Coding Club
Unleash Your Potential - Namagunga Girls Coding ClubUnleash Your Potential - Namagunga Girls Coding Club
Unleash Your Potential - Namagunga Girls Coding ClubKalema Edgar
 
"ML in Production",Oleksandr Bagan
"ML in Production",Oleksandr Bagan"ML in Production",Oleksandr Bagan
"ML in Production",Oleksandr BaganFwdays
 
The Fit for Passkeys for Employee and Consumer Sign-ins: FIDO Paris Seminar.pptx
The Fit for Passkeys for Employee and Consumer Sign-ins: FIDO Paris Seminar.pptxThe Fit for Passkeys for Employee and Consumer Sign-ins: FIDO Paris Seminar.pptx
The Fit for Passkeys for Employee and Consumer Sign-ins: FIDO Paris Seminar.pptxLoriGlavin3
 
Connect Wave/ connectwave Pitch Deck Presentation
Connect Wave/ connectwave Pitch Deck PresentationConnect Wave/ connectwave Pitch Deck Presentation
Connect Wave/ connectwave Pitch Deck PresentationSlibray Presentation
 
Gen AI in Business - Global Trends Report 2024.pdf
Gen AI in Business - Global Trends Report 2024.pdfGen AI in Business - Global Trends Report 2024.pdf
Gen AI in Business - Global Trends Report 2024.pdfAddepto
 
"Debugging python applications inside k8s environment", Andrii Soldatenko
"Debugging python applications inside k8s environment", Andrii Soldatenko"Debugging python applications inside k8s environment", Andrii Soldatenko
"Debugging python applications inside k8s environment", Andrii SoldatenkoFwdays
 
Commit 2024 - Secret Management made easy
Commit 2024 - Secret Management made easyCommit 2024 - Secret Management made easy
Commit 2024 - Secret Management made easyAlfredo García Lavilla
 
SAP Build Work Zone - Overview L2-L3.pptx
SAP Build Work Zone - Overview L2-L3.pptxSAP Build Work Zone - Overview L2-L3.pptx
SAP Build Work Zone - Overview L2-L3.pptxNavinnSomaal
 
New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024BookNet Canada
 
Developer Data Modeling Mistakes: From Postgres to NoSQL
Developer Data Modeling Mistakes: From Postgres to NoSQLDeveloper Data Modeling Mistakes: From Postgres to NoSQL
Developer Data Modeling Mistakes: From Postgres to NoSQLScyllaDB
 
TeamStation AI System Report LATAM IT Salaries 2024
TeamStation AI System Report LATAM IT Salaries 2024TeamStation AI System Report LATAM IT Salaries 2024
TeamStation AI System Report LATAM IT Salaries 2024Lonnie McRorey
 
Advanced Computer Architecture – An Introduction
Advanced Computer Architecture – An IntroductionAdvanced Computer Architecture – An Introduction
Advanced Computer Architecture – An IntroductionDilum Bandara
 
Streamlining Python Development: A Guide to a Modern Project Setup
Streamlining Python Development: A Guide to a Modern Project SetupStreamlining Python Development: A Guide to a Modern Project Setup
Streamlining Python Development: A Guide to a Modern Project SetupFlorian Wilhelm
 
unit 4 immunoblotting technique complete.pptx
unit 4 immunoblotting technique complete.pptxunit 4 immunoblotting technique complete.pptx
unit 4 immunoblotting technique complete.pptxBkGupta21
 
A Deep Dive on Passkeys: FIDO Paris Seminar.pptx
A Deep Dive on Passkeys: FIDO Paris Seminar.pptxA Deep Dive on Passkeys: FIDO Paris Seminar.pptx
A Deep Dive on Passkeys: FIDO Paris Seminar.pptxLoriGlavin3
 
Moving Beyond Passwords: FIDO Paris Seminar.pdf
Moving Beyond Passwords: FIDO Paris Seminar.pdfMoving Beyond Passwords: FIDO Paris Seminar.pdf
Moving Beyond Passwords: FIDO Paris Seminar.pdfLoriGlavin3
 
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024BookNet Canada
 
DSPy a system for AI to Write Prompts and Do Fine Tuning
DSPy a system for AI to Write Prompts and Do Fine TuningDSPy a system for AI to Write Prompts and Do Fine Tuning
DSPy a system for AI to Write Prompts and Do Fine TuningLars Bell
 
How AI, OpenAI, and ChatGPT impact business and software.
How AI, OpenAI, and ChatGPT impact business and software.How AI, OpenAI, and ChatGPT impact business and software.
How AI, OpenAI, and ChatGPT impact business and software.Curtis Poe
 
WordPress Websites for Engineers: Elevate Your Brand
WordPress Websites for Engineers: Elevate Your BrandWordPress Websites for Engineers: Elevate Your Brand
WordPress Websites for Engineers: Elevate Your Brandgvaughan
 

Recently uploaded (20)

Unleash Your Potential - Namagunga Girls Coding Club
Unleash Your Potential - Namagunga Girls Coding ClubUnleash Your Potential - Namagunga Girls Coding Club
Unleash Your Potential - Namagunga Girls Coding Club
 
"ML in Production",Oleksandr Bagan
"ML in Production",Oleksandr Bagan"ML in Production",Oleksandr Bagan
"ML in Production",Oleksandr Bagan
 
The Fit for Passkeys for Employee and Consumer Sign-ins: FIDO Paris Seminar.pptx
The Fit for Passkeys for Employee and Consumer Sign-ins: FIDO Paris Seminar.pptxThe Fit for Passkeys for Employee and Consumer Sign-ins: FIDO Paris Seminar.pptx
The Fit for Passkeys for Employee and Consumer Sign-ins: FIDO Paris Seminar.pptx
 
Connect Wave/ connectwave Pitch Deck Presentation
Connect Wave/ connectwave Pitch Deck PresentationConnect Wave/ connectwave Pitch Deck Presentation
Connect Wave/ connectwave Pitch Deck Presentation
 
Gen AI in Business - Global Trends Report 2024.pdf
Gen AI in Business - Global Trends Report 2024.pdfGen AI in Business - Global Trends Report 2024.pdf
Gen AI in Business - Global Trends Report 2024.pdf
 
"Debugging python applications inside k8s environment", Andrii Soldatenko
"Debugging python applications inside k8s environment", Andrii Soldatenko"Debugging python applications inside k8s environment", Andrii Soldatenko
"Debugging python applications inside k8s environment", Andrii Soldatenko
 
Commit 2024 - Secret Management made easy
Commit 2024 - Secret Management made easyCommit 2024 - Secret Management made easy
Commit 2024 - Secret Management made easy
 
SAP Build Work Zone - Overview L2-L3.pptx
SAP Build Work Zone - Overview L2-L3.pptxSAP Build Work Zone - Overview L2-L3.pptx
SAP Build Work Zone - Overview L2-L3.pptx
 
New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
 
Developer Data Modeling Mistakes: From Postgres to NoSQL
Developer Data Modeling Mistakes: From Postgres to NoSQLDeveloper Data Modeling Mistakes: From Postgres to NoSQL
Developer Data Modeling Mistakes: From Postgres to NoSQL
 
TeamStation AI System Report LATAM IT Salaries 2024
TeamStation AI System Report LATAM IT Salaries 2024TeamStation AI System Report LATAM IT Salaries 2024
TeamStation AI System Report LATAM IT Salaries 2024
 
Advanced Computer Architecture – An Introduction
Advanced Computer Architecture – An IntroductionAdvanced Computer Architecture – An Introduction
Advanced Computer Architecture – An Introduction
 
Streamlining Python Development: A Guide to a Modern Project Setup
Streamlining Python Development: A Guide to a Modern Project SetupStreamlining Python Development: A Guide to a Modern Project Setup
Streamlining Python Development: A Guide to a Modern Project Setup
 
unit 4 immunoblotting technique complete.pptx
unit 4 immunoblotting technique complete.pptxunit 4 immunoblotting technique complete.pptx
unit 4 immunoblotting technique complete.pptx
 
A Deep Dive on Passkeys: FIDO Paris Seminar.pptx
A Deep Dive on Passkeys: FIDO Paris Seminar.pptxA Deep Dive on Passkeys: FIDO Paris Seminar.pptx
A Deep Dive on Passkeys: FIDO Paris Seminar.pptx
 
Moving Beyond Passwords: FIDO Paris Seminar.pdf
Moving Beyond Passwords: FIDO Paris Seminar.pdfMoving Beyond Passwords: FIDO Paris Seminar.pdf
Moving Beyond Passwords: FIDO Paris Seminar.pdf
 
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
 
DSPy a system for AI to Write Prompts and Do Fine Tuning
DSPy a system for AI to Write Prompts and Do Fine TuningDSPy a system for AI to Write Prompts and Do Fine Tuning
DSPy a system for AI to Write Prompts and Do Fine Tuning
 
How AI, OpenAI, and ChatGPT impact business and software.
How AI, OpenAI, and ChatGPT impact business and software.How AI, OpenAI, and ChatGPT impact business and software.
How AI, OpenAI, and ChatGPT impact business and software.
 
WordPress Websites for Engineers: Elevate Your Brand
WordPress Websites for Engineers: Elevate Your BrandWordPress Websites for Engineers: Elevate Your Brand
WordPress Websites for Engineers: Elevate Your Brand
 

Don't Snooze Yet - A Netflix Original

  • 2.
  • 3. 5 min.. 10 min.. 15 min.. You find yourself still scrolling but don’t hit play
  • 5. We’ll talk about 5 things. Understanding Netflix Understanding user pain points Proposing a solution Measures of success & Launch Way forward
  • 7. Mission “Entertaining everyone & making the world smile.” Understanding Netflix User Pain Points Proposing A Solution Measuring success Way forward
  • 8. Company Shareholders care about? Understanding Netflix User Pain Points Proposing A Solution Measuring success Way forward Subscribers! MORE SUBSCRIBERS! Huge Competitive Threat
  • 9. Loyal & Growing Customer Base Reed Hastings cares about? Understanding Netflix User Pain Points Proposing A Solution Measuring success Way forward Content for everyone Accurate Recommendations & Personalised Experience
  • 10. Guiding Metrics Discovery Time Viewing Time Drop-off Rate Understanding Netflix User Pain Points Proposing A Solution Measuring success Way forward
  • 11. Netflix 10% Not Netflix 90% TV Viewership Annual TV viewership: 1B Hours* Number of TV households: 120 M* Average TV viewing hours/day per HH: 8.33 Netflix users (2018): 66 M Average Netflix viewing hours per household per day: 1.65 In the US Understanding Netflix User Pain Points Proposing A Solution Measuring success Way forward Interesting Facts To Note Source: Netflix ’18 Q4 Earnings Call We can estimate *Source: Neilson Research
  • 12. Interesting Facts To Note 6000+ titles on Netflix (in the US alone) In 2018 alone Netflix added 700+ titles It could take upto 3.88 years to watch everything on Netflix (if watched 24/7 without breaks) Understanding Netflix User Pain Points Proposing A Solution Measuring success Way forward
  • 13. THERE IS STILL MORE THAN 6 HOURS WORTH OF OPPORTUNITY PER HOUSEHOLD! Understanding Netflix User Pain Points Proposing A Solution Measuring success Way forward
  • 14. Who are the users? On the Dimension of Time Spent Watching Netflix The Daily The Hardly Evers Understanding Netflix User Pain Points Proposing A Solution Measuring success Way forward The Weekenders
  • 16. Let’s talk to some users.
  • 17. Survey Sample Set includes: 100 Respondents US based Age group: 21 - 40 Equal gender representation Understanding Netflix User Pain Points Proposing A Solution Measuring success Way forward Aim: Validating if users experience fatigue on Netflix. In person interviews Observing users
  • 18. Finding #1 User Behaviour: Average Discovery Time (DT) for most users = 15 min+ 0 0.15 0.3 0.45 0.6 0.75 Less than 15 min 10-15 min 30 min More than 45+ min Understanding Netflix User Pain Points Proposing A Solution Measuring success Way forward Note: Total avg. viewing hours on Netflix per user: 1.65 hours
  • 19. User Behaviour (when they don't find anything to watch): Most Switch to another platform / Don’t watch anything Finding #2 0 0.1 0.2 0.3 0.4 Quit to another platform Quit and don't watch anything Message a friend for recommendations Watch something from NF recommendations ] Understanding Netflix User Pain Points Proposing A Solution Measuring success Way forward
  • 20. Finding #3 User Behaviour Frequency (when they don't find anything to watch): Most users said they switch/quit frequently Never 3% Quite Frequently 80% Sometimes 12% Almost Always 5% Note: Quite Frequently defined as: 2 / 5 or 40% of the times Understanding Netflix User Pain Points Proposing A Solution Measuring success Way forward
  • 21. Finding #4 0 0.15 0.3 0.45 0.6 Friends Recommendations Netflix Recommendations Own Curated Lists Social Media, Blogs, Online Reviews Users Trust: Recommendations from friends > Recommendations from Netflix ] Understanding Netflix User Pain Points Proposing A Solution Measuring success Way forward
  • 22. 0 0.1 0.2 0.3 0.4 Quit to another platform Quit and don't watch anything Message a friend for recommendations Watch something from NF recommendations Finding #5 User behaviour (when they don't find anything to watch): Most don’t text a friend in the moment Understanding Netflix User Pain Points Proposing A Solution Measuring success Way forward ]
  • 23. Summary of Company & User Research More Titles + Imperfect Recommendations Increase in Discovery Time User Fatigue Drop Off/Switch • Netflix content is growing at a very fast pace. • Users spend significant amount of total avg. viewing time on discovery. • Netflix recommendations haven’t been perfect. • Friends are better trusted source for recommendations. Summary of the User’s Problem Understanding Netflix User Pain Points Proposing A Solution Measuring success Way forward
  • 24. My Conclusion Since machine based recommendations are great but not perfect, yet, user fatigue is real. Understanding Netflix User Pain Points Proposing A Solution Measuring success Way forward
  • 26. Identifying the user fatigue point is crucial. Adding trusted human aspect - at the points of fatigue - to machine based recommendations can increase viewership and decrease drop offs. Understanding Netflix User Pain Points Proposing A Solution Measuring success Way forward
  • 27. Meet Neil - The Explorer Software Engineer, 32 Lives in Mountain View, CA A Netflix Daily User Loves watching & finding out about new shows & movies Also subscribes to Amazon Prime, Hulu, HBO On weekends, he enjoys hanging out with friends Pain Point: “I watch anything and everything. But my Netflix feed has too much content similar to what I watched but didn't like. My friends & I discuss interesting new content all the time that I don’t see on my feed. A fresh set of unbiased recommendations would be helpful at times when I’m just tired of scrolling.” Understanding Netflix User Pain Points Proposing A Solution Measuring success Way forward
  • 28. Introducing Understanding Netflix User Pain Points Proposing A Solution Measuring success Way forward
  • 29. Don’t Snooze Yet – A Netflix Original Understanding Netflix User Pain Points Proposing A Solution Measuring success Way forward
  • 30. How will it work? 2 Key Aspects: • Identifying unique fatigue point for every user. • Leverages on ‘The Social Effect’ at fatigue point. (Facebook is used as an example here) Understanding Netflix User Pain Points Proposing A Solution Measuring success Way forward
  • 31. Unique Fatigue Point Estimated per user based on machine learning algorithms built on data models that aggregates and captures signals* as: • Drop off time: Amount of time a user spends looking for shows before abandoning the platform without playing anything. • Viewing time per session:  How long did the user watch the show, and did he interrupt to go back to browsing. • Discovery time per session :  Amount of time user spends to browse shows before watching something. • Content Watched : What did the user end up watching, something we recommended, something random and new, or shows he’s watched already but repeats frequently. *Based on an assumption that Netflix already captures the above usage logs Understanding Netflix User Pain Points Proposing A Solution Measuring success Way forward
  • 32. Link to Feauture Demo The Social Effect
  • 34. Measuring success Discovery Time Viewing Time Drop-off Rate Understanding Netflix User Pain Points Proposing A Solution Measuring success & Launch Way forward At Product Level
  • 35. Fatigue Detection Don’t Snooze Yet Screen Social Media Plugin Measuring success At Feature Level Activation Retention My List How many times does Don’t Snooze Yet screen show up for Neil per viewing hour? Did Neil drop off before Don’t Snooze Yet screen showed up? How many times does Neil play from Don’t Snooze Yet screen? How many times does Neil play from what a followed friend watched? Has Neil integrated his social media to Netflix? Does he add friends’ shows to his own list? Is he adding more shows/movies to his lists? Is adding categories? Is Neil interacting with Don’t Snooze Yet Screen? Understanding Netflix User Pain Points Proposing A Solution Measuring success & Launch Way forward
  • 36. Launch Understanding Netflix User Pain Points Proposing A Solution Measuring success & Launch Way forward Beta Test MVP Phase 1: Internal Test with Netflix employees Phase 2: Users who have agreed to Participate in Tests. US based 100k users (standard Netflix testing sample size) Don’t Snooze Yet Screen on TV only Social Plugin enabled across all devices
  • 37. Share your Netflix stories with your friends now! Expect lots of memes! • Social Media • Push Notifications • In-app / Web notifications • Blog • Email Go To Market Understanding Netflix User Pain Points Proposing A Solution Measuring success & Launch Way forward Across all devices
  • 38. Don’t Snooze Yet! Share your Netflix stories now! Understanding Netflix User Pain Points Proposing A Solution Measuring success & Launch Way forward 3 of your friends are watching Narcos Mexico • Social Media • Push Notifications • In-app / Web notifications • Blog • Email Go To Market
  • 39. See what your friends are watching now. Add their shows to your list. And Watch Better. See what your friends are watching now. Create your own recommendations. Watch better. Understanding Netflix User Pain Points Proposing A Solution Measuring success & Launch Way forward • Social Media • Push Notifications • In-app / Web notifications • Blog • Email Go To Market
  • 40. Don’t Snooze Yet! Share your Netflix stories now! Adding recommendations from trusted friends to the Netflix experience. Understanding Netflix User Pain Points Proposing A Solution Measuring success & Launch Way forward • Social Media • Push Notifications • In-app / Web notifications • Blog • Email Go To Market
  • 41. Don’t Snooze Yet! Share your Netflix stories now! 3 of your Friends Have Been Watching Narcos Mexico, Neil! Understanding Netflix User Pain Points Proposing A Solution Measuring success & Launch Way forward • Social Media • Push Notifications • In-app / Web notifications • Blog • Email Go To Market
  • 42. Estimated Roadmap Phase 1 (MVP): Don’t Snooze Yet Product Design/UX Engineering Sprint 1 Wireframe Prototype: 1. DSY Screen 2. User Profiles, Social Plugin 3. My List Update Sprint 2 Fatigue Identification / Data Model Sprint 3 Back End/Server-side Implementation UI Development for all screens Integration Sprint 4 QA Launch User Testing; User Feedback Coordination with Marketing to prep for announcement Firefighting Across Teams Test! Understanding Netflix User Pain Points Proposing A Solution Measuring success & Launch Way forward Design Validation Beta Testing
  • 44. Phase 2 Main Feed Further down the road: Short Term Understanding Netflix User Pain Points Proposing A Solution Measuring success & Launch Way forward
  • 45. • Making fatigue point identification 100% accurate. • Training data for better machine learning based recommendations model. • Capturing data on kind of content users prefer / don’t prefer to watch. Other Benefits Understanding Netflix User Pain Points Proposing A Solution Measuring success & Launch Way forward
  • 46. Further down the road: Long Term Phase 1 Phase 2 Phase 3 Don't Snooze Yet Main Feed Pattern Based Recommendations Neil likes watching Netflix when he is getting dinner. He usually picks something out from his Dinner Time shows. Recommend something from there to him on his main feed. Right recommendations to the right user at the right time. Understanding Netflix User Pain Points Proposing A Solution Measuring success & Launch Way forward
  • 47. "I think about it really as us winning time away, entertainment time, from other activities. Instead of doing Xbox or Fortnite or YouTube or HBO or a long list, we want to win and provide a better experience.” - Reed Hastings
  • 49. Hi, I am Sneha Die-Hard Netflix Fan. A Product Enthusiast About me sul.sneha@gmail.com

Editor's Notes

  1. I’m going to talk to you about our beloved company - Netflix. Imagine this.
  2. You come back home from a long day at work. Food is ready. Netflix is on.
  3. You start browsing. You scroll through different titles. You’re on your list. 5 min. 10 min. 15 min. Food is over.
  4. So let’s jump right in.
  5. Netflix wants to: entertain everyone and make everyone smile.
  6. Shareholders want: subscribers and more subscribers.
  7. But we also care about: best content for everyone, beautiful CX with accurate recommendations leading to happy, growing customers.
  8. For the context of this presentation we will look at The time people watch shows on Netflix The time between searching and playing a show on Netflix And the number of times people leave the platform
  9. Netflix takes up 10% of the 1B hours of TV viewership in the US - about 1.65 hours per day.
  10. There is too much content on Netflix. Growing at a very fast pace.
  11. A lot of opportunity to grow.
  12. I decided to segment our users into these 3 buckets according to time they spend on Netflix.
  13. With this in mind, I conducted a survey some in person interviews with current users. Focusing on the US market to start with. I found 5 things.
  14. When asked what they do when they don’t find anything on Netflix to watch, most said they switch or quit.
  15. Very few text friends.
  16. Also, note. A lot of users are on 1 other platform at least.
  17. With these findings in mind, I have come to the conclusion that…
  18. A New Feature which we’ll call ‘Dont Snooze Yet’
  19. So we are back to the browsing screen, 15 min in - fatigue is setting in. And then this shows up. A handpicked list of few shows that is fed in from your own several watch lists and from what your trusted circle of friends have been watching recently. When you don’t know what to watch, this will be that friend you could but won’t text. Let’s take a step back and see how this would work. We’ll start with revamping an understated feature on Netflix - “My List” You can now create several of your own lists - according to your mood, time you watch certain shows etc so that a specific kinds of shows and movies are easily accessible when you are looking for them. When you add shows to your list, Netflix would aut-tag the shows, giving the lists their identity of genre so we can show you recommendations of lists of other people who watch similar genre. You would be able to plug in to your social profile and follow your friends or even discover new people with similar tastes. You will be able to see what they are watching, there lists and people they follow. Lastly, you could set your list or what you are currently watching to public or private mode also. That is the first look of our brand new feature: Don’t Snooze Yet.
  20. This aims to directly impact the discovery time and increase viewing time, decreasing the drop off rates. Better recommendations (with the ML + the human component) would lead to increased effective viewing time.
  21. We will roll out the feature to our users and leverage on our strong social media presence.
  22. Push Notifications.
  23. Reach out via email too.
  24. I am estimating about 8 weeks from start to launch for the MVP and will be working across Product, Design and Engineering.
  25. If the MVP is a success, we will roll the feature out on the main feed itself as a part of phase 2.
  26. The long term idea is to: Use user usage and preference data to train our ML models better and make better content decisions and offer pattern based recommendations.
  27. To conclude, this will be in line with our larger company vision and goal.
  28. I’m Sneha. I am here to solve this problem.