Targeting Marketing Modelling Stack with Hierarchical, Randomized, and Propensity Score Models
•
0 j'aime•543 vues
Slides from my presentation to Analyze Boulder Meetup group on Feb 5, 2014.
Recommandé
Recommandé
Contenu connexe
En vedette
En vedette (6)
Similaire à Targeting Marketing Modelling Stack with Hierarchical, Randomized, and Propensity Score Models
Similaire à Targeting Marketing Modelling Stack with Hierarchical, Randomized, and Propensity Score Models (20)
Dernier
Dernier (20)
Targeting Marketing Modelling Stack with Hierarchical, Randomized, and Propensity Score Models
- 1. A Targeting Marketing Analytical Modelling Stack SCOTTY NELSON
- 2. Outline Business problem/context Modelling stack: Optimal learning Sparse data ROI measurement Future directions
- 3. Business Context Client is a loyalty program for small/medium sized businesses Members shop using a registered credit card at participating merchants and receive cash-back offers Targeted email campaigns send offers most relevant to the members in order to drive business to the merchants Available member/merchant transaction history is very sparse Currently modelling ~1.6 million members and ~1500 merchant offers
- 4. Business Context 2. Send targeted offers to members 1. Identify best offers for each member 5. Refine targeting models 3. Members incentivized to shop at merchants 4. Measure impact of campaign on actual member spend
- 5. Summary of Modelling Requirements Issue What We Need Proposed Solution Sparse Data Personalized member level merchant recommendations & price sensitivities Hierarchical models Learning Adjusts recommendations to reflect uncertainty (don’t put all your eggs in one basket) Randomized probability matching ROI Measurement $ impact of the campaign Weighted inverse propensity score models
- 6. Hierarchical Models REI Sports We want individual level merchant scores but we don’t have a complete set of data on any one member This blows up traditional regression models scores (there are more parameters than data) Hierarchical models solve this issue by “drawing strength” across similar members in a statistically valid way Has connections to kernel smoothing and collaborative filtering Neptune Member 1 Southern Sun Restaurants Zoe Ma Ma Prior REI Sports Neptune Member 2 Southern Sun Restaurants Zoe Ma Ma
- 7. Randomized Probability Matching Motivating example: Let’s say we have a slot machine with 5 different arms(A, B, C, D, E) We play D and it wins Does that mean we should always play D? Randomized probability matching (Scott, 2010) Samples strategies proportional to their posterior probability of being ‘the best’ Balances exploitation vs exploration of the (unknown) payoff distribution for the different strategies A B C D Source: “Optimal Learning”, Powell & Ryzhov E
- 8. ROI Measurement How to measure ROI? Run the campaign compare before/after sales? What about people who would have bought anyways? Dealing with selection bias: propensity score matching Solution: Create a synthetic control group by re-weighting observations according to a estimated probability of treatment This creates a balanced panel appropriate for comparing those who received the offer, and those who didn’t
- 9. Summary of Modelling Stack Issue Proposed Solution What It Gives Us Why it Works Sparse Data Hierarchical models Personalized member level merchant recommendations & price sensitivities Blends individual and pooled information in a statistically valid manner Learning Randomized Adjusts probability matching recommendations to reflect uncertainty (don’t put all your eggs in one basket) Balances exploitation of proven recommendations, with accumulation of more information on underperforming recommendations ROI Measurement Weighted inverse propensity score models Controls for confounding selection bias Outputs the $ impact of the campaign
- 10. Questions? Contact me at scotty@convergenceanalytics.co