Coming right from the Recommender Systems conference in San Francisco, I present some latest developments in the field of large scale recommendation engines and machine learning.

1. @torbenbrodt #recsys
Recommender Trends
ACM RecSys 2014
Silicon Valley USA
Torben Brodt
plista GmbH
inspired by ..Stammtisch
Nov 13th 2014

2. @torbenbrodt #recsys
Silicon Valley
Image by New Media at the University of Maine

3. @torbenbrodt #recsys
RecSys 2014 was ..
● 1 day workshop
● 3 day tech conference (see )
● 1 day conference

4. @torbenbrodt #recsys
biased with my experience
● Head of Data Engineering
● > 6y plista
○ News, advertising, real-time
● Open!
DevOps Math
Core

5. @torbenbrodt #recsys
Contents
1. Product
2. Algorithms
3. Metrics
4. Openness
5. Crazy Stuff
6. Missing

6. @torbenbrodt #recsys
Product, ”Data Driven Decisions”
“We take a proposal for an original production or
for a piece of content we’re going to buy and we
plug in all the data we can abou tit into our
models. We’re able to predict reach and hours
for that piece of content even before it exists
with reasonable precision in a way that helps us
to say, ‘this is worth funding’ or ‘that’s not worth
funding,’ ”
NEIL HUNT Netflix

7. @torbenbrodt #recsys
Product, “Search & Recommendation
should (not?) converge”
HECTOR GARCIA-MOLINA
Professor, Stanford University
DEBORA DONATO
StumbleUpon

8. @torbenbrodt #recsys
Product, “Use Human Experts”
ERIC COLSON
Stitch Fix
Humans send you customized
outfits. Machines suggest clothes
and judge stuff.

9. @torbenbrodt #recsys
Product, “Explain your knowledge”
● Xbox explains why their
recommendations are utile
● Cortana builds ML model of user and still
allows to change it
Build Trust!

10. @torbenbrodt #recsys
Product, “Care about Privacy”
once you lose your customer because of
privacy, you will never get him back
solutions
● store user history on client side
● ..

11. @torbenbrodt #recsys
Product, ”Allow User Interaction”
HECTOR GARCIA-MOLINA
Professor, Computer Science and Electrical Engineering
Departments of Stanford University

12. @torbenbrodt #recsys
Product, “active learning”
Why do vague
passive learning when you
can ask the user?
.. implicitly or explicitly
http://en.wikipedia.org/wiki/Active_learning_(machine_learning)
SMRITI BHAGAT
Technicolor

13. @torbenbrodt #recsys
Algorithms

14. @torbenbrodt #recsys
Algorithms, ”Matrix Factorization”
[...] faster by replacing inner product with
PCA trees
NOAM KOENIGSTEIN
Microsoft R&D

15. @torbenbrodt #recsys
Algorithms, “Ensembles”
● Multi Armed Bandits
● Ensemble Methods
● Global Optimization
https://github.com/Yelp/MOE

16. @torbenbrodt #recsys
Algorithms, “How does MOE work”
DR. SCOTT CLARK
Yelp
1. Build Gaussian Process (GP) with points
sampled so far
2. Optimize covariance hyperparameters of GP
3. Find point(s) of highest Expected Improvement
within parameter domain
4. Return optimal next best point(s) to sample
https://github.com/Yelp/MOE

17. @torbenbrodt #recsys
Algorithms, “Topic Modelling”
● LDA is standard
● datascience tasks
○ where to cut
○ how many topics
● where to use?
http://en.wikipedia.org/wiki/Topic_model

18. @torbenbrodt #recsys
Algorithms, “Content”
● Sense identifiers (int) instead of keywords
● Word sense disambiguation

19. @torbenbrodt #recsys
Metrics, “Stakeholders”
● Business Value
● Consumer Value
● Conflicting goals?
● Diversity?
NEIL HUNT Netflix

20. @torbenbrodt #recsys
Metrics, “Dwell Time”
● Client Side
implementation
● Yahoo ensures dwell-time
is comparable across
different context (device,
etc)
● it correlates to clicks, but
is more meaningful
XING YI
Yahoo Labs

21. @torbenbrodt #recsys
Metrics, “Increasing signals”
Get the full lifetime journey
● reservation
● rating
● billing / tipping
JEREMY SCHIFF
OpenTable

22. @torbenbrodt #recsys
Openness, “Software Side”
Companies share software
● credits to Twitter, Yelp, others
Finally Paper results can be reused (github)

23. @torbenbrodt #recsys
Openness, “Data Side”
Wikipedia, DBPedia, common crawl
Companies share Data & Challenges
● credits to Netflix, Tmall, Criteo

24. @torbenbrodt #recsys
Openness, “Connectivity”
Everything is possible!
To Me and to You
● Connect to Facebook
○ access open graph
● Get Fulltext without 10k servers
● Use Apache Mahout, Azure ML, etc

25. @torbenbrodt #recsys
Openness, “Connectivity”
● Give students the chance
to learn
● CoLaboratory Notebook
http://venturebeat.com/2014/08/08/google-whips-up-a-
chrome-app-to-let-data-scientists-work-together/

26. @torbenbrodt #recsys
Openness, “Connectivity”
● Azure Marketplace
allows to exchange
machine learning models
● RapidMiner makes
workflows reproducable
https://datamarket.azure.com/browse/data

27. @torbenbrodt #recsys
Crazy Stuff
Industry Sessions…
● Facebook News
● Shopkick
● Stumble Upon
● climate institute
● ...

28. @torbenbrodt #recsys
Crazy Stuff, “music genome project”
1 song = 450 musical
characteristics from
trained music analyst
ERIK M. SCHMIDT
Pandora

29. @torbenbrodt #recsys
Crazy Stuff, “LinkedIn A/B testing”
● XLNT Platform
● Key Component !
● Continuous Deployment
YA XU
LinkedIn
https://engineering.linkedin.com/ab-testing/xlnt-
platform-driving-ab-testing-linkedin

30. @torbenbrodt #recsys
Crazy Stuff, “Google Deep Learning”
● Application?
○ Pixels, Audio, Searches,
Translation
● Embeddings
● Language Models
● Scalability
JEFF DEAN
Google

31. @torbenbrodt #recsys
Missing? “Uncovered Topics”

32. @torbenbrodt #recsys
Missing, “Probabilistic Data Structures”
probabilistic counting, hyperLogLog, etc
http://research.neustar.biz/
https://streamdrill.com/

33. @torbenbrodt #recsys
Missing
Large Scale?
● Computational Costs
● Real-Time Recs

34. @torbenbrodt #recsys
Questions?
Torben Brodt
plista GmbH

35. @torbenbrodt #recsys
● hard to convince mgmt (?!)
● start measuring
example
● coupons 1/week might
decrease revenue
JEREMY SCHIFF
OpenTable
Metrics, “Long Term Satisfaction”

36. @torbenbrodt #recsys
Resume, ”we enhance services”
Large Size Companies cannot exist without
data science
● Netflix
● Zalando
● etc