ML practitioners and advocates are increasingly finding themselves becoming gatekeepers of the modern world. The models you create have power to get people arrested or vindicated, get loans approved or rejected, determine what interest rate should be charged for such loans, who should be shown to you in your long list of pursuits on your Tinder, what news do you read, who gets called for a job phone screen or even a college admission... the list goes on. My goal in this talk is to summarize the kinds of disparate outcomes that are caused by cargo cult machine learning, and recent academic efforts to address some of them.

1. Fairness in Machine Learning
Delip Rao

3. Metrics

4. Every ML practitioners dream scenario
Well defined eval objective
Lots of clean data
Rich data with lot of attributes

5. Incorporating Ethnicity improves Engagement
Metrics
But should you do it?

8. Report Link

11. link

12. Goldstein, the “computer expert”

13. Dramatic Changes in Machine Learning
Landscape

14. Rise of fast/cheap data collection, processing

15. Rise of popular, easy-to-use tools

16. Rise of Data Scientist Factories

18. Two Questions
Should everything that can be predicted, be predicted?
If you really have to predict, what should you be aware of?

19. “Catalog of Evils”
Dwork et al 2011

20. Sc
S
protected
class
population

21. Blatant Explicit Discrimination
Feature4231:Race=’Black’

22. Discrimination Based on Redundant Encoding
Feature4231:Race=’Black’
Features = {‘loc’, ‘income’, ..}
Polynomial kernel with degree 2
Feature6578:Loc=’EastOakland’^Income=’<10k’

24. Big Data
There is no
data like more
data.

25. Big Data
Classifier error rate
Number of training
examples in your data

26. Most ML Objective functions create
models accurate for the majority class at
the expense of the protected class

27. Cultural differences can throw a wrench in your
models

28. Look at Error Cases vs. Error Rates
Macros - Accuracy, RMSE, F1, etc
vs.
Individuals

29. Becoming Responsible Gatekeepers

30. We are pretty good at learning
function approximations today

31. Image Credit: Jason Eisner, the three cultures of ML, 2016
NNs &
Decision Trees

32. Learning methods that introduce fairness
Ways to characterize fairness
Need

33. How can we characterize fairness?
What does fairness even mean?
Group Fairness vs. Individual Fairness

34. How can we characterize fairness?
One way to characterize group fairness is to ensure both majority and the
protected population have similar outcomes.
or
P(FavorableOutcome | S) : P(FavorableOutcome | Sc) = 1 : 1

35. How can we characterize fairness?
One way to characterize group fairness is to ensure both majority and the
protected population have similar outcomes.
or
P(FavorableOutcome | S) : P(FavorableOutcome | Sc) = 1 : 1
often this is hard to achieve.
For example, for jobs, the EEOC specifies this ratio should be no less than 0.8 : 1
(aka 80% rule).

36. Characterizing Fairness of a black box classifier
One way: Is classifier outcome correlated with membership in
S?

37. Fairness as a constraint
Is classifier outcome correlated with membership in S?
Sensitive attributes
Decision function
Want

38. Fairness as a constraint
Constraint to be added:

39. Supervised Learning with Fairness Constraint
minimize
such that
Zafar et al, ICML 2015

40. “If we allowed a model to be
used for college admissions in
1870, we’d still have 0.7% of
women going to college.”
Recommended Reading

41. Reading List
There’s much material on fairness in data-driven decision/policy making from
literature in
- law
- sociology
- political science
- computer science/machine learning
- economics
(the machine learning literature is nascent, only around 2009 onwards)

42. Reading List (Fairness in ML)
Pedreschi, Dino, Salvatore Ruggieri, and Franco Turini. "Measuring Discrimination in Socially-Sensitive
Decision Records." SDM. 2009.
Kamiran, Faisal, and Toon Calders. "Classifying without discriminating."Computer, Control and
Communication, 2009. IC4 2009. 2nd International Conference on. IEEE, 2009.
Dwork, Cynthia, et al. "Fairness through awareness." Proceedings of the 3rd Innovations in
Theoretical Computer Science Conference. ACM, 2012
Romei, Andrea, and Salvatore Ruggieri. "A multidisciplinary survey on discrimination analysis."
The Knowledge Engineering Review 29.05 (2014)

43. Reading List (Fairness in ML)
Friedler, Sorelle, Carlos Scheidegger, and Suresh Venkatasubramanian. "Certifying and removing
disparate impact." CoRR (2014).
Barocas, Solon and Selbst, Andrew D., Big Data's Disparate Impact (August 14, 2015). California
Law Review, Vol. 104,
Zafar, Muhammad Bilal, et al. "Fairness Constraints: A Mechanism for Fair Classification." arXiv preprint
arXiv:1507.05259 (2015).
Zliobaite, Indre. "On the relation between accuracy and fairness in binary classification." arXiv preprint
arXiv:1505.05723 (2015).

44. Other resources
NSF’s “Big Data Innovation Hubs” were created in part to address these
challenges
http://www.nsf.gov/news/news_summ.jsp?cntn_id=136784
Stanford Law Review touches upon this topic regularly
http://www.stanfordlawreview.org/online/privacy-and-big-data
Fairness blog
http://fairness.haverford.edu
Academic: FATML workshops (NIPS 2014, ICML 2015)
www.fatml.org

45. Lessons
Discrimination is an emergent property of any learning algorithm
Watch out for discrimination (implicitly) encoded in features
Big Data can cause Big Problems
Watch out for the proportion of the protected classes
Always do error analysis with protected classes in mind
Notions of fairness are nascent at best. Involve as many people to improve
understanding.
There is no one best notion of fairness

46. questions
@deliprao / delip@joostware.com