by Sergii Shelpuk

1. Machine Learning
The Cutting Edge
Sergii Shelpuk
Director, Data Science
SoftServe, Inc.
sshel@softserveinc.com

2. Classification Problem
Recognize what is a bike and what is a moon

3. Classification Problem
Classifier
©A. Ng

4. Classification Problem
pixel
intensity

5. Classification Problem
Raw data does not represent the picture well.
You need some smart features
contains wheels
contains seas

6. Feature Extraction
Classifier
Featureextractor
©A. Ng

7. Feature Extraction
Can we do better?

8. Neural Networks
a
a
a
a
a
a
a
a
a
a
a
a
a
a
features
bike
moon

9. Neural Networks

10. Neural Networks
aX
a0
a1
a2
w0
w1
w2
Activation function:
aX = f(a0, a1, a2, w0, w1, w2)
Example (logistic):
aX = 1 / (1 + e-(a0*w0+a1*w1+a2*w2))

11. Autoencoder

12. Autoencoder
© H. Lee et al.

13. Autoencoder
© Q Le et al.

14. Deep Learning Neural Network
Pre-trained as Autoencoder
Typical classification
neural network
Moon

15. Deep Learning Neural NetworkVideoText/NLPImages
©A. Ng

16. Deep Learning Neural Network
Hints and Tips
 Using unlabeled data
 Avoiding overfitting
 Computational efficiency

17. Using Unlabeled Data
wheels
handlebar

18. Avoiding Overfitting
Sparsity constraint limits variance of autoencoder

19. Avoiding Overfitting
Dropout ensures generalization of the neural network

20. Computational Efficiency
 Thousands of cores
 Base Clock: 300-900 MHz
 Memory: 2-6 Gb
 Performance: up to 3.5 Tflops
 Instruction-level parallelism
 Shared memory
 Up to 4 devices in cluster
GPU computing provides cheapest computational power

21. Feature Learning: MNIST
Data:
Features:

22. Feature Learning: Galaxy Zoo
Data: Features:

23. Thank you!