Soumettre la recherche
Mettre en ligne
Deep learning: what? how? why? How to win a Kaggle competition
•
Télécharger en tant que PPTX, PDF
•
4 j'aime
•
896 vues
3
317070
Suivre
Deep learning: what? how? why? How to win a Kaggle competition
Lire moins
Lire la suite
Technologie
Signaler
Partager
Signaler
Partager
1 sur 75
Télécharger maintenant
Recommandé
Introduction to deep learning in python and Matlab
Introduction to deep learning in python and Matlab
Imry Kissos
LSTM Tutorial
LSTM Tutorial
Ralph Schlosser
Python for Image Understanding: Deep Learning with Convolutional Neural Nets
Python for Image Understanding: Deep Learning with Convolutional Neural Nets
Roelof Pieters
[2A4]DeepLearningAtNAVER
[2A4]DeepLearningAtNAVER
NAVER D2
Language translation with Deep Learning (RNN) with TensorFlow
Language translation with Deep Learning (RNN) with TensorFlow
S N
Intel Nervana Artificial Intelligence Meetup 11/30/16
Intel Nervana Artificial Intelligence Meetup 11/30/16
Intel Nervana
DIY Deep Learning with Caffe Workshop
DIY Deep Learning with Caffe Workshop
odsc
Deep Learning - Convolutional Neural Networks
Deep Learning - Convolutional Neural Networks
Christian Perone
Recommandé
Introduction to deep learning in python and Matlab
Introduction to deep learning in python and Matlab
Imry Kissos
LSTM Tutorial
LSTM Tutorial
Ralph Schlosser
Python for Image Understanding: Deep Learning with Convolutional Neural Nets
Python for Image Understanding: Deep Learning with Convolutional Neural Nets
Roelof Pieters
[2A4]DeepLearningAtNAVER
[2A4]DeepLearningAtNAVER
NAVER D2
Language translation with Deep Learning (RNN) with TensorFlow
Language translation with Deep Learning (RNN) with TensorFlow
S N
Intel Nervana Artificial Intelligence Meetup 11/30/16
Intel Nervana Artificial Intelligence Meetup 11/30/16
Intel Nervana
DIY Deep Learning with Caffe Workshop
DIY Deep Learning with Caffe Workshop
odsc
Deep Learning - Convolutional Neural Networks
Deep Learning - Convolutional Neural Networks
Christian Perone
Deep Learning in Python with Tensorflow for Finance
Deep Learning in Python with Tensorflow for Finance
Ben Ball
Synthetic dialogue generation with Deep Learning
Synthetic dialogue generation with Deep Learning
S N
Deep Learning through Examples
Deep Learning through Examples
Sri Ambati
Introduction to deep learning @ Startup.ML by Andres Rodriguez
Introduction to deep learning @ Startup.ML by Andres Rodriguez
Intel Nervana
Modeling Electronic Health Records with Recurrent Neural Networks
Modeling Electronic Health Records with Recurrent Neural Networks
Josh Patterson
Electricity price forecasting with Recurrent Neural Networks
Electricity price forecasting with Recurrent Neural Networks
Taegyun Jeon
Startup.Ml: Using neon for NLP and Localization Applications
Startup.Ml: Using neon for NLP and Localization Applications
Intel Nervana
Object Detection (D2L5 Insight@DCU Machine Learning Workshop 2017)
Object Detection (D2L5 Insight@DCU Machine Learning Workshop 2017)
Universitat Politècnica de Catalunya
A Platform for Accelerating Machine Learning Applications
A Platform for Accelerating Machine Learning Applications
NVIDIA Taiwan
Urs Köster - Convolutional and Recurrent Neural Networks
Urs Köster - Convolutional and Recurrent Neural Networks
Intel Nervana
Towards Machine Comprehension of Spoken Content
Towards Machine Comprehension of Spoken Content
NVIDIA Taiwan
Caffe framework tutorial2
Caffe framework tutorial2
Park Chunduck
Deep Learning Cases: Text and Image Processing
Deep Learning Cases: Text and Image Processing
Grigory Sapunov
Distributed implementation of a lstm on spark and tensorflow
Distributed implementation of a lstm on spark and tensorflow
Emanuel Di Nardo
Deep learning in Computer Vision
Deep learning in Computer Vision
David Dao
Anil Thomas - Object recognition
Anil Thomas - Object recognition
Intel Nervana
[DSC 2016] 系列活動:李宏毅 / 一天搞懂深度學習
[DSC 2016] 系列活動:李宏毅 / 一天搞懂深度學習
台灣資料科學年會
Introduction to Deep Learning with Python
Introduction to Deep Learning with Python
indico data
Software Frameworks for Deep Learning (D1L7 2017 UPC Deep Learning for Comput...
Software Frameworks for Deep Learning (D1L7 2017 UPC Deep Learning for Comput...
Universitat Politècnica de Catalunya
[SNU Computer Vision Course Project] Image Style Recognition
[SNU Computer Vision Course Project] Image Style Recognition
Hunjae Jung
Deep Learning in a nutshell
Deep Learning in a nutshell
HopeBay Technologies, Inc.
Wsdm17 value-at-risk-bidding
Wsdm17 value-at-risk-bidding
Jun Wang
Contenu connexe
Tendances
Deep Learning in Python with Tensorflow for Finance
Deep Learning in Python with Tensorflow for Finance
Ben Ball
Synthetic dialogue generation with Deep Learning
Synthetic dialogue generation with Deep Learning
S N
Deep Learning through Examples
Deep Learning through Examples
Sri Ambati
Introduction to deep learning @ Startup.ML by Andres Rodriguez
Introduction to deep learning @ Startup.ML by Andres Rodriguez
Intel Nervana
Modeling Electronic Health Records with Recurrent Neural Networks
Modeling Electronic Health Records with Recurrent Neural Networks
Josh Patterson
Electricity price forecasting with Recurrent Neural Networks
Electricity price forecasting with Recurrent Neural Networks
Taegyun Jeon
Startup.Ml: Using neon for NLP and Localization Applications
Startup.Ml: Using neon for NLP and Localization Applications
Intel Nervana
Object Detection (D2L5 Insight@DCU Machine Learning Workshop 2017)
Object Detection (D2L5 Insight@DCU Machine Learning Workshop 2017)
Universitat Politècnica de Catalunya
A Platform for Accelerating Machine Learning Applications
A Platform for Accelerating Machine Learning Applications
NVIDIA Taiwan
Urs Köster - Convolutional and Recurrent Neural Networks
Urs Köster - Convolutional and Recurrent Neural Networks
Intel Nervana
Towards Machine Comprehension of Spoken Content
Towards Machine Comprehension of Spoken Content
NVIDIA Taiwan
Caffe framework tutorial2
Caffe framework tutorial2
Park Chunduck
Deep Learning Cases: Text and Image Processing
Deep Learning Cases: Text and Image Processing
Grigory Sapunov
Distributed implementation of a lstm on spark and tensorflow
Distributed implementation of a lstm on spark and tensorflow
Emanuel Di Nardo
Deep learning in Computer Vision
Deep learning in Computer Vision
David Dao
Anil Thomas - Object recognition
Anil Thomas - Object recognition
Intel Nervana
[DSC 2016] 系列活動:李宏毅 / 一天搞懂深度學習
[DSC 2016] 系列活動:李宏毅 / 一天搞懂深度學習
台灣資料科學年會
Introduction to Deep Learning with Python
Introduction to Deep Learning with Python
indico data
Software Frameworks for Deep Learning (D1L7 2017 UPC Deep Learning for Comput...
Software Frameworks for Deep Learning (D1L7 2017 UPC Deep Learning for Comput...
Universitat Politècnica de Catalunya
[SNU Computer Vision Course Project] Image Style Recognition
[SNU Computer Vision Course Project] Image Style Recognition
Hunjae Jung
Tendances
(20)
Deep Learning in Python with Tensorflow for Finance
Deep Learning in Python with Tensorflow for Finance
Synthetic dialogue generation with Deep Learning
Synthetic dialogue generation with Deep Learning
Deep Learning through Examples
Deep Learning through Examples
Introduction to deep learning @ Startup.ML by Andres Rodriguez
Introduction to deep learning @ Startup.ML by Andres Rodriguez
Modeling Electronic Health Records with Recurrent Neural Networks
Modeling Electronic Health Records with Recurrent Neural Networks
Electricity price forecasting with Recurrent Neural Networks
Electricity price forecasting with Recurrent Neural Networks
Startup.Ml: Using neon for NLP and Localization Applications
Startup.Ml: Using neon for NLP and Localization Applications
Object Detection (D2L5 Insight@DCU Machine Learning Workshop 2017)
Object Detection (D2L5 Insight@DCU Machine Learning Workshop 2017)
A Platform for Accelerating Machine Learning Applications
A Platform for Accelerating Machine Learning Applications
Urs Köster - Convolutional and Recurrent Neural Networks
Urs Köster - Convolutional and Recurrent Neural Networks
Towards Machine Comprehension of Spoken Content
Towards Machine Comprehension of Spoken Content
Caffe framework tutorial2
Caffe framework tutorial2
Deep Learning Cases: Text and Image Processing
Deep Learning Cases: Text and Image Processing
Distributed implementation of a lstm on spark and tensorflow
Distributed implementation of a lstm on spark and tensorflow
Deep learning in Computer Vision
Deep learning in Computer Vision
Anil Thomas - Object recognition
Anil Thomas - Object recognition
[DSC 2016] 系列活動:李宏毅 / 一天搞懂深度學習
[DSC 2016] 系列活動:李宏毅 / 一天搞懂深度學習
Introduction to Deep Learning with Python
Introduction to Deep Learning with Python
Software Frameworks for Deep Learning (D1L7 2017 UPC Deep Learning for Comput...
Software Frameworks for Deep Learning (D1L7 2017 UPC Deep Learning for Comput...
[SNU Computer Vision Course Project] Image Style Recognition
[SNU Computer Vision Course Project] Image Style Recognition
En vedette
Deep Learning in a nutshell
Deep Learning in a nutshell
HopeBay Technologies, Inc.
Wsdm17 value-at-risk-bidding
Wsdm17 value-at-risk-bidding
Jun Wang
MDEC Data Matters Series: machine learning and Deep Learning, A Primer
MDEC Data Matters Series: machine learning and Deep Learning, A Primer
Poo Kuan Hoong
Introduction to deep learning
Introduction to deep learning
Zeynep Su Kurultay
EmadYacoubPresentation
EmadYacoubPresentation
PhD. Emad Yacoub
Ready Reference: World Factbook Guide
Ready Reference: World Factbook Guide
Kayla Kuffel
Indjija град који волим
Indjija град који волим
jelenavesna
Transcript three pages
Transcript three pages
Rod Wise
MASTER RAD
MASTER RAD
Stefan Savic
Impossible Sale
Impossible Sale
Amy Wilbur
Articulacion
Articulacion
Mel D
Resume Geetha
Resume Geetha
Geethanjali R
En vedette
(12)
Deep Learning in a nutshell
Deep Learning in a nutshell
Wsdm17 value-at-risk-bidding
Wsdm17 value-at-risk-bidding
MDEC Data Matters Series: machine learning and Deep Learning, A Primer
MDEC Data Matters Series: machine learning and Deep Learning, A Primer
Introduction to deep learning
Introduction to deep learning
EmadYacoubPresentation
EmadYacoubPresentation
Ready Reference: World Factbook Guide
Ready Reference: World Factbook Guide
Indjija град који волим
Indjija град који волим
Transcript three pages
Transcript three pages
MASTER RAD
MASTER RAD
Impossible Sale
Impossible Sale
Articulacion
Articulacion
Resume Geetha
Resume Geetha
Similaire à Deep learning: what? how? why? How to win a Kaggle competition
Deep learning to the rescue - solving long standing problems of recommender ...
Deep learning to the rescue - solving long standing problems of recommender ...
Balázs Hidasi
08 neural networks
08 neural networks
ankit_ppt
Artificial neural networks introduction
Artificial neural networks introduction
SungminYou
Deep learning with TensorFlow
Deep learning with TensorFlow
Barbara Fusinska
Separating Hype from Reality in Deep Learning with Sameer Farooqui
Separating Hype from Reality in Deep Learning with Sameer Farooqui
Databricks
Troubleshooting Deep Neural Networks - Full Stack Deep Learning
Troubleshooting Deep Neural Networks - Full Stack Deep Learning
Sergey Karayev
Think Machine Learning with Scikit-Learn (Python)
Think Machine Learning with Scikit-Learn (Python)
Chetan Khatri
Smallsat 2021
Smallsat 2021
klepsydratechnologie
Getting Started with Machine Learning
Getting Started with Machine Learning
Humberto Marchezi
machine learning in the age of big data: new approaches and business applicat...
machine learning in the age of big data: new approaches and business applicat...
Armando Vieira
Lunch session: Quantum Computing
Lunch session: Quantum Computing
Rolf Huisman
gan.pdf
gan.pdf
Dr.rukmani Devi
Deep learning from a novice perspective
Deep learning from a novice perspective
Anirban Santara
Understanding Basics of Machine Learning
Understanding Basics of Machine Learning
Pranav Ainavolu
Deep learning - a primer
Deep learning - a primer
Uwe Friedrichsen
Deep learning - a primer
Deep learning - a primer
Shirin Elsinghorst
InfoGAN and Generative Adversarial Networks
InfoGAN and Generative Adversarial Networks
Zak Jost
DeepLearningLecture.pptx
DeepLearningLecture.pptx
ssuserf07225
Neural network basic and introduction of Deep learning
Neural network basic and introduction of Deep learning
Tapas Majumdar
Angular and Deep Learning
Angular and Deep Learning
Oswald Campesato
Similaire à Deep learning: what? how? why? How to win a Kaggle competition
(20)
Deep learning to the rescue - solving long standing problems of recommender ...
Deep learning to the rescue - solving long standing problems of recommender ...
08 neural networks
08 neural networks
Artificial neural networks introduction
Artificial neural networks introduction
Deep learning with TensorFlow
Deep learning with TensorFlow
Separating Hype from Reality in Deep Learning with Sameer Farooqui
Separating Hype from Reality in Deep Learning with Sameer Farooqui
Troubleshooting Deep Neural Networks - Full Stack Deep Learning
Troubleshooting Deep Neural Networks - Full Stack Deep Learning
Think Machine Learning with Scikit-Learn (Python)
Think Machine Learning with Scikit-Learn (Python)
Smallsat 2021
Smallsat 2021
Getting Started with Machine Learning
Getting Started with Machine Learning
machine learning in the age of big data: new approaches and business applicat...
machine learning in the age of big data: new approaches and business applicat...
Lunch session: Quantum Computing
Lunch session: Quantum Computing
gan.pdf
gan.pdf
Deep learning from a novice perspective
Deep learning from a novice perspective
Understanding Basics of Machine Learning
Understanding Basics of Machine Learning
Deep learning - a primer
Deep learning - a primer
Deep learning - a primer
Deep learning - a primer
InfoGAN and Generative Adversarial Networks
InfoGAN and Generative Adversarial Networks
DeepLearningLecture.pptx
DeepLearningLecture.pptx
Neural network basic and introduction of Deep learning
Neural network basic and introduction of Deep learning
Angular and Deep Learning
Angular and Deep Learning
Dernier
Connect Wave/ connectwave Pitch Deck Presentation
Connect Wave/ connectwave Pitch Deck Presentation
Slibray Presentation
SAP Build Work Zone - Overview L2-L3.pptx
SAP Build Work Zone - Overview L2-L3.pptx
NavinnSomaal
Advanced Computer Architecture – An Introduction
Advanced Computer Architecture – An Introduction
Dilum Bandara
DSPy a system for AI to Write Prompts and Do Fine Tuning
DSPy a system for AI to Write Prompts and Do Fine Tuning
Lars Bell
TeamStation AI System Report LATAM IT Salaries 2024
TeamStation AI System Report LATAM IT Salaries 2024
Lonnie McRorey
Leverage Zilliz Serverless - Up to 50X Saving for Your Vector Storage Cost
Leverage Zilliz Serverless - Up to 50X Saving for Your Vector Storage Cost
Zilliz
"Debugging python applications inside k8s environment", Andrii Soldatenko
"Debugging python applications inside k8s environment", Andrii Soldatenko
Fwdays
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
BookNet Canada
Streamlining Python Development: A Guide to a Modern Project Setup
Streamlining Python Development: A Guide to a Modern Project Setup
Florian Wilhelm
H2O.ai CEO/Founder: Sri Ambati Keynote at Wells Fargo Day
H2O.ai CEO/Founder: Sri Ambati Keynote at Wells Fargo Day
Sri Ambati
Powerpoint exploring the locations used in television show Time Clash
Powerpoint exploring the locations used in television show Time Clash
charlottematthew16
TrustArc Webinar - How to Build Consumer Trust Through Data Privacy
TrustArc Webinar - How to Build Consumer Trust Through Data Privacy
TrustArc
Dev Dives: Streamline document processing with UiPath Studio Web
Dev Dives: Streamline document processing with UiPath Studio Web
UiPathCommunity
Take control of your SAP testing with UiPath Test Suite
Take control of your SAP testing with UiPath Test Suite
DianaGray10
WordPress Websites for Engineers: Elevate Your Brand
WordPress Websites for Engineers: Elevate Your Brand
gvaughan
SIP trunking in Janus @ Kamailio World 2024
SIP trunking in Janus @ Kamailio World 2024
Lorenzo Miniero
The Ultimate Guide to Choosing WordPress Pros and Cons
The Ultimate Guide to Choosing WordPress Pros and Cons
Pixlogix Infotech
Hyperautomation and AI/ML: A Strategy for Digital Transformation Success.pdf
Hyperautomation and AI/ML: A Strategy for Digital Transformation Success.pdf
Precisely
Nell’iperspazio con Rocket: il Framework Web di Rust!
Nell’iperspazio con Rocket: il Framework Web di Rust!
Commit University
Vertex AI Gemini Prompt Engineering Tips
Vertex AI Gemini Prompt Engineering Tips
Miki Katsuragi
Dernier
(20)
Connect Wave/ connectwave Pitch Deck Presentation
Connect Wave/ connectwave Pitch Deck Presentation
SAP Build Work Zone - Overview L2-L3.pptx
SAP Build Work Zone - Overview L2-L3.pptx
Advanced Computer Architecture – An Introduction
Advanced Computer Architecture – An Introduction
DSPy a system for AI to Write Prompts and Do Fine Tuning
DSPy a system for AI to Write Prompts and Do Fine Tuning
TeamStation AI System Report LATAM IT Salaries 2024
TeamStation AI System Report LATAM IT Salaries 2024
Leverage Zilliz Serverless - Up to 50X Saving for Your Vector Storage Cost
Leverage Zilliz Serverless - Up to 50X Saving for Your Vector Storage Cost
"Debugging python applications inside k8s environment", Andrii Soldatenko
"Debugging python applications inside k8s environment", Andrii Soldatenko
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
Streamlining Python Development: A Guide to a Modern Project Setup
Streamlining Python Development: A Guide to a Modern Project Setup
H2O.ai CEO/Founder: Sri Ambati Keynote at Wells Fargo Day
H2O.ai CEO/Founder: Sri Ambati Keynote at Wells Fargo Day
Powerpoint exploring the locations used in television show Time Clash
Powerpoint exploring the locations used in television show Time Clash
TrustArc Webinar - How to Build Consumer Trust Through Data Privacy
TrustArc Webinar - How to Build Consumer Trust Through Data Privacy
Dev Dives: Streamline document processing with UiPath Studio Web
Dev Dives: Streamline document processing with UiPath Studio Web
Take control of your SAP testing with UiPath Test Suite
Take control of your SAP testing with UiPath Test Suite
WordPress Websites for Engineers: Elevate Your Brand
WordPress Websites for Engineers: Elevate Your Brand
SIP trunking in Janus @ Kamailio World 2024
SIP trunking in Janus @ Kamailio World 2024
The Ultimate Guide to Choosing WordPress Pros and Cons
The Ultimate Guide to Choosing WordPress Pros and Cons
Hyperautomation and AI/ML: A Strategy for Digital Transformation Success.pdf
Hyperautomation and AI/ML: A Strategy for Digital Transformation Success.pdf
Nell’iperspazio con Rocket: il Framework Web di Rust!
Nell’iperspazio con Rocket: il Framework Web di Rust!
Vertex AI Gemini Prompt Engineering Tips
Vertex AI Gemini Prompt Engineering Tips
Deep learning: what? how? why? How to win a Kaggle competition
1.
2.
In about 45
minutes ZMUV
3.
In about 30
minutes Bigger & Deeper Is better
4.
In about 15
minutes
5.
Who works in Machine
learning?
6.
Who am I Jonas
Degrave Phd student UGent
7.
8.
Who are we? former Reservoir
Lab Data Science Lab IDLab
9.
10.
What do we
do? Machine learning Robotics Brain-inspired computing
11.
What did we
do? Totalling $160k in prizes
12.
Testimonials
13.
Neural networks in
5 minutes
14.
Neural networks in
5 minutes Input layer Hidden layers Output layer
15.
Gradient descent
16.
Backpropagation
17.
Deep learning Input layer Hidden
layer Output layer Hidden layer Hidden layer
18.
History Artificial Neural Net:
1949 Backpropagation: 1975 Deep Learning: 2012
19.
What used to
be the problem Input layer Hidden layer Output layer Hidden layer Hidden layer
20.
Vanishing gradients And all information is
gone
21.
For long, we didn’t
know
22.
GPU’s Rectifiers Maxpool Dropout They fight vanishing gradients!
23.
24.
State of the
art for all problems with spatially correlated data No more feature engineering!
25.
Old school bingo Boltzmann
Machines Energy Tanh or sigmoid activation Feature engineering Deep belief networks
26.
27.
Train set Make the
sets, make them well Validation set & Test set
28.
Choose your error
function Always optimize the error function where possible! Use error function for your problem
29.
Error Validation Training Time Validation Training Make Train & validation
curves
30.
Underfitting & overfitting
31.
Validation Training Time Underfitting & overfitting
32.
Regularize Bigger & Deeper “Larger
networks tend to work better. Make your network bigger and bigger until the accuracy stops increasing. Then regularize the hell out of it. Then make it bigger still.” – Yoshua Bengio
33.
My first architecture Start
with standard components: Conv-layers, dense layers, max-pooling, dropout
34.
Sparsity Make sure, that for
each sample, only a few parameters are used
35.
36.
Dropout
37.
Maxpool y x
38.
Rectifier (aka Relu)
39.
Convolution layers
40.
No bigger than
3x3 3x3 layer 9 parameters 3x3 receptive field 5x5 layer 25 parameters 5x5 receptive field 2 stacked 3x3 layer 19 parameters 5x5 receptive field
41.
Output function
42.
My first architecture ~
1 million parameters
43.
Let us optimize
44.
Gradient Descent Trainset Gradient
45.
Stochastic Gradient DescentTrainset Gradient Batch
46.
Adam’s update ruleTrainset Gradien tBatch Gradien t Gradien t Gradien t Weight Update Step
47.
Local minimum You want
generalization, not the global minimum on the train set!
48.
My first architecture ~
1 million parameters
49.
Initialization
50.
Weight matrices Random orthogonal
initialization With correct amplitude Most libraries provide this Does not lose information
51.
Output layer You have
prior information! Initialize with zeros!
52.
Bias Bias sets the
initial sparsity!
53.
Think about your initialization!
54.
My first architecture ~
1 million parameters
55.
Does Train on 1
sample it work ? Train on 2 samples
56.
Learning rate Overshooting Learn
too slow
57.
Learning rate
58.
Data preprocessing ZMUV your
data Zero mean Unit Variance
59.
60.
Input layer Hidden layer Output
layer Hidden layer Hidden layer Batch normalization batchnorm batchnorm batchnorm batchnorm
61.
62.
Data augmentation
63.
Unsupervised learning Learn on the
test set Pseudo-labeling Ladder networks
64.
Insert a priori
information into architecture
65.
Insert a priori
information
66.
It’s an art
67.
Regularize Bigger & Deeper “Rinse
and repeat” – Jonas Degrave
68.
Ensemble The average prediction
will always be better than the worst prediction.
69.
Ensemble Optimized On Validation set
70.
Submit
71.
Computing time Deadlines are
fixed End performance is proportional to number of iterations, NOT training time per model
72.
Major take-aways Everything has
a reason Don’t buy into hypes If it can’t be explained in 1 minute why it works, it probably isn’t working.
73.
Skip connections zeros
74.
Wide convolutions
Télécharger maintenant