This talk is about how we applied deep learning techinques to achieve state-of-the-art results in various NLP tasks like sentiment analysis and aspect identification, and how we deployed these models at Flipkart
19. FEATURE ENGINEERING
Functions which transform input (raw) data into a
feature space
Discriminative – for decision boundary
Feature engineering is painful
Deep Neural Networks: Identify the features
automatically
26. PROBLEMS WITH STATISTICAL
MODELS
Word ordering information lost
Data sparsity
Words as atomic symbols
Very hard to find higher level features
Features other than BOW
27. HOW TO ENCODE THE
MEANING OF A WORD?
Wordnet: Dictionary of synonyms
Synonyms: Adept, expert, good, practiced,
proficient, skillful
34. WORD EMBEDDING:
VISUALIZATIONS
Trained in a completely unsupervised way
Reduce data sparsity
Semantic Hashing
Appear to carry semantic information
about the words
Freely available for Out of Box usage
35. COMPOSITIONALITY
How do we go beyond words (sentences and
paragraphs)?
This turns out to be a very hard problem
Simple Approaches
Word Vector Averaging
Weighted Word Vector Averaging
37. CONVOLUTIONAL NEURAL
NETWORKS
Excellent feature extractors in image
Features are detected regardless of position in
image
NLP Almost from Scratch: Collobert et al 2011
First applied CNN for NLP
52. DRAWBACKS & LEARNINGS
Computationally Expensive
How to scale training?
How to scale prediction?
Libraries for Deep Learning
Theano
PyLearn2
Torch
53. “I THINK YOU SHOULD BE MORE EXPLICIT HERE IN STEP TWO”
55. BEYOND TEXT CLASSIFICATION
Text Classification covers a lot of NLP
problems (or problems can be reduced to it)
Word Embedding
Unsupervised Learning
Sequence Learning
RNN, LSTM
Information Extraction
Personalization….
Very hard problem for computers
Science of deriving meaning from Natural Language
Still, not enough good systems in production
Information Extraction
Personalization….
Loosely inspired by what (little) we know about the biological brain
Why image is hard?
Information Extraction
Personalization….
Information Extraction
Personalization….
Real life: 1000sof D space
Real life: 1000sof D space
Information Extraction
Personalization….
Elaborate more on pain of feature engineeing
Hundreds of thousands of features in real life
Information Extraction
Personalization….
Information Extraction
Personalization….
Put unsup chart
How to solve classification problems and getting semantic representations of Natural Language using DL?
Revise
Information Extraction
Personalization….
Bigram trigram
Manual feature engineering disadvantages – not generic
POS Tags
Brown clusters
Negation
Manually created lexicons ….
Mention LSA
Cat and dog have lot of semantic similarity compared to say cat and ambulance