모바일VR 사용자 인터페이스를 위한 데이터 기반 기계 학습 - 딥픽셀 이제훈 대표

V i s u a l I n t e l l i g e n c e C o m p a n y © 2017. Deepixel Inc. all rights reserved.
모바일 VR 사용자 인터페이스를
위한 데이터 기반 기계 학습
이제훈
2017 데이터 진흥주간 | 2017 데이터 그랜드 컨퍼런스 | 지능형 데이터 for AI | 2017년 11월 7일

2
© 2017. Deepixel Inc. all rights reserved.
Contents
Data-driven Machine Learning
I n t e l l i g e n ce , M a c h i n e Le a r n i n g | D e e p Le a r n i n g , B i g D at a
Mobile VR User Interface
P I X I E , Re i n v e n t e d 3 D N at u r a l I n t e r f a ce f o r M o b i le V R
Concluding Remarks
P18
P3
P29

Data-driven Machine Learning
Int elligence, M a chine Lea rning | Deep Lea rning, Big Dat a

4
Intelligence
•
• Turing test: A test of a machine's ability to exhibit intelligent behavior equivalent to, or
indistinguishable from, that of a human [Alan Turing, MIND, 1950]
• https://en.wikipedia.org/wiki/Turing_test

5
Intelligence
• How can we add intelligence to the machine?
Data
Algorithm/Rule
Output
Machine Machine
Algorithm/Rule
Data
Output
Machine LearningTraditional Programming
C

6
Machine Learning
• Field of computer science that gives computers the ability to learn without
being explicitly programmed (Arthur Samuel, 1959)
• Data-driven approach: Generalization and pattern discovery from training data
• Data is crucial to machine learning.
• Traditional programming
• Impossible to make a perfect rule
Too many rules
Too many factors influencing the rules
Obscure rules
• https://en.wikipedia.org/wiki/Machine_learning

7
© 2017. Deepixel Inc. all rights reserved.• https://machinelearningmastery.com/
A mind map of machine learning algorithms (by type)

8
Deep learning
• Large neural network with a cascade of multiple layers of nonlinear processing units
• Improvement of
• Algorithms: Unsupervised pre-training [G. Hinton, TCS, 2007], Dropout [G. Dhal, ICASSP, 2013], and so on.
• H/W : GPU
• Big data: Lots of datasets from the Internet, SNS, and so on.

9
Deep Learning X Big Data
Cat face, Google, 10M 200x200 captures from YouTube
Large scale unsupervised learning
[Quoc V. Le et al., ICML, 2012]
AlphaGo, Google DeepMind, ~30M moves, large
numbers of games against other instances of itself
Deep learning w/ supervised learning and reinforcement
learning [D. Silver et al., Nature, 2016]
• https://googleblog.blogspot.kr/2012/06/using-large-scale-brain-simulations-for.html
• http://www.nature.com/nature/journal/v529/n7587/index.html

10
Datasets for Machine Learning | Deep Learning
• https://en.wikipedia.org/wiki/List_of_datasets_for_machine_learning_research
• FERET (facial recognition
technology)
• CMU Pose, Illumination, and
Expression (PIE)
• YouTube Faces DB
• Yale Face Database
• TV Human Interaction Dataset
• Berkeley Multimodal Human
Action Database (MHAD)
• Microsoft Common Objects in
Context (COCO)
• ImageNet
• MNIST database
• Stanford Dogs Dataset
• Microsoft Sequential Image
Narrative Dataset (SIND)
• Berkeley 3-D Object Dataset
• Etc.

11
THE MNIST(Modified National Institute of Standards and Technology) DATABASE
Handwritten digits for handwriting and character recognition
60,000 images
• http://yann.lecun.com/exdb/mnist/

12
© 2017. Deepixel Inc. all rights reserved.• http://www.vision.caltech.edu/Image_Datasets/Caltech_10K_WebFaces/
Caltech Web Faces
Frontal face images for facial recognition
10,000 images

13
ImageNet
Labeled objects, SIFT features for object detection/recognition LSVRC
(Large Scale Visual Recognition Challenge)
14,197,122+ images (Y2010)
• http://image-net.org/index

14
Deep Learning w/ or w/o Big Data
• Deep learning is a powerful tool but, it requires lots of annotated data.
• Annotated data may be
• Expensive (human labor)
• Very difficult to collect
• Not publicly available
• How to effectively collect large scale data with little human labor?
• Data augmentation
• Synthetic data
• Data cleaning
• Etc.

15
Data Collection for Deep Learning
• End-to-End Learning of Deep Visual Representations for Image Retrieval [Gordo et al., IJCV, 2017]
To automatically clean datasets by pruning edges of matched images with the pairwise matching graph from public
datasets with noisy.
• Frankenstein: Learning Deep Face Representations using Small Data [Hu et al., TIP, 2017]
To generate very large training datasets of synthetic images by composition of real face images in a given dataset, which
swaps the facial components of different face images to generate a new face.

16
Data Collection for Deep Learning
• The SYNTHIA Dataset: A Large Collection of Synthetic Images for Semantic Segmentation of Urban Scenes.
[Ros et al., CVPR, 2016]
To render a virtual city created with the Unity development platform and automatically generate realistic synthetic
images with pixel-level annotations.
• Data-Augmentation for Reducing Dataset Bias in Person Re-identification [McLaughlin et al., AVSS, 2015]
Random modification including background simulation of the raw data to increase dataset diversity.
C

17
Deep Learning w/ or w/o Big Data (again)
• Deep learning is one of the best
methods in machine learning.
• Big data is crucial to deep learning.
• Data scarcity problem
• Alternative methods for collecting
data

Mobile VR User Interface
P I X I E , Reinvent ed 3D N at ura l Int erfa ce for M obile VR

19
• https://vr.google.com/
• https://www.vive.com/
Mobile VR PC/Console VR

20
Mobile VR Interface
• https://uxdesign.cc/

22
PIXIE Reinvented 3D Natural Interface for Mobile VR

23
Hand motion tracking in a VR/AR environment
• http://www.samsung.com/us/campaign_smarttv/
• https://www.microsoft.com/en-us/hololens/
Smart TV AR Headset

24
Our Approach
• Challenging Issues
• Difficult to collect the desired training dataset:
the back of the hand
designed shape of the hand for UX/UI
dataset with various colors and poses of the hand
• 3D pose estimation with no additional device other than a smartphone with a single camera
• Real-time operation ( 60ms) in a mobile environment
• Our Approaches
• Development of DEEP(Data Extraction Equipment and Program) to collect data
• -big-
• Algorithm optimization to reduce computational complexity

25
DEEP(Data Extraction Equipment & Program)

26
DEEP dataset
Hand and fingertip action videos and images for PIXIE
50+ video files
1h+ running time
20,000+ images

27
Algorithm
• https://en.wikipedia.org/wiki/Local_binary_patterns
• https://en.wikipedia.org/wiki/Support_vector_machine

28
C
https://www.youtube.com/watch?v=aCP6__kv1zM

29
Concluding Remarks
• Design suitable methods and choose the right machine learning
algorithm for your problem
according to
the size of datasets you have,
performance of available hardware, and
challenging issues of the given problem.
• Collect your dataset with a clever way
, if you choose a machine learning algorithm as your core solution.

모바일VR 사용자 인터페이스를 위한 데이터 기반 기계 학습 - 딥픽셀 이제훈 대표

Recommandé

Recommandé

Contenu connexe

Similaire à 모바일VR 사용자 인터페이스를 위한 데이터 기반 기계 학습 - 딥픽셀 이제훈 대표

Similaire à 모바일VR 사용자 인터페이스를 위한 데이터 기반 기계 학습 - 딥픽셀 이제훈 대표 (20)

Plus de eungjin cho

Plus de eungjin cho (20)

Dernier

Dernier (20)

모바일VR 사용자 인터페이스를 위한 데이터 기반 기계 학습 - 딥픽셀 이제훈 대표