Deep Qualia: Philosophy of Statistics, Deep Learning, and Blockchain
Deep learning: What is it, why is it important, and what do I need to know?
The aim of this talk is to discuss deep learning as an advanced computational method and its philosophical implications. Computing is a fundamental model by which we are understanding more about ourselves and the world. We think that reality is composed of patterns, which can be detected by machine learning methods.
Deep learning is a complexity optimization technique in which algorithms learn from data by modeling high-level abstractions and assigning probabilities to nodes as they characterize the system and make predictions. An important challenge in deep learning is that these methods work in certain domains (image, speech, and text recognition), but we do not have a good explanation for why, which impedes a wider application of these solutions.
Another recent advance in computational methods is blockchain technology which allows the secure transfer of assets and information, and the automated coordination of operations via a trackable remunerative ledger and smart contracts (automatically-executing Internet-based programs).
This talk looks at how deep learning technology, particularly as coupled with blockchain systems, might be used to produce a new kind of global computing platform. The goal is for blockchain deep learning systems to address higher-dimensional computing challenges that require learning and dynamic response in domains such as economics and financial risk, epidemiology, social modeling, public health (cancer, aging), dark matter, atomic reactions, network-modeling (transportation, energy, smart cities), artificial intelligence, and consciousness.
Scaling API-first – The story of a global engineering organization
Philosophy of Deep Learning
1. Melanie Swan
Philosophy & Economic Theory
New School for Social Research, NY NY
melanie@BlockchainStudies.org
Pfizer, New York NY, March 30, 2017
Slides: http://slideshare.net/LaBlogga
Philosophy of Deep Learning:
Deep Qualia, Statistics, and Blockchain
Image credit: Nvidia
2. 30 Mar 2017
Deep Learning
ASA P value misuse statement
1
Source: http://www.nature.com/news/statisticians-issue-warning-over-misuse-of-p-values-1.19503,
http://amstat.tandfonline.com/doi/abs/10.1080/00031305.2016.1154108
ASA principles to guide P value use
The P value alone cannot determine whether a
hypothesis is true or whether results are important
3. 30 Mar 2017
Deep Learning 2
Melanie Swan
Philosophy and Economic Theory, New School
for Social Research, New York NY
Founder, Institute for Blockchain Studies
Singularity University Instructor; Institute for Ethics and
Emerging Technology Affiliate Scholar; EDGE
Essayist; FQXi Advisor
Traditional Markets Background
Economics and Financial
Theory Leadership
New Economies research group
Source: http://www.melanieswan.com, http://blockchainstudies.org/NSNE.pdf, http://blockchainstudies.org/Metaphilosophy_CFP.pdf
https://www.facebook.com/groups/NewEconomies
4. 30 Mar 2017
Deep Learning
Deep Learning vocabulary
What do these terms mean?
Deep Learning, Machine Learning, Artificial Intelligence
Deep Belief Net
Perceptron, Artificial Neuron
MLP/RELU: Multilayer Perceptron
Artificial Neural Net
TensorFlow, Caffe, Theano, Torch, DL4J
Recurrent Neural Nets
Boltzmann Machine, Feedforward Neural Net
Open Source Deep Learning Frameworks
Google DeepDream, Google Brain, Google DeepMind
3
5. 30 Mar 2017
Deep Learning
Key take-aways
1. What is deep learning?
Advanced statistical method using logistic regression
Deep learning is a sub-field of machine learning and
artificial intelligence
2. Why is deep learning important?
Crucial method of algorithmic data manipulation
3. What do I need to know (as a data scientist)?
Awareness of new methods like deep learning needed to
keep pace with data growth
4
6. 30 Mar 2017
Deep Learning
Deep Learning and Data Science
5
Not optional: older algorithms cannot perform to
generate requisite insights
Source: http://blog.algorithmia.com/introduction-to-deep-learning-2016
7. 30 Mar 2017
Deep Learning
Agenda
Deep Learning Basics
Definition, operation, drawbacks
Implications of Deep Learning
Deep Learning and the Brain
Deep Learning Blockchain Networks
Philosophy of Deep Learning
6
Image Source: http://www.opennn.net
8. 30 Mar 2017
Deep Learning
Deep Learning Context
7
Source: Machine Learning Guide, 9. Deep Learning
9. 30 Mar 2017
Deep Learning
Deep Learning Definition
“machines that learn to represent the world” – Yann LeCun
Deep learning is a class of machine learning algorithms
that use a cascade of layers of processing units to
extract features from data
Each layer uses the output from the previous layer as input
Two kinds of learning algorithms
Supervised (classify labeled data)
Unsupervised (find patterns in unlabeled data)
Two phases: training (existing data) and test (new data)
8
Source: Wikiepdia, http://spectrum.ieee.org/automaton/robotics/artificial-intelligence/facebook-ai-director-yann-lecun-on-deep-
learning
10. 30 Mar 2017
Deep Learning
What is Learning? When algorithms detect a
system’s features or rules
9
Single-purpose AI: Deep Blue, 1997
Hard-coded rules
Multi-purpose AI structure: AlphaGo, 2016
Algorithm-detected rules, reusable template
Deep Learning machine
General purpose AI: Deep Qualia, 2xxx?
Novel situation problem-solving,
Algorithm edits/writes rules
Question-answering AI: Watson, 2011
Natural-language processing
Deep Learning prototype
11. 30 Mar 2017
Deep Learning
Deep Learning: what is the problem space?
10
Source: Yann LeCun, CVPR 2015 keynote (Computer Vision ), "What's wrong with Deep Learning" http://t.co/nPFlPZzMEJ
Level 1 – basic application areas
Image, text, speech recognition
Multi-factor recognition (label image with text)
Sentiment analysis
Level 2 – complex application areas
Autonomous driving
Disease diagnosis, tumor recognition, X-ray/MRI interpretation
Seismic analysis (earthquake, energy, oil and gas)
12. 30 Mar 2017
Deep Learning
Deep Learning Taxonomy
High-level fundamentals of machine learning
11
Source: Machine Learning Guide, 9. Deep Learning;
AI (artificial intelligence)
Machine learning Other methods
Supervised learning
(labeled data:
classification)
Unsupervised learning
(unlabeled data: pattern
recognition)
Reinforcement learning
Shallow learning
(1-2 layers)
Deep learning
(5-20 layers (expensive))
Recurrent nets (text, speech)
Convolutional nets (images)
Neural Nets (NN) Other methods
Bayesian inference
Support Vector Machines
Decision trees
K-means clustering
K-nearest neighbor
13. 30 Mar 2017
Deep Learning
What is the problem? Computer Vision
(and speech and text recognition)
12
Source: Quoc Le, https://arxiv.org/abs/1112.6209; Yann LeCun, NIPS 2016,
https://drive.google.com/file/d/0BxKBnD5y2M8NREZod0tVdW5FLTQ/view
Marv Minsky, 1966
“summer project”
Jeff Hawkins, 2004,
Hierarchical Temporal
Memory (HTM)
Quoc Le, 2011, Google
Brain cat recognition
Yann LeCun, 2016,
Predictive Learning,
Convolutional net for driving
14. 30 Mar 2017
Deep Learning
Image Recognition: Basic Concept
13
Source: https://developer.clarifai.com/modelshttps://developer.clarifai.com/models
How many orange pixels?
Apple or Orange? Melanoma risk or healthy skin?
Degree of contrast in photo colors?
15. 30 Mar 2017
Deep Learning
Regression (review)
Linear regression
Predict continuous set of values
(house prices)
Logistic regression
Predict binary outcomes (0,1)
14
Logistic regression
(sigmoid function)
Linear regression
16. 30 Mar 2017
Deep Learning
Deep Learning Architecture
15
Source: Michael A. Nielsen, Neural Networks and Deep Learning
17. 30 Mar 2017
Deep Learning
Example: Image recognition
1. Obtain training data set
2. Digitize pixels (convert images to numbers)
Divide image into 28x28 grid, assign a value (0-255) to each
square based on brightness
3. Read into vector (array) (28x28 = 784 elements per image)
16
Source: Quoc V. Le, A Tutorial on Deep Learning, Part 1: Nonlinear Classifiers and The Backpropagation Algorithm, 2015, Google
Brain, https://cs.stanford.edu/~quocle/tutorial1.pdf
18. 30 Mar 2017
Deep Learning
Deep Learning Architecture
4. Load spreadsheet of vectors into deep learning system
Each row of spreadsheet is an input
17
Source: http://deeplearning.stanford.edu/tutorial; MNIST dataset: http://yann.lecun.com/exdb/mnist
1. Input 2. Hidden layers 3. Output
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
Vector data
19. 30 Mar 2017
Deep Learning
What happens in the Hidden Layers?
18
Source: Michael A. Nielsen, Neural Networks and Deep Learning
First layer learns primitive features (line, edge, tiniest
unit of sound) by finding combinations of the input vector
data that occur more frequently than by chance
Logistic regression performed and encoded at each processing
node (Y/N (0,1)), does this example have this feature?
Feeds these basic features to next layer, which trains
itself to recognize slightly more complicated features
(corner, combination of speech sounds)
Feeds features to new layers until recognizes full objects
20. 30 Mar 2017
Deep Learning
Feature Recognition in the Hidden Layers
19
Source: Jann LeCun, http://www.pamitc.org/cvpr15/files/lecun-20150610-cvpr-keynote.pdf
21. 30 Mar 2017
Deep Learning
What happens in the Hidden Layers?
20
Source: Nvidia
First hidden layer extracts all possible low-level features
from data (lines, edges, contours), next layers abstract
into more complex features of possible relevance
22. 30 Mar 2017
Deep Learning
Deep Learning
Core concept:
Deep Learning
systems learn
increasingly
complex features
21
Source: Andrew Ng
23. 30 Mar 2017
Deep Learning
Deep Learning
Google Deep Brain recognizes cats
22
Source: Quoc V. Le et al, Building high-level features using large scale unsupervised learning, 2011, https://arxiv.org/abs/1112.6209
24. 30 Mar 2017
Deep Learning
Deep Learning Architecture
23
Source: Michael A. Nielsen, Neural Networks and Deep Learning
1. Input 2. Hidden layers 3. Output guess
(0,1)
25. 30 Mar 2017
Deep Learning
Deep Learning Math
Test new data after system iterates
24
1. Input 2. Hidden layers 3. Output
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
Source: http://deeplearning.stanford.edu/tutorial; MNIST dataset: http://yann.lecun.com/exdb/mnist
Linear algebra: matrix multiplications of input vectors
Statistics: logistic regression units (Y/N (0,1)), probability
weighting and updating, inference for outcome prediction
Calculus: optimization (minimization), gradient descent in
back-propagation to avoid local minima with saddle points
Feed-forward pass
(0,1)
0.5
Back-propagation pass; update probabilities
.5.5
.5.5.5
0
01
.75
.25
Inference
Guess
Actual
26. 30 Mar 2017
Deep Learning
Hidden Layer Unit, Perceptron, Neuron
25
Source: http://deeplearning.stanford.edu/tutorial; MNIST dataset: http://yann.lecun.com/exdb/mnist
1. Input 2. Hidden layers 3. Output
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
Unit (processing unit, logistic regression
unit), perceptron (“multilayer perceptron”),
artificial neuron
27. 30 Mar 2017
Deep Learning
Kinds of Deep Learning Systems
What Deep Learning net to choose?
26
Source: Yann LeCun, CVPR 2015 keynote (Computer Vision ), "What's wrong with Deep Learning" http://t.co/nPFlPZzMEJ
Supervised algorithms (classify labeled data)
Image (object) recognition
Convolutional net (image processing), deep belief
network, recursive neural tensor network
Text analysis (name recognition, sentiment
analysis)
Recurrent net (iteration; character level text),
recursive neural tensor network
Speech recognition
Recurrent net
Unsupervised algorithms (find patterns in
unlabeled data)
Boltzmann machine or autoencoder
28. 30 Mar 2017
Deep Learning
Advanced
Deep Learning Architectures
27
Source: http://prog3.com/sbdm/blog/zouxy09/article/details/8781396
Deep Belief Network
Connections between layers not units
Establish weighting guesses for
processing units before run deep
learning system
Used to pre-train systems to assign
initial probability weights (more efficient)
Deep Boltzmann Machine
Stochastic recurrent neural network
Runs learning on internal
representations
Represent and solve combinatoric
problems
Deep
Boltzmann
Machine
Deep
Belief
Network
29. 30 Mar 2017
Deep Learning
Convolutional net: Image Enhancement
Google DeepDream: Convolutional neural network
enhances (potential) patterns in images; deliberately
over-processing images
28
Source: Georges Seurat, Un dimanche après-midi à l'Île de la Grande Jatte, 1884-1886;
http://web.cs.hacettepe.edu.tr/~aykut/classes/spring2016/bil722; Google DeepDream uses algorithmic pareidolia (seeing an image
when none is present) to create a dream-like hallucinogenic appearance
30. 30 Mar 2017
Deep Learning
How big are Deep Learning systems?
Google Deep Brain cat recognition, 2011
1 billion connections, 10 million images (200x200 pixel),
1,000 machines (16,000 cores), 3 days, each instantiation of
the network spanned 170 servers, 20,000 object categories
State of the art, 2015-2016
Nvidia facial recognition example, 2016, 100 million images,
10 layers, 18 parameters, 30 exaflops, 30 GPU days
Google, 11.2-billion parameter system
Lawrence Livermore Lab, 15-billion parameter system
Digital Reasoning, 2015, cognitive computing (Nashville TN),
160 billion parameters, trained on three multi-core
computers overnight
29
Source: https://futurism.com/biggest-neural-network-ever-pushes-ai-deep-learning, Digital Reasoning paper:
https://arxiv.org/pdf/1506.02338v3.pdf
31. 30 Mar 2017
Deep Learning
Deep Learning, Deep Flaws?
Even though now possible, still early days
Expensive and inefficient, big systems
Only available to massive data processing
operations (Google, Facebook, Microsoft, Baidu)
Black box: we don’t know how it works
Reusable model but still can’t multi-task
Atari example: cannot learn multiple games
Drop Asteroids to learn Frogger
Add common sense to intelligence
Background information, reasoning, planning
Memory (update and remember states of the world)
…Deep Learning is still a Specialty System
30
AlphaGo
applied to
Atari games
Source: http://www.theverge.com/2016/10/10/13224930/ai-deep-learning-limitations-drawbacks
32. 30 Mar 2017
Deep Learning
We had the math, what took so long?
A) Hardware, software, processing
advances; and B) more data
Key advances in hardware chips
GPU chips (graphics processing unit):
3D graphics cards designed to do fast
matrix multiplication
Google TPU chip (tensor processing
unit): custom ASICs for machine
learning, used in AlphaGo
Training the amount of data
required was too slow to be useful
Now can train neural nets quickly, still
expensive
31
Tensor
(Scalar (x,y,z), Vector (x,y,z)3, Tensor (x,y,z)9)
Google TPU chip (Tensor
Processing Unit), 2016
Source: http://www.techradar.com/news/computing-components/processors/google-s-tensor-processing-unit-explained-this-is-what-
the-future-of-computing-looks-like-1326915
33. 30 Mar 2017
Deep Learning
Agenda
Deep Learning Basics
Definition, operation, drawbacks
Implications of Deep Learning
Deep Learning and the Brain
Deep Learning Blockchain Networks
Philosophy of Deep Learning
32
Image Source: http://www.opennn.net
35. 30 Mar 2017
Deep Learning
Deep learning neural networks are inspired by the
structure of the cerebral cortex
The processing unit, perceptron, artificial neuron is the
mathematical representation of a biological neuron
In the cerebral cortex, there can be several layers of
interconnected perceptrons
34
Deep Qualia machine? General purpose AI
Mutual inspiration of neurological and computing research
36. 30 Mar 2017
Deep Learning
Deep Qualia machine?
Visual cortex is hierarchical with intermediate layers
The ventral (recognition) pathway in the visual cortex has multiple
stages: Retina - LGN - V1 - V2 - V4 - PIT – AIT
Human brain simulation projects
Swiss Blue Brain project, European Human Brain Project
35
Source: Jann LeCun, http://www.pamitc.org/cvpr15/files/lecun-20150610-cvpr-keynote.pdf
37. 30 Mar 2017
Deep Learning
Agenda
Deep Learning Basics
Definition, operation, drawbacks
Implications of Deep Learning
Deep Learning and the Brain
Deep Learning Blockchain Networks
Philosophy of Deep Learning
36
Image Source: http://www.opennn.net
39. 30 Mar 2017
Deep Learning
Blockchain Technology
38
Source: http://www.amazon.com/Bitcoin-Blueprint-New-World-Currency/dp/1491920491
40. 30 Mar 2017
Deep Learning
What is Blockchain Technology?
Blockchain technology is an Internet-
based ledger system for submitting,
logging, and tracking transactions
Allows the secure transfer of assets
(like money) and information,
computationally, without a human
intermediary
Secure asset transfer protocol, like email
First application is currency (Bitcoin) and
FinTech re-engineering, subsequent
applications in algorithmic data processing
39
Source: Blockchain Smartnetworks, https://www.slideshare.net/lablogga/blockchain-smartnetworks
41. 30 Mar 2017
Deep Learning
Deep Learning Blockchain Networks
Help resolve Deep Learning challenges
40
Source: http://www.melanieswan.com, http://blockchainstudies.org/NSNE.pdf, http://blockchainstudies.org/Metaphilosophy_CFP.pdf
Deep Learning systems need greater capacity
Put Deep Learning systems on the Internet in a secure-
trackable, remunerable way; distributed not parallel systems
Deep Learning systems need more complexity and
side modules
Instantiate common sense, memory, planning modules
Deep Learning systems do not reveal what happens
in the hidden layers
Track arbitrarily-many transactions with smart contracts
Core blockchain functionality employed
Automated coordination of massive amounts of operations
via smart contracts (automatically-executing Internet-based
programs)
42. 30 Mar 2017
Deep Learning
Deep Learning systems go online with Blockchain
Key point is to put Deep
Learning systems on the Internet
Blockchain is perfect technology
to control secure access, yet
have all of the 24/7 availability,
flexibility, scale, and side
modules needed
Provide global infrastructure to
work on current problems
Genomic disease, protein modeling,
financial risk assessment,
astronomical data analysis
41
43. 30 Mar 2017
Deep Learning
Combine Deep Learning and Blockchain Technology
Deep learning technology, particularly coupled with blockchain
systems, might create a new kind of global computing platform
Deep Learning and Blockchains are similar
Indicative of a shift toward having increasingly sophisticated and
automated computational tools
Mode of operation of both is making (statistically-supported)
guesses about reality states of the world
Predictive inference (deep learning) and cryptographic nonce-
guesses (blockchain)
Current sense-making model of the world, we are guessing at more
complex forms of reality
42
Advanced Computational Infrastructure
Deep Learning Blockchain Networks
44. 30 Mar 2017
Deep Learning
Agenda
Deep Learning Basics
Definition, operation, drawbacks
Implications of Deep Learning
Deep Learning and the Brain
Deep Learning Blockchain Networks
Philosophy of Deep Learning
43
Image Source: http://www.opennn.net
46. 30 Mar 2017
Deep Learning 45
Human’s Role in the World is Changing
Sparse data we control Abundant data controls us?
Deep Learning is emphasizing the
presence of Big Data
47. 30 Mar 2017
Deep Learning
Philosophy of Deep Learning - Definition
46
The Philosophy of Deep Learning is
the branch of philosophy concerned
with the definition, methods, and
implications of Deep Learning
Internal Industry Practice
Internal to the field as a generalized
articulation of the concepts, theory, and
systems that comprise the overall use of
deep learning algorithms
External Social Impact
External to the field, considering the
impact of deep learning more broadly
on individuals, society, and the world
48. 30 Mar 2017
Deep Learning
3 Kinds of Philosophic Concerns
Ontology (existence, reality)
What is it? What is deep learning?
What does it mean?
Epistemology (knowledge)
What knowledge are we gaining from
deep learning?
What is the proof standard?
Axiology or Valorization (ethics,
aesthetics)
What is noticed, overlooked?
What is ethical practice?
What is beauty, elegance?
47
Sources: http://www.melanieswan.com/documents/Philosophy_of_Big_Data_SWAN.pdf
49. 30 Mar 2017
Deep Learning
Explanation: does the map fit the territory?
48
1626 map of “the Island of California”
Source: California Is An Island Off the Northerne Part of America; John Speed, "America," 1626, London
Explanandum
What is being
explained
Explanans
The
explanation
50. 30 Mar 2017
Deep Learning
How do we understand reality?
Methods, models, and
tools
Descartes, Optics, 1637
Deep Learning, 2017
49
51. 30 Mar 2017
Deep Learning
Agenda
Deep Learning Basics
Definition, operation, drawbacks
Implications of Deep Learning
Deep Learning and the Brain
Deep Learning Blockchain Networks
Philosophy of Deep Learning
50
Image Source: http://www.opennn.net
52. 30 Mar 2017
Deep Learning
Key take-aways
What is deep learning?
Advanced statistical method using logistic regression
Deep learning is a sub-field of machine learning and
artificial intelligence
Why is deep learning important?
Crucial method of algorithmic data manipulation
What do I need to know (as a data scientist)?
Awareness of new methods like deep learning needed to
keep pace with data growth
51
53. 30 Mar 2017
Deep Learning
Conclusion
Deep learning systems are machine
learning algorithms that learn
increasingly complex feature sets from
data via hidden layers
Deep qualia systems might be a step
forward in brain simulation in computer
networks and general intelligence
Next-generation global infrastructure:
Deep Learning Blockchain Networks
merging deep learning systems and
blockchain technology
52
54. 30 Mar 2017
Deep Learning
Resources
53
Distill, a visual,
interactive journal for
machine learning
research
http://distill.pub/
55. Melanie Swan
Philosophy & Economic Theory
New School for Social Research, NY NY
melanie@BlockchainStudies.org
Philosophy of Deep Learning:
Deep Qualia, Statistics, and Blockchain
Pfizer, New York NY, March 30, 2017
Slides: http://slideshare.net/LaBlogga
Thank You! Questions?
Image credit: Nvidia