Big Data Graph Analytics

Copyright © 2015 Oracle and/or its affiliates. All rights reserved.
Big Data Graph Analytics
Cristian Spigariol
Cloud Enterprise Architect

Types of uses of Machine Learning in all Industries
Typical use case scenarios
Classification
(predict among a set of options)
• Find and preventing customer
churn
• Target the right customer with
the right offer
• Predict customer response to an
affinity card program
Regression
(estimate a missing value)
• predict how much a customer
will spend
Clustering
(find unknown patterns)
• Detect anomalous or suspicious
activities
Association Rules
(find correlations)
• Predict correlation among items
Graph Analysis
(understand interactions)
• Understanding influencers in
social networks

Machine Learning through Graphs
• Graphs systems focus on relationships rather than entities
– They are key to understand highly connected systems and relative
behaviours (i. e. areas of strong/weak interaction) by examining how
relationships spread throughout the graph
• Graphs algorithms are self-consistent
– The answer to complex problems resides in how entities (nodes) interact
and not in the entities themself or in external resources
– Graph algorithms are effective even with graphs based on entities with few
properties
• Cover a broad range of applications
– Their simple and flexible data model is able to describe a broad range of
use cases, from financial systems, human neural networks to
infrastructural networks (transportation, telco, electricity)

Marketing Analyses using Graphs
Graph algorithms can strongly improve the effectiveness of marketing
analyses.
 In customer profiling we can extend the individual profile of a
given customer by considering his/her ability to influence the circle
of friends
 In marketing campaigns the identification of influencers can
amplify the echo of the relative promotional activities and increase
the conversion rate
 In marketing campaigns the identification of strongly connected
communities (people who interact on the basis of shared
behaviors) can be the the basis for customers segmentation.

Network Analysis using Graphs
Detection of weak links is aimed at identifying nodes in the
transportation/telco/energy network that have a high numbers
of flows that come through them and that are not balanced with
a proper number of alternative paths (Betweenness centrality).
Graphs algorithms are extremely useful to optimize network.
Network flows analysis consists in assigning to each connection
(i. e. link between two nodes) a capacity and evaluate the total
amount of flows that passes on it. The amount of flows on an
edge cannot exceed the capacity of the edge.

Copyright © 2015 Oracle and/or its affiliates. All rights reserved. Oracle Confidentihal – Internal 6
Collaborative Filtering Pattern
Find out similarities
Select
potential
targets
Rank
output by
relevance
If a person A has the same opinion as a person B on an issue, A is more likely to have B's opinion on a
different issue than that of a randomly chosen person (Wikipedia).
Find out people that
present the same
behavior with respect to
the person A.
Person A
Select the items chosen
by the people similar to
the person A (i. e.
potential targets).
Among the potential targets
weight the items that
present the highest
relevance rank.
Higher Rank

Recommendation using Graphs Analytical Algorithms
Ricky
Simon
Lucia
Circle of trust
Ricky
0.4
Simon
0.3
Lucia
0.1
JohnMaria
...
...
...
By using a centrality algorithm (Personalized
PageRank) we can determine the most
influent people in the circle of connections
originated by Alice.
We move from similarities to trust!
Alice

Prod#3
Ricky
Simon
Lucia
Circle of trust Targets
Prod#4
Prod#9
Prod#7
We determine the
potential targets
by selecting the
products already
boughts by the
trusted people
(bipartite graph).

Prod#3
(2)
Ricky
Simon
Lucia
Prod#4
(1)
Prod#9
(1)
Prod#7
(2)
We start the
relevance algorithm
(salsa) by measuring
the relevance score,
that is the sum of the
preferences received
by each product.

Ricky
(2)
Simon
(4)
Lucia
(4)
We then walk
connections back-to-
front to measure
Hub Score as the
sum of the relevance
ranks of products
bought
Measure the ability of
each trusted person
to intercept the tastes
of the circle
Prod#3
(2)
Prod#4
(1)
Prod#9
(1)
Prod#7
(2)

Prod#3
(6)
Prod#4
(4)
Prod#7
(8)
Prod#9
(4)
The new relevance
score is measured
with the weighted
sum of the
preferences (hub
ranks) received.
Ricky
(2)
Simon
(4)
Lucia
(4)
The Prod#9 has the
highest likelihood to
be well-accepted by
Alice since it has been
chosen by the most
“knowledgeable”
trusted people
Alice

The algorithm can be iterated
many times. Each iteration will
reinforce the rank score and
the relevance score.
The higher the number of
iterations the higher the
effectiveness of the algorithm

Graphs Analytical Algorithms – Possible use cases
This recommendation approach is at the basis of the WTF service at Twitter.
It can be proficiently be applied to different industries as for example:
 to recommend insurance policies based on the most relevant opinions
of “trusted“ people
 to up-sell telco services with the same trust+expertise approach.

Do it yourself? If so please consider...
Complexity Productivity
Architecture Integration
Not trivial algorithms, need domain
specific knowledge
Bug Fixing, Tuning for precision and
performance, Support
Graph algorithms need in-memory
parallel execution as well as a low-
latency NoSQL storage
You need to integrate your solution
with the Big Data cluster to feed
your Graph database

A rich set of built-in, parallel algorithms Parallel graph mutation operations
Detecting Components and
Communities
Tarjan’s, Kosaraju’s,
Weakly Connected
Components, Label
Propagation (w/ variants),
Spasification
Ranking and Walking
Pagerank, Personalized
Pagerank,
Betwenness Centrality (w/ variants),
Closeness Centrality, Degree
Centrality,
Eigenvector Centrality, HITS,
Random walking and sampling (w/
variants)
Evaluating Community Structures
∑ ∑
Conductance,
Modularity
Clustering Coefficient
(Triangle Counting)
Path-Finding
Hop-Distance (BFS)
Dijkstra’s,
Bi-directional Dijkstra’s
Bellman-Ford’s
Link Prediction SALSA
(Twitter’s Who-to-follow)
Other Classics Vertex Cover
a
d
b e
g
c i
f
h
The original graph
a
d
b e
g
c i
f
h
Undirected Graph
Simplify Graph
a
d
b e
g
c i
f
h
Left Set: “a,b,e”
a d
b
e
g
c
i
Bipartite
Graph
ge b d i a f c h
Sort-By-Degree (Renumbering)
Filtered
Subgraph
d
b
g
i
e
15
Oracle Big Data Spatial and Graph – Memory Analyst

Machine Learning is Data Driven
• Data is the fuel of ML
algorithms
• The effectiveness of ML
algorithms is strictly tied to the
amount of available
data
• To translate ML results into a
competitive advantage we need
a paradigm shift in the way
information management
solution are designed and
managed.

Copyright © 2015 Oracle and/or its affiliates. All rights reserved. Oracle Confidential – Internal 17
A Data Driven Strategy with Machine Learning
Adopt
Standards
Don’t
Move data
Broadcast
ML results
Experiment
and Act
New
Paradigm
• Data is heavy – don’t move data
• Move elaboration to data instead
• Reduce the complexity
• Facilitate integration
• Speak the language of Data
Scientists (R, Python, Scala,
Spark, Gremlin)
• Take advantage of new ML
packages release (e. g. CRAN,
MLlib)
• Define your models (Lab) and
then move them in the
mainstream (Prod)
• Score your models continuoulsly
(both in batch and in streming)
• Take them up to date
• Spread ML results thorughout
the user communities
• Predictions are new inputs for in-
place processes or analyses
(additional KPIs, properties, etc..)

Big Data Graph Analytics

Recommandé

Contenu connexe

Tendances

Tendances (20)

Similaire à Big Data Graph Analytics

Similaire à Big Data Graph Analytics (20)

Dernier

Dernier (20)

Big Data Graph Analytics