Graph Gurus Episode 11: Accumulators for Complex Graph Analytics

Graph Gurus
Episode 11
Using Accumulators in GSQL
for Complex Graph Analytics

© 2019 TigerGraph. All Rights Reserved
Welcome
● Attendees are muted
● If you have any Zoom issues please contact the panelists via chat
● We will have 10 min for Q&A at the end so please send your questions
at any time using the Q&A tab in the Zoom menu
● The webinar will be recorded and sent via email
2

Developer Edition Available
We now offer Docker versions and VirtualBox versions of the TigerGraph
Developer Edition, so you can now run on
● MacOS
● Windows 10
● Linux
Developer Edition Download https://www.tigergraph.com/developer/
3
Version 2.3
Released Today

© 2019 TigerGraph. All Rights
Reserved
Today's Gurus
4
Victor Lee
Director of Product Management
● BS in Electrical Engineering and Computer Science
from UC Berkeley, MS in Electrical Engineering from
Stanford University
● PhD in Computer Science from Kent State University
focused on graph data mining
● 15+ years in tech industry
Gaurav Deshpande
Vice President of Marketing
● Built out and positioned IBM’s Big Data and Analytics
portfolio, driving 45 percent year-over-year growth.
● Led 2 startups through explosive growth - i2
Technologies (IPO) & Trigo Technologies (largest MDM
acquisition by IBM)
● Big Data Analytics Veteran, 14 patents in supply chain
management and big data analytics

How Can TigerGraph Help You?
Improve Operational EfficiencyReduce Costs & Manage RisksIncrease Revenue
• Recommendation Engine
• Real-time Customer 360/
MDM
• Product & Service Marketing
• Fraud Detection
• Anti-Money Laundering
(AML)
• Risk Assessment & Monitoring
• Cyber Security
• Enterprise Knowledge Graph
• Network, IT and Cloud
Resource Optimization
• Energy Management System
• Supply Chain Analysis
Analyze all interactions
in real-time to sell more
Reduce costs and assess and
monitor risks effectively
Manage resources for
maximum output
Foundational Use Cases: Geospatial Analysis, Time Series Analysis, AI and Machine Learning

Driving Business Outcomes with TigerGraph
Increase Revenue
Grew to Billion+ in annual
revenue in 5 years with
TigerGraph
Reduce Costs &
Manage Risks
Recommendation Engine
Detects fraud in real-time
for 300+ Million calls per
day with TigerGraph
Fraud Detection
Improve Operational
Efficiency
Improve Operational
Efficiency
Optimizes infrastructure to
minimize outages for
critical workloads with
TigerGraph
Network & IT Resource Optimization

Accumulators
7
Example: A teacher collects test papers
from all students and calculates an
average score.
Teacher: node with an accumulator
Student: node
Student-to-teacher: edge
Test paper: message sent to
accumulator
Average Score: final value of
accumulator
Phase 1: teacher collects all the test
papers
Phase 2: teacher grades it and
calculates the average score.

Example 1: Counting Movies Seen by
Friends
Analogy: You ask each of your friends to tally how many movies they
saw last year (They work at the same time → Parallelism!)
● This is the Gathering phase
● Next, you need to
Consolidate into one list.
→ Eliminate the duplicates Lee
Bob
Chris
Sue
If we are only counting (not recording names),
How can we eliminate duplicates?
P
Q
R
S
T

Eliminating Duplication: visited flag
● Traditional method: Each target
Item has a true/false "flag", initialized to "false"
● If a flag is "false":
○ It may be counted;
Set the flag to "true"
● If a flag is "true":
○ Skip it
● ⇒ Each item is counted once
Lee
Bob
Chris
Sue
If we are only counting (not recording names),
How can we eliminate duplicates?

Counting Movies with Accumulators
An accumulator is a runtime, shared variable with a built-in update
function.
1. Each friend's tally is a local Accumulator.
2. The visited flag is also a local Accumulator.
a. Bob, Chris, and Sue all visit "R", but we don't know who will first.
3. GSQL queries are designed to let each friend work in parallel, but
at its own pace, and in any order
4. Consolidate the Lists into one global Accumulator.
Bob: 2 (P, Q) Chris: 1 (S)
Sue: 2 (R,T)

Optimization: One global list/tally
Instead of each friend keeping a separate tally…
Keep only one tally.
● Eliminates the consolidate step
● Saves memory
Each friend still works separately,
but they record their efforts in one common place.
Lee
Bob
Chris
Sue

friendsMovieCount GSQL Query
CREATE QUERY friendsMovieCount (VERTEX me) {
OrAccum @visited = false; // @ means local
SumAccum<INT> @@movieCount; // @@ means global
#~~~~~~~~~~~~~~
Start = {me};
Friends = SELECT f # 1. Identify my friends
FROM Start:s -(friendOf:e)- Person:f;
#~~~~~~~~~~~~~~
Movies = SELECT m # 2. Identify the movies they've seen
FROM Friends:f -(saw:r)- Movie:m
ACCUM
IF m.@visited == false THEN
@@movieCount += 1, m.@visited += true; # 3. Count each movie once
#~~~~~~~~~~~~~~
PRINT @@movieCount;
}

Review - Accumulator Properties
● Scope: Local (per vertex) or global
● Multiple data types and functions available
○ For flag: Boolean true/false data, OR function
○ For tally: Integer data, Sum function
● Supports multi-worker processing for parallelism
○ Shared: Anyone can read
○ Shared: Anyone can update submit an update request
○ All the accumulated updates are processed at once, at the end.

Types of Accumulators
The GSQL language provides many different accumulators, which follow the
same rules for receiving and accessing data. However each of them has their
unique way of aggregating values.
14
Old Value:
2
New Value:
11
1, 3, 5
1
3 5
SumAccum<int>
Old Value:
2
New Value:
5
1, 3, 5
1 3
MaxAccum<int>
Old Value:
2
New Value:
1
1, 3, 5
1 3
MinAccum<int>
Old Value:
2
New Value:
2.75
1, 3, 5
1 3
AvgAccum
5 5 5
Computes and stores the
cumulative sum of numeric
values or the cumulative
concatenation of text values.
cumulative maximum of
a series of values.
cumulative mean of a
series of numeric values.
cumulative minimum of
a series of values.

The GSQL language provides many different accumulators, which follow the
same rules for receiving and accessing data. However each of them has their
unique way of aggregating values.
15
Old Value:
[2]
New Value:
[2,1,3,5]
1, 3, 3, 5
1 3 5
SetAccum<int>
Old Value:
[2]
New Value:
[2,1,5,3,3]
1, 5, 3, 3
1 3
ListAccum<int>
Old Value:
[1->1]
New Value:
1->6
5->2
1->2
1->3
5->2
1->2 1->3
MapAccum<int,SumAccum<int>>
Old Value:
[userD,150]
New Value:
[userC,300,
UserD,150,
userA,100]
userC,300
userA,100
(“userA”, 100)
HeapAccum<Tuple>
5 5->23 3 (“userC”, 300)
Maintains a collection of
unique elements.
Maintains a sequential
collection of elements.
Maintains a collection of
(key → value) pairs.
Maintains a sorted collection of
tuples and enforces a
maximum number of tuples in
the collection
Types of Accumulators, continued

Use Case: Graph Algorithms
PageRank analyzes the WHOLE graph
● For each node V:
PR(V) = sum [ PR(A)
/deg(A)
+ PR(B)
/deg(B)
+ PR(C)
/deg(C)
]
○ Summing contributions from several independent nodes
→ Ideal task for accumulators and parallel processing
● Compute a score for each node → parallelism
● Iterative method: Keep recalculating PR(v) for the full graph,
unless the max change in PR < threshold → MaxAccum
V
A
B
C

© 2018 TigerGraph. All Rights Reserved 17
Input:
A person p, two integer parameters k1
and k2
Algorithm steps:
1. Find all movies p has rated;
2. Find all persons rated same movies as p;
3. Based on the movie ratings, find the k1
persons that have most similar tastes with p;
4. Find all movies these k1
persons rated that p hasn’t rated yet;
5. Recommend the top k2
movies with highest average rating by the k1
persons;
Output:
At most k2
movies to be recommended to person p.
p
…...
rate
rate
rate
PRatedMovies
PSet
…...
PeopleRatedSameMovies
rate
rate
rate
rate
…...
rate
rate
rate
rate
Implementing Movie Recommendation Algorithm

More Use Cases: Anti-Fraud
TestDrive Anti-Fraud: https://testdrive.tigergraph.com/main/dashboard
CIrcle Detection:
● Is money passing from A → B → C → … → back to A?
See Graph Gurus #4 on our YouTube Channel:
https://www.youtube.com/tigergraph

Demo
19

Summary
• Graph Traversal and Graph Analytics can be
complex, but also amenable to parallelism.
• Accumulators, coupled with the TigerGraph MPP
engine, provide simple and efficient parallelized
aggregation.
• Accumulators come in a variety of shapes and sizes,
to fit all different needs.
20

Q&A
Please send your questions via the Q&A menu in Zoom
21

NEW! Graph Gurus Developer Office Hours
22
The Graph Gurus Webinar in late March or catch up on previous
episodes: https://www.tigergraph.com/webinars-and-events/
Every Thursday at 11:00 am Pacific
Talk directly with our engineers every
week. During office hours, you get
answers to any questions pertaining to
graph modeling and GSQL
programming.
https://info.tigergraph.com/officehours

Additional Resources
23
New Developer Portal
https://www.tigergraph.com/developers/
Download the Developer Edition or Enterprise Free Trial
https://www.tigergraph.com/download/
Guru Scripts
https://github.com/tigergraph/ecosys/tree/master/guru_scripts
Join our Developer Forum
https://groups.google.com/a/opengsql.org/forum/#!forum/gsql-users
@TigerGraphDB youtube.com/tigergraph facebook.com/TigerGraphDB linkedin.com/company/TigerGraph

Graph Gurus Episode 11: Accumulators for Complex Graph Analytics

Recommandé

Recommandé

Contenu connexe

Tendances

Tendances (20)

Similaire à Graph Gurus Episode 11: Accumulators for Complex Graph Analytics

Similaire à Graph Gurus Episode 11: Accumulators for Complex Graph Analytics (20)

Plus de TigerGraph

Plus de TigerGraph (20)

Dernier

Dernier (20)

Graph Gurus Episode 11: Accumulators for Complex Graph Analytics