Contenu connexe Similaire à Kineograph: Taking the Pulse of a Fast-Changing and Connected World Similaire à Kineograph: Taking the Pulse of a Fast-Changing and Connected World (20) Kineograph: Taking the Pulse of a Fast-Changing and Connected World1. Kineograph: Taking the Pulse of a
Fast-Changing and Connected World
Speaker: LIN Qian
http://www.comp.nus.edu.sg/~linqian
8. Master Progress
Continuous table
Data feeds
Ingest
nodes Snapshooter
Graph
nodes Global consistent snapshots
Graph Storage
Computation
Incremental computation on a
static graph snapshot
13. Snapshot
ingest nodes
graph nodes
global progress table
16. Progress table
s1 1
3
2
0
… … Global tx
vector
Ingest nodes s1 … sn sn 7
3
4
Snapshooter
Partition u Partition v
1 2 4 s1 2 3 5 s1
… Epoch specified by progress
…
Graph nodes
…
table and snapshooter
4 6 7 sn 5 6 8 sn
17. Graph update / compute
Pipeline
Incoming
Tweets … … Time
Snapshot Si-1 Si Si+1
Construction
Graph Epoch Ci
Computation ti-1 ti ti’ ti’’
Timeliness
18. Consistency
no global serialization
(diff. from 2PL or t.s. ordering)
23. Incremental Graph
Computation
Updates from
other vertices
N
Detect Vertex Compute New Change
Init
Status Vertex Values Significantly?
Graph-Scale Propagate Y
Aggregation Updates
38. Twitter feeds
8M vertices, 29M edges
100M tweets with 100K/sec
power-law