Graph Gurus Episode 13: Visualizing Bitcoin Blockchain with Tiger Graph

Graph Gurus
Episode 13
Visualizing Bitcoin Blockchain with TigerGraph

© 2019 TigerGraph. All Rights Reserved
Welcome
● Attendees are muted
● If you have any Zoom issues please contact the panelists via chat
● We will have 10 min for Q&A at the end so please send your questions
at any time using the Q&A tab in the Zoom menu
● The webinar will be recorded and sent via email
2

Developer Edition Available
We now offer Docker versions and VirtualBox versions of the TigerGraph
Developer Edition, so you can now run on
● MacOS
● Windows 10
● Linux
Developer Edition Download https://www.tigergraph.com/developer/
3
Version 2.3
Available Now

© 2019 TigerGraph. All Rights
Reserved
Today's Gurus
4
Sai Sameer Pusapaty
Sophomore, MIT
● Majoring in Computer Science and Engineering (6-3)
● Past projects in topics including Machine Learning,
Parallel Processing, Big Data
● Extern at TigerGraph (Winter 2019). Worked previously
at Visa (Summer 2018)
Benyue (Emma) Liu
Senior Product Manager
● BS in Engineering from Harvey Mudd College, MS in
Engineering Systems from MIT
● Prior work experience at Oracle and MarkLogic
● Focus - Cloud, Containers, Enterprise Infra, Monitoring,
Management, Connectors

Agenda
5
● How to Load Bitcoin Dataset into TigerGraph
● Demo Bitcoin BlockChain in TigerGraph
● TigerGraph 2.3 Kafka Loader for Future Integration

MIT Externship
6
● 4 week Internship during winter Independent Activities Period
(IAP)

Visualizing Bitcoin Blockchain with
TigerGraph
Sai Sameer Pusapaty
Sophomore, MIT

Briefly on Blockchain
• Past: central authority for money management
• I.e. banks, Visa, Paypal
• These systems require “trust”
• New idea: trustless system
• Not completely accurate name
• Rather than a “human” system, a network of computers divide
the “trust”
• Public-private key cryptography
• Private key used to make a signature
• Public key used to confirm the signature

Bitcoin transaction
TXN
3 BTC
3 BTC
2 BTC
TXN4 BTC
4 BTC 3 BTC
3 BTC
2 BTC
FALSE TRUE

Briefly on Blockchain 199
843 132
344
432
424
345
665
645
634
432
432
324
256 345
634
???
• Transaction is sent to pool of
unverified txns
• Set of transactions put into a block
• Verification is done across a network
of computers by solving a complex
math problem -- mining
• Goal of mining is to generate a
proper block hash

Looking at the Data Transactions
Blocks
Addresses

Past Attempts with Graph DBs
• Report 1:
• Neo4j, 600GB db to store 100GB raw data
• 60+ days of exporting
• Report 2:
• used 400-core computer to parse 80GB
• used a 12-core VM with 64 GB RAM to analyze resulting graph
• Other:
• Graph Blockchain -> Apache Cassandra
• GraphSense -> AgensGraph Engine

Niche of TigerGraph
• Able to scale with the
exponential size of
blockchain
• Can perform more
complex queries as
network increases
• Relatively lightweight
(especially in regards
to other graph db
alternatives)

Within the Data
Transaction
● Txn_hash
● Coinbase
● ...
Input List
Output List
Inputs (outputs
of previous txns)
● Previous hash
● output_id
Outputs
● Address
● Amount
File
Block
Block
Block
Block
Block Header
● Block_hash
● Previous hash
● Nonce
● Merkle_root
● Timestamp
● ...
Transaction List

Structure of the Graph
Block
Txn
Address
Output

Structure of the Graph
Block
AddressTransaction
Output

Program Structure: Golang
txn
Block
Channel
Batch
of
blocks txn txn txn
txn txn txn txn
Blocks
Txn
Channel
Txns Inputs Outputs
spawned
Goroutines
Folder
containing .dat
files
Wait
Group

LET’S EXPLORE!
19

Demo Machine and Loading Stats
• Hardware:
• AWS EC2 r5.24xlarge (96vCPU and 768G memory)
• Original Bitcoin Data Size: 569G
• Loading Time: 2h 50min
• TigerGraph Graph Data Size: 262G
• Memory Usage During Loading: ~200G
Github:https://github.com/tigergraph/bitcoin-to-tigergraph

Let’s explore!
explore
Alleged address of Satoshi Nakamoto: 1A1zP1eP5QGefi2DMPTfTL5SLmv7DivfNa
Pizza transaction: cca7507897abc89628f450e8b1e0c6fca4ec3f7b34cccf55f3f531c659ff4d79
Address of the Silk Road guy: 1Ez69SnzzmePmZX3WpEzMKTrcBF2gpNQ55
Largest Bitcoin Transaction Address: 1M8s2S5bgAzSSzVTeL7zruvMPLvzSkEAuv
More addresses: http://www.theopenledger.com/9-most-famous-bitcoin-addresses/

Address Deanonymization

Source: https://people.csail.mit.edu/spillai/data/papers/bitcoin-transaction-graph-analysis.pdf

Extended Ideas
• Each block is a frozen state in time
• Acts as time series data
• Examine behavior of addresses over time
• Can be used to determine black market addresses
• Can potentially be used to predict the USD price of
cryptocurrencies
• https://medium.com/@clearblocks/valuing-bitcoin-and-ethereum
-with-metcalfes-law-aaa743f469f6
• Metcalfe’s Law: value of a network is proportional to the number
of active users squared

Kafka Loader Integration
Benyue (Emma) Liu, TigerGraph
25

NEW in TigerGraph 2.3: Kafka Loader
● Increase Data Availability and Accelerate Time to Value
• Load streaming and batched data from user's Kafka server
• Consistent with GSQL file loading syntax and MultiGraph support
● Embrace Benefits of Kafka Ecosystem
• Scalable data loading and Built-in fault tolerance
• Data buffer - Kafka is in a separate cluster
• Extensible - open up data pipeline from many other data sources
26
Kafka Loader

Kafka and TigerGraph Data Pipeline
Static
Data
Sources
Streaming
Data
Sources
Kafka
Loader

Bitcoin Streaming Data Pipeline
Preprocessed
Bitcoin
Live
Data
Kafka
Loader

Summary
• How to Load Bitcoin Dataset into TigerGraph
• Demo Bitcoin BlockChain in TigerGraph
• Future BitCoin Data Pipeline Integration with Kafka Loader
from TigerGraph 2.3 (See Graph Guru 12)
Github:https://github.com/tigergraph/bitcoin-to-tigergraph
29
Version 2.3
Available Now

Q&A
Please send your questions via the Q&A menu in Zoom
30

NEW! Graph Gurus Developer Office Hours
31
Catch up on previous episodes of Graph Gurus:
https://www.tigergraph.com/webinars-and-events/
Every Thursday at 11:00 am Pacific
Talk directly with our engineers every
week. During office hours, you get
answers to any questions pertaining to
graph modeling and GSQL
programming.
https://info.tigergraph.com/officehours

Additional Resources
32
New Developer Portal
https://www.tigergraph.com/developers/
Download the Developer Edition or Enterprise Free Trial
https://www.tigergraph.com/download/
Guru Scripts
https://github.com/tigergraph/ecosys/tree/master/guru_scripts
Join our Developer Forum
https://groups.google.com/a/opengsql.org/forum/#!forum/gsql-users
@TigerGraphDB youtube.com/tigergraph facebook.com/TigerGraphDB linkedin.com/company/TigerGraph

Kafka Loader High Level Architecture
• Connect to External Kafka Cluster
• User Commands Through GSQL
Server
• Configuration Files:
• Config 1: Kakfa Cluster
Configuration
• Config 2: Topic/Partition/Offset
Info
34

Kafka Loader : Three Steps
35
Consistent with GSQL Data Loading Steps
Step 1: Define the Data Source
Step 2: Create a Loading Job
Step 3: Run the Loading Job

Prerequisites: Kafka Configuration Files
36
Connect to External
Kafka Data Source
Through
Kafka Cluster
Configuration file
In Step 1
(Kafka broker's domain
name and port)
Define Kafka Data
Source Structure
Through
Kafka Topic/Partition
Configuration File
In Step 2
(Kafka topic, partition list,
and start offset)

Manage Loading Job
• SHOW LOADING STATUS
• ABORT LOADING JOB
• RESUME LOADING JOB
37
Consistent with GSQL Data Loading Syntax

Graph Gurus Episode 13: Visualizing Bitcoin Blockchain with Tiger Graph

Recommandé

Recommandé

Contenu connexe

Tendances

Tendances (20)

Similaire à Graph Gurus Episode 13: Visualizing Bitcoin Blockchain with Tiger Graph

Similaire à Graph Gurus Episode 13: Visualizing Bitcoin Blockchain with Tiger Graph (20)

Plus de TigerGraph

Plus de TigerGraph (20)

Dernier

Dernier (20)

Graph Gurus Episode 13: Visualizing Bitcoin Blockchain with Tiger Graph