2. OutLine
Introduction
Data Model
API
Building Blocks
Implementation
Refinements
Performance Evaluation
Real Applications
Lessons
Related Work
Conclusions
3. OutLine
Introduction
Data Model
API
Building Blocks
Implementation
Refinements
Performance Evaluation
Real Applications
Lessons
Related Work
Conclusions
5. Data Model
BigTable Is A Sparse Sorted Map
Key: <Row, Column Family, Qualifier, Timestamp>
6. API
How to Support Single-Row transaction
Atomic Read-modify-write
Using API for MapReduce
Input source & Output target
7. Building Blocks
GFS
SSTable Data File
Commit Log (WAL & Redo Log)
Chubby
Distributed Lock Service
Root Tablet Location
Tablet server Manager
Schema & ACL Metadata
Why GFS not use chubby For Master?
If Chubby not exist, how to design the bigtable?
8. Implementations
Client
How To Protect Lighted Master ?
Master
Tablet server Manager
Tablet Assignment
Garbage Collection
Schema Change
Do We Need Slave Master For Availability?
Tablet Server
Reader & Writer
10. Implementations
Why select Three-level hierarchy?
Why Persistent the location info?
Why analogous as B+tree?
The Most RPC times during One simple Query?
13. Implementations
Compactions
Three Type Compactions Differences
If client Write 1GB data, How much data in GFS?
14. Refinements
How To Reduce IO?
Column Store
Compression
Block Size
Caching
Caching Consistency
Bloom Filter
Why Must Using Bloom Filter For LSM?
Group Commit
Prons Vs Cons
15. Refinements
Log
GFS exist duplicated record?
What Happened when Tablet Server Crashes?
Tablet Recovery
Tablet Migrate
Compare with Tair & OB
Tablet Split
Tablet Merge
How can do this online?
16. Performance
Compare With Single Server Performance
Low Latency Vs High Through output
Why Performance/Server Decrease?
18. Lessons
Vulnerable to many types of failure
RPC crc checksum
Removing assumption
Say No to adding new features
System-level Monitoring
Simple Design
19. Related Work
CAP in BigTable?
Sharing Nothing or Sharing Disk or Sharing
Memory?
CStore or BigTable or OceanBase