Explore the world of TiDB with Kabilesh PR, Co-Founder of Mydbops, as he unveils the potential of this open-source distributed SQL database. Dive into the architecture, scalability solutions, and production readiness of TiDB, and discover how it addresses MySQL scalability bottlenecks through sharding. Gain insights into its stateless SQL interface, transactional storage with TiKV, and analytical capabilities with TiFlash. Learn about TiDB's native sharding features, use cases across various industries, and its readiness for production environments. Delve into its limitations and discover how TiDB can transform your data management landscape.

1. Unleashing Data At Scale With TiDB
Sep 09th, 2023
LSPE - 48
Kabilesh PR
Co-Founder, Mydbops

2. Interested in Open Source technologies
Pingcap Certified TiDB Associate
Tech Speaker/Blogger
AWS certified Database speciality
AWS community builder / SME
Co-Founder, Mydbops
Kabilesh PR
About Me

3. Mydbops
at a
glance
300+
Clients
3000+
Servers
ISO & PCI DSS
Certified
7 Yrs of
Expertise
AWS Advanced
Consulting Partner
TiDB PingCap
Partner

4. Mydbops Services
Consulting
Services
Managed
Services
MySQL, MongoDB, PostgreSQL, TiDB & Cassandra
Targeted
Engagement
24 x 7
DBA Team

5. Agenda
What is TiDB ?
Architecture
Production Readiness
Sharding

6. What is TiDB ?
An Open Source distributed SQL database by PingCAP
•
Scalable in both ways
•
Supports both Transactional and Analytical (HTAP)
•
Compatible with MySQL protocol
•
Highly available
•
Fault Tolerant
•

7. MySQL scalability - Bottlenecks
Monolithic - Single node Limitation
Horizontal scaling - Sharding

8. Architecture
TiDB cluster - SQL layer
Tikv - Storage layer
Placement Driver(PD) - Orchestrator
TiFlash - Analytical columnar storage

9. Architecture

10. TiDB - SQL Interface
Stateless
•
Handles Client connections & authentication
•
Parsing & compiling
•
Coverting relational to KV pair (Tikv)
•
Executing SQL
•
Online DDL execution
•
Garbage collections
•
Cache table
•

11. TiKV - Transactional Storage
Stateful
•
Persistent storage with RocksDB
•
Distributed storage & transaction support
•
Strong Consistency
•
MVCC support
•
Inbuilt HA
•
Co-Processor (PP)
•

12. TiKV - Regions and Raft
Region - Represents a range of key-value data
•
96 Mb in size
•
A region is replicated to multiple nodes
•
Regions are automatically balanced across TiKV nodes
•
Each region has a leader(1) and followers(2)
•
Leader and Followers form Raft group
•
Leader handles RW ops
•
Followers Participate in Election & Data Reads
•

13. Services (LB)
Stateful Sets
Operator Maturity
TiKV - Regions and Raft

14. Placement Driver - PD
Metadata management - Global view
•
Data placement
•
load balancing
•
TransactionID allocation
•
Global Timestamp (TSO)
•
Scheduling by collecting cluster info
•
PD consists of 3 node at-least
•

15. TiFlash - Analytical Store
Columnar storage
Replicated Async from Tikv
Intelligent engine selection
Optimised for aggregation, join & table scans
MPP (Multiple Parallel Processing)

16. Detailed Architecture

17. Sharding
Supports native sharding & distribution
Relational to Key-value transformation
Distributed based on Global key
Global_key = Primary_key + Table_id(generated)
For Non-int primary key row_id is generated

18. Sharding - Relational to KV

19. Sharding - Global-key

20. Sharding - Region split & distribution

21. Use-Cases
https://www.pingcap.com/customers/
SAAS Application
IOT data , Time series
Ecommerce
Logistics
Gaming
Fintech

22. Production Readiness
Wire Compatible with MySQL 5.7
UTF8MB4 by default with Binary Collation
Profile your workload with TiDB
Support for CDC with Ti-Binlog
Monitoring integration with Grafana + Prometheus + Alert manager
Full + Incremental , Logical Backup with PITR supported

23. Limitations
Stored procedures & Functions
Triggers
Events
User-defined functions
Fulltext and Spatial index
Column-level privileges

24. Reach Us : Info@mydbops.com
Thank You
Image Courtesy : PingCAP