My slides from First Ceph user meetup @ Helsinki , Feb 2017

1. Introduction and Beyond

2. Karan Singh
Sr. Storage Architect
Red Hat , Finland
Karan@redhat.com
•Author for Learning Ceph and Ceph CookBook
•www.ksingh.co.in
# whoami

3. What is Software Defined Storage (SDS) ?
•Fancy name for Storage Virtualization
•All the intelligence is built into software
•Commodity Hardware
•! (Big Iron , Secret , Black Boxes) in your DC

4. What are different storage options ?
FILE STORAGE
File systems allow users to
organize data stored in blocks
using hierarchical folders and
files.
OBJECT STORAGE
Object stores distribute data
algorithmically throughout a
cluster of media, without a rigid
structure.
BLOCK STORAGE
Physical storage media appears
to computers as a series of
sequential blocks of a uniform
size.

5. Ceph
Introduction & Architecture

6. Why Ceph is Cool ?
• Open Source Software Defined Storage
• Unified Storage Platform ( Block , Object and File Storage )
• Runs on Commodity Hardware
• Self Managing, Self Healing
• Massively Scalable
• No Single Point of failure
•Designed for cloud infrastructure and emerging workloads
•Awesome release names

7. CPU
Memory
HDDs
SSD
Network
Linux OS
Ceph
Ceph Storage Cluster
...
...
...
SERVER 1 SERVER 2 SERVER 3 SERVER N
Block StorageObject Storage File Storage
Clients
CPU
Memory
HDD
SSD
Network
Linux OS
Ceph
CPU
Memory
HDD
SSD
Network
Linux OS
Ceph
CPU
Memory
HDD
SSD
Network
Linux OS
Ceph
Commodity
Hardware
Linux OS
Ceph SDS
Distributed
Storage
Unified
Storage
Solution
Ceph Storage Cluster

8. LIBRADOS
A library allowing apps to directly access RADOS (C, C++, Java, Python, Ruby
RADOS
A software-based reliable, autonomous, distributed object store comprised of
self-healing, self-managing, intelligent storage nodes and lightweight monitors
RBD
A reliable, fully distributed block device
with cloud platform integration
RGW
A web services gateway for object
storage, compatible with S3 and Swift
APP HOST/VM
CephFS
A distributed file system with POSIX
semantics & scale-out metadata
Ceph Architecture

9. OSDs (Object Storage Daemon)
• 10s to 10000s in a cluster
• Typically one daemon per physical HDD
• Serve stored data to clients
• Intelligently peer for replication & recovery
Monitors
• Maintain cluster membership and state
• Provide consensus for distributed decision-making
• Small, odd number
• Do not store data
Ceph Components

10. RADOS CLUSTER
APPLICATION
RADOS Cluster a.k.a Ceph Cluster

11. APPLICATION
OBJECTS
CRUSH ALGORITHM
LIBRADOS
How data is stored in Ceph

12. Ceph Access Methods

13. LIBRADOS
A library allowing apps to directly access RADOS (C, C++, Java, Python, Ruby)
RADOS
A software-based reliable, autonomous, distributed object store comprised of
self-healing, self-managing, intelligent storage nodes and lightweight monitors
RBD
A reliable, fully distributed block device
with cloud platform integration
RGW
A web services gateway for object
storage, compatible with S3 and Swift
APP HOST/VM
CephFS
A distributed file system with POSIX
semantics & scale-out metadata
Librados : Native Cluster Access
APP

14. LIBRADOS
A library allowing apps to directly access RADOS (C, C++, Java, Python, Ruby)
RADOS
A software-based reliable, autonomous, distributed object store comprised of
self-healing, self-managing, intelligent storage nodes and lightweight monitors
RBD
A reliable, fully distributed block device
with cloud platform integration
RGW
A web services gateway for object
storage, compatible with S3 and Swift
APP HOST/VM
CephFS
A distributed file system with POSIX
semantics & scale-out metadata
Librados : Base Access Layer

15. LIBRADOS
A library allowing apps to directly access RADOS (C, C++, Java, Python, Ruby)
RADOS
A software-based reliable, autonomous, distributed object store comprised of
self-healing, self-managing, intelligent storage nodes and lightweight monitors
RBD
A reliable, fully distributed block device
with cloud platform integration
RGW
A web services gateway for object
storage, compatible with S3 and Swift
APP HOST/VM
CephFS
A distributed file system with POSIX
semantics & scale-out metadata
RBD : Block Storage Access

16. VMHYPERVISOR
LIBRBD
HYPERVISOR
LIBRBD
RADOS CLUSTER
RBD : Separating Compute from Storage

17. LIBRADOS
A library allowing apps to directly access RADOS (C, C++, Java, Python, Ruby)
RADOS
A software-based reliable, autonomous, distributed object store comprised of
self-healing, self-managing, intelligent storage nodes and lightweight monitors
RBD
A reliable, fully distributed block device
with cloud platform integration
RGW
A web services gateway for object
storage, compatible with S3 and Swift
APP HOST/VM
CephFS
A distributed file system with POSIX
semantics & scale-out metadata
Ceph Rados Gateway : Object Storage Access

18. RADOSGWRADOSGW
LIBRADOS
REST
Socket
CEPH CLUSTER
LIBRADOS
Ceph Rados Gateway : Object Storage Access

19. LIBRADOS
A library allowing apps to directly access RADOS (C, C++, Java, Python, Ruby)
RADOS
A software-based reliable, autonomous, distributed object store comprised of
self-healing, self-managing, intelligent storage nodes and lightweight monitors
RBD
A reliable, fully distributed block device
with cloud platform integration
RGW
A web services gateway for object
storage, compatible with S3 and Swift
APP HOST/VM
CephFS
A distributed file system with POSIX
semantics & scale-out metadata
CephFS : File Storage

20. Linux Host
LIBCEPHFS
CEPH CLUSTER
CephFS : File Storage

21. Ceph Use Cases

22. GLANCE SWIFT NOVA CINDER MANILA
RADOSGW
INSTANCE
IMAGE
LIBRBD
SWIFT OBJECT BOOT IMAGE
LIBRBD
BLOCK
STORAGE
LIBRBD
FILE SHARE
CEPHFS
STORE
GET
PUT
...
BOOT
BLOCK
DEVICE
MOUNT
The complete storage solution for OpenStack

23. CEPH CLUSTER
Web Scale Object Storage
RGW RGW RGW RGWRGW
Load Balancer Load Balancer

24. Keystone
NeutronNova
Cinder
Horizon
RabbitMQ
OpenStack
Controller
Ceph
Monitor
Glance
OSPd, Ceph
installer
nova compute,
tenant VMs, ceph-
osd
MariaDB-
Galera
NIC3
NIC2
NIC1
NIC1
NIC1
NIC2
NIC4
NIC3
NIC4
HCI : OpenStack and Ceph
https://goo.gl/8gv5un

25. Ceph : What's New ?

26. BlueStore : New OSD Backend
● BlueStore = Block + New Store
○ KV DB (rocksdb) for metadata
○ All data written directly to raw device
○ Can combine HDD, SSD , NVMe
● Replaces FileStore, which was designed for HDDs
● Eliminates the need for an underlying filesystem or dedicated journal device
● Provides a 2-3X performance boost
● Full data checksum ( CRC32, hash )
● Inline Compression ( zlib , snappy , zstc)

27. Blue Store
Compression
Over The Wire encryption
Management
Ceph-mgr
Tendrl GUI Management
Console
QoS
Ceph in Containers
RBD
Erasure Coding Support
Volume Mirroring
iSCSI
Ceph : What's New ?
RGW
RGW Metadata Indexing
LDAP / AD
Keystone v3
Object expiration
Static Websites
NFS Gateway for RGW
MDS
Active / Active MDS support
for CephFS
Repair and DR tools
OpenStack Manila Support