Openstack Swift Introduction

Swift
Highly available, distributed, eventually consistent object
storage
Youn Sung Park

March 16, 2015 Presentation Title
2
1. Introduction
2. Features - Basic
3. Features - Advanced
Contents

Contents
• What is Swift?
Object Storage
Swift component
March 16,
2015
Swift
4
Introduction

Object Storage
• Object Storage – What is Object Storage?
March 16,
2015
Swift
5
Introduction
Swift
- Highly available, distributed, eventually
consistent object storage

Object Storage
March 16,
2015
Swift
6
Introduction

Object Storage
March 16,
2015
Swift
7
Introduction
Object storage
vs File storage
vs Volume storage

Object Storage
• Object Storage – Legacy Storage
March 16,
2015
Swift
8
Introduction
WAS WAS WAS
NAS / SAN NAS / SAN NAS / SAN NAS / SAN
…
…

Object Storage
March 16,
2015
Swift
9
Introduction
WAS WAS WAS
NAS / SAN NAS / SAN NAS / SAN NAS / SAN
…
…
Storage manager

Object Storage
• Fault tolerance – RAID5, 6, 10, ...
• Interface – NFS for file, iSCSI for volume
• Multi tenancy - ??
• QoS - ??
• Authentication - ??
• Data Placement - ??
• Metadata management - ??
• Capacity management - ??
• Rebalancing - ??
March 16,
2015
Swift
10
Introduction

Object Storage
• Object Storage
March 16,
2015
Swift
11
Introduction
WAS WAS WAS
Object Storage
…

Object Storage
• Fault tolerance – Replication
• Interface – REST API
• Multi tenancy – Defaut. SWAuth, Keystone, Tempauth, LDAP(?)
• QoS – Rate limit, Bandwidth limit, Software defined ...
• Authentication - SWAuth, Keystone, Tempauth
• Data Placement – The Ring algorithm
• Metadata management – Ring and database
• Capacity management – The Ring algorithm
• Rebalancing – The Ring algorithm
March 16,
2015
Swift
12
Introduction

Components
• Swift component – Server application
March 16,
2015
Swift
13
Introduction
Proxy Server
Account Server
Container Server
Object Server
Sqlite
Container List
Object List
Objects
Sqlite

Components
• Swift component – Daemon
• Replicator – Replicate objects and make a system in a consistent state.
• Recover disk failure, network outages situation.
• Updater – Update metadata
• Recover failure caused by container, account metadata high load.
• Auditor – Delete problematic account, container or objects and replicate from other server
• Recover dbs or files which have bit rot problem.
March 16,
2015
Swift
14
Introduction

Components
• Swift component – Key algorithm: Data placement, Metadata management
March 16,
2015
Swift
15
Introduction
RingContainer name/file name Location of a file
ex) 3 copy
Server A / sdc
Server H / sdy
Server D / sdp

Contents
• Swift features
History – Releases
Architecture
High availability
High scalability
Eventually consistent & Self healing
REST API
March 16,
2015
Swift
17
Features - Basic

History - Releases
March 16,
2015
Swift
18
Features - Basic

Architecture
• Architecture
March 16,
2015
Swift
19
Features - Basic
Proxy Server Proxy Server Proxy Server
A/C/O Server
A/C/O Server
A/C/O Server
Zone1
A/C/O Server
…
A/C/O Server
A/C/O Server
A/C/O Server
Zone2
A/C/O Server
…
A/C/O Server
A/C/O Server
A/C/O Server
Zone3
A/C/O Server
…
A/C/O Server
A/C/O Server
A/C/O Server
Zone4
A/C/O Server
…
A/C/O Server
A/C/O Server
A/C/O Server
Zone5
A/C/O Server
…
…
…
Sqlite
Account server
Container server
Object server
Sqlite
Ring Ring Ring

Features
• High availability, Fault tolerance
March 16,
2015
Swift
20
Features - Basic
A/C/O Server
A/C/O Server
A/C/O Server
Zone1
A/C/O Server
…
A/C/O Server
A/C/O Server
A/C/O Server
Zone2
A/C/O Server
…
A/C/O Server
A/C/O Server
A/C/O Server
Zone3
A/C/O Server
…
A/C/O Server
A/C/O Server
A/C/O Server
Zone4
A/C/O Server
…
A/C/O Server
A/C/O Server
A/C/O Server
Zone5
A/C/O Server
…
…
…
Sqlite
Account server
Container server
Object server
Sqlite
Ring Ring Ring

Features
• High scalability
March 16,
2015
Swift
21
Features - Basic
A/C/O Server
HDDHDDHDDHDD
A/C/O Server
HDDHDDHDDHDD
HDDHDDHDDHDD
More disk
Zone1
Zone1
Zone1
Zone1
Zone5
Zone1
Zone1
Zone1
Zone1
Zone1
Zone1
Zone7
More zones
A/C/O Server
A/C/O Server
Zone
A/C/O Server
A/C/O Server
Zone
A/C/O Server
A/C/O Server
More servers

Features
• Eventually consistent & self healing
March 16,
2015
Swift
22
Features - Basic
A/C/O Server
A/C/O Server
A/C/O Server
Zone1
A/C/O Server
…
A/C/O Server
A/C/O Server
A/C/O Server
Zone2
A/C/O Server
…
A/C/O Server
A/C/O Server
A/C/O Server
Zone3
A/C/O Server
…
A/C/O Server
A/C/O Server
A/C/O Server
Zone4
A/C/O Server
…
A/C/O Server
A/C/O Server
A/C/O Server
Zone5
A/C/O Server
…
…
…
Sqlite
Account server
Container server
Object server
Sqlite
Fail
Eventual consistency
Ring Ring Ring

Features
• Eventually consistent & self healing
March 16,
2015
Swift
23
Proxy Server Proxy Server
A/C/O Server
A/C/O Server
Zone1
A/C/O Server
…
A/C/O Server
A/C/O Server
A/C/O Server
Zone2
A/C/O Server
…
A/C/O Server
A/C/O Server
A/C/O Server
Zone3
A/C/O Server
…
A/C/O Server
A/C/O Server
A/C/O Server
Zone4
A/C/O Server
…
A/C/O Server
A/C/O Server
A/C/O Server
Zone5
A/C/O Server
…
…
…
Sqlite
Account server
Container server
Object server
Sqlite
Server
failure
Self healing
- replication
A/C/O Server
Proxy Server
Features - Basic
Ring RingRing

Contents
• Swift features
History – Features
Features
Large object support
Static web hosting
S3 compatible API
Object expiration
Temp url
Object versioning
Global cluster
Storage policy
March 16,
2015
Swift
25
Features - Advanced

History - Features
March 16,
2015
Swift
26
Features - Advanced
Swift 2.2 Release
Storage policy support
Swift 1.10 Release
Global clusters support
Swift 1.8 Release
Different replica policy
for A/C/O
StatsD update
Swift 1.7 Release
Object versioning support
Statd integration
Swift 1.4.8 Release
Object expiration support
Temp url support
Swift 1.4.3 Release
S3 compatible API support
Swift 1.3 Release
Static website support
Swift 1.2 Release
Large object support
Swift 1.0 Release
Swift 1.12 Release
Account ACL support

Features
• Large object support
• Swift limitation
• Single object: 5GB
• Split object & manage large object
• Manage segmented objects by manifest file
• Ref
• http://docs.openstack.org/developer/swift/overview_large_objects.html
March 16,
2015
Swift
27
Features - Advanced

Features
• Static web hosting
• Upload static web file and make web site
• Upload web site file with index and error files
• Use statiscweb middleware
• Ref
• http://docs.openstack.org/developer/swift/middleware.html#staticweb
March 16,
2015
Swift
28
Features - Advanced

Features
• S3 compatible API
• Support S3 API
• Support limited API less than 40%
• Use swift3 middleware
• Ref
• https://github.com/stackforge/swift3
March 16,
2015
Swift
29
Features - Advanced

Features
• Object expiration
• Schedule deletion of objects
• Use X-Delete-At and X-Delete-After header while using an object PUT or POST
•X-Delete-At: Delete object at specified time
•X-Delete-After: Delete object after specified time
• Ref
• http://docs.openstack.org/developer/swift/overview_expiring_objects.html
March 16,
2015
Swift
30
Features - Advanced

Features
• Temp url
• Provide url to access in limited time
• Need temp_url_expires time in header
• Use temporary URL middleware
• Ref
• http://docs.openstack.org/developer/swift/api/temporary_url_middleware.html
March 16,
2015
Swift
31
Features - Advanced

Features
• Global cluster
• Make a single cluster in distant region
• Read/Write affinity
• Deferred replication
• Ref
• http://docs.openstack.org/developer/swift/admin_guide.html
• https://swiftstack.com/blog/2012/09/16/globally-distributed-openstack-swift-cluster/
March 16,
2015
Swift
32
Features - Advanced

Features
• Storage policy
• Support various policy in sing storage cluster
• Use multiple ring file
• Ref
• http://docs.openstack.org/developer/swift/admin_guide.html
March 16,
2015
Swift
33
Features - Advanced

Openstack Swift Introduction

Recommandé

Recommandé

Contenu connexe

Tendances

Tendances (20)

En vedette

En vedette (20)

Similaire à Openstack Swift Introduction

Similaire à Openstack Swift Introduction (20)

Dernier

Dernier (20)

Openstack Swift Introduction