1. Live Podcasting on AWS
marco pracucci
Follow me on Twitter @pracucci
giovedì 8 marzo 2012

2. Why AWS
Scalability
Handle peaks smoothly
and keep focus on business
giovedì 8 marzo 2012

3. AWS Usage
The slide of acronyms
EC2 Servers web, api, db, streaming, ... 20 - 30 instances
giovedì 8 marzo 2012

4. AWS Usage
The slide of acronyms
EC2 Servers web, api, db, streaming, ... 20 - 30 instances
ELB Balance HTTP traffic 3 ELB
giovedì 8 marzo 2012

5. AWS Usage
The slide of acronyms
EC2 Servers web, api, db, streaming, ... 20 - 30 instances
ELB Balance HTTP traffic 3 ELB
EBS Dynamic data storage pg, mongo, ... 500 GB
giovedì 8 marzo 2012

6. AWS Usage
The slide of acronyms
EC2 Servers web, api, db, streaming, ... 20 - 30 instances
ELB Balance HTTP traffic 3 ELB
EBS Dynamic data storage pg, mongo, ... 500 GB
S3 Static data storage mp3, images, ... 15 TB
giovedì 8 marzo 2012

7. AWS Usage
The slide of acronyms
EC2 Servers web, api, db, streaming, ... 20 - 30 instances
ELB Balance HTTP traffic 3 ELB
EBS Dynamic data storage pg, mongo, ... 500 GB
S3 Static data storage mp3, images, ... 15 TB
CF Static data CDN 50M req / month
giovedì 8 marzo 2012

8. AWS Usage
The slide of acronyms
EC2 Servers web, api, db, streaming, ... 20 - 30 instances
ELB Balance HTTP traffic 3 ELB
EBS Dynamic data storage pg, mongo, ... 500 GB
S3 Static data storage mp3, images, ... 15 TB
CF Static data CDN 50M req / month
R53 Authoritative DNS 15M req / month
giovedì 8 marzo 2012

9. Scaling Spreaker
with AWS
a real day example
giovedì 8 marzo 2012

10. giovedì 8 marzo 2012
Requests / sec
0
40
80
120
160
200
00:0
0
01:0
0
02:0
0
03:0
0
04:0
0
05:0
0
06:0
0
07:0
0
08:0
0
09:0
0
10:0
0
11:0
0
12:0
0
13:0
0
14:0
0
Scaling Spreaker
15:0
0
16:0
0
17:0
0
The ideal traffic
18:0
0
19:0
0
20:0
0
21:0
0
22:0
0
23:0
0

11. giovedì 8 marzo 2012
Requests / sec
0
70
140
210
280
350
00:0
0
01:0
0
02:0
0
03:0
0
04:0
0
05:0
0
06:0
0
07:0
0
08:0
0
09:0
0
10:0
0
11:0
0
12:0
0
13:0
0
14:0
0
Scaling Spreaker
15:0
0
16:0
0
17:0
0
18:0
0
19:0
0
20:0
0
The almost real traffic
21:0
0
22:0
0
23:0
0

12. giovedì 8 marzo 2012
Requests / sec
0
70
140
210
280
350
00:0
0
01:0
0
02:0
0
03:0
0
04:0
0
05:0
0
06:0
0
07:0
0
08:0
0
09:0
0
10:0
0
11:0
0
12:0
0
13:0
0
14:0
0
Scaling Spreaker
15:0
0
16:0
Unpredictable
0
17:0
0
The real traffic
18:0
0
19:0
0
20:0
0
21:0
0
22:0
0
23:0
0

13. Requirements
Scaling Spreaker
Fast Automatized
New instances up and No human intervention
running in less then 5 mins
giovedì 8 marzo 2012

14. Auto-Scaling
Scaling Spreaker
TRACK UP / DOWN
ANALYZE
METRICS SCALE
giovedì 8 marzo 2012

15. Auto-Scaling
Scaling Spreaker
TRACK UP / DOWN
ANALYZE
METRICS SCALE
Zenoss + Logstash Skynet Spreaker + EC2 API
giovedì 8 marzo 2012

16. Auto-Scaling
Scaling Spreaker
TRACK UP / DOWN
ANALYZE
METRICS SCALE
Zenoss + Logstash Skynet Spreaker + EC2 API
{
You can also use
CloudWatch
giovedì 8 marzo 2012

17. Auto-Scaling
Scaling Spreaker
1. Split the infrastructure into roles
web
streaming
api
giovedì 8 marzo 2012

18. Auto-Scaling
Scaling Spreaker
1. Split the infrastructure into roles
db-master
web
gearman
streaming
db-slave
redis
api
recording search
memcache
encoder
mongo
web worker
giovedì 8 marzo 2012

19. Auto-Scaling
Scaling Spreaker
2. Assign roles to each instance (using EC2 tags)
Instance 1 Instance 2 Instance 3
web web streaming
api api
giovedì 8 marzo 2012

20. Auto-Scaling
Scaling Spreaker
3. Start roles at instance boot
a script fetch roles,
install code (latest build),
configure and start services
giovedì 8 marzo 2012

21. Auto-Scaling
Scaling Spreaker
4. Ready to auto-scale
To add another web server,
turn on a new instance with web role
giovedì 8 marzo 2012