3. Kalibrr on AWS
! Kalibrr’s Infrastructure
• EC2 instances to host their Pyramid and SockJS-Tornado-based web server
• SES for mail, S3 for content
• EC2 instances are configured with EBS,
and we load balance using ELB
• Manage our DNS records with Route 53
AWS Cloud Kata for Start-Ups and Developers
4. How AWS Helped Get us to MVP
! B2C Launch March 2013
•
0 > 25,000 Users in 2 months
•
Average user is on the site for 45 minutes
•
Up to 200 simultaneous users taking assessments
•
No downtime, no issues with latency = We looked like a bigger company than we really were
•
Now we’re working on our B2B business and selling our platform directly to MNCs – speed,
uptime, reliability are going to be key to us winning (most competitors run natively)
AWS Cloud Kata for Start-Ups and Developers
5. Kalibrr & AWS
! Why Kalibrr Chose AWS?
• Easy, cheap and allows us to focus on the product and not on the
infrastructure
• You’re up in minutes, you’re redundant and load balanced with ease
! How We Launched on AWS?
• IaaS (Infrastructure as a Service) + Great support from Amazon
!
Kalibrr’s Savings
• No hardware to be purchased
• Focused more on product development and less on Dev Ops
AWS Cloud Kata for Start-Ups and Developers
9. 503
Service Temporarily Unavailable
The server is temporarily unable
to service your request due to
maintenance downtime or capacity
problems. Please try again later.
18. Database Options
Self-Managed
Database Server
on Amazon EC2
Your choice of
database running on
Amazon EC2
Bring Your Own
License (BYOL)
Fully-Managed
Amazon RDS
Relational Database
as a managed
service
Flexible licensing:
BYOL or License
Included
Amazon
DynamoDB
Managed NoSQL
database service
using SSD storage
Seamless scalability
Zero administration
19. But how do I choose what
DB technology I need?
SQL? NoSQL?
24. Why SQL?
Established and well worn technology
Lots of existing code, communities, books, tools, etc
Clear patterns to scalability
You aren’t going to break SQL DBs in your first 10
million users. No really, you won’t
25. Amazon
Rela&onal
Database
Service
(RDS)
Feature
PlaAorm
support
Preconfigured
Automated
patching
Details
Create
MySQL,
SQL
Server
and
Oracle
Get
started
instantly
with
sensible
default
seAngs
Keep
your
database
plaEorm
up
to
date
automa&cally
Backups
Automa&c
backups
and
point
in
&me
recovery
using
snapshots
Manual
DB
snapshots
Failover
Automated
failover
to
slave
hosts
in
event
of
a
failure
• Database-as-a-Service
• No need to install or manage database
instances
• Scalable and fault tolerant configurations
ReplicaFon
Easily
create
read-‐replicas
of
your
data
and
seamlessly
replicate
data
across
availability
zones
26. Auto-Scaling
Automatic resizing of
compute clusters based on
demand
Feature
Amazon
CloudWatch
Trigger
auto-‐scaling
policy
Details
Control
Define
minimum
and
maximum
instance
pool
sizes
and
when
scaling
and
cool
down
occurs.
Integrated
to
Amazon
CloudWatch
Use
metrics
gathered
by
CloudWatch
to
drive
scaling.
Instance
types
Run
Auto
Scaling
for
On-‐Demand
and
Spot
Instances.
Compa&ble
with
VPC.
as-create-auto-scaling-group MyGroup
--launch-configuration MyConfig
--availability-zones us-east-1a
--min-size 4
--max-size 200
37. Database Query Performance
a)
b)
c)
d)
Choose
the
fastest
route
Offload
your
applica&on
servers
Cache
it
if
you
can
Single
digit
latencies
where
it
maWers
Dynamo
DB
Query
Performance
Average single-digit milliseconds server
side latencies
Runs on solid state drives, and is built to
maintain consistent, fast latencies at any
scale
DynamoDB
Low
latency
Large
scale
Zero
admin
Predictable
performance
Scale
39. Production 1.0 Architecture
Well-‐designed,
2
Tier
architecture
Highly
Available
due
to
Mul&ple
Availability
Zone
Load
Balancing
&
Auto-‐Scaling
for
full
scalability
Fully
managed
Database
included
Capable
of
serving
>10K-‐100Ks
users
41. Production 1.0 Architecture
Wasted
server
capacity
for
sta&c
content
Reliability
and
durability
are
not
yet
op&mal
End-‐user
experience
could
be
improved
thru
offloading
&
caching
44. Simple Storage Service (S3)
Feature
Flexible
object
store
Access
control
Server-‐side
encrypFon
MulF-‐part
uploads
Object
versioning
Object
expiry
Durable storage, any object
99.999999999% durability of objects
Unlimited storage of objects of any type
Up to 5TB size per object
Access
logging
Web
content
hosFng
NoFficaFons
Import/Export
Details
Buckets
act
like
drives,
folder
structures
within
Granular
control
over
object
permissions
256bit
AES
encryp&on
of
objects
Improved
throughput
&
control
Archive
old
objects
and
version
new
ones
Automa&cally
remove
old
objects
Full
audit
log
of
bucket/object
ac&ons
Serve
content
as
web
site
with
built
in
page
handling
Receive
no&fica&ons
on
key
events
Physical
device
import/export
service
45. CloudFront
• World-wide content distribution
network
• Easily distribute content to end
users with low latency, high data
transfer speeds, and no
commitments
Feature
Fast
Integrated
with
other
services
Dynamic
content
Streaming
Details
Mul&ple
world-‐wide
edge
loca&ons
to
serve
content
as
close
to
your
users
as
possible
Works
seamlessly
with
S3
and
EC2
origin
servers
Supports
sta&c
and
dynamic
content
from
origin
servers
Supports
rtmp
from
S3
and
includes
support
for
live
streaming
from
Adobe
FMS
and
Microsoe
Media
Server
47. Production 1.2 Architecture
Well-‐designed,
2
Tier
architecture
Highly
Available
due
to
Mul&ple
Availability
Zone
Load
Balancing
&
Auto-‐Scaling
for
full
scalability
Fully
managed
Database
included
Sta&c
content
stored
in
durable,
consistent
way
Improved
end-‐user
experience
through
CDN
Capable
of
serving
>100K-‐1M+
users
53. Elastic MapReduce (EMR)
• Managed, elastic Hadoop cluster
• Integrates with S3 & DynamoDB
• Leverage Hive & Pig analytics scripts
Feature
Scalable
Integrated
with
other
services
Comprehensive
Cost
effecFve
Monitoring
Details
Use
as
many
or
as
few
compute
instances
running
Hadoop
as
you
want.
Modify
the
number
of
instances
while
your
job
flow
is
running
Works
seamlessly
with
S3
as
origin
and
output.
Integrates
with
DynamoDB
Supports
languages
such
as
Hive
and
Pig
for
defining
analy&cs,
and
allows
complex
defini&ons
in
Cascading,
Java,
Ruby,
Perl,
Python,
PHP,
R,
or
C++
Works
with
Spot
instance
types
Monitor
job
flows
from
with
the
management
console
54. Foursquare…
…generates a lot of Data
Founded in 2009
112M in Venture Capital
33 million users
1.3 million businesses using the service
3.5 billion check-ins
15M+ venues,
Terabytes of log data
55. Uses EMR for
Evaluation of new features
Machine learning
Exploratory analysis
Daily customer usage reporting
Long-term trend analysis
56. Benefits of EMR
Ease-of-Use
“We have decreased the processing time for urgent data-analysis”
Flexibility
To deal with changing requirements & dynamically expand reporting clusters
Costs
“We have reduced our analytics costs by over 50%”
58. Production 1.3 Architecture
Well-‐designed,
2
Tier
architecture
Highly
Available
due
to
Mul&ple
Availability
Zone
Load
Balancing
&
Auto-‐Scaling
for
full
scalability
Sta&c
content
stored
in
durable,
consistent
way
Improved
end-‐user
experience
through
CDN
Big
Data
analy&cs
built
in
for
con&nuous
op&miza&on
Capable
of
serving
>1m-‐10M+
users
