SlideShare une entreprise Scribd logo
1  sur  167
Télécharger pour lire hors ligne
INFRASTRUCTURE
as CODE
Running Microservices on AWS with
Docker, Terraform, and ECS
Why infrastructure-as-code
matters: a short story.
You are starting a
new project
I know, I’ll use Ruby on Rails!
> gem install rails
> gem install rails
Fetching: i18n-0.7.0.gem (100%)
Fetching: json-1.8.3.gem (100%)
Building native extensions. This could take a while...
ERROR: Error installing rails:
ERROR: Failed to build gem native extension.
/usr/bin/ruby1.9.1 extconf.rb
creating Makefile
make
sh: 1: make: not found
Ah, I just need to install make
> sudo apt-get install make
...
Success!
> gem install rails
> gem install rails
Fetching: nokogiri-1.6.7.2.gem (100%)
Building native extensions. This could take a while...
ERROR: Error installing rails:
ERROR: Failed to build gem native extension.
/usr/bin/ruby1.9.1 extconf.rb
checking if the C compiler accepts ... yes
Building nokogiri using packaged libraries.
Using mini_portile version 2.0.0.rc2
checking for gzdopen() in -lz... no
zlib is missing; necessary for building libxml2
*** extconf.rb failed ***
Hmm. Time to visit StackOverflow.
> sudo apt-get install zlib1g-dev
...
Success!
> gem install rails
> gem install rails
Building native extensions. This could take a while...
ERROR: Error installing rails:
ERROR: Failed to build gem native extension.
/usr/bin/ruby1.9.1 extconf.rb
checking if the C compiler accepts ... yes
Building nokogiri using packaged libraries.
Using mini_portile version 2.0.0.rc2
checking for gzdopen() in -lz... yes
checking for iconv... yes
Extracting libxml2-2.9.2.tar.gz into tmp/x86_64-pc-linux-
gnu/ports/libxml2/2.9.2... OK
*** extconf.rb failed ***
nokogiri y u never install correctly?
(Spend 2 hours trying random
StackOverflow suggestions)
> gem install rails
> gem install rails
...
Success!
Finally!
> rails new my-project
> cd my-project
> rails start
> rails new my-project
> cd my-project
> rails start
/source/my-project/bin/spring:11:in `<top (required)>':
undefined method `path_separator' for Gem:Module
(NoMethodError)
from bin/rails:3:in `load'
from bin/rails:3:in `<main>'
Infrastructure as code: running microservices on AWS using Docker, Terraform, and ECS
Eventually, you get it working
Now you have to deploy your
Rails app in production
You use the AWS Console to
deploy an EC2 instance
> ssh ec2-user@ec2-12-34-56-78.compute-1.amazonaws.com
__| __|_ )
_| ( / Amazon Linux AMI
___|___|___|
[ec2-user@ip-172-31-61-204 ~]$ gem install rails
> ssh ec2-user@ec2-12-34-56-78.compute-1.amazonaws.com
__| __|_ )
_| ( / Amazon Linux AMI
___|___|___|
[ec2-user@ip-172-31-61-204 ~]$ gem install rails
ERROR: Error installing rails:
ERROR: Failed to build gem native extension.
/usr/bin/ruby1.9.1 extconf.rb
Infrastructure as code: running microservices on AWS using Docker, Terraform, and ECS
Eventually you get it working
Infrastructure as code: running microservices on AWS using Docker, Terraform, and ECS
Now you urgently have to update
all your Rails installs
> bundle update rails
> bundle update rails
Building native extensions. This could take a while...
ERROR: Error installing rails:
ERROR: Failed to build gem native extension.
/usr/bin/ruby1.9.1 extconf.rb
checking if the C compiler accepts ... yes
Building nokogiri using packaged libraries.
Using mini_portile version 2.0.0.rc2
checking for gzdopen() in -lz... yes
checking for iconv... yes
Extracting libxml2-2.9.2.tar.gz into tmp/x86_64-pc-linux-
gnu/ports/libxml2/2.9.2... OK
*** extconf.rb failed ***
Infrastructure as code: running microservices on AWS using Docker, Terraform, and ECS
The problem isn’t Rails
> ssh ec2-user@ec2-12-34-56-78.compute-1.amazonaws.com
__| __|_ )
_| ( / Amazon Linux AMI
___|___|___|
[ec2-user@ip-172-31-61-204 ~]$ gem install rails
The problem is that you’re
configuring servers manually
And that you’re deploying
infrastructure manually
A better alternative: infrastructure-
as-code
In this talk, we’ll go through a
real-world example:
We’ll configure & deploy two
microservices on Amazon ECS
With two infrastructure-as-code
tools: Docker and Terraform
TERRAFORM
I’m
Yevgeniy
Brikman
ybrikman.com
Co-founder of
Gruntwork
gruntwork.io
gruntwork.io
We offer DevOps
as a Service
gruntwork.io
And DevOps
as a Library
PAST LIVES
Author of
Hello,
Startup
hello-startup.net
And
Terraform:
Up & Running
terraformupandrunning.com
Slides and code from this talk:
ybrikman.com/speaking
1. Microservices
2. Docker
3. Terraform
4. ECS
5. Recap
Outline
1. Microservices
2. Docker
3. Terraform
4. ECS
5. Recap
Outline
Code is the enemy: the more you
have, the slower you go
Project Size
Lines of code
Bug Density
Bugs per thousand lines
of code
< 2K 0 – 25
2K – 6K 0 – 40
16K – 64K 0.5 – 50
64K – 512K 2 – 70
> 512K 4 – 100
As the code grows, the number of
bugs grows even faster
“Software
development doesn't
happen in a chart, an
IDE, or a design tool;
it happens in your
head.”
The mind can only handle so
much complexity at once
One solution is to break the code
into microservices
In a monolith, you use function
calls within one process
moduleA.func()
moduleB.func() moduleC.func() moduleD.func()
moduleE.func()
http://service.a
http://service.b http://service.c http://service.d
http://service.e
With services, you pass messages
between processes
Advantages of services:
1. Isolation
2. Technology agnostic
3. Scalability
Disadvantages of services:
1. Operational overhead
2. Performance overhead
3. I/O, error handling
4. Backwards compatibility
5. Global changes, transactions,
referential integrity all very hard
For more info, see: Splitting Up a
Codebase into Microservices and
Artifacts
For this talk, we’ll use two
example microservices
require 'sinatra'
get "/" do
"Hello, World!"
end
A sinatra backend that returns
“Hello, World”
class ApplicationController < ActionController::Base
def index
url = URI.parse(backend_addr)
req = Net::HTTP::Get.new(url.to_s)
res = Net::HTTP.start(url.host, url.port) {|http|
http.request(req)
}
@text = res.body
end
end
A rails frontend that calls the
sinatra backend
<h1>Rails Frontend</h1>
<p>
Response from the backend: <strong><%= @text %></strong>
</p>
And renders the response as
HTML
1. Microservices
2. Docker
3. Terraform
4. ECS
5. Recap
Outline
Docker allows you to build and
run code in containers
Containers are like lightweight
Virtual Machines (VMs)
VMs virtualize the hardware and run an
entire guest OS on top of the host OS
VM
Hardware
Host OS
Host User Space
Virtual Machine
Virtualized
hardware
Guest OS
Guest User
Space
App
VM
Virtualized
hardware
Guest OS
Guest User
Space
App
VM
Virtualized
hardware
Guest OS
Guest User
Space
App
This provides good isolation, but lots of
CPU, memory, disk, & startup overhead
VM
Hardware
Host OS
Host User Space
Virtual Machine
Virtualized
hardware
Guest OS
Guest User
Space
App
VM
Virtualized
hardware
Guest OS
Guest User
Space
App
VM
Virtualized
hardware
Guest OS
Guest User
Space
App
Containers virtualize User Space (shared
memory, processes, mount, network)
Container
VM
Hardware
Host OS
Host User Space
Virtual Machine
Virtualized
hardware
Guest OS
Guest User
Space
App
Hardware
Host OS
Host User Space
Container Engine
Virtualized
User Space
VM
Virtualized
hardware
Guest OS
Guest User
Space
App
VM
Virtualized
hardware
Guest OS
Guest User
Space
App
App
Container
Virtualized
User Space
App
Container
Virtualized
User Space
App
Container
VM
Hardware
Host OS
Host User Space
Virtual Machine
Virtualized
hardware
Guest OS
Guest User
Space
App
Hardware
Host OS
Host User Space
Container Engine
Virtualized
User Space
VM
Virtualized
hardware
Guest OS
Guest User
Space
App
VM
Virtualized
hardware
Guest OS
Guest User
Space
App
App
Container
Virtualized
User Space
App
Container
Virtualized
User Space
App
Isolation isn’t as good but much less CPU,
memory, disk, startup overhead
> docker run –it ubuntu bash
root@12345:/# echo "I'm in $(cat /etc/issue)”
I'm in Ubuntu 14.04.4 LTS
Running Ubuntu in a Docker
container
> time docker run ubuntu echo "Hello, World"
Hello, World
real 0m0.183s
user 0m0.009s
sys 0m0.014s
Containers boot very quickly.
Easily run a dozen at once.
You can define a Docker image
as code in a Dockerfile
FROM gliderlabs/alpine:3.3
RUN apk --no-cache add ruby ruby-dev
RUN gem install sinatra --no-ri --no-rdoc
RUN mkdir -p /usr/src/app
COPY . /usr/src/app
WORKDIR /usr/src/app
EXPOSE 4567
CMD ["ruby", "app.rb"]
Here is the Dockerfile for the
Sinatra backend
FROM gliderlabs/alpine:3.3
RUN apk --no-cache add ruby ruby-dev
RUN gem install sinatra --no-ri --no-rdoc
RUN mkdir -p /usr/src/app
COPY . /usr/src/app
WORKDIR /usr/src/app
EXPOSE 4567
CMD ["ruby", "app.rb"]
It specifies dependencies, code,
config, and how to run the app
> docker build -t brikis98/sinatra-backend .
Step 0 : FROM gliderlabs/alpine:3.3
---> 0a7e169bce21
(...)
Step 8 : CMD ruby app.rb
---> 2e243eba30ed
Successfully built 2e243eba30ed
Build the Docker image
> docker run -it -p 4567:4567 brikis98/sinatra-backend
INFO WEBrick 1.3.1
INFO ruby 2.2.4 (2015-12-16) [x86_64-linux-musl]
== Sinatra (v1.4.7) has taken the stage on 4567 for
development with backup from WEBrick
INFO WEBrick::HTTPServer#start: pid=1 port=4567
Run the Docker image
> docker push brikis98/sinatra-backend
The push refers to a repository [docker.io/brikis98/sinatra-
backend] (len: 1)
2e243eba30ed: Image successfully pushed
7e2e0c53e246: Image successfully pushed
919d9a73b500: Image successfully pushed
(...)
v1: digest: sha256:09f48ed773966ec7fe4558 size: 14319
You can share your images by
pushing them to Docker Hub
Now you can reuse the same
image in dev, stg, prod, etc
> docker pull rails:4.2.6
And you can reuse images created
by others.
FROM rails:4.2.6
RUN mkdir -p /usr/src/app
COPY . /usr/src/app
WORKDIR /usr/src/app
RUN bundle install
EXPOSE 3000
CMD ["rails", "start"]
The rails-frontend is built on top of
the official rails Docker image
No more insane install procedures!
rails_frontend:
image: brikis98/rails-frontend
ports:
- "3000:3000"
links:
- sinatra_backend:sinatra_backend
sinatra_backend:
image: brikis98/sinatra-backend
ports:
- "4567:4567"
Define your entire dev stack as
code with docker-compose
rails_frontend:
image: brikis98/rails-frontend
ports:
- "3000:3000"
links:
- sinatra_backend
sinatra_backend:
image: brikis98/sinatra-backend
ports:
- "4567:4567"
Docker links provide a simple
service discovery mechanism
> docker-compose up
Starting infrastructureascodetalk_sinatra_backend_1
Recreating infrastructureascodetalk_rails_frontend_1
sinatra_backend_1 | INFO WEBrick 1.3.1
sinatra_backend_1 | INFO ruby 2.2.4 (2015-12-16)
sinatra_backend_1 | Sinatra has taken the stage on 4567
rails_frontend_1 | INFO WEBrick 1.3.1
rails_frontend_1 | INFO ruby 2.3.0 (2015-12-25)
rails_frontend_1 | INFO WEBrick::HTTPServer#start: port=3000
Run your entire dev stack with one
command
Advantages of Docker:
1. Easy to create & share images
2. Images run the same way in all
environments (dev, test, prod)
3. Easily run the entire stack in dev
4. Minimal overhead
5. Better resource utilization
Disadvantages of Docker:
1. Maturity. Ecosystem developing
very fast, but still a ways to go
2. Tricky to manage persistent data in
a container
3. Tricky to pass secrets to containers
1. Microservices
2. Docker
3. Terraform
4. ECS
5. Recap
Outline
Terraform is a tool for
provisioning infrastructure
Terraform supports many
providers (cloud agnostic)
And many resources for each
provider
You define infrastructure as code
in Terraform templates
provider "aws" {
region = "us-east-1"
}
resource "aws_instance" "example" {
ami = "ami-408c7f28"
instance_type = "t2.micro"
}
This template creates a single EC2
instance in AWS
> terraform plan
+ aws_instance.example
ami: "" => "ami-408c7f28"
instance_type: "" => "t2.micro"
key_name: "" => "<computed>"
private_ip: "" => "<computed>"
public_ip: "" => "<computed>"
Plan: 1 to add, 0 to change, 0 to destroy.
Use the plan command to see
what you’re about to deploy
> terraform apply
aws_instance.example: Creating...
ami: "" => "ami-408c7f28"
instance_type: "" => "t2.micro"
key_name: "" => "<computed>"
private_ip: "" => "<computed>"
public_ip: "" => "<computed>”
aws_instance.example: Creation complete
Apply complete! Resources: 1 added, 0 changed, 0 destroyed.
Use the apply command to apply
the changes
Now our EC2 instance is running!
resource "aws_instance" "example" {
ami = "ami-408c7f28"
instance_type = "t2.micro"
tags {
Name = "terraform-example"
}
}
Let’s give the EC2 instance a tag
with a readable name
> terraform plan
~ aws_instance.example
tags.#: "0" => "1"
tags.Name: "" => "terraform-example"
Plan: 0 to add, 1 to change, 0 to destroy.
Use the plan command again to
verify your changes
> terraform apply
aws_instance.example: Refreshing state...
aws_instance.example: Modifying...
tags.#: "0" => "1"
tags.Name: "" => "terraform-example"
aws_instance.example: Modifications complete
Apply complete! Resources: 0 added, 1 changed, 0 destroyed.
Use the apply command again to
deploy those changes
Now our EC2 instance has a tag!
resource "aws_elb" "example" {
name = "example"
availability_zones = ["us-east-1a", "us-east-1b"]
instances = ["${aws_instance.example.id}"]
listener {
lb_port = 80
lb_protocol = "http"
instance_port = "${var.instance_port}"
instance_protocol = "http”
}
}
Let’s add an Elastic Load Balancer
(ELB).
resource "aws_elb" "example" {
name = "example"
availability_zones = ["us-east-1a", "us-east-1b"]
instances = ["${aws_instance.example.id}"]
listener {
lb_port = 80
lb_protocol = "http"
instance_port = "${var.instance_port}"
instance_protocol = "http”
}
}
Terraform supports variables,
such as var.instance_port
resource "aws_elb" "example" {
name = "example"
availability_zones = ["us-east-1a", "us-east-1b"]
instances = ["${aws_instance.example.id}"]
listener {
lb_port = 80
lb_protocol = "http"
instance_port = "${var.instance_port}"
instance_protocol = "http"
}
}
As well as dependencies like
aws_instance.example.id
resource "aws_elb" "example" {
name = "example"
availability_zones = ["us-east-1a", "us-east-1b"]
instances = ["${aws_instance.example.id}"]
listener {
lb_port = 80
lb_protocol = "http"
instance_port = "${var.instance_port}"
instance_protocol = "http"
}
}
It builds a dependency graph and
applies it in parallel.
After running apply, we have an ELB!
> terraform destroy
aws_instance.example: Refreshing state... (ID: i-f3d58c70)
aws_elb.example: Refreshing state... (ID: example)
aws_elb.example: Destroying...
aws_elb.example: Destruction complete
aws_instance.example: Destroying...
aws_instance.example: Destruction complete
Apply complete! Resources: 0 added, 0 changed, 2 destroyed.
Use the destroy command to
delete all your resources
For more info, check out The Comprehensive Guide
to Terraform
Advantages of Terraform:
1. Concise, readable syntax
2. Reusable code: inputs, outputs,
modules
3. Plan command!
4. Cloud agnostic
5. Very active development
Disadvantages of Terraform:
1. Maturity
2. Collaboration on Terraform state is
hard (but terragrunt makes it easier)
3. No rollback
4. Poor secrets management
1. Microservices
2. Docker
3. Terraform
4. ECS
5. Recap
Outline
EC2 Container Service (ECS) is a
way to run Docker on AWS
ECS Overview
EC2 Instance
ECS Cluster
ECS Scheduler
ECS Agent
ECS Tasks
ECS Task Definition
{
"cluster": "example",
"serviceName": ”foo",
"taskDefinition": "",
"desiredCount": 2
}
ECS Service Definition
{
"name": "example",
"image": "foo/example",
"cpu": 1024,
"memory": 2048,
"essential": true,
}
ECS Cluster: several servers
managed by ECS
EC2 Instance
ECS Cluster
Typically, the servers are in an
Auto Scaling Group
EC2 Instance
Auto Scaling Group
Which can automatically
relaunch failed servers
EC2 Instance
Auto Scaling Group
Each server must run the ECS
Agent
EC2 Instance
ECS Cluster
ECS Agent
ECS Task: Docker container(s)
to run, resources they need
EC2 Instance
ECS Cluster
ECS Agent
ECS Task Definition
{
"name": "example",
"image": "foo/example",
"cpu": 1024,
"memory": 2048,
"essential": true,
}
ECS Service: long-running ECS
Task & ELB settings
EC2 Instance
ECS Cluster
ECS Agent
ECS Task Definition
{
"name": "example",
"image": "foo/example",
"cpu": 1024,
"memory": 2048,
"essential": true,
}
{
"cluster": "example",
"serviceName": ”foo",
"taskDefinition": "",
"desiredCount": 2
}
ECS Service Definition
ECS Scheduler: Deploys Tasks
across the ECS Cluster
EC2 Instance
ECS Cluster
ECS Agent
ECS Task Definition
{
"name": "example",
"image": "foo/example",
"cpu": 1024,
"memory": 2048,
"essential": true,
}
{
"cluster": "example",
"serviceName": ”foo",
"taskDefinition": "",
"desiredCount": 2
}
ECS Service Definition
ECS Scheduler ECS Tasks
It will also automatically
redeploy failed Services
EC2 Instance
ECS Cluster
ECS Agent
ECS Task Definition
{
"name": "example",
"image": "foo/example",
"cpu": 1024,
"memory": 2048,
"essential": true,
}
{
"cluster": "example",
"serviceName": ”foo",
"taskDefinition": "",
"desiredCount": 2
}
ECS Service Definition
ECS Scheduler ECS Tasks
You can associate an ALB or
ELB with each ECS service
EC2 Instance
ECS Cluster
ECS Agent
ECS Tasks
User
This lets you distribute traffic
across multiple ECS Tasks
EC2 Instance
ECS Cluster
ECS Agent
ECS Tasks
User
Which allows zero-downtime
deployment
EC2 Instance
ECS Cluster
ECS Agent
ECS TasksUser
v1
v1
v1 v2
As well as a simple form of
service discovery
EC2 Instance
ECS Cluster
ECS Agent
ECS Tasks
You can use CloudWatch
alarms to trigger auto scaling
EC2 Instance
ECS Cluster
ECS Agent
ECS Tasks
CloudWatch
You can scale up by running
more ECS Tasks
EC2 Instance
ECS Cluster
ECS Agent
ECS Tasks
CloudWatch
And by adding more EC2
Instances
EC2 Instance
ECS Cluster
ECS Agent
ECS Tasks
CloudWatch
And scale back down when load
is lower
EC2 Instance
ECS Cluster
ECS Agent
ECS Tasks
CloudWatch
Let’s deploy our microservices in
ECS using Terraform
Define the ECS Cluster as an
Auto Scaling Group (ASG)
EC2 Instance
ECS Cluster
resource "aws_ecs_cluster" "example_cluster" {
name = "example-cluster"
}
resource "aws_autoscaling_group" "ecs_cluster_instances" {
name = "ecs-cluster-instances"
min_size = 3
max_size = 3
launch_configuration =
"${aws_launch_configuration.ecs_instance.name}"
}
Ensure each server in the ASG
runs the ECS Agent
EC2 Instance
ECS Cluster
ECS Agent
# The launch config defines what runs on each EC2 instance
resource "aws_launch_configuration" "ecs_instance" {
name_prefix = "ecs-instance-"
instance_type = "t2.micro"
# This is an Amazon ECS AMI, which has an ECS Agent
# installed that lets it talk to the ECS cluster
image_id = "ami-a98cb2c3”
}
The launch config runs AWS ECS
Linux on each server in the ASG
Define an ECS Task for each
microservice
EC2 Instance
ECS Cluster
ECS Agent
ECS Task Definition
{
"name": "example",
"image": "foo/example",
"cpu": 1024,
"memory": 2048,
"essential": true,
}
resource "aws_ecs_task_definition" "rails_frontend" {
family = "rails-frontend"
container_definitions = <<EOF
[{
"name": "rails-frontend",
"image": "brikis98/rails-frontend:v1",
"cpu": 1024,
"memory": 768,
"essential": true,
"portMappings": [{"containerPort": 3000, "hostPort": 3000}]
}]
EOF
}
Rails frontend ECS Task
resource "aws_ecs_task_definition" "sinatra_backend" {
family = "sinatra-backend"
container_definitions = <<EOF
[{
"name": "sinatra-backend",
"image": "brikis98/sinatra-backend:v1",
"cpu": 1024,
"memory": 768,
"essential": true,
"portMappings": [{"containerPort": 4567, "hostPort": 4567}]
}]
EOF
}
Sinatra Backend ECS Task
Define an ECS Service for each
ECS Task
EC2 Instance
ECS Cluster
ECS Agent
ECS Task Definition
{
"name": "example",
"image": "foo/example",
"cpu": 1024,
"memory": 2048,
"essential": true,
}
{
"cluster": "example",
"serviceName": ”foo",
"taskDefinition": "",
"desiredCount": 2
}
ECS Service Definition
resource "aws_ecs_service" "rails_frontend" {
family = "rails-frontend"
cluster = "${aws_ecs_cluster.example_cluster.id}"
task_definition =
"${aws_ecs_task_definition.rails-fronted.arn}"
desired_count = 2
}
Rails Frontend ECS Service
resource "aws_ecs_service" "sinatra_backend" {
family = "sinatra-backend"
cluster = "${aws_ecs_cluster.example_cluster.id}"
task_definition =
"${aws_ecs_task_definition.sinatra_backend.arn}"
desired_count = 2
}
Sinatra Backend ECS Service
Associate an ELB with each
ECS Service
EC2 Instance
ECS Cluster
ECS Agent
ECS Tasks
User
resource "aws_elb" "rails_frontend" {
name = "rails-frontend"
listener {
lb_port = 80
lb_protocol = "http"
instance_port = 3000
instance_protocol = "http"
}
}
Rails Frontend ELB
resource "aws_ecs_service" "rails_frontend" {
(...)
load_balancer {
elb_name = "${aws_elb.rails_frontend.id}"
container_name = "rails-frontend"
container_port = 3000
}
}
Associate the ELB with the Rails
Frontend ECS Service
resource "aws_elb" "sinatra_backend" {
name = "sinatra-backend"
listener {
lb_port = 4567
lb_protocol = "http"
instance_port = 4567
instance_protocol = "http"
}
}
Sinatra Backend ELB
resource "aws_ecs_service" "sinatra_backend" {
(...)
load_balancer {
elb_name = "${aws_elb.sinatra_backend.id}"
container_name = "sinatra-backend"
container_port = 4567
}
}
Associate the ELB with the Sinatra
Backend ECS Service
Set up service discovery
between the microservices
EC2 Instance
ECS Cluster
ECS Agent
ECS Tasks
resource "aws_ecs_task_definition" "rails_frontend" {
family = "rails-frontend"
container_definitions = <<EOF
[{
...
"environment": [{
"name": "SINATRA_BACKEND_PORT",
"value": "tcp://${aws_elb.sinatra_backend.dns_name}:4567"
}]
}]
EOF
}
Pass the Sinatra Bckend ELB URL
as env var to Rails Frontend
It’s time to deploy!
EC2 Instance
ECS Cluster
ECS Agent
ECS Task Definition
{
"name": "example",
"image": "foo/example",
"cpu": 1024,
"memory": 2048,
"essential": true,
}
{
"cluster": "example",
"serviceName": ”foo",
"taskDefinition": "",
"desiredCount": 2
}
ECS Service Definition
ECS Scheduler ECS Tasks
> terraform apply
aws_ecs_cluster.example_cluster: Creating...
name: "" => "example-cluster"
aws_ecs_task_definition.sinatra_backend: Creating...
...
Apply complete! Resources: 17 added, 0 changed, 0 destroyed.
Use the apply command to deploy
the ECS Cluster & Tasks
See the cluster in the ECS console
Track events for each Service
As well as basic metrics
Test the rails-frontend
resource "aws_ecs_task_definition" "sinatra_backend" {
family = "sinatra-backend"
container_definitions = <<EOF
[{
"name": "sinatra-backend",
"image": "brikis98/sinatra-backend:v2",
...
}
To deploy a new image, just
update the docker tag
> terraform plan
~ aws_ecs_service.sinatra_backend
task_definition: "arn...sinatra-backend:3" => "<computed>"
-/+ aws_ecs_task_definition.sinatra_backend
arn: "arn...sinatra-backend:3" => "<computed>"
container_definitions: "bb5352f" => "2ff6ae" (forces new resource)
revision: "3" => "<computed>”
Plan: 1 to add, 1 to change, 1 to destroy.
Use the plan command to verify
the changes
Apply the changes and you’ll see v2.
Advantages of ECS:
1. One of the simplest Docker cluster
management tools
2. Almost no extra cost if on AWS
3. Pluggable scheduler
4. Auto-restart of instances & Tasks
5. Automatic ALB/ELB integration
Disadvantages of ECS:
1. UI is so-so
2. Minimal monitoring built-in
3. ALB is broken
1. Microservices
2. Docker
3. Terraform
4. ECS
5. Recap
Outline
Benefits of infrastructure-as-code:
1. Reuse
2. Automation
3. Version control
4. Code review
5. Testing
6. Documentation
Slides and code from this talk:
ybrikman.com/speaking
For more
info, see
Hello,
Startup
hello-startup.net
And
Terraform:
Up & Running
terraformupandrunning.com
gruntwork.io
For DevOps help, see
Gruntwork
Questions?

Contenu connexe

Tendances

Docker introduction &amp; benefits
Docker introduction &amp; benefitsDocker introduction &amp; benefits
Docker introduction &amp; benefitsAmit Manwade
 
Intro to containerization
Intro to containerizationIntro to containerization
Intro to containerizationBalint Pato
 
An introduction to terraform
An introduction to terraformAn introduction to terraform
An introduction to terraformJulien Pivotto
 
Best Practices of Infrastructure as Code with Terraform
Best Practices of Infrastructure as Code with TerraformBest Practices of Infrastructure as Code with Terraform
Best Practices of Infrastructure as Code with TerraformDevOps.com
 
What Is A Docker Container? | Docker Container Tutorial For Beginners| Docker...
What Is A Docker Container? | Docker Container Tutorial For Beginners| Docker...What Is A Docker Container? | Docker Container Tutorial For Beginners| Docker...
What Is A Docker Container? | Docker Container Tutorial For Beginners| Docker...Simplilearn
 
Docker introduction
Docker introductionDocker introduction
Docker introductionPhuc Nguyen
 
Docker 101: An Introduction
Docker 101: An IntroductionDocker 101: An Introduction
Docker 101: An IntroductionPOSSCON
 
Docker: From Zero to Hero
Docker: From Zero to HeroDocker: From Zero to Hero
Docker: From Zero to Herofazalraja
 
Virtualization Vs. Containers
Virtualization Vs. ContainersVirtualization Vs. Containers
Virtualization Vs. Containersactualtechmedia
 
Kubernetes Concepts And Architecture Powerpoint Presentation Slides
Kubernetes Concepts And Architecture Powerpoint Presentation SlidesKubernetes Concepts And Architecture Powerpoint Presentation Slides
Kubernetes Concepts And Architecture Powerpoint Presentation SlidesSlideTeam
 
Getting started with Docker
Getting started with DockerGetting started with Docker
Getting started with DockerRavindu Fernando
 
Introduction to Docker storage, volume and image
Introduction to Docker storage, volume and imageIntroduction to Docker storage, volume and image
Introduction to Docker storage, volume and imageejlp12
 
Reusable, composable, battle-tested Terraform modules
Reusable, composable, battle-tested Terraform modulesReusable, composable, battle-tested Terraform modules
Reusable, composable, battle-tested Terraform modulesYevgeniy Brikman
 
Docker Compose by Aanand Prasad
Docker Compose by Aanand Prasad Docker Compose by Aanand Prasad
Docker Compose by Aanand Prasad Docker, Inc.
 

Tendances (20)

Docker introduction &amp; benefits
Docker introduction &amp; benefitsDocker introduction &amp; benefits
Docker introduction &amp; benefits
 
Intro to containerization
Intro to containerizationIntro to containerization
Intro to containerization
 
An introduction to terraform
An introduction to terraformAn introduction to terraform
An introduction to terraform
 
Best Practices of Infrastructure as Code with Terraform
Best Practices of Infrastructure as Code with TerraformBest Practices of Infrastructure as Code with Terraform
Best Practices of Infrastructure as Code with Terraform
 
What Is A Docker Container? | Docker Container Tutorial For Beginners| Docker...
What Is A Docker Container? | Docker Container Tutorial For Beginners| Docker...What Is A Docker Container? | Docker Container Tutorial For Beginners| Docker...
What Is A Docker Container? | Docker Container Tutorial For Beginners| Docker...
 
Docker introduction
Docker introductionDocker introduction
Docker introduction
 
Docker 101: An Introduction
Docker 101: An IntroductionDocker 101: An Introduction
Docker 101: An Introduction
 
Docker: From Zero to Hero
Docker: From Zero to HeroDocker: From Zero to Hero
Docker: From Zero to Hero
 
Introduction to docker
Introduction to dockerIntroduction to docker
Introduction to docker
 
Virtualization Vs. Containers
Virtualization Vs. ContainersVirtualization Vs. Containers
Virtualization Vs. Containers
 
Kubernetes Concepts And Architecture Powerpoint Presentation Slides
Kubernetes Concepts And Architecture Powerpoint Presentation SlidesKubernetes Concepts And Architecture Powerpoint Presentation Slides
Kubernetes Concepts And Architecture Powerpoint Presentation Slides
 
Kubernetes 101
Kubernetes 101Kubernetes 101
Kubernetes 101
 
Kubernetes
KubernetesKubernetes
Kubernetes
 
Advanced Terraform
Advanced TerraformAdvanced Terraform
Advanced Terraform
 
Getting started with Docker
Getting started with DockerGetting started with Docker
Getting started with Docker
 
Introduction to container based virtualization with docker
Introduction to container based virtualization with dockerIntroduction to container based virtualization with docker
Introduction to container based virtualization with docker
 
Kubernetes Introduction
Kubernetes IntroductionKubernetes Introduction
Kubernetes Introduction
 
Introduction to Docker storage, volume and image
Introduction to Docker storage, volume and imageIntroduction to Docker storage, volume and image
Introduction to Docker storage, volume and image
 
Reusable, composable, battle-tested Terraform modules
Reusable, composable, battle-tested Terraform modulesReusable, composable, battle-tested Terraform modules
Reusable, composable, battle-tested Terraform modules
 
Docker Compose by Aanand Prasad
Docker Compose by Aanand Prasad Docker Compose by Aanand Prasad
Docker Compose by Aanand Prasad
 

En vedette

2017 holiday survey: An annual analysis of the peak shopping season
2017 holiday survey: An annual analysis of the peak shopping season2017 holiday survey: An annual analysis of the peak shopping season
2017 holiday survey: An annual analysis of the peak shopping seasonDeloitte United States
 
What is Artificial Intelligence | Artificial Intelligence Tutorial For Beginn...
What is Artificial Intelligence | Artificial Intelligence Tutorial For Beginn...What is Artificial Intelligence | Artificial Intelligence Tutorial For Beginn...
What is Artificial Intelligence | Artificial Intelligence Tutorial For Beginn...Edureka!
 
Gruntwork Executive Summary
Gruntwork Executive SummaryGruntwork Executive Summary
Gruntwork Executive SummaryYevgeniy Brikman
 
Privacy is an Illusion and you’re all losers! - Cryptocow - Infosecurity 2013
Privacy is an Illusion and you’re all losers! - Cryptocow - Infosecurity 2013Privacy is an Illusion and you’re all losers! - Cryptocow - Infosecurity 2013
Privacy is an Illusion and you’re all losers! - Cryptocow - Infosecurity 2013Cain Ransbottyn
 
Harry Surden - Artificial Intelligence and Law Overview
Harry Surden - Artificial Intelligence and Law OverviewHarry Surden - Artificial Intelligence and Law Overview
Harry Surden - Artificial Intelligence and Law OverviewHarry Surden
 
Taming the ever-evolving Compliance Beast : Lessons learnt at LinkedIn [Strat...
Taming the ever-evolving Compliance Beast : Lessons learnt at LinkedIn [Strat...Taming the ever-evolving Compliance Beast : Lessons learnt at LinkedIn [Strat...
Taming the ever-evolving Compliance Beast : Lessons learnt at LinkedIn [Strat...Shirshanka Das
 
Inside Google's Numbers in 2017
Inside Google's Numbers in 2017Inside Google's Numbers in 2017
Inside Google's Numbers in 2017Rand Fishkin
 
Top 5 Deep Learning and AI Stories - October 6, 2017
Top 5 Deep Learning and AI Stories - October 6, 2017Top 5 Deep Learning and AI Stories - October 6, 2017
Top 5 Deep Learning and AI Stories - October 6, 2017NVIDIA
 
AI and Machine Learning Demystified by Carol Smith at Midwest UX 2017
AI and Machine Learning Demystified by Carol Smith at Midwest UX 2017AI and Machine Learning Demystified by Carol Smith at Midwest UX 2017
AI and Machine Learning Demystified by Carol Smith at Midwest UX 2017Carol Smith
 
Getting Started with Go
Getting Started with GoGetting Started with Go
Getting Started with GoSteven Francia
 
Microservices Technology Stack
Microservices Technology StackMicroservices Technology Stack
Microservices Technology StackEberhard Wolff
 
Self-contained Systems: A Different Approach to Microservices
Self-contained Systems: A Different Approach to MicroservicesSelf-contained Systems: A Different Approach to Microservices
Self-contained Systems: A Different Approach to MicroservicesEberhard Wolff
 
Challenges for Information Security Theory
Challenges for Information Security TheoryChallenges for Information Security Theory
Challenges for Information Security TheoryICAC09
 
25 Quotes That Will Make You a Better Freelancer
25 Quotes That Will Make You a Better Freelancer25 Quotes That Will Make You a Better Freelancer
25 Quotes That Will Make You a Better Freelancercontently
 
Information Security - Back to Basics - Own Your Vulnerabilities
Information Security - Back to Basics - Own Your VulnerabilitiesInformation Security - Back to Basics - Own Your Vulnerabilities
Information Security - Back to Basics - Own Your VulnerabilitiesJack Nichelson
 
Information Security Governance: Concepts, Security Management & Metrics
Information Security Governance: Concepts, Security Management & MetricsInformation Security Governance: Concepts, Security Management & Metrics
Information Security Governance: Concepts, Security Management & MetricsMarius FAILLOT DEVARRE
 
System Security Beyond the Libraries
System Security Beyond the LibrariesSystem Security Beyond the Libraries
System Security Beyond the LibrariesEoin Woods
 

En vedette (20)

The AI Rush
The AI RushThe AI Rush
The AI Rush
 
2017 holiday survey: An annual analysis of the peak shopping season
2017 holiday survey: An annual analysis of the peak shopping season2017 holiday survey: An annual analysis of the peak shopping season
2017 holiday survey: An annual analysis of the peak shopping season
 
What is Artificial Intelligence | Artificial Intelligence Tutorial For Beginn...
What is Artificial Intelligence | Artificial Intelligence Tutorial For Beginn...What is Artificial Intelligence | Artificial Intelligence Tutorial For Beginn...
What is Artificial Intelligence | Artificial Intelligence Tutorial For Beginn...
 
Gruntwork Executive Summary
Gruntwork Executive SummaryGruntwork Executive Summary
Gruntwork Executive Summary
 
Privacy is an Illusion and you’re all losers! - Cryptocow - Infosecurity 2013
Privacy is an Illusion and you’re all losers! - Cryptocow - Infosecurity 2013Privacy is an Illusion and you’re all losers! - Cryptocow - Infosecurity 2013
Privacy is an Illusion and you’re all losers! - Cryptocow - Infosecurity 2013
 
Harry Surden - Artificial Intelligence and Law Overview
Harry Surden - Artificial Intelligence and Law OverviewHarry Surden - Artificial Intelligence and Law Overview
Harry Surden - Artificial Intelligence and Law Overview
 
Taming the ever-evolving Compliance Beast : Lessons learnt at LinkedIn [Strat...
Taming the ever-evolving Compliance Beast : Lessons learnt at LinkedIn [Strat...Taming the ever-evolving Compliance Beast : Lessons learnt at LinkedIn [Strat...
Taming the ever-evolving Compliance Beast : Lessons learnt at LinkedIn [Strat...
 
Inside Google's Numbers in 2017
Inside Google's Numbers in 2017Inside Google's Numbers in 2017
Inside Google's Numbers in 2017
 
Top 5 Deep Learning and AI Stories - October 6, 2017
Top 5 Deep Learning and AI Stories - October 6, 2017Top 5 Deep Learning and AI Stories - October 6, 2017
Top 5 Deep Learning and AI Stories - October 6, 2017
 
AI and Machine Learning Demystified by Carol Smith at Midwest UX 2017
AI and Machine Learning Demystified by Carol Smith at Midwest UX 2017AI and Machine Learning Demystified by Carol Smith at Midwest UX 2017
AI and Machine Learning Demystified by Carol Smith at Midwest UX 2017
 
New AWS Services
New AWS ServicesNew AWS Services
New AWS Services
 
Getting Started with Go
Getting Started with GoGetting Started with Go
Getting Started with Go
 
Microservices Technology Stack
Microservices Technology StackMicroservices Technology Stack
Microservices Technology Stack
 
Self-contained Systems: A Different Approach to Microservices
Self-contained Systems: A Different Approach to MicroservicesSelf-contained Systems: A Different Approach to Microservices
Self-contained Systems: A Different Approach to Microservices
 
Innovation can be Trained
Innovation can be TrainedInnovation can be Trained
Innovation can be Trained
 
Challenges for Information Security Theory
Challenges for Information Security TheoryChallenges for Information Security Theory
Challenges for Information Security Theory
 
25 Quotes That Will Make You a Better Freelancer
25 Quotes That Will Make You a Better Freelancer25 Quotes That Will Make You a Better Freelancer
25 Quotes That Will Make You a Better Freelancer
 
Information Security - Back to Basics - Own Your Vulnerabilities
Information Security - Back to Basics - Own Your VulnerabilitiesInformation Security - Back to Basics - Own Your Vulnerabilities
Information Security - Back to Basics - Own Your Vulnerabilities
 
Information Security Governance: Concepts, Security Management & Metrics
Information Security Governance: Concepts, Security Management & MetricsInformation Security Governance: Concepts, Security Management & Metrics
Information Security Governance: Concepts, Security Management & Metrics
 
System Security Beyond the Libraries
System Security Beyond the LibrariesSystem Security Beyond the Libraries
System Security Beyond the Libraries
 

Similaire à Infrastructure as code: running microservices on AWS using Docker, Terraform, and ECS

Docker Security workshop slides
Docker Security workshop slidesDocker Security workshop slides
Docker Security workshop slidesDocker, Inc.
 
Agile Brown Bag - Vagrant & Docker: Introduction
Agile Brown Bag - Vagrant & Docker: IntroductionAgile Brown Bag - Vagrant & Docker: Introduction
Agile Brown Bag - Vagrant & Docker: IntroductionAgile Partner S.A.
 
Developing and Deploying PHP with Docker
Developing and Deploying PHP with DockerDeveloping and Deploying PHP with Docker
Developing and Deploying PHP with DockerPatrick Mizer
 
Scaling Docker Containers using Kubernetes and Azure Container Service
Scaling Docker Containers using Kubernetes and Azure Container ServiceScaling Docker Containers using Kubernetes and Azure Container Service
Scaling Docker Containers using Kubernetes and Azure Container ServiceBen Hall
 
Start tracking your ruby infrastructure
Start tracking your ruby infrastructureStart tracking your ruby infrastructure
Start tracking your ruby infrastructureSergiy Kukunin
 
How Puppet Enables the Use of Lightweight Virtualized Containers - PuppetConf...
How Puppet Enables the Use of Lightweight Virtualized Containers - PuppetConf...How Puppet Enables the Use of Lightweight Virtualized Containers - PuppetConf...
How Puppet Enables the Use of Lightweight Virtualized Containers - PuppetConf...Puppet
 
Docker - A Ruby Introduction
Docker - A Ruby IntroductionDocker - A Ruby Introduction
Docker - A Ruby IntroductionTyler Johnston
 
Real World Experience of Running Docker in Development and Production
Real World Experience of Running Docker in Development and ProductionReal World Experience of Running Docker in Development and Production
Real World Experience of Running Docker in Development and ProductionBen Hall
 
JDO 2019: Tips and Tricks from Docker Captain - Łukasz Lach
JDO 2019: Tips and Tricks from Docker Captain - Łukasz LachJDO 2019: Tips and Tricks from Docker Captain - Łukasz Lach
JDO 2019: Tips and Tricks from Docker Captain - Łukasz LachPROIDEA
 
Containers, Docker, and Microservices: the Terrific Trio
Containers, Docker, and Microservices: the Terrific TrioContainers, Docker, and Microservices: the Terrific Trio
Containers, Docker, and Microservices: the Terrific TrioJérôme Petazzoni
 
From development environments to production deployments with Docker, Compose,...
From development environments to production deployments with Docker, Compose,...From development environments to production deployments with Docker, Compose,...
From development environments to production deployments with Docker, Compose,...Jérôme Petazzoni
 
Rails web api 开发
Rails web api 开发Rails web api 开发
Rails web api 开发shaokun
 
Cloud meets Fog & Puppet A Story of Version Controlled Infrastructure
Cloud meets Fog & Puppet A Story of Version Controlled InfrastructureCloud meets Fog & Puppet A Story of Version Controlled Infrastructure
Cloud meets Fog & Puppet A Story of Version Controlled InfrastructureHabeeb Rahman
 
The Docker "Gauntlet" - Introduction, Ecosystem, Deployment, Orchestration
The Docker "Gauntlet" - Introduction, Ecosystem, Deployment, OrchestrationThe Docker "Gauntlet" - Introduction, Ecosystem, Deployment, Orchestration
The Docker "Gauntlet" - Introduction, Ecosystem, Deployment, OrchestrationErica Windisch
 
Docker for developers on mac and windows
Docker for developers on mac and windowsDocker for developers on mac and windows
Docker for developers on mac and windowsDocker, Inc.
 
Kubernetes laravel and kubernetes
Kubernetes   laravel and kubernetesKubernetes   laravel and kubernetes
Kubernetes laravel and kubernetesWilliam Stewart
 
Why everyone is excited about Docker (and you should too...) - Carlo Bonamic...
Why everyone is excited about Docker (and you should too...) -  Carlo Bonamic...Why everyone is excited about Docker (and you should too...) -  Carlo Bonamic...
Why everyone is excited about Docker (and you should too...) - Carlo Bonamic...Codemotion
 
What’s New in Docker - Victor Vieux, Docker
What’s New in Docker - Victor Vieux, DockerWhat’s New in Docker - Victor Vieux, Docker
What’s New in Docker - Victor Vieux, DockerDocker, Inc.
 

Similaire à Infrastructure as code: running microservices on AWS using Docker, Terraform, and ECS (20)

Docker Security workshop slides
Docker Security workshop slidesDocker Security workshop slides
Docker Security workshop slides
 
Agile Brown Bag - Vagrant & Docker: Introduction
Agile Brown Bag - Vagrant & Docker: IntroductionAgile Brown Bag - Vagrant & Docker: Introduction
Agile Brown Bag - Vagrant & Docker: Introduction
 
Developing and Deploying PHP with Docker
Developing and Deploying PHP with DockerDeveloping and Deploying PHP with Docker
Developing and Deploying PHP with Docker
 
Scaling Docker Containers using Kubernetes and Azure Container Service
Scaling Docker Containers using Kubernetes and Azure Container ServiceScaling Docker Containers using Kubernetes and Azure Container Service
Scaling Docker Containers using Kubernetes and Azure Container Service
 
Docker, c'est bonheur !
Docker, c'est bonheur !Docker, c'est bonheur !
Docker, c'est bonheur !
 
Start tracking your ruby infrastructure
Start tracking your ruby infrastructureStart tracking your ruby infrastructure
Start tracking your ruby infrastructure
 
How Puppet Enables the Use of Lightweight Virtualized Containers - PuppetConf...
How Puppet Enables the Use of Lightweight Virtualized Containers - PuppetConf...How Puppet Enables the Use of Lightweight Virtualized Containers - PuppetConf...
How Puppet Enables the Use of Lightweight Virtualized Containers - PuppetConf...
 
Docker - A Ruby Introduction
Docker - A Ruby IntroductionDocker - A Ruby Introduction
Docker - A Ruby Introduction
 
Sheep it
Sheep itSheep it
Sheep it
 
Real World Experience of Running Docker in Development and Production
Real World Experience of Running Docker in Development and ProductionReal World Experience of Running Docker in Development and Production
Real World Experience of Running Docker in Development and Production
 
JDO 2019: Tips and Tricks from Docker Captain - Łukasz Lach
JDO 2019: Tips and Tricks from Docker Captain - Łukasz LachJDO 2019: Tips and Tricks from Docker Captain - Łukasz Lach
JDO 2019: Tips and Tricks from Docker Captain - Łukasz Lach
 
Containers, Docker, and Microservices: the Terrific Trio
Containers, Docker, and Microservices: the Terrific TrioContainers, Docker, and Microservices: the Terrific Trio
Containers, Docker, and Microservices: the Terrific Trio
 
From development environments to production deployments with Docker, Compose,...
From development environments to production deployments with Docker, Compose,...From development environments to production deployments with Docker, Compose,...
From development environments to production deployments with Docker, Compose,...
 
Rails web api 开发
Rails web api 开发Rails web api 开发
Rails web api 开发
 
Cloud meets Fog & Puppet A Story of Version Controlled Infrastructure
Cloud meets Fog & Puppet A Story of Version Controlled InfrastructureCloud meets Fog & Puppet A Story of Version Controlled Infrastructure
Cloud meets Fog & Puppet A Story of Version Controlled Infrastructure
 
The Docker "Gauntlet" - Introduction, Ecosystem, Deployment, Orchestration
The Docker "Gauntlet" - Introduction, Ecosystem, Deployment, OrchestrationThe Docker "Gauntlet" - Introduction, Ecosystem, Deployment, Orchestration
The Docker "Gauntlet" - Introduction, Ecosystem, Deployment, Orchestration
 
Docker for developers on mac and windows
Docker for developers on mac and windowsDocker for developers on mac and windows
Docker for developers on mac and windows
 
Kubernetes laravel and kubernetes
Kubernetes   laravel and kubernetesKubernetes   laravel and kubernetes
Kubernetes laravel and kubernetes
 
Why everyone is excited about Docker (and you should too...) - Carlo Bonamic...
Why everyone is excited about Docker (and you should too...) -  Carlo Bonamic...Why everyone is excited about Docker (and you should too...) -  Carlo Bonamic...
Why everyone is excited about Docker (and you should too...) - Carlo Bonamic...
 
What’s New in Docker - Victor Vieux, Docker
What’s New in Docker - Victor Vieux, DockerWhat’s New in Docker - Victor Vieux, Docker
What’s New in Docker - Victor Vieux, Docker
 

Plus de Yevgeniy Brikman

Cloud adoption fails - 5 ways deployments go wrong and 5 solutions
Cloud adoption fails - 5 ways deployments go wrong and 5 solutionsCloud adoption fails - 5 ways deployments go wrong and 5 solutions
Cloud adoption fails - 5 ways deployments go wrong and 5 solutionsYevgeniy Brikman
 
How to test infrastructure code: automated testing for Terraform, Kubernetes,...
How to test infrastructure code: automated testing for Terraform, Kubernetes,...How to test infrastructure code: automated testing for Terraform, Kubernetes,...
How to test infrastructure code: automated testing for Terraform, Kubernetes,...Yevgeniy Brikman
 
Lessons learned from writing over 300,000 lines of infrastructure code
Lessons learned from writing over 300,000 lines of infrastructure codeLessons learned from writing over 300,000 lines of infrastructure code
Lessons learned from writing over 300,000 lines of infrastructure codeYevgeniy Brikman
 
The Truth About Startups: What I wish someone had told me about entrepreneurs...
The Truth About Startups: What I wish someone had told me about entrepreneurs...The Truth About Startups: What I wish someone had told me about entrepreneurs...
The Truth About Startups: What I wish someone had told me about entrepreneurs...Yevgeniy Brikman
 
Comprehensive Terraform Training
Comprehensive Terraform TrainingComprehensive Terraform Training
Comprehensive Terraform TrainingYevgeniy Brikman
 
Startup Ideas and Validation
Startup Ideas and ValidationStartup Ideas and Validation
Startup Ideas and ValidationYevgeniy Brikman
 
A Guide to Hiring for your Startup
A Guide to Hiring for your StartupA Guide to Hiring for your Startup
A Guide to Hiring for your StartupYevgeniy Brikman
 
Node.js vs Play Framework (with Japanese subtitles)
Node.js vs Play Framework (with Japanese subtitles)Node.js vs Play Framework (with Japanese subtitles)
Node.js vs Play Framework (with Japanese subtitles)Yevgeniy Brikman
 
Composable and streamable Play apps
Composable and streamable Play appsComposable and streamable Play apps
Composable and streamable Play appsYevgeniy Brikman
 
Play Framework: async I/O with Java and Scala
Play Framework: async I/O with Java and ScalaPlay Framework: async I/O with Java and Scala
Play Framework: async I/O with Java and ScalaYevgeniy Brikman
 
The Play Framework at LinkedIn
The Play Framework at LinkedInThe Play Framework at LinkedIn
The Play Framework at LinkedInYevgeniy Brikman
 
Startup DNA: the formula behind successful startups in Silicon Valley (update...
Startup DNA: the formula behind successful startups in Silicon Valley (update...Startup DNA: the formula behind successful startups in Silicon Valley (update...
Startup DNA: the formula behind successful startups in Silicon Valley (update...Yevgeniy Brikman
 

Plus de Yevgeniy Brikman (20)

Cloud adoption fails - 5 ways deployments go wrong and 5 solutions
Cloud adoption fails - 5 ways deployments go wrong and 5 solutionsCloud adoption fails - 5 ways deployments go wrong and 5 solutions
Cloud adoption fails - 5 ways deployments go wrong and 5 solutions
 
How to test infrastructure code: automated testing for Terraform, Kubernetes,...
How to test infrastructure code: automated testing for Terraform, Kubernetes,...How to test infrastructure code: automated testing for Terraform, Kubernetes,...
How to test infrastructure code: automated testing for Terraform, Kubernetes,...
 
Lessons learned from writing over 300,000 lines of infrastructure code
Lessons learned from writing over 300,000 lines of infrastructure codeLessons learned from writing over 300,000 lines of infrastructure code
Lessons learned from writing over 300,000 lines of infrastructure code
 
The Truth About Startups: What I wish someone had told me about entrepreneurs...
The Truth About Startups: What I wish someone had told me about entrepreneurs...The Truth About Startups: What I wish someone had told me about entrepreneurs...
The Truth About Startups: What I wish someone had told me about entrepreneurs...
 
Comprehensive Terraform Training
Comprehensive Terraform TrainingComprehensive Terraform Training
Comprehensive Terraform Training
 
Agility Requires Safety
Agility Requires SafetyAgility Requires Safety
Agility Requires Safety
 
Startup Ideas and Validation
Startup Ideas and ValidationStartup Ideas and Validation
Startup Ideas and Validation
 
A Guide to Hiring for your Startup
A Guide to Hiring for your StartupA Guide to Hiring for your Startup
A Guide to Hiring for your Startup
 
Startup DNA: Speed Wins
Startup DNA: Speed WinsStartup DNA: Speed Wins
Startup DNA: Speed Wins
 
Node.js vs Play Framework (with Japanese subtitles)
Node.js vs Play Framework (with Japanese subtitles)Node.js vs Play Framework (with Japanese subtitles)
Node.js vs Play Framework (with Japanese subtitles)
 
Node.js vs Play Framework
Node.js vs Play FrameworkNode.js vs Play Framework
Node.js vs Play Framework
 
Rapid prototyping
Rapid prototypingRapid prototyping
Rapid prototyping
 
Composable and streamable Play apps
Composable and streamable Play appsComposable and streamable Play apps
Composable and streamable Play apps
 
Play Framework: async I/O with Java and Scala
Play Framework: async I/O with Java and ScalaPlay Framework: async I/O with Java and Scala
Play Framework: async I/O with Java and Scala
 
The Play Framework at LinkedIn
The Play Framework at LinkedInThe Play Framework at LinkedIn
The Play Framework at LinkedIn
 
Kings of Code Hack Battle
Kings of Code Hack BattleKings of Code Hack Battle
Kings of Code Hack Battle
 
Hackdays and [in]cubator
Hackdays and [in]cubatorHackdays and [in]cubator
Hackdays and [in]cubator
 
Startup DNA: the formula behind successful startups in Silicon Valley (update...
Startup DNA: the formula behind successful startups in Silicon Valley (update...Startup DNA: the formula behind successful startups in Silicon Valley (update...
Startup DNA: the formula behind successful startups in Silicon Valley (update...
 
Dust.js
Dust.jsDust.js
Dust.js
 
LinkedIn Overview
LinkedIn OverviewLinkedIn Overview
LinkedIn Overview
 

Dernier

Enterprise Content Managements Solutions
Enterprise Content Managements SolutionsEnterprise Content Managements Solutions
Enterprise Content Managements SolutionsIQBG inc
 
Splashtop Enterprise Brochure - Remote Computer Access and Remote Support Sof...
Splashtop Enterprise Brochure - Remote Computer Access and Remote Support Sof...Splashtop Enterprise Brochure - Remote Computer Access and Remote Support Sof...
Splashtop Enterprise Brochure - Remote Computer Access and Remote Support Sof...Splashtop Inc
 
Large Scale Architecture -- The Unreasonable Effectiveness of Simplicity
Large Scale Architecture -- The Unreasonable Effectiveness of SimplicityLarge Scale Architecture -- The Unreasonable Effectiveness of Simplicity
Large Scale Architecture -- The Unreasonable Effectiveness of SimplicityRandy Shoup
 
Mobile App Development company Houston
Mobile  App  Development  company HoustonMobile  App  Development  company Houston
Mobile App Development company Houstonjennysmithusa549
 
If your code could speak, what would it tell you? Let GitHub Copilot Chat hel...
If your code could speak, what would it tell you? Let GitHub Copilot Chat hel...If your code could speak, what would it tell you? Let GitHub Copilot Chat hel...
If your code could speak, what would it tell you? Let GitHub Copilot Chat hel...Maxim Salnikov
 
openEuler Community Overview - a presentation showing the current scale
openEuler Community Overview - a presentation showing the current scaleopenEuler Community Overview - a presentation showing the current scale
openEuler Community Overview - a presentation showing the current scaleShane Coughlan
 
Technical improvements. Reasons. Methods. Estimations. CJ
Technical improvements.  Reasons. Methods. Estimations. CJTechnical improvements.  Reasons. Methods. Estimations. CJ
Technical improvements. Reasons. Methods. Estimations. CJpolinaucc
 
VuNet software organisation powerpoint deck
VuNet software organisation powerpoint deckVuNet software organisation powerpoint deck
VuNet software organisation powerpoint deckNaval Singh
 
Unlocking the Power of IoT: A comprehensive approach to real-time insights
Unlocking the Power of IoT: A comprehensive approach to real-time insightsUnlocking the Power of IoT: A comprehensive approach to real-time insights
Unlocking the Power of IoT: A comprehensive approach to real-time insightsconfluent
 
Mobile App Development process | Expert Tips
Mobile App Development process | Expert TipsMobile App Development process | Expert Tips
Mobile App Development process | Expert Tipsmichealwillson701
 
8 key point on optimizing web hosting services in your business.pdf
8 key point on optimizing web hosting services in your business.pdf8 key point on optimizing web hosting services in your business.pdf
8 key point on optimizing web hosting services in your business.pdfOffsiteNOC
 
Building Generative AI-infused apps: what's possible and how to start
Building Generative AI-infused apps: what's possible and how to startBuilding Generative AI-infused apps: what's possible and how to start
Building Generative AI-infused apps: what's possible and how to startMaxim Salnikov
 
Unlocking AI: Navigating Open Source vs. Commercial Frontiers
Unlocking AI:Navigating Open Source vs. Commercial FrontiersUnlocking AI:Navigating Open Source vs. Commercial Frontiers
Unlocking AI: Navigating Open Source vs. Commercial FrontiersRaphaël Semeteys
 
Flutter the Future of Mobile App Development - 5 Crucial Reasons.pdf
Flutter the Future of Mobile App Development - 5 Crucial Reasons.pdfFlutter the Future of Mobile App Development - 5 Crucial Reasons.pdf
Flutter the Future of Mobile App Development - 5 Crucial Reasons.pdfMind IT Systems
 
BusinessGPT - SECURITY AND GOVERNANCE FOR GENERATIVE AI.pptx
BusinessGPT  - SECURITY AND GOVERNANCE  FOR GENERATIVE AI.pptxBusinessGPT  - SECURITY AND GOVERNANCE  FOR GENERATIVE AI.pptx
BusinessGPT - SECURITY AND GOVERNANCE FOR GENERATIVE AI.pptxAGATSoftware
 
Take Advantage of Mx Tracking Flight Scheduling Solutions to Streamline Your ...
Take Advantage of Mx Tracking Flight Scheduling Solutions to Streamline Your ...Take Advantage of Mx Tracking Flight Scheduling Solutions to Streamline Your ...
Take Advantage of Mx Tracking Flight Scheduling Solutions to Streamline Your ...MyFAA
 
Revolutionize Your Field Service Management with FSM Grid
Revolutionize Your Field Service Management with FSM GridRevolutionize Your Field Service Management with FSM Grid
Revolutionize Your Field Service Management with FSM GridMathew Thomas
 
Boost Efficiency: Sabre API Integration Made Easy
Boost Efficiency: Sabre API Integration Made EasyBoost Efficiency: Sabre API Integration Made Easy
Boost Efficiency: Sabre API Integration Made Easymichealwillson701
 
03.2024_North America VMUG Optimizing RevOps using the power of ChatGPT in Ma...
03.2024_North America VMUG Optimizing RevOps using the power of ChatGPT in Ma...03.2024_North America VMUG Optimizing RevOps using the power of ChatGPT in Ma...
03.2024_North America VMUG Optimizing RevOps using the power of ChatGPT in Ma...jackiepotts6
 

Dernier (20)

Enterprise Content Managements Solutions
Enterprise Content Managements SolutionsEnterprise Content Managements Solutions
Enterprise Content Managements Solutions
 
Splashtop Enterprise Brochure - Remote Computer Access and Remote Support Sof...
Splashtop Enterprise Brochure - Remote Computer Access and Remote Support Sof...Splashtop Enterprise Brochure - Remote Computer Access and Remote Support Sof...
Splashtop Enterprise Brochure - Remote Computer Access and Remote Support Sof...
 
Large Scale Architecture -- The Unreasonable Effectiveness of Simplicity
Large Scale Architecture -- The Unreasonable Effectiveness of SimplicityLarge Scale Architecture -- The Unreasonable Effectiveness of Simplicity
Large Scale Architecture -- The Unreasonable Effectiveness of Simplicity
 
Mobile App Development company Houston
Mobile  App  Development  company HoustonMobile  App  Development  company Houston
Mobile App Development company Houston
 
If your code could speak, what would it tell you? Let GitHub Copilot Chat hel...
If your code could speak, what would it tell you? Let GitHub Copilot Chat hel...If your code could speak, what would it tell you? Let GitHub Copilot Chat hel...
If your code could speak, what would it tell you? Let GitHub Copilot Chat hel...
 
openEuler Community Overview - a presentation showing the current scale
openEuler Community Overview - a presentation showing the current scaleopenEuler Community Overview - a presentation showing the current scale
openEuler Community Overview - a presentation showing the current scale
 
Technical improvements. Reasons. Methods. Estimations. CJ
Technical improvements.  Reasons. Methods. Estimations. CJTechnical improvements.  Reasons. Methods. Estimations. CJ
Technical improvements. Reasons. Methods. Estimations. CJ
 
VuNet software organisation powerpoint deck
VuNet software organisation powerpoint deckVuNet software organisation powerpoint deck
VuNet software organisation powerpoint deck
 
Unlocking the Power of IoT: A comprehensive approach to real-time insights
Unlocking the Power of IoT: A comprehensive approach to real-time insightsUnlocking the Power of IoT: A comprehensive approach to real-time insights
Unlocking the Power of IoT: A comprehensive approach to real-time insights
 
Mobile App Development process | Expert Tips
Mobile App Development process | Expert TipsMobile App Development process | Expert Tips
Mobile App Development process | Expert Tips
 
8 key point on optimizing web hosting services in your business.pdf
8 key point on optimizing web hosting services in your business.pdf8 key point on optimizing web hosting services in your business.pdf
8 key point on optimizing web hosting services in your business.pdf
 
Building Generative AI-infused apps: what's possible and how to start
Building Generative AI-infused apps: what's possible and how to startBuilding Generative AI-infused apps: what's possible and how to start
Building Generative AI-infused apps: what's possible and how to start
 
20140812 - OBD2 Solution
20140812 - OBD2 Solution20140812 - OBD2 Solution
20140812 - OBD2 Solution
 
Unlocking AI: Navigating Open Source vs. Commercial Frontiers
Unlocking AI:Navigating Open Source vs. Commercial FrontiersUnlocking AI:Navigating Open Source vs. Commercial Frontiers
Unlocking AI: Navigating Open Source vs. Commercial Frontiers
 
Flutter the Future of Mobile App Development - 5 Crucial Reasons.pdf
Flutter the Future of Mobile App Development - 5 Crucial Reasons.pdfFlutter the Future of Mobile App Development - 5 Crucial Reasons.pdf
Flutter the Future of Mobile App Development - 5 Crucial Reasons.pdf
 
BusinessGPT - SECURITY AND GOVERNANCE FOR GENERATIVE AI.pptx
BusinessGPT  - SECURITY AND GOVERNANCE  FOR GENERATIVE AI.pptxBusinessGPT  - SECURITY AND GOVERNANCE  FOR GENERATIVE AI.pptx
BusinessGPT - SECURITY AND GOVERNANCE FOR GENERATIVE AI.pptx
 
Take Advantage of Mx Tracking Flight Scheduling Solutions to Streamline Your ...
Take Advantage of Mx Tracking Flight Scheduling Solutions to Streamline Your ...Take Advantage of Mx Tracking Flight Scheduling Solutions to Streamline Your ...
Take Advantage of Mx Tracking Flight Scheduling Solutions to Streamline Your ...
 
Revolutionize Your Field Service Management with FSM Grid
Revolutionize Your Field Service Management with FSM GridRevolutionize Your Field Service Management with FSM Grid
Revolutionize Your Field Service Management with FSM Grid
 
Boost Efficiency: Sabre API Integration Made Easy
Boost Efficiency: Sabre API Integration Made EasyBoost Efficiency: Sabre API Integration Made Easy
Boost Efficiency: Sabre API Integration Made Easy
 
03.2024_North America VMUG Optimizing RevOps using the power of ChatGPT in Ma...
03.2024_North America VMUG Optimizing RevOps using the power of ChatGPT in Ma...03.2024_North America VMUG Optimizing RevOps using the power of ChatGPT in Ma...
03.2024_North America VMUG Optimizing RevOps using the power of ChatGPT in Ma...
 

Infrastructure as code: running microservices on AWS using Docker, Terraform, and ECS

  • 1. INFRASTRUCTURE as CODE Running Microservices on AWS with Docker, Terraform, and ECS
  • 3. You are starting a new project
  • 4. I know, I’ll use Ruby on Rails!
  • 6. > gem install rails Fetching: i18n-0.7.0.gem (100%) Fetching: json-1.8.3.gem (100%) Building native extensions. This could take a while... ERROR: Error installing rails: ERROR: Failed to build gem native extension. /usr/bin/ruby1.9.1 extconf.rb creating Makefile make sh: 1: make: not found
  • 7. Ah, I just need to install make
  • 8. > sudo apt-get install make ... Success!
  • 10. > gem install rails Fetching: nokogiri-1.6.7.2.gem (100%) Building native extensions. This could take a while... ERROR: Error installing rails: ERROR: Failed to build gem native extension. /usr/bin/ruby1.9.1 extconf.rb checking if the C compiler accepts ... yes Building nokogiri using packaged libraries. Using mini_portile version 2.0.0.rc2 checking for gzdopen() in -lz... no zlib is missing; necessary for building libxml2 *** extconf.rb failed ***
  • 11. Hmm. Time to visit StackOverflow.
  • 12. > sudo apt-get install zlib1g-dev ... Success!
  • 13. > gem install rails
  • 14. > gem install rails Building native extensions. This could take a while... ERROR: Error installing rails: ERROR: Failed to build gem native extension. /usr/bin/ruby1.9.1 extconf.rb checking if the C compiler accepts ... yes Building nokogiri using packaged libraries. Using mini_portile version 2.0.0.rc2 checking for gzdopen() in -lz... yes checking for iconv... yes Extracting libxml2-2.9.2.tar.gz into tmp/x86_64-pc-linux- gnu/ports/libxml2/2.9.2... OK *** extconf.rb failed ***
  • 15. nokogiri y u never install correctly?
  • 16. (Spend 2 hours trying random StackOverflow suggestions)
  • 17. > gem install rails
  • 18. > gem install rails ... Success!
  • 20. > rails new my-project > cd my-project > rails start
  • 21. > rails new my-project > cd my-project > rails start /source/my-project/bin/spring:11:in `<top (required)>': undefined method `path_separator' for Gem:Module (NoMethodError) from bin/rails:3:in `load' from bin/rails:3:in `<main>'
  • 23. Eventually, you get it working
  • 24. Now you have to deploy your Rails app in production
  • 25. You use the AWS Console to deploy an EC2 instance
  • 26. > ssh ec2-user@ec2-12-34-56-78.compute-1.amazonaws.com __| __|_ ) _| ( / Amazon Linux AMI ___|___|___| [ec2-user@ip-172-31-61-204 ~]$ gem install rails
  • 27. > ssh ec2-user@ec2-12-34-56-78.compute-1.amazonaws.com __| __|_ ) _| ( / Amazon Linux AMI ___|___|___| [ec2-user@ip-172-31-61-204 ~]$ gem install rails ERROR: Error installing rails: ERROR: Failed to build gem native extension. /usr/bin/ruby1.9.1 extconf.rb
  • 29. Eventually you get it working
  • 31. Now you urgently have to update all your Rails installs
  • 33. > bundle update rails Building native extensions. This could take a while... ERROR: Error installing rails: ERROR: Failed to build gem native extension. /usr/bin/ruby1.9.1 extconf.rb checking if the C compiler accepts ... yes Building nokogiri using packaged libraries. Using mini_portile version 2.0.0.rc2 checking for gzdopen() in -lz... yes checking for iconv... yes Extracting libxml2-2.9.2.tar.gz into tmp/x86_64-pc-linux- gnu/ports/libxml2/2.9.2... OK *** extconf.rb failed ***
  • 36. > ssh ec2-user@ec2-12-34-56-78.compute-1.amazonaws.com __| __|_ ) _| ( / Amazon Linux AMI ___|___|___| [ec2-user@ip-172-31-61-204 ~]$ gem install rails The problem is that you’re configuring servers manually
  • 37. And that you’re deploying infrastructure manually
  • 38. A better alternative: infrastructure- as-code
  • 39. In this talk, we’ll go through a real-world example:
  • 40. We’ll configure & deploy two microservices on Amazon ECS
  • 41. With two infrastructure-as-code tools: Docker and Terraform TERRAFORM
  • 49. Slides and code from this talk: ybrikman.com/speaking
  • 50. 1. Microservices 2. Docker 3. Terraform 4. ECS 5. Recap Outline
  • 51. 1. Microservices 2. Docker 3. Terraform 4. ECS 5. Recap Outline
  • 52. Code is the enemy: the more you have, the slower you go
  • 53. Project Size Lines of code Bug Density Bugs per thousand lines of code < 2K 0 – 25 2K – 6K 0 – 40 16K – 64K 0.5 – 50 64K – 512K 2 – 70 > 512K 4 – 100
  • 54. As the code grows, the number of bugs grows even faster
  • 55. “Software development doesn't happen in a chart, an IDE, or a design tool; it happens in your head.”
  • 56. The mind can only handle so much complexity at once
  • 57. One solution is to break the code into microservices
  • 58. In a monolith, you use function calls within one process moduleA.func() moduleB.func() moduleC.func() moduleD.func() moduleE.func()
  • 60. Advantages of services: 1. Isolation 2. Technology agnostic 3. Scalability
  • 61. Disadvantages of services: 1. Operational overhead 2. Performance overhead 3. I/O, error handling 4. Backwards compatibility 5. Global changes, transactions, referential integrity all very hard
  • 62. For more info, see: Splitting Up a Codebase into Microservices and Artifacts
  • 63. For this talk, we’ll use two example microservices
  • 64. require 'sinatra' get "/" do "Hello, World!" end A sinatra backend that returns “Hello, World”
  • 65. class ApplicationController < ActionController::Base def index url = URI.parse(backend_addr) req = Net::HTTP::Get.new(url.to_s) res = Net::HTTP.start(url.host, url.port) {|http| http.request(req) } @text = res.body end end A rails frontend that calls the sinatra backend
  • 66. <h1>Rails Frontend</h1> <p> Response from the backend: <strong><%= @text %></strong> </p> And renders the response as HTML
  • 67. 1. Microservices 2. Docker 3. Terraform 4. ECS 5. Recap Outline
  • 68. Docker allows you to build and run code in containers
  • 69. Containers are like lightweight Virtual Machines (VMs)
  • 70. VMs virtualize the hardware and run an entire guest OS on top of the host OS VM Hardware Host OS Host User Space Virtual Machine Virtualized hardware Guest OS Guest User Space App VM Virtualized hardware Guest OS Guest User Space App VM Virtualized hardware Guest OS Guest User Space App
  • 71. This provides good isolation, but lots of CPU, memory, disk, & startup overhead VM Hardware Host OS Host User Space Virtual Machine Virtualized hardware Guest OS Guest User Space App VM Virtualized hardware Guest OS Guest User Space App VM Virtualized hardware Guest OS Guest User Space App
  • 72. Containers virtualize User Space (shared memory, processes, mount, network) Container VM Hardware Host OS Host User Space Virtual Machine Virtualized hardware Guest OS Guest User Space App Hardware Host OS Host User Space Container Engine Virtualized User Space VM Virtualized hardware Guest OS Guest User Space App VM Virtualized hardware Guest OS Guest User Space App App Container Virtualized User Space App Container Virtualized User Space App
  • 73. Container VM Hardware Host OS Host User Space Virtual Machine Virtualized hardware Guest OS Guest User Space App Hardware Host OS Host User Space Container Engine Virtualized User Space VM Virtualized hardware Guest OS Guest User Space App VM Virtualized hardware Guest OS Guest User Space App App Container Virtualized User Space App Container Virtualized User Space App Isolation isn’t as good but much less CPU, memory, disk, startup overhead
  • 74. > docker run –it ubuntu bash root@12345:/# echo "I'm in $(cat /etc/issue)” I'm in Ubuntu 14.04.4 LTS Running Ubuntu in a Docker container
  • 75. > time docker run ubuntu echo "Hello, World" Hello, World real 0m0.183s user 0m0.009s sys 0m0.014s Containers boot very quickly. Easily run a dozen at once.
  • 76. You can define a Docker image as code in a Dockerfile
  • 77. FROM gliderlabs/alpine:3.3 RUN apk --no-cache add ruby ruby-dev RUN gem install sinatra --no-ri --no-rdoc RUN mkdir -p /usr/src/app COPY . /usr/src/app WORKDIR /usr/src/app EXPOSE 4567 CMD ["ruby", "app.rb"] Here is the Dockerfile for the Sinatra backend
  • 78. FROM gliderlabs/alpine:3.3 RUN apk --no-cache add ruby ruby-dev RUN gem install sinatra --no-ri --no-rdoc RUN mkdir -p /usr/src/app COPY . /usr/src/app WORKDIR /usr/src/app EXPOSE 4567 CMD ["ruby", "app.rb"] It specifies dependencies, code, config, and how to run the app
  • 79. > docker build -t brikis98/sinatra-backend . Step 0 : FROM gliderlabs/alpine:3.3 ---> 0a7e169bce21 (...) Step 8 : CMD ruby app.rb ---> 2e243eba30ed Successfully built 2e243eba30ed Build the Docker image
  • 80. > docker run -it -p 4567:4567 brikis98/sinatra-backend INFO WEBrick 1.3.1 INFO ruby 2.2.4 (2015-12-16) [x86_64-linux-musl] == Sinatra (v1.4.7) has taken the stage on 4567 for development with backup from WEBrick INFO WEBrick::HTTPServer#start: pid=1 port=4567 Run the Docker image
  • 81. > docker push brikis98/sinatra-backend The push refers to a repository [docker.io/brikis98/sinatra- backend] (len: 1) 2e243eba30ed: Image successfully pushed 7e2e0c53e246: Image successfully pushed 919d9a73b500: Image successfully pushed (...) v1: digest: sha256:09f48ed773966ec7fe4558 size: 14319 You can share your images by pushing them to Docker Hub
  • 82. Now you can reuse the same image in dev, stg, prod, etc
  • 83. > docker pull rails:4.2.6 And you can reuse images created by others.
  • 84. FROM rails:4.2.6 RUN mkdir -p /usr/src/app COPY . /usr/src/app WORKDIR /usr/src/app RUN bundle install EXPOSE 3000 CMD ["rails", "start"] The rails-frontend is built on top of the official rails Docker image
  • 85. No more insane install procedures!
  • 86. rails_frontend: image: brikis98/rails-frontend ports: - "3000:3000" links: - sinatra_backend:sinatra_backend sinatra_backend: image: brikis98/sinatra-backend ports: - "4567:4567" Define your entire dev stack as code with docker-compose
  • 87. rails_frontend: image: brikis98/rails-frontend ports: - "3000:3000" links: - sinatra_backend sinatra_backend: image: brikis98/sinatra-backend ports: - "4567:4567" Docker links provide a simple service discovery mechanism
  • 88. > docker-compose up Starting infrastructureascodetalk_sinatra_backend_1 Recreating infrastructureascodetalk_rails_frontend_1 sinatra_backend_1 | INFO WEBrick 1.3.1 sinatra_backend_1 | INFO ruby 2.2.4 (2015-12-16) sinatra_backend_1 | Sinatra has taken the stage on 4567 rails_frontend_1 | INFO WEBrick 1.3.1 rails_frontend_1 | INFO ruby 2.3.0 (2015-12-25) rails_frontend_1 | INFO WEBrick::HTTPServer#start: port=3000 Run your entire dev stack with one command
  • 89. Advantages of Docker: 1. Easy to create & share images 2. Images run the same way in all environments (dev, test, prod) 3. Easily run the entire stack in dev 4. Minimal overhead 5. Better resource utilization
  • 90. Disadvantages of Docker: 1. Maturity. Ecosystem developing very fast, but still a ways to go 2. Tricky to manage persistent data in a container 3. Tricky to pass secrets to containers
  • 91. 1. Microservices 2. Docker 3. Terraform 4. ECS 5. Recap Outline
  • 92. Terraform is a tool for provisioning infrastructure
  • 94. And many resources for each provider
  • 95. You define infrastructure as code in Terraform templates
  • 96. provider "aws" { region = "us-east-1" } resource "aws_instance" "example" { ami = "ami-408c7f28" instance_type = "t2.micro" } This template creates a single EC2 instance in AWS
  • 97. > terraform plan + aws_instance.example ami: "" => "ami-408c7f28" instance_type: "" => "t2.micro" key_name: "" => "<computed>" private_ip: "" => "<computed>" public_ip: "" => "<computed>" Plan: 1 to add, 0 to change, 0 to destroy. Use the plan command to see what you’re about to deploy
  • 98. > terraform apply aws_instance.example: Creating... ami: "" => "ami-408c7f28" instance_type: "" => "t2.micro" key_name: "" => "<computed>" private_ip: "" => "<computed>" public_ip: "" => "<computed>” aws_instance.example: Creation complete Apply complete! Resources: 1 added, 0 changed, 0 destroyed. Use the apply command to apply the changes
  • 99. Now our EC2 instance is running!
  • 100. resource "aws_instance" "example" { ami = "ami-408c7f28" instance_type = "t2.micro" tags { Name = "terraform-example" } } Let’s give the EC2 instance a tag with a readable name
  • 101. > terraform plan ~ aws_instance.example tags.#: "0" => "1" tags.Name: "" => "terraform-example" Plan: 0 to add, 1 to change, 0 to destroy. Use the plan command again to verify your changes
  • 102. > terraform apply aws_instance.example: Refreshing state... aws_instance.example: Modifying... tags.#: "0" => "1" tags.Name: "" => "terraform-example" aws_instance.example: Modifications complete Apply complete! Resources: 0 added, 1 changed, 0 destroyed. Use the apply command again to deploy those changes
  • 103. Now our EC2 instance has a tag!
  • 104. resource "aws_elb" "example" { name = "example" availability_zones = ["us-east-1a", "us-east-1b"] instances = ["${aws_instance.example.id}"] listener { lb_port = 80 lb_protocol = "http" instance_port = "${var.instance_port}" instance_protocol = "http” } } Let’s add an Elastic Load Balancer (ELB).
  • 105. resource "aws_elb" "example" { name = "example" availability_zones = ["us-east-1a", "us-east-1b"] instances = ["${aws_instance.example.id}"] listener { lb_port = 80 lb_protocol = "http" instance_port = "${var.instance_port}" instance_protocol = "http” } } Terraform supports variables, such as var.instance_port
  • 106. resource "aws_elb" "example" { name = "example" availability_zones = ["us-east-1a", "us-east-1b"] instances = ["${aws_instance.example.id}"] listener { lb_port = 80 lb_protocol = "http" instance_port = "${var.instance_port}" instance_protocol = "http" } } As well as dependencies like aws_instance.example.id
  • 107. resource "aws_elb" "example" { name = "example" availability_zones = ["us-east-1a", "us-east-1b"] instances = ["${aws_instance.example.id}"] listener { lb_port = 80 lb_protocol = "http" instance_port = "${var.instance_port}" instance_protocol = "http" } } It builds a dependency graph and applies it in parallel.
  • 108. After running apply, we have an ELB!
  • 109. > terraform destroy aws_instance.example: Refreshing state... (ID: i-f3d58c70) aws_elb.example: Refreshing state... (ID: example) aws_elb.example: Destroying... aws_elb.example: Destruction complete aws_instance.example: Destroying... aws_instance.example: Destruction complete Apply complete! Resources: 0 added, 0 changed, 2 destroyed. Use the destroy command to delete all your resources
  • 110. For more info, check out The Comprehensive Guide to Terraform
  • 111. Advantages of Terraform: 1. Concise, readable syntax 2. Reusable code: inputs, outputs, modules 3. Plan command! 4. Cloud agnostic 5. Very active development
  • 112. Disadvantages of Terraform: 1. Maturity 2. Collaboration on Terraform state is hard (but terragrunt makes it easier) 3. No rollback 4. Poor secrets management
  • 113. 1. Microservices 2. Docker 3. Terraform 4. ECS 5. Recap Outline
  • 114. EC2 Container Service (ECS) is a way to run Docker on AWS
  • 115. ECS Overview EC2 Instance ECS Cluster ECS Scheduler ECS Agent ECS Tasks ECS Task Definition { "cluster": "example", "serviceName": ”foo", "taskDefinition": "", "desiredCount": 2 } ECS Service Definition { "name": "example", "image": "foo/example", "cpu": 1024, "memory": 2048, "essential": true, }
  • 116. ECS Cluster: several servers managed by ECS EC2 Instance ECS Cluster
  • 117. Typically, the servers are in an Auto Scaling Group EC2 Instance Auto Scaling Group
  • 118. Which can automatically relaunch failed servers EC2 Instance Auto Scaling Group
  • 119. Each server must run the ECS Agent EC2 Instance ECS Cluster ECS Agent
  • 120. ECS Task: Docker container(s) to run, resources they need EC2 Instance ECS Cluster ECS Agent ECS Task Definition { "name": "example", "image": "foo/example", "cpu": 1024, "memory": 2048, "essential": true, }
  • 121. ECS Service: long-running ECS Task & ELB settings EC2 Instance ECS Cluster ECS Agent ECS Task Definition { "name": "example", "image": "foo/example", "cpu": 1024, "memory": 2048, "essential": true, } { "cluster": "example", "serviceName": ”foo", "taskDefinition": "", "desiredCount": 2 } ECS Service Definition
  • 122. ECS Scheduler: Deploys Tasks across the ECS Cluster EC2 Instance ECS Cluster ECS Agent ECS Task Definition { "name": "example", "image": "foo/example", "cpu": 1024, "memory": 2048, "essential": true, } { "cluster": "example", "serviceName": ”foo", "taskDefinition": "", "desiredCount": 2 } ECS Service Definition ECS Scheduler ECS Tasks
  • 123. It will also automatically redeploy failed Services EC2 Instance ECS Cluster ECS Agent ECS Task Definition { "name": "example", "image": "foo/example", "cpu": 1024, "memory": 2048, "essential": true, } { "cluster": "example", "serviceName": ”foo", "taskDefinition": "", "desiredCount": 2 } ECS Service Definition ECS Scheduler ECS Tasks
  • 124. You can associate an ALB or ELB with each ECS service EC2 Instance ECS Cluster ECS Agent ECS Tasks User
  • 125. This lets you distribute traffic across multiple ECS Tasks EC2 Instance ECS Cluster ECS Agent ECS Tasks User
  • 126. Which allows zero-downtime deployment EC2 Instance ECS Cluster ECS Agent ECS TasksUser v1 v1 v1 v2
  • 127. As well as a simple form of service discovery EC2 Instance ECS Cluster ECS Agent ECS Tasks
  • 128. You can use CloudWatch alarms to trigger auto scaling EC2 Instance ECS Cluster ECS Agent ECS Tasks CloudWatch
  • 129. You can scale up by running more ECS Tasks EC2 Instance ECS Cluster ECS Agent ECS Tasks CloudWatch
  • 130. And by adding more EC2 Instances EC2 Instance ECS Cluster ECS Agent ECS Tasks CloudWatch
  • 131. And scale back down when load is lower EC2 Instance ECS Cluster ECS Agent ECS Tasks CloudWatch
  • 132. Let’s deploy our microservices in ECS using Terraform
  • 133. Define the ECS Cluster as an Auto Scaling Group (ASG) EC2 Instance ECS Cluster
  • 134. resource "aws_ecs_cluster" "example_cluster" { name = "example-cluster" } resource "aws_autoscaling_group" "ecs_cluster_instances" { name = "ecs-cluster-instances" min_size = 3 max_size = 3 launch_configuration = "${aws_launch_configuration.ecs_instance.name}" }
  • 135. Ensure each server in the ASG runs the ECS Agent EC2 Instance ECS Cluster ECS Agent
  • 136. # The launch config defines what runs on each EC2 instance resource "aws_launch_configuration" "ecs_instance" { name_prefix = "ecs-instance-" instance_type = "t2.micro" # This is an Amazon ECS AMI, which has an ECS Agent # installed that lets it talk to the ECS cluster image_id = "ami-a98cb2c3” } The launch config runs AWS ECS Linux on each server in the ASG
  • 137. Define an ECS Task for each microservice EC2 Instance ECS Cluster ECS Agent ECS Task Definition { "name": "example", "image": "foo/example", "cpu": 1024, "memory": 2048, "essential": true, }
  • 138. resource "aws_ecs_task_definition" "rails_frontend" { family = "rails-frontend" container_definitions = <<EOF [{ "name": "rails-frontend", "image": "brikis98/rails-frontend:v1", "cpu": 1024, "memory": 768, "essential": true, "portMappings": [{"containerPort": 3000, "hostPort": 3000}] }] EOF } Rails frontend ECS Task
  • 139. resource "aws_ecs_task_definition" "sinatra_backend" { family = "sinatra-backend" container_definitions = <<EOF [{ "name": "sinatra-backend", "image": "brikis98/sinatra-backend:v1", "cpu": 1024, "memory": 768, "essential": true, "portMappings": [{"containerPort": 4567, "hostPort": 4567}] }] EOF } Sinatra Backend ECS Task
  • 140. Define an ECS Service for each ECS Task EC2 Instance ECS Cluster ECS Agent ECS Task Definition { "name": "example", "image": "foo/example", "cpu": 1024, "memory": 2048, "essential": true, } { "cluster": "example", "serviceName": ”foo", "taskDefinition": "", "desiredCount": 2 } ECS Service Definition
  • 141. resource "aws_ecs_service" "rails_frontend" { family = "rails-frontend" cluster = "${aws_ecs_cluster.example_cluster.id}" task_definition = "${aws_ecs_task_definition.rails-fronted.arn}" desired_count = 2 } Rails Frontend ECS Service
  • 142. resource "aws_ecs_service" "sinatra_backend" { family = "sinatra-backend" cluster = "${aws_ecs_cluster.example_cluster.id}" task_definition = "${aws_ecs_task_definition.sinatra_backend.arn}" desired_count = 2 } Sinatra Backend ECS Service
  • 143. Associate an ELB with each ECS Service EC2 Instance ECS Cluster ECS Agent ECS Tasks User
  • 144. resource "aws_elb" "rails_frontend" { name = "rails-frontend" listener { lb_port = 80 lb_protocol = "http" instance_port = 3000 instance_protocol = "http" } } Rails Frontend ELB
  • 145. resource "aws_ecs_service" "rails_frontend" { (...) load_balancer { elb_name = "${aws_elb.rails_frontend.id}" container_name = "rails-frontend" container_port = 3000 } } Associate the ELB with the Rails Frontend ECS Service
  • 146. resource "aws_elb" "sinatra_backend" { name = "sinatra-backend" listener { lb_port = 4567 lb_protocol = "http" instance_port = 4567 instance_protocol = "http" } } Sinatra Backend ELB
  • 147. resource "aws_ecs_service" "sinatra_backend" { (...) load_balancer { elb_name = "${aws_elb.sinatra_backend.id}" container_name = "sinatra-backend" container_port = 4567 } } Associate the ELB with the Sinatra Backend ECS Service
  • 148. Set up service discovery between the microservices EC2 Instance ECS Cluster ECS Agent ECS Tasks
  • 149. resource "aws_ecs_task_definition" "rails_frontend" { family = "rails-frontend" container_definitions = <<EOF [{ ... "environment": [{ "name": "SINATRA_BACKEND_PORT", "value": "tcp://${aws_elb.sinatra_backend.dns_name}:4567" }] }] EOF } Pass the Sinatra Bckend ELB URL as env var to Rails Frontend
  • 150. It’s time to deploy! EC2 Instance ECS Cluster ECS Agent ECS Task Definition { "name": "example", "image": "foo/example", "cpu": 1024, "memory": 2048, "essential": true, } { "cluster": "example", "serviceName": ”foo", "taskDefinition": "", "desiredCount": 2 } ECS Service Definition ECS Scheduler ECS Tasks
  • 151. > terraform apply aws_ecs_cluster.example_cluster: Creating... name: "" => "example-cluster" aws_ecs_task_definition.sinatra_backend: Creating... ... Apply complete! Resources: 17 added, 0 changed, 0 destroyed. Use the apply command to deploy the ECS Cluster & Tasks
  • 152. See the cluster in the ECS console
  • 153. Track events for each Service
  • 154. As well as basic metrics
  • 156. resource "aws_ecs_task_definition" "sinatra_backend" { family = "sinatra-backend" container_definitions = <<EOF [{ "name": "sinatra-backend", "image": "brikis98/sinatra-backend:v2", ... } To deploy a new image, just update the docker tag
  • 157. > terraform plan ~ aws_ecs_service.sinatra_backend task_definition: "arn...sinatra-backend:3" => "<computed>" -/+ aws_ecs_task_definition.sinatra_backend arn: "arn...sinatra-backend:3" => "<computed>" container_definitions: "bb5352f" => "2ff6ae" (forces new resource) revision: "3" => "<computed>” Plan: 1 to add, 1 to change, 1 to destroy. Use the plan command to verify the changes
  • 158. Apply the changes and you’ll see v2.
  • 159. Advantages of ECS: 1. One of the simplest Docker cluster management tools 2. Almost no extra cost if on AWS 3. Pluggable scheduler 4. Auto-restart of instances & Tasks 5. Automatic ALB/ELB integration
  • 160. Disadvantages of ECS: 1. UI is so-so 2. Minimal monitoring built-in 3. ALB is broken
  • 161. 1. Microservices 2. Docker 3. Terraform 4. ECS 5. Recap Outline
  • 162. Benefits of infrastructure-as-code: 1. Reuse 2. Automation 3. Version control 4. Code review 5. Testing 6. Documentation
  • 163. Slides and code from this talk: ybrikman.com/speaking