Continuous delivery is all the rage these days, but without self-healing, highly available, and fault-tolerant infrastructure to deploy your applications to, it’s really only one piece of a much larger picture. Apache Mesos was born at UC Berkeley and grew into a robust, highly scalable cluster orchestrator while running thousands of nodes at Twitter. Support for Docker containers was added in 2013, and since then, it’s been adopted by companies like Netflix and Apple to run their critical infrastructure. Mesosphere has built the open source Datacenter Operating System (DC/OS) around Apache Mesos to provide all the supplementary tooling necessary to take Mesos to a production environment. Jenkins with DC/OS allows you to spin up build agents dynamically, an approach which has allowed companies like PayPal to cut the footprint of their build farms by hundreds of nodes, saving money on infrastructure by increasing utilization and reducing obstacles to providing teams with the resources they need when they need them. Sunil Shah and Roger Ignazio introduce DC/OS and demonstrate how to integrate it with the stalwart continuous integration server Jenkins, allowing you to set up a continuous delivery pipeline that takes an application composed of microservices from code repository to Docker Hub to a staging or production server with seamless automation. Sunil and Roger walk attendees through setting up their own pipeline using Jenkins on a DC/OS cluster, from installation and configuration of Jenkins to setting up a build to actually deploying it to a live environment where it can serve traffic, and also cover the internals of practical microservice architecture, including component-level deployment, application-level persistence, and intraprocess communication via service discovery.

1. © 2016 Mesosphere, Inc. All Rights Reserved.
VELOCITY NYC | SEPTEMBER 20, 2016
CONTINUOUS
DELIVERY WITH
DC/OS AND
JENKINS

2. © 2016 Mesosphere, Inc. All Rights Reserved.
SUNIL SHAH
Product Manager at Mesosphere
@ssk2
ROGER IGNAZIO
Tech Lead at Mesosphere
@rogerignazio
WHO WE ARE

3. © 2016 Mesosphere, Inc. All Rights Reserved.
mesosinaction.com
Use the code vecymes for
42% off!
BUY MY BOOK!

4. © 2016 Mesosphere, Inc. All Rights Reserved.
AGENDA
Lecture
● Introduction to Apache Mesos and DC/OS
● Components that make up modern infra.
● Running Jenkins as a service on DC/OS
● Continuously deploying applications to DC/OS
Demos & Lab
● Installing and configuring Jenkins
● Installing and configuring a load balancer
● Creating a new CI/CD pipeline
● Putting it all together (CD in practice)

5. © 2016 Mesosphere, Inc. All Rights Reserved.
DEVELOPER AGILITY,
DEFINED

6. © 2016 Mesosphere, Inc. All Rights Reserved.
Developer agility empowers developers to
● ship their apps to production
● leverage the power of Mesos and DC/OS
● fix bugs rapidly
without downtime!
DEVELOPER AGILITY, DEFINED

7. © 2016 Mesosphere, Inc. All Rights Reserved.
Continuous integration is not the same as
continuous delivery or continuous deployment.
DEVELOPER AGILITY, DEFINED

8. © 2016 Mesosphere, Inc. All Rights Reserved.
An artifact repo stores built binaries, whereas an
image registry stores templates for deployment.
DEVELOPER AGILITY, DEFINED

9. © 2016 Mesosphere, Inc. All Rights Reserved.
DEVELOPER AGILITY, DEFINED

10. © 2016 Mesosphere, Inc. All Rights Reserved.
INTRO TO
APACHE MESOS
AND DC/OS

11. © 2016 Mesosphere, Inc. All Rights Reserved.
A QUICK PRIMER ON CONTAINERS
Physical Server
Virtual Machine–Based
Application Deployment
Container–Based
Application Deployment
Hypervisor
OS
VM VM
OS
App App
Physical Server
Hypervisor
OS
VM VM
OS
App App
Physical Server
Operating System
App App
Physical Server
Operating System
App App
Mesos
Isolate apps by running
multiple VMs per physical
server; still need to
manage each guest OS!
Isolate apps using
features of the host OS,
such as Linux cgroups.

12. © 2016 Mesosphere, Inc. All Rights Reserved.
A QUICK PRIMER ON CONTAINERS
Linux cgroup
Dependencies
Linux Host OS
Application
Docker Container
Dependencies
Host OS
Application
Linux cgroupsDocker Containers
VM
Dependencies
Hypervisor
Application
Virtual Machines
Guest OS
Docker Engine

13. © 2016 Mesosphere, Inc. All Rights Reserved.
A BIT OF CLARIFICATION
https://mesos.apache.org https://dcos.io

14. © 2016 Mesosphere, Inc. All Rights Reserved.
● General purpose cluster resource manager
● Represents many machines as a single entity
● Advertises resources directly to frameworks
● Works at scale: Apple, Twitter, Airbnb, Netflix, …
WHAT IS MESOS?

15. © 2016 Mesosphere, Inc. All Rights Reserved.
● Two-tier scheduling across resource types
● cpus, mem, disk, and ports by default
● Masters are highly available, agents are fault tolerant
● Checkpointing, agent recovery
● Resource isolation between processes
● Linux cgroups, Docker, …
● Language bindings: C++, Java, Python, Go, …
WHAT IS MESOS? (CONTINUED)

16. © 2016 Mesosphere, Inc. All Rights Reserved.
MESOS ARCHITECTURE
Launch task(s)
Mesos Master Quorum
Mesos Master
(Standby)
ZK
Mesos Master
(Standby)
ZK
Mesos Master
(Leader)
ZK
Framework B
Scheduler
Agent 1
Framework A
Executor
Task(s)
Agent n
Framework A
Executor
Task(s)
Framework B
Executor
Task(s)
Resource offer
Resource offer
Framework A
Scheduler
Launch task(s)

17. © 2016 Mesosphere, Inc. All Rights Reserved.
ANATOMY OF A RESOURCE OFFER (TWO-TIER SCHEDULING)
Mesos Master(s)
Mesos Agent
16 CPUs
128 GB RAM
1 TB disk
Mesos Agent
16 CPUs
128 GB RAM
1 TB disk
Mesos Agent
16 CPUs
128 GB RAM
1 TB disk
Master offers resources
to scheduler
Scheduler
Executor
Available compute resources
Resource offer accepted, launch executors/tasks
Tasks
Executor
Tasks
Executor
Tasks
Scheduler accepts or
declines an offer

18. © 2016 Mesosphere, Inc. All Rights Reserved.
● Service discovery and load balancing
● BIND, Mesos-DNS, Consul-Mesos, Marathon-LB
NEW (OLD) PROBLEMS

19. © 2016 Mesosphere, Inc. All Rights Reserved.
● Service discovery and load balancing
● BIND, Mesos-DNS, Consul-Mesos, Marathon-LB
● Monitoring and metrics collection
● Collectd, Nagios, Prometheus, Snap
NEW (OLD) PROBLEMS

20. © 2016 Mesosphere, Inc. All Rights Reserved.
● Service discovery and load balancing
● BIND, Mesos-DNS, Consul-Mesos, Marathon-LB
● Monitoring and metrics collection
● Collectd, Nagios, Prometheus, Snap
● Persistent storage (filesystems, databases,
etc)
● Ceph, HDFS, Amazon EBS / EFS / S3, NFS, Cassandra
NEW (OLD) PROBLEMS

21. © 2016 Mesosphere, Inc. All Rights Reserved.
● Service discovery and load balancing
● BIND, Mesos-DNS, Consul-Mesos, Marathon-LB
● Monitoring and metrics collection
● Collectd, Nagios, Prometheus, Snap
● Persistent storage (filesystems, databases,
etc)
● Ceph, HDFS, Amazon EBS / EFS / S3, NFS, Cassandra
● Administration: named URIs vs. ports, IPAM
● Nginx, HAProxy, Mesos-DNS, dhcpd, Minuteman
NEW (OLD) PROBLEMS

22. © 2016 Mesosphere, Inc. All Rights Reserved.
https://dcos.io
https://github.com/dcos
DC/OS:
BUILT ON
MESOS

23. © 2016 Mesosphere, Inc. All Rights Reserved.
DC/OS:
BUILT ON
MESOS

24. © 2016 Mesosphere, Inc. All Rights Reserved.
All of the benefits of Mesos, plus
● Built-in service discovery and load balancing
● Support for stateful services
● Turn-key installation of distributed systems
● Cloud-agnostic installer
● Web and command-line interfaces
● All components are integration tested and supported by
Mesosphere, Inc.
MESOS AND DC/OS: BETTER TOGETHER

25. © 2016 Mesosphere, Inc. All Rights Reserved.
JENKINS ON
DC/OS

26. © 2016 Mesosphere, Inc. All Rights Reserved.
MULTIPLE JENKINS MASTERS
Team A Jenkins
. . .
Jenkins Agent 1
Jenkins Agent 2
. . .
Jenkins Agent N
Team B Jenkins
Jenkins Agent 1
Jenkins Agent 2
. . .
Jenkins Agent N
Team X Jenkins
Jenkins Agent 1
Jenkins Agent 2
. . .
Jenkins Agent N

27. © 2016 Mesosphere, Inc. All Rights Reserved.
MULTIPLE JENKINS MASTERS → STATIC PARTITIONING
Team A Jenkins
. . .
Jenkins Agent 1
Jenkins Agent 2
. . .
Jenkins Agent N
Team B Jenkins
Jenkins Agent 1
Jenkins Agent 2
. . .
Jenkins Agent N
Team X Jenkins
Jenkins Agent 1
Jenkins Agent 2
. . .
Jenkins Agent N
90% Utilized
(Normal)
140% Utilized
(40 builds in queue)
0% Utilized
(Idle)

28. © 2016 Mesosphere, Inc. All Rights Reserved.
RUNNING JENKINS ON DC/OS → FAIR SHARING
Team A Jenkins . . .
DC/OS Agent 1
DC/OS Agent 4
DC/OS Agent N
Team B Jenkins
DC/OS Agent 2
DC/OS Agent 5
Team X Jenkins
DC/OS Agent 3
DC/OS Agent 6
. . .
DC/OS

29. © 2016 Mesosphere, Inc. All Rights Reserved.
CONTINUOUSLY
DEPLOYING
APPLICATIONS TO
DC/OS

30. © 2016 Mesosphere, Inc. All Rights Reserved.
TRADITIONAL RELEASE PROCESS
● Planning & implementing new
technologies
● Waiting for people &
infrastructure
● Building environment specific
CI/CD for each project
● Moving apps from dev to staging
to prod
DEV(OPS) TEAMS SPEND SIGNIFICANT TIME AND EFFORT ON:
PROJECT
PLANNING
WEEK 1 WEEK 2 WEEK 3 WEEKS 4 TO 8 WEEK 9
CUSTOMER
FEEDBACKCONFIGURATIONRESOURCE
ALLOCATION
Dev
Test Stage
Release

31. © 2016 Mesosphere, Inc. All Rights Reserved.
MODERN RELEASE PROCESS
WEEK 1 WEEK 2 WEEK 3 WEEK 4
PROJECT
PLANNING
CUSTOMER
FEEDBACK
CUSTOMER
FEEDBACK
CUSTOMER
FEEDBACK
RELEASE 1 RELEASE 2 RELEASE 3 RELEASE 4
Better products through a
repeatable release cadence
Happier customers through
continuous feedback
Dev
Test Stage
Release Dev
Test Stage
Release
Dev
Test Stage
Release Dev
Test Stage
Release

32. © 2016 Mesosphere, Inc. All Rights Reserved.
● Scheduling — advertising available compute resources
● Deployments — getting an application onto a node
● Health checks — ensuring the app/service is healthy
● Service discovery — connecting to dependent services
● Persistence — running stateful services in containers
DEPLOYING APPLICATIONS: BASIC REQUIREMENTS

33. © 2016 Mesosphere, Inc. All Rights Reserved.
A sysadmin provisions one or more
physical/virtual servers to host the app
Mesos resource offers (two-tier
scheduling) offers available resources
directly to frameworks
DEPLOYING APPLICATIONS: SCHEDULING
Before DC/OS With DC/OS

34. © 2016 Mesosphere, Inc. All Rights Reserved.
By hand or using Puppet / Chef /
Ansible
Jenkins SSHing to the machine and
running a shell script
Note: all dependencies must also be present!
Marathon deploys containers, ideally
using a CI/CD tool to create/update
app definitions
Docker containers packages app and
dependencies
DEPLOYING APPLICATIONS: DEPLOYMENTS
Before DC/OS With DC/OS

35. © 2016 Mesosphere, Inc. All Rights Reserved.
Nagios pages a sysadmin
Marathon performs health checks,
restarts unhealthy/failed instances
DEPLOYING APPLICATIONS: HEALTH CHECKS
Before DC/OS With DC/OS

36. © 2016 Mesosphere, Inc. All Rights Reserved.
Static hostnames / IP addresses in a
spreadsheet or config management
A sysadmin configures a load balancer
manually or with Puppet / Chef /
Ansible
Mesos-DNS provides DNS resolution for
running services (hostname / IP
address, ports, etc)
Load balancer configs built
dynamically using cluster state
DEPLOYING APPLICATIONS: SERVICE DISCOVERY
Before DC/OS With DC/OS

37. © 2016 Mesosphere, Inc. All Rights Reserved.
Individual servers with RAID 1/5/6/10,
expensive SANs, NFS, etc.
Dedicated, statically partitioned Ceph
or Gluster storage clusters
Mesos external/persistent volumes
(REX-Ray), HDFS, etc.
Self-healing Ceph or Gluster on Mesos /
DC/OS
DEPLOYING APPLICATIONS: PERSISTENCE
Before DC/OS With DC/OS

38. © 2016 Mesosphere, Inc. All Rights Reserved.
Q: Given that we are all engineers,
which tools would we like to work with?

39. © 2016 Mesosphere, Inc. All Rights Reserved.
DEMOS & LAB

40. © 2016 Mesosphere, Inc. All Rights Reserved. 40
PIPELINE COMPONENTS
Continuous
Integration
Container
Registry
Container
Orchestrator
Version Control
System
Load Balancer
Production
Environment
Continuous Delivery Pipeline
git push
DC/OS

41. © 2016 Mesosphere, Inc. All Rights Reserved. 41
PIPELINE COMPONENTS
Jenkins
DockerHub
MarathonGitHub Marathon-lbMesos
Continuous Delivery Pipeline
git push
DC/OS

42. © 2016 Mesosphere, Inc. All Rights Reserved. 42
PIPELINE CONFIGURATION
Jenkinsfile
Dockerfile
marathon.json
Continuous Delivery Pipeline
git push
DC/OS

43. © 2016 Mesosphere, Inc. All Rights Reserved.
A SNEAK PREVIEW

44. © 2016 Mesosphere, Inc. All Rights Reserved.
● Head over to github.com/mesosphere/vny
● Follow the exercises!
● Use the username/password: velocityuser/velocitypassword
YOUR TURN

45. © 2016 Mesosphere, Inc. All Rights Reserved.
Roger Ignazio
roger@mesosphere.com
@rogerignazio
Sunil Shah
sunil@mesosphere.com
@ssk2
THANK YOU!
Learn more by visiting DCOS.io and Mesosphere.com