This document discusses Docker Inc's transition to running their infrastructure on Docker EE (Docker Enterprise Edition). Some key points: - Previously Docker Inc ran their infrastructure (Docker Hub, Docker Store, etc) on a collection of Docker hosts managed imperatively without orchestration. - They transitioned to running everything on Docker EE for the benefits of declarative orchestration using Docker Swarm, access controls, and resource management. - The transition was done by changing the orchestration layer only, without changing applications, containers, logs, metrics, etc. This allowed them to gradually transition services to Docker EE. - Running as a customer of their own product has provided Docker Inc valuable insights into best practices, feature requests

1. Docker on Docker
Add
picture
here
Brett Inman
Infrastructure Engineer
Marcus Martins
Senior Manager - Orchestration
Docker Inc Docker Inc

2. Docker on Docker
● Where we were
● How we moved to Docker EE
● What we learned
● How we’re making Docker EE better

3. SaaS at Docker
● Docker Hub
● Docker Store
● Docker Cloud
> 1B pulls a month
> 500K builds a month
> 5000 requests / sec

4. Philosophy
● High Availability
● Self-healing
● Self-service
● Build it - own it

5. Where we were

6. Previous Infrastructure
Infra provides:
● Group of Docker Hosts per team
● Docker TLS Certs
● Centralized logging
● Centralized metrics
● Service discovery / routing

7. How dev used the old
setup
Dogfooding Docker since 2013

8. How dev used the old setup
The good parts:
● Fully containerized
● Docker as the only interface
● No SSH access

9. How dev used the old setup
The bad parts:
● Multiple deployments tools
● Imperative deployments

10. How dev used the old setup
The bad parts:
● Hosts as pets
● Manual resource management

11. What was missing

12. Docker EE
Add picture
here
● Docker Swarm
● User
Management
● Resource
Access Control

13. How we did it

14. Didn’t Change Anything*

15. Focus
Changed:
● Orchestration
Did not change:
● Code
● Containers
● Logs
● Metrics
● Service Discovery
● Routing

16. Enabling the transition

17. Self-service transition
● Goal: give service teams control
● Use both deployments at will
● Swarmkit mesh networking - magic, but:
○ Hard cutover transition for us
○ Can’t pull metrics from individual containers

18. Our Routing
● Registrator
● Consul
● Consul-template
● Haproxy

19. Host Port Publishing
--publish mode=host
Enables classic engine behavior of
exposing container port on host for
Swarm services

20. Automation
● Bootstrap lock
● Discovery - Load balancer or tags

21. Quorum failure
without intervention

22. Resources Constraints
Human is no longer the scheduler

24. Reservation
=
Limit
docker service create
--reserve-memory 640m
--limit-memory 640m

25. System
Reservation
docker service create
--name system-reservation
--reserve-memory 1G
--limit-memory 1G
--mode global
--init
ubuntu sleep infinity

26. Cloud Permissions
● IAM is at host level - not container
● Each team gets an autoscaling group
● Leverage UCP Collections

27. New deployment model

28. Declarative version: '3.3'
services:
api:
image: example/api:2.1.2
ports:
- mode: host
protocol: tcp
target: 80
environment:
DEBUG: False
ENVIRONMENT: product
...
● Compose file
● Deploy with `docker
stack deploy`
● All stacks defined in
source control

29. Updates version: '3.2'
services:
api:
image: example/api:2.1.2
deploy:
replicas: 20
…
update_config:
delay: 10s
parallelism: 2
order: stop-first
max_failure_ratio: 0
failure_action: rollback
● Rolling updates
● Automated rollback

30. Access Control version: '3.2'
services:
api:
image: example/api:2.1.2
deploy:
replicas: 2
…
labels:
com.docker.ucp.access.label:/Api
…
● Using Docker EE
Collections

31. What’s better?
Decouple host from application
● Host replacement
● Host failures
● Build it own it - developers don’t own hosts

32. Host
Replacement
Three touches:
● Ops
● Dev
● Ops

33. Host
Replacement
One touch:
● Ops

34. What else?
● Easy to scale application
● Standardization
● Docker-native

35. Where we are now

36. Now
● All of Docker SaaS is running on
Docker EE
● 80 worker nodes
● 60 swarm services
● 1000 tasks
● Multiple deployments a day

37. Improving Docker EE

38. Best Practices
Topics including:
● Managing Resources
● Provisioning and Automation
● Logging and Monitoring
Coming soon to success.docker.com

39. Product guidance
● In-house customer at scale in production
● Architecture
● 35+ feature requests
● Host port publishing
● More soon!

40. Canary
● Running every Docker EE Release Candidate
● 50+ bugs filed
● Customer Zero

41. Docker EE
with Kubernetes
in production

42. Thank you
● Questions?

43. Docker EE
Hosted Demo
Add picture
here
docker.com/trial
● Free 4 Hour Demo
● No Servers Required
● Full Docker EE
Cluster Access