1. Docker EE will include an unmodified Kubernetes distribution to provide orchestration capabilities alongside Docker Swarm. 2. When running mixed workloads across orchestrators, resource contention is a risk and it is recommended to separate workloads by orchestrator on each node for now. 3. Docker EE aims to address the shortcomings of running mixed workloads to better support this in the future.

1. Kubernetes in Docker
Alex Mavrogiannis
Daniel Hiltgen
Docker EE Engineering

2. Agenda
1. Recap EE demo from Keynote
2. General CE/EE Architectures
3. EE: Topics on mixed workloads
4. EE: AuthN/AuthZ
5. Q&A

3. Demo Recap

4. General CE/EE Architecture

5. Linuxkit VM
Kubernetes CLI
(Swarm-Mode) Kubernetes
etcd
Docker CLI
kubeadm
Docker CE to include Kubernetes (Windows and Mac)
Stacks
CRD
Single Docker Engine
vpnkitHost fs mounts hyperkit / hyperv

6. Docker EE to include Kubernetes
Docker Enterprise Edition
Production Ready Windows and IBM P/Z Support
Pods, batch jobs, blue-green deployments,
horizontal pod auto-scaling
Docker Swarm Swarm-Mode Kubernetes
Private Image Registry
Secure Access and User
Management
App and Cluster Management
Image Security Scanning Content Trust and Verification
Policy Management

7. Orchestrator: Docker Swarm
● github.com/docker/swarm
● Cluster-wide imperative API based on the Single-node API of the Docker Engine
● High Availability and peer discovery managed through a pluggable discovery backend:
etcd, consul
● Leader caches cluster state: containers, volumes, networks etc.
● Scheduling decisions based on the reservations and limits of all cached Docker Containers.

8. Orchestrator: Docker Engine with Swarm-Mode Enabled
● github.com/docker/swarmkit
● Declarative State through the “Service” construct
● Built-in Routing Mesh & Overlay networking
● Scheduling decisions based on all the reservations of all swarm services across all nodes.
● Built-in in-memory Raft Store for all state (persisted to disk)
● Built in CA, per-node cryptographic node identity, mTLS between all endpoints

9. Orchestrator: Kubernetes
● github.com/kubernetes/kubernetes
● Scheduling: Pods
● Declarative State through “Controllers”: Deployment, ReplicaSet, DaemonSet …
● Flat Networking model delegated to plugins
● Scheduling decisions based on usage, reservations and limits of all kubernetes workloads.
○ Usage monitored through “cadvisor”, a cgroup monitoring tool

10. GUI
Universal Control Plane
Trusted Registry Kubernetes CLI
Docker Engine
Swarm-Mode
Docker Swarm Kubernetes
etcd
CA OIDC Provider
Docker CLI
Agent Reconciler
Docker EE to include Kubernetes

11. Docker EE Architectural Highlights
● Unmodified Kubernetes components run as Docker containers
● Swarm Managers are Kubernetes Masters
● Swarmkit node inventory is source of truth
● Cryptographic Node Identity and mTLS used throughout
● UCP Agent/Reconciler manages component lifecycle
○ Manager / Worker states
○ Certificate validity

12. Plugin Interfaces
● General: Native API extensibility supported
○ API server and kubelet flags not modifiable
● Networking:
○ Support for CNI plugin during install
○ Ingress
● Storage: Docker Volume Plugins supported via built-in flexvolume driver, CSI in future
● Metrics: Heapster Storage Backends or Prometheus

13. Topics on Mixed Workloads

14. Resource Contention
● Allocatable Resources: The set of CPU and Memory resources available for scheduling by
an orchestrator
● Multiple orchestrators = Different definitions of allocatable resources
○ Docker Swarm: Respectful of CPU/Memory limits, but container cache may be stale
○ Docker Engine with Swarm-Mode: Only aware of its own reservations
○ Kubernetes: Effective handling of out-of-resource situations, but only for kubernetes
workloads
● When a node is at/near capacity:
○ All CPU shares throttled equally
○ The OS’s OOM killer kills processes
○ All orchestrators will reschedule on OOM, but potential workload interruption

15. Resource Contention (cont.)
For production workloads
● For now we recommended one orchestrator per node
● Working on UX to provide simple orchestrator selection per node
Future:
● Working to address shortcomings to better support mixed orchestration

16. Workload Interoperability
● Networking
○ Layer 3 not connected between kubernetes & swarm
○ Batteries-included kubernetes ingress controller
○ Layer 7 routing for swarm workloads
○ Configure external DNS
● Storage: Kubernetes workloads with docker volumes via flexvolume

17. AuthN / AuthZ

18. AuthN
AuthZ

19. In Summary...
● Docker will include an unmodified Kubernetes
distribution.
● Resource Contention mitigated via workload
separation
● In EE, Authentication and Authorization integrated
via standard plugin interfaces.

20. Thank You!
alexmavr
dhiltgen