SlideShare une entreprise Scribd logo
1  sur  22
Télécharger pour lire hors ligne
Linux Plumbers Conference 2014, Düsseldorf 
Docker and the Linux kernel 
Cristian S., Docker Inc. 
cristian@docker.com
What is Docker ?
The Matrix From Hell
Another Matrix From Hell
Solution: 
the intermodal shipping container
Solved!
Solution to the deployment problem: 
the Linux container
Solved!
High level overview 
• Uses namespaces & cgroups 
• Runs on mainline kernels 
• Lower overhead than VMs 
• Can run a full system with an init or a 
single service/process 
• Snapshot based approach to build 
one image on top of another
Docker's code 
• Licensed under the Apache 2 license 
• There's no paid/premium/commercial 
version 
• Docker and its code are free and will 
be free 
• Lives at: 
https://github.com/docker/docker/
Use cases 
• Application development, 
testing, packaging & deployment 
• PaaS/SaaS/cloud infrastructure 
• Application & service isolation 
• Stress testing & benchmarking 
(including the Linux kernel)
Under the hood 
• Exec drivers provide the execution environment 
(virtualization/container tech) 
– native (libcontainer based) and LXC 
– are platform dependent 
– native is the default 
• Graph drivers are the storage providers 
– aufs, devicemapper, btrfs and vfs 
– vfs shouldn't be used (only used by the tests by 
default) 
– PRs open on GitHub for ZFS and OverlayFS 
• Existing graph & exec drivers only support Linux
Kernel requirements 
• Kernel 3.8 is the absolute minimum 
(except RHEL's 2.6.32) 
• Stable & supported kernels >= 3.10 are 
recommended 
• BTRFS has special requirements 
• Kernel configuration can be checked using 
https://raw.githubusercontent.com/docker/ 
docker/master/contrib/check-config.sh
AUFS graph driver stability & 
performance 
• Operations with containers are faster than devicemapper & 
(sometimes) btrfs 
• Known problems 
– Stale NFS file handle, cap_set_file, invalid argument on mount 
– Causes troubles on btrfs, remote file systems and many file 
systems which aren't ext3/ext4 
– Direct IO problems & poor performance for IO intensive workloads 
• Limitations 
– Requires aufs-tools for auplink to dereference hard links 
– No support for hard links across layers 
– Limit of 127 layers 
– Can't be used on Fedora/RHEL/all systems which don't apply AUFS 
patches and don't ship aufs-tools 
• AUFS is developed outside of the mainline kernel tree 
• Update the kernel using distro updates
Devicemapper graph driver stability 
& performance 
• Uses loopback mounted block devices by default 
• Allows EXT4 or XFS to be used 
• Known problems 
– EBUSY errors (fix to be tested) 
– Potential file system corruption bug with ext4 
● Might be caused by loopback mounted block devices 
– Older kernels: space not freed when freed on the file 
system, various kernel errors and problems 
• Limitations 
– Devicemapper requires configuration of the storage to avoid 
the use of loopback mounted block devices 
• Update the kernel using distro updates
btrfs graph driver stability & 
performance 
• BTRFS bugs are a problem for Docker 
• Using the RAID like features of BTRFS is likely to cause data loss 
• Known problems 
– Data can be corrupted if exotic mount options are used 
– Kernel 3.8 & older than the latest stable kernels can cause data loss & 
corruption 
– The file system becomes slower the more data is written & stored on it 
– Performance degrades quickly & fragmentation is a problem 
– Balancing the file system to fix fragmentation could trigger some bugs 
• BTRFS is used automatically if Docker's root folder is on BTRFS 
• Update the kernel using distro updates 
• Using the latest minor version of supported kernels is recommended
Performance & stability 
• Generally speaking, the kernel has been stable 
– Fixes have been made to file systems, namespaces, cgroups, 
netfilter, aufs, btrfs, devicemapper and other kernel components 
– Recent kernels and the newest minor versions of LTS kernels have 
fixed a lot of problems, including devicemapper and btrfs problems 
– Some PID 1 issues are still being discussed 
– Changes around the cgroups and the namespaces have been 
made; this has also helped improve stability for containers 
• Performance is actively being studied 
– Docker's code has been improved to use less memory & be faster 
– Performance needs to be studied on the kernel side to achieve 
better scalability by a) making fewer syscalls in Docker where 
possible, b) making those syscalls faster in the kernel
Kernel recommendations 
• Avoid running kernels no longer supported 
by your Linux distribution (kernel 3.8 lts-raring 
from Ubuntu 12.04.x) 
• Open Docker issues on github; post errors 
(full kernel panics, btrfs check output, 
Docker daemon logs, `docker info`, 
`docker version` and `uname -a` output) 
when encountering errors 
• File bug reports with your distribution
What can Docker do for the kernel? 
• Can be used to test the kernel's stability and 
evaluate its performance 
• Changes made to the kernel can be tested 
easily against Docker 
• Can be used to test the running kernel to 
avoid breaking user space compatibility 
• Makes it simple to mix workloads for stress 
testing and performance testing 
• Exposes some hard to trigger kernel problems
What can a kernel developer do 
with Docker? 
• Find bottlenecks in the kernel's code 
• Run a KVM VM in a container 
• Stress testing for file systems, network, 
namespaces and the kernel in general 
• Large scale testing with containers 
• Hardware testing in containers 
• Network testing with advanced 
topologies
How can someone contribute to Docker? 
What about kernel developers? 
• You don't necessarily have to write code 
• Providing feedback, doing code review or 
telling the right kernel developer to do 
that are also useful helpful 
• Suggestions on how to debug specific 
kernel bugs and bugs in general are 
welcome 
• Contributing to Docker is also helping the 
kernel because Docker relies on the kernel
Thank you! Questions? 
http://docker.com 
@docker

Contenu connexe

Tendances

Docker introduction for the beginners
Docker introduction for the beginnersDocker introduction for the beginners
Docker introduction for the beginnersJuneyoung Oh
 
Introduction to Docker Compose
Introduction to Docker ComposeIntroduction to Docker Compose
Introduction to Docker ComposeAjeet Singh Raina
 
Docker introduction
Docker introductionDocker introduction
Docker introductionPhuc Nguyen
 
How to write a Dockerfile
How to write a DockerfileHow to write a Dockerfile
How to write a DockerfileKnoldus Inc.
 
Docker 101 - Nov 2016
Docker 101 - Nov 2016Docker 101 - Nov 2016
Docker 101 - Nov 2016Docker, Inc.
 
Docker Tutorial For Beginners | What Is Docker And How It Works? | Docker Tut...
Docker Tutorial For Beginners | What Is Docker And How It Works? | Docker Tut...Docker Tutorial For Beginners | What Is Docker And How It Works? | Docker Tut...
Docker Tutorial For Beginners | What Is Docker And How It Works? | Docker Tut...Simplilearn
 
Docker Introduction
Docker IntroductionDocker Introduction
Docker IntroductionSparkbit
 
Getting started with Docker
Getting started with DockerGetting started with Docker
Getting started with DockerRavindu Fernando
 
Docker: From Zero to Hero
Docker: From Zero to HeroDocker: From Zero to Hero
Docker: From Zero to Herofazalraja
 
Introduction to Docker
Introduction to DockerIntroduction to Docker
Introduction to DockerAditya Konarde
 
An Introduction to Kubernetes
An Introduction to KubernetesAn Introduction to Kubernetes
An Introduction to KubernetesImesh Gunaratne
 
Docker introduction (1)
Docker introduction (1)Docker introduction (1)
Docker introduction (1)Gourav Varma
 
Docker Introduction
Docker IntroductionDocker Introduction
Docker IntroductionPeng Xiao
 
Dockers and containers basics
Dockers and containers basicsDockers and containers basics
Dockers and containers basicsSourabh Saxena
 

Tendances (20)

Docker introduction for the beginners
Docker introduction for the beginnersDocker introduction for the beginners
Docker introduction for the beginners
 
Introduction to Docker Compose
Introduction to Docker ComposeIntroduction to Docker Compose
Introduction to Docker Compose
 
Docker introduction
Docker introductionDocker introduction
Docker introduction
 
How to write a Dockerfile
How to write a DockerfileHow to write a Dockerfile
How to write a Dockerfile
 
Docker 101 - Nov 2016
Docker 101 - Nov 2016Docker 101 - Nov 2016
Docker 101 - Nov 2016
 
Docker Tutorial For Beginners | What Is Docker And How It Works? | Docker Tut...
Docker Tutorial For Beginners | What Is Docker And How It Works? | Docker Tut...Docker Tutorial For Beginners | What Is Docker And How It Works? | Docker Tut...
Docker Tutorial For Beginners | What Is Docker And How It Works? | Docker Tut...
 
Docker Introduction
Docker IntroductionDocker Introduction
Docker Introduction
 
Getting started with Docker
Getting started with DockerGetting started with Docker
Getting started with Docker
 
Docker: From Zero to Hero
Docker: From Zero to HeroDocker: From Zero to Hero
Docker: From Zero to Hero
 
Docker Basics
Docker BasicsDocker Basics
Docker Basics
 
Docker basics
Docker basicsDocker basics
Docker basics
 
Introduction to Docker
Introduction to DockerIntroduction to Docker
Introduction to Docker
 
Docker compose
Docker composeDocker compose
Docker compose
 
Docker in real life
Docker in real lifeDocker in real life
Docker in real life
 
An Introduction to Kubernetes
An Introduction to KubernetesAn Introduction to Kubernetes
An Introduction to Kubernetes
 
Docker introduction (1)
Docker introduction (1)Docker introduction (1)
Docker introduction (1)
 
Kubernetes 101
Kubernetes 101Kubernetes 101
Kubernetes 101
 
Docker Introduction
Docker IntroductionDocker Introduction
Docker Introduction
 
Introduction to container based virtualization with docker
Introduction to container based virtualization with dockerIntroduction to container based virtualization with docker
Introduction to container based virtualization with docker
 
Dockers and containers basics
Dockers and containers basicsDockers and containers basics
Dockers and containers basics
 

Similaire à Docker and the Linux Kernel

Docker and kubernetes
Docker and kubernetesDocker and kubernetes
Docker and kubernetesDongwon Kim
 
Docking postgres
Docking postgresDocking postgres
Docking postgresrycamor
 
Linux: Everyting-as-a-service
Linux: Everyting-as-a-serviceLinux: Everyting-as-a-service
Linux: Everyting-as-a-serviceRohit Sansiya
 
Hands on kubernetes_container_orchestration
Hands on kubernetes_container_orchestrationHands on kubernetes_container_orchestration
Hands on kubernetes_container_orchestrationAmir Hossein Sorouri
 
Lessons Learned Running Hadoop and Spark in Docker Containers
Lessons Learned Running Hadoop and Spark in Docker ContainersLessons Learned Running Hadoop and Spark in Docker Containers
Lessons Learned Running Hadoop and Spark in Docker ContainersBlueData, Inc.
 
Docker introduction
Docker introductionDocker introduction
Docker introductionWalter Liu
 
Central Iowa Linux Users Group: November Meeting -- Container showdown
Central Iowa Linux Users Group: November Meeting -- Container showdownCentral Iowa Linux Users Group: November Meeting -- Container showdown
Central Iowa Linux Users Group: November Meeting -- Container showdownAndrew Denner
 
Gentoo Linux, or Why in the World You Should Compile Everything
Gentoo Linux, or Why in the World You Should Compile EverythingGentoo Linux, or Why in the World You Should Compile Everything
Gentoo Linux, or Why in the World You Should Compile EverythingDonnie Berkholz
 
Building community with CentOS Stream
Building community with CentOS StreamBuilding community with CentOS Stream
Building community with CentOS StreamDavide Cavalca
 
Learning of docker storage driver (container file system)
Learning of docker storage driver  (container file system)Learning of docker storage driver  (container file system)
Learning of docker storage driver (container file system)Deepak Kumar
 
Getting Started with Docker
Getting Started with DockerGetting Started with Docker
Getting Started with DockerGeeta Vinnakota
 
State of the Container Ecosystem
State of the Container EcosystemState of the Container Ecosystem
State of the Container EcosystemVinay Rao
 
December 2020 CIALUG: Local Kubernetes for Dummies-So you want to move on fro...
December 2020 CIALUG: Local Kubernetes for Dummies-So you want to move on fro...December 2020 CIALUG: Local Kubernetes for Dummies-So you want to move on fro...
December 2020 CIALUG: Local Kubernetes for Dummies-So you want to move on fro...Andrew Denner
 
Unraveling Docker Security: Lessons From a Production Cloud
Unraveling Docker Security: Lessons From a Production CloudUnraveling Docker Security: Lessons From a Production Cloud
Unraveling Docker Security: Lessons From a Production CloudSalman Baset
 
Tokyo OpenStack Summit 2015: Unraveling Docker Security
Tokyo OpenStack Summit 2015: Unraveling Docker SecurityTokyo OpenStack Summit 2015: Unraveling Docker Security
Tokyo OpenStack Summit 2015: Unraveling Docker SecurityPhil Estes
 
Lessons learned from running Spark on Docker
Lessons learned from running Spark on DockerLessons learned from running Spark on Docker
Lessons learned from running Spark on DockerDataWorks Summit
 
Java in containers
Java in containersJava in containers
Java in containersMartin Baez
 
LCNA14: Why Use Xen for Large Scale Enterprise Deployments? - Konrad Rzeszute...
LCNA14: Why Use Xen for Large Scale Enterprise Deployments? - Konrad Rzeszute...LCNA14: Why Use Xen for Large Scale Enterprise Deployments? - Konrad Rzeszute...
LCNA14: Why Use Xen for Large Scale Enterprise Deployments? - Konrad Rzeszute...The Linux Foundation
 

Similaire à Docker and the Linux Kernel (20)

Docker and kubernetes
Docker and kubernetesDocker and kubernetes
Docker and kubernetes
 
Docking postgres
Docking postgresDocking postgres
Docking postgres
 
Linux: Everyting-as-a-service
Linux: Everyting-as-a-serviceLinux: Everyting-as-a-service
Linux: Everyting-as-a-service
 
Hands on kubernetes_container_orchestration
Hands on kubernetes_container_orchestrationHands on kubernetes_container_orchestration
Hands on kubernetes_container_orchestration
 
Docker Dojo
Docker DojoDocker Dojo
Docker Dojo
 
Lessons Learned Running Hadoop and Spark in Docker Containers
Lessons Learned Running Hadoop and Spark in Docker ContainersLessons Learned Running Hadoop and Spark in Docker Containers
Lessons Learned Running Hadoop and Spark in Docker Containers
 
First steps on CentOs7
First steps on CentOs7First steps on CentOs7
First steps on CentOs7
 
Docker introduction
Docker introductionDocker introduction
Docker introduction
 
Central Iowa Linux Users Group: November Meeting -- Container showdown
Central Iowa Linux Users Group: November Meeting -- Container showdownCentral Iowa Linux Users Group: November Meeting -- Container showdown
Central Iowa Linux Users Group: November Meeting -- Container showdown
 
Gentoo Linux, or Why in the World You Should Compile Everything
Gentoo Linux, or Why in the World You Should Compile EverythingGentoo Linux, or Why in the World You Should Compile Everything
Gentoo Linux, or Why in the World You Should Compile Everything
 
Building community with CentOS Stream
Building community with CentOS StreamBuilding community with CentOS Stream
Building community with CentOS Stream
 
Learning of docker storage driver (container file system)
Learning of docker storage driver  (container file system)Learning of docker storage driver  (container file system)
Learning of docker storage driver (container file system)
 
Getting Started with Docker
Getting Started with DockerGetting Started with Docker
Getting Started with Docker
 
State of the Container Ecosystem
State of the Container EcosystemState of the Container Ecosystem
State of the Container Ecosystem
 
December 2020 CIALUG: Local Kubernetes for Dummies-So you want to move on fro...
December 2020 CIALUG: Local Kubernetes for Dummies-So you want to move on fro...December 2020 CIALUG: Local Kubernetes for Dummies-So you want to move on fro...
December 2020 CIALUG: Local Kubernetes for Dummies-So you want to move on fro...
 
Unraveling Docker Security: Lessons From a Production Cloud
Unraveling Docker Security: Lessons From a Production CloudUnraveling Docker Security: Lessons From a Production Cloud
Unraveling Docker Security: Lessons From a Production Cloud
 
Tokyo OpenStack Summit 2015: Unraveling Docker Security
Tokyo OpenStack Summit 2015: Unraveling Docker SecurityTokyo OpenStack Summit 2015: Unraveling Docker Security
Tokyo OpenStack Summit 2015: Unraveling Docker Security
 
Lessons learned from running Spark on Docker
Lessons learned from running Spark on DockerLessons learned from running Spark on Docker
Lessons learned from running Spark on Docker
 
Java in containers
Java in containersJava in containers
Java in containers
 
LCNA14: Why Use Xen for Large Scale Enterprise Deployments? - Konrad Rzeszute...
LCNA14: Why Use Xen for Large Scale Enterprise Deployments? - Konrad Rzeszute...LCNA14: Why Use Xen for Large Scale Enterprise Deployments? - Konrad Rzeszute...
LCNA14: Why Use Xen for Large Scale Enterprise Deployments? - Konrad Rzeszute...
 

Plus de Docker, Inc.

Containerize Your Game Server for the Best Multiplayer Experience
Containerize Your Game Server for the Best Multiplayer Experience Containerize Your Game Server for the Best Multiplayer Experience
Containerize Your Game Server for the Best Multiplayer Experience Docker, Inc.
 
How to Improve Your Image Builds Using Advance Docker Build
How to Improve Your Image Builds Using Advance Docker BuildHow to Improve Your Image Builds Using Advance Docker Build
How to Improve Your Image Builds Using Advance Docker BuildDocker, Inc.
 
Build & Deploy Multi-Container Applications to AWS
Build & Deploy Multi-Container Applications to AWSBuild & Deploy Multi-Container Applications to AWS
Build & Deploy Multi-Container Applications to AWSDocker, Inc.
 
Securing Your Containerized Applications with NGINX
Securing Your Containerized Applications with NGINXSecuring Your Containerized Applications with NGINX
Securing Your Containerized Applications with NGINXDocker, Inc.
 
How To Build and Run Node Apps with Docker and Compose
How To Build and Run Node Apps with Docker and ComposeHow To Build and Run Node Apps with Docker and Compose
How To Build and Run Node Apps with Docker and ComposeDocker, Inc.
 
Distributed Deep Learning with Docker at Salesforce
Distributed Deep Learning with Docker at SalesforceDistributed Deep Learning with Docker at Salesforce
Distributed Deep Learning with Docker at SalesforceDocker, Inc.
 
The First 10M Pulls: Building The Official Curl Image for Docker Hub
The First 10M Pulls: Building The Official Curl Image for Docker HubThe First 10M Pulls: Building The Official Curl Image for Docker Hub
The First 10M Pulls: Building The Official Curl Image for Docker HubDocker, Inc.
 
Monitoring in a Microservices World
Monitoring in a Microservices WorldMonitoring in a Microservices World
Monitoring in a Microservices WorldDocker, Inc.
 
COVID-19 in Italy: How Docker is Helping the Biggest Italian IT Company Conti...
COVID-19 in Italy: How Docker is Helping the Biggest Italian IT Company Conti...COVID-19 in Italy: How Docker is Helping the Biggest Italian IT Company Conti...
COVID-19 in Italy: How Docker is Helping the Biggest Italian IT Company Conti...Docker, Inc.
 
Predicting Space Weather with Docker
Predicting Space Weather with DockerPredicting Space Weather with Docker
Predicting Space Weather with DockerDocker, Inc.
 
Become a Docker Power User With Microsoft Visual Studio Code
Become a Docker Power User With Microsoft Visual Studio CodeBecome a Docker Power User With Microsoft Visual Studio Code
Become a Docker Power User With Microsoft Visual Studio CodeDocker, Inc.
 
How to Use Mirroring and Caching to Optimize your Container Registry
How to Use Mirroring and Caching to Optimize your Container RegistryHow to Use Mirroring and Caching to Optimize your Container Registry
How to Use Mirroring and Caching to Optimize your Container RegistryDocker, Inc.
 
Monolithic to Microservices + Docker = SDLC on Steroids!
Monolithic to Microservices + Docker = SDLC on Steroids!Monolithic to Microservices + Docker = SDLC on Steroids!
Monolithic to Microservices + Docker = SDLC on Steroids!Docker, Inc.
 
Kubernetes at Datadog Scale
Kubernetes at Datadog ScaleKubernetes at Datadog Scale
Kubernetes at Datadog ScaleDocker, Inc.
 
Labels, Labels, Labels
Labels, Labels, Labels Labels, Labels, Labels
Labels, Labels, Labels Docker, Inc.
 
Using Docker Hub at Scale to Support Micro Focus' Delivery and Deployment Model
Using Docker Hub at Scale to Support Micro Focus' Delivery and Deployment ModelUsing Docker Hub at Scale to Support Micro Focus' Delivery and Deployment Model
Using Docker Hub at Scale to Support Micro Focus' Delivery and Deployment ModelDocker, Inc.
 
Build & Deploy Multi-Container Applications to AWS
Build & Deploy Multi-Container Applications to AWSBuild & Deploy Multi-Container Applications to AWS
Build & Deploy Multi-Container Applications to AWSDocker, Inc.
 
From Fortran on the Desktop to Kubernetes in the Cloud: A Windows Migration S...
From Fortran on the Desktop to Kubernetes in the Cloud: A Windows Migration S...From Fortran on the Desktop to Kubernetes in the Cloud: A Windows Migration S...
From Fortran on the Desktop to Kubernetes in the Cloud: A Windows Migration S...Docker, Inc.
 
Developing with Docker for the Arm Architecture
Developing with Docker for the Arm ArchitectureDeveloping with Docker for the Arm Architecture
Developing with Docker for the Arm ArchitectureDocker, Inc.
 

Plus de Docker, Inc. (20)

Containerize Your Game Server for the Best Multiplayer Experience
Containerize Your Game Server for the Best Multiplayer Experience Containerize Your Game Server for the Best Multiplayer Experience
Containerize Your Game Server for the Best Multiplayer Experience
 
How to Improve Your Image Builds Using Advance Docker Build
How to Improve Your Image Builds Using Advance Docker BuildHow to Improve Your Image Builds Using Advance Docker Build
How to Improve Your Image Builds Using Advance Docker Build
 
Build & Deploy Multi-Container Applications to AWS
Build & Deploy Multi-Container Applications to AWSBuild & Deploy Multi-Container Applications to AWS
Build & Deploy Multi-Container Applications to AWS
 
Securing Your Containerized Applications with NGINX
Securing Your Containerized Applications with NGINXSecuring Your Containerized Applications with NGINX
Securing Your Containerized Applications with NGINX
 
How To Build and Run Node Apps with Docker and Compose
How To Build and Run Node Apps with Docker and ComposeHow To Build and Run Node Apps with Docker and Compose
How To Build and Run Node Apps with Docker and Compose
 
Hands-on Helm
Hands-on Helm Hands-on Helm
Hands-on Helm
 
Distributed Deep Learning with Docker at Salesforce
Distributed Deep Learning with Docker at SalesforceDistributed Deep Learning with Docker at Salesforce
Distributed Deep Learning with Docker at Salesforce
 
The First 10M Pulls: Building The Official Curl Image for Docker Hub
The First 10M Pulls: Building The Official Curl Image for Docker HubThe First 10M Pulls: Building The Official Curl Image for Docker Hub
The First 10M Pulls: Building The Official Curl Image for Docker Hub
 
Monitoring in a Microservices World
Monitoring in a Microservices WorldMonitoring in a Microservices World
Monitoring in a Microservices World
 
COVID-19 in Italy: How Docker is Helping the Biggest Italian IT Company Conti...
COVID-19 in Italy: How Docker is Helping the Biggest Italian IT Company Conti...COVID-19 in Italy: How Docker is Helping the Biggest Italian IT Company Conti...
COVID-19 in Italy: How Docker is Helping the Biggest Italian IT Company Conti...
 
Predicting Space Weather with Docker
Predicting Space Weather with DockerPredicting Space Weather with Docker
Predicting Space Weather with Docker
 
Become a Docker Power User With Microsoft Visual Studio Code
Become a Docker Power User With Microsoft Visual Studio CodeBecome a Docker Power User With Microsoft Visual Studio Code
Become a Docker Power User With Microsoft Visual Studio Code
 
How to Use Mirroring and Caching to Optimize your Container Registry
How to Use Mirroring and Caching to Optimize your Container RegistryHow to Use Mirroring and Caching to Optimize your Container Registry
How to Use Mirroring and Caching to Optimize your Container Registry
 
Monolithic to Microservices + Docker = SDLC on Steroids!
Monolithic to Microservices + Docker = SDLC on Steroids!Monolithic to Microservices + Docker = SDLC on Steroids!
Monolithic to Microservices + Docker = SDLC on Steroids!
 
Kubernetes at Datadog Scale
Kubernetes at Datadog ScaleKubernetes at Datadog Scale
Kubernetes at Datadog Scale
 
Labels, Labels, Labels
Labels, Labels, Labels Labels, Labels, Labels
Labels, Labels, Labels
 
Using Docker Hub at Scale to Support Micro Focus' Delivery and Deployment Model
Using Docker Hub at Scale to Support Micro Focus' Delivery and Deployment ModelUsing Docker Hub at Scale to Support Micro Focus' Delivery and Deployment Model
Using Docker Hub at Scale to Support Micro Focus' Delivery and Deployment Model
 
Build & Deploy Multi-Container Applications to AWS
Build & Deploy Multi-Container Applications to AWSBuild & Deploy Multi-Container Applications to AWS
Build & Deploy Multi-Container Applications to AWS
 
From Fortran on the Desktop to Kubernetes in the Cloud: A Windows Migration S...
From Fortran on the Desktop to Kubernetes in the Cloud: A Windows Migration S...From Fortran on the Desktop to Kubernetes in the Cloud: A Windows Migration S...
From Fortran on the Desktop to Kubernetes in the Cloud: A Windows Migration S...
 
Developing with Docker for the Arm Architecture
Developing with Docker for the Arm ArchitectureDeveloping with Docker for the Arm Architecture
Developing with Docker for the Arm Architecture
 

Dernier

UiPath Studio Web workshop series - Day 8
UiPath Studio Web workshop series - Day 8UiPath Studio Web workshop series - Day 8
UiPath Studio Web workshop series - Day 8DianaGray10
 
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...DianaGray10
 
VoIP Service and Marketing using Odoo and Asterisk PBX
VoIP Service and Marketing using Odoo and Asterisk PBXVoIP Service and Marketing using Odoo and Asterisk PBX
VoIP Service and Marketing using Odoo and Asterisk PBXTarek Kalaji
 
Videogame localization & technology_ how to enhance the power of translation.pdf
Videogame localization & technology_ how to enhance the power of translation.pdfVideogame localization & technology_ how to enhance the power of translation.pdf
Videogame localization & technology_ how to enhance the power of translation.pdfinfogdgmi
 
Salesforce Miami User Group Event - 1st Quarter 2024
Salesforce Miami User Group Event - 1st Quarter 2024Salesforce Miami User Group Event - 1st Quarter 2024
Salesforce Miami User Group Event - 1st Quarter 2024SkyPlanner
 
Comparing Sidecar-less Service Mesh from Cilium and Istio
Comparing Sidecar-less Service Mesh from Cilium and IstioComparing Sidecar-less Service Mesh from Cilium and Istio
Comparing Sidecar-less Service Mesh from Cilium and IstioChristian Posta
 
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...Will Schroeder
 
20230202 - Introduction to tis-py
20230202 - Introduction to tis-py20230202 - Introduction to tis-py
20230202 - Introduction to tis-pyJamie (Taka) Wang
 
Secure your environment with UiPath and CyberArk technologies - Session 1
Secure your environment with UiPath and CyberArk technologies - Session 1Secure your environment with UiPath and CyberArk technologies - Session 1
Secure your environment with UiPath and CyberArk technologies - Session 1DianaGray10
 
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdf
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdfIaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdf
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdfDaniel Santiago Silva Capera
 
9 Steps For Building Winning Founding Team
9 Steps For Building Winning Founding Team9 Steps For Building Winning Founding Team
9 Steps For Building Winning Founding TeamAdam Moalla
 
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...Aggregage
 
COMPUTER 10 Lesson 8 - Building a Website
COMPUTER 10 Lesson 8 - Building a WebsiteCOMPUTER 10 Lesson 8 - Building a Website
COMPUTER 10 Lesson 8 - Building a Websitedgelyza
 
Building AI-Driven Apps Using Semantic Kernel.pptx
Building AI-Driven Apps Using Semantic Kernel.pptxBuilding AI-Driven Apps Using Semantic Kernel.pptx
Building AI-Driven Apps Using Semantic Kernel.pptxUdaiappa Ramachandran
 
Introduction to Matsuo Laboratory (ENG).pptx
Introduction to Matsuo Laboratory (ENG).pptxIntroduction to Matsuo Laboratory (ENG).pptx
Introduction to Matsuo Laboratory (ENG).pptxMatsuo Lab
 
Building Your Own AI Instance (TBLC AI )
Building Your Own AI Instance (TBLC AI )Building Your Own AI Instance (TBLC AI )
Building Your Own AI Instance (TBLC AI )Brian Pichman
 
Cybersecurity Workshop #1.pptx
Cybersecurity Workshop #1.pptxCybersecurity Workshop #1.pptx
Cybersecurity Workshop #1.pptxGDSC PJATK
 
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdfUiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdfDianaGray10
 
Crea il tuo assistente AI con lo Stregatto (open source python framework)
Crea il tuo assistente AI con lo Stregatto (open source python framework)Crea il tuo assistente AI con lo Stregatto (open source python framework)
Crea il tuo assistente AI con lo Stregatto (open source python framework)Commit University
 

Dernier (20)

UiPath Studio Web workshop series - Day 8
UiPath Studio Web workshop series - Day 8UiPath Studio Web workshop series - Day 8
UiPath Studio Web workshop series - Day 8
 
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...
 
VoIP Service and Marketing using Odoo and Asterisk PBX
VoIP Service and Marketing using Odoo and Asterisk PBXVoIP Service and Marketing using Odoo and Asterisk PBX
VoIP Service and Marketing using Odoo and Asterisk PBX
 
Videogame localization & technology_ how to enhance the power of translation.pdf
Videogame localization & technology_ how to enhance the power of translation.pdfVideogame localization & technology_ how to enhance the power of translation.pdf
Videogame localization & technology_ how to enhance the power of translation.pdf
 
Salesforce Miami User Group Event - 1st Quarter 2024
Salesforce Miami User Group Event - 1st Quarter 2024Salesforce Miami User Group Event - 1st Quarter 2024
Salesforce Miami User Group Event - 1st Quarter 2024
 
Comparing Sidecar-less Service Mesh from Cilium and Istio
Comparing Sidecar-less Service Mesh from Cilium and IstioComparing Sidecar-less Service Mesh from Cilium and Istio
Comparing Sidecar-less Service Mesh from Cilium and Istio
 
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...
 
20230202 - Introduction to tis-py
20230202 - Introduction to tis-py20230202 - Introduction to tis-py
20230202 - Introduction to tis-py
 
Secure your environment with UiPath and CyberArk technologies - Session 1
Secure your environment with UiPath and CyberArk technologies - Session 1Secure your environment with UiPath and CyberArk technologies - Session 1
Secure your environment with UiPath and CyberArk technologies - Session 1
 
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdf
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdfIaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdf
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdf
 
9 Steps For Building Winning Founding Team
9 Steps For Building Winning Founding Team9 Steps For Building Winning Founding Team
9 Steps For Building Winning Founding Team
 
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...
 
201610817 - edge part1
201610817 - edge part1201610817 - edge part1
201610817 - edge part1
 
COMPUTER 10 Lesson 8 - Building a Website
COMPUTER 10 Lesson 8 - Building a WebsiteCOMPUTER 10 Lesson 8 - Building a Website
COMPUTER 10 Lesson 8 - Building a Website
 
Building AI-Driven Apps Using Semantic Kernel.pptx
Building AI-Driven Apps Using Semantic Kernel.pptxBuilding AI-Driven Apps Using Semantic Kernel.pptx
Building AI-Driven Apps Using Semantic Kernel.pptx
 
Introduction to Matsuo Laboratory (ENG).pptx
Introduction to Matsuo Laboratory (ENG).pptxIntroduction to Matsuo Laboratory (ENG).pptx
Introduction to Matsuo Laboratory (ENG).pptx
 
Building Your Own AI Instance (TBLC AI )
Building Your Own AI Instance (TBLC AI )Building Your Own AI Instance (TBLC AI )
Building Your Own AI Instance (TBLC AI )
 
Cybersecurity Workshop #1.pptx
Cybersecurity Workshop #1.pptxCybersecurity Workshop #1.pptx
Cybersecurity Workshop #1.pptx
 
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdfUiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
 
Crea il tuo assistente AI con lo Stregatto (open source python framework)
Crea il tuo assistente AI con lo Stregatto (open source python framework)Crea il tuo assistente AI con lo Stregatto (open source python framework)
Crea il tuo assistente AI con lo Stregatto (open source python framework)
 

Docker and the Linux Kernel

  • 1. Linux Plumbers Conference 2014, Düsseldorf Docker and the Linux kernel Cristian S., Docker Inc. cristian@docker.com
  • 5. Solution: the intermodal shipping container
  • 7. Solution to the deployment problem: the Linux container
  • 9. High level overview • Uses namespaces & cgroups • Runs on mainline kernels • Lower overhead than VMs • Can run a full system with an init or a single service/process • Snapshot based approach to build one image on top of another
  • 10. Docker's code • Licensed under the Apache 2 license • There's no paid/premium/commercial version • Docker and its code are free and will be free • Lives at: https://github.com/docker/docker/
  • 11. Use cases • Application development, testing, packaging & deployment • PaaS/SaaS/cloud infrastructure • Application & service isolation • Stress testing & benchmarking (including the Linux kernel)
  • 12. Under the hood • Exec drivers provide the execution environment (virtualization/container tech) – native (libcontainer based) and LXC – are platform dependent – native is the default • Graph drivers are the storage providers – aufs, devicemapper, btrfs and vfs – vfs shouldn't be used (only used by the tests by default) – PRs open on GitHub for ZFS and OverlayFS • Existing graph & exec drivers only support Linux
  • 13. Kernel requirements • Kernel 3.8 is the absolute minimum (except RHEL's 2.6.32) • Stable & supported kernels >= 3.10 are recommended • BTRFS has special requirements • Kernel configuration can be checked using https://raw.githubusercontent.com/docker/ docker/master/contrib/check-config.sh
  • 14. AUFS graph driver stability & performance • Operations with containers are faster than devicemapper & (sometimes) btrfs • Known problems – Stale NFS file handle, cap_set_file, invalid argument on mount – Causes troubles on btrfs, remote file systems and many file systems which aren't ext3/ext4 – Direct IO problems & poor performance for IO intensive workloads • Limitations – Requires aufs-tools for auplink to dereference hard links – No support for hard links across layers – Limit of 127 layers – Can't be used on Fedora/RHEL/all systems which don't apply AUFS patches and don't ship aufs-tools • AUFS is developed outside of the mainline kernel tree • Update the kernel using distro updates
  • 15. Devicemapper graph driver stability & performance • Uses loopback mounted block devices by default • Allows EXT4 or XFS to be used • Known problems – EBUSY errors (fix to be tested) – Potential file system corruption bug with ext4 ● Might be caused by loopback mounted block devices – Older kernels: space not freed when freed on the file system, various kernel errors and problems • Limitations – Devicemapper requires configuration of the storage to avoid the use of loopback mounted block devices • Update the kernel using distro updates
  • 16. btrfs graph driver stability & performance • BTRFS bugs are a problem for Docker • Using the RAID like features of BTRFS is likely to cause data loss • Known problems – Data can be corrupted if exotic mount options are used – Kernel 3.8 & older than the latest stable kernels can cause data loss & corruption – The file system becomes slower the more data is written & stored on it – Performance degrades quickly & fragmentation is a problem – Balancing the file system to fix fragmentation could trigger some bugs • BTRFS is used automatically if Docker's root folder is on BTRFS • Update the kernel using distro updates • Using the latest minor version of supported kernels is recommended
  • 17. Performance & stability • Generally speaking, the kernel has been stable – Fixes have been made to file systems, namespaces, cgroups, netfilter, aufs, btrfs, devicemapper and other kernel components – Recent kernels and the newest minor versions of LTS kernels have fixed a lot of problems, including devicemapper and btrfs problems – Some PID 1 issues are still being discussed – Changes around the cgroups and the namespaces have been made; this has also helped improve stability for containers • Performance is actively being studied – Docker's code has been improved to use less memory & be faster – Performance needs to be studied on the kernel side to achieve better scalability by a) making fewer syscalls in Docker where possible, b) making those syscalls faster in the kernel
  • 18. Kernel recommendations • Avoid running kernels no longer supported by your Linux distribution (kernel 3.8 lts-raring from Ubuntu 12.04.x) • Open Docker issues on github; post errors (full kernel panics, btrfs check output, Docker daemon logs, `docker info`, `docker version` and `uname -a` output) when encountering errors • File bug reports with your distribution
  • 19. What can Docker do for the kernel? • Can be used to test the kernel's stability and evaluate its performance • Changes made to the kernel can be tested easily against Docker • Can be used to test the running kernel to avoid breaking user space compatibility • Makes it simple to mix workloads for stress testing and performance testing • Exposes some hard to trigger kernel problems
  • 20. What can a kernel developer do with Docker? • Find bottlenecks in the kernel's code • Run a KVM VM in a container • Stress testing for file systems, network, namespaces and the kernel in general • Large scale testing with containers • Hardware testing in containers • Network testing with advanced topologies
  • 21. How can someone contribute to Docker? What about kernel developers? • You don't necessarily have to write code • Providing feedback, doing code review or telling the right kernel developer to do that are also useful helpful • Suggestions on how to debug specific kernel bugs and bugs in general are welcome • Contributing to Docker is also helping the kernel because Docker relies on the kernel
  • 22. Thank you! Questions? http://docker.com @docker