Operations practices have historically lagged behind development. Agile and Extreme Programming have become common practice for development teams. In the last decade, the DevOps and SRE movements have brought these concepts to operations, borrowing heavily from Lean principles such as Kanban and Value Stream Mapping. So, how does all of this play out if we’re using Kubernetes? In this class, Paul Czarkowski, Principal Technologist at Pivotal, will explain how Kubernetes enables a new cloud-native way of operating software. Attend to learn: ● what cloud-native operations are; ● how to build a cloud-native CI/CD stack; and ● how to deploy and upgrade an application from source to production on Kubernetes. Presenter: Paul Czarkowski, Principal Technologist, Pivotal Software

1. © Copyright 2018 Pivotal Software, Inc. All rights Reserved.
Paul Czarkowski
@pczarkowski
pczarkowski@pivotal.io
Cloud Native Operations
With PKS (Kubernetes)

2. © Copyright 2018 Pivotal Software, Inc. All rights Reserved.
Paul Czarkowski
@pczarkowski
pczarkowski@pivotal.io
Cloud Native Operations
With PKS (Kubernetes)

3. Operations is the Secret sauce

4. Operations as a Competitive Advantage

5. Just not Traditional Operations.

6. https://go.forrester.com/blogs/ceos-and-software/

7. The Agile/DevOps Journey

8. BUSINESS DEVELOPMENT TEST / QA Operations
Months Months Months

9. BUSINESS DEVELOPMENT TEST / QA Operations
Weeks Months Months

10. BUSINESS DEVELOPMENT
TEST / QA
Operations
Weeks Months

11. BUSINESS DEVELOPMENT
TEST / QA
Operations
Weeks Months

14. BUSINESS DEVELOPMENT
TEST / QA
Operations
Weeks

15. https://www.sketchplanations.com/post/175174474143/conways-law-im-paraphrasing-here-but-the-gist

16. Hack Conway’s Law

17. CHANGE

18. API
Users
Systems
Admin
Network
Engineer
SecurityDBA QA
Storage
Admin

19. API
Users
Storage Compute NetworkDatabase AccessArtifacts
Creative Commons
[1] Jon Trillana
[2] Simon Child
1 2

21. Operations as a Platform

22. Platform as a Product

23. Infra
Services
App
Platform
Platform
Team
Application
Team
Build common services
for App Teams
Take business
requirements and turn
them into features
IaaS
Virtual Infrastructure
Physical Infrastructure
Abstract infrastructure
complexity with easy
consumption
DBaaSELK
App2App1 App3
Middleware
ML
Creds/CertsMessaging
???
Container Services
Container Hosts | Kubernetes
Infrastructure
Team

24. Observability
Orchestration
and scheduling
CI / CD
Access
Control

27. APP
APP
APP
APP
Production

28. This is Cloud Native Operations

29. Pivotal is a Platform Company

30. Embedded OS
(Windows & Linux)
NSX-T
CPI (15 methods)
v1
v2
v3
...
CVEs
Product Updates
Java | .NET | NodeJS
Pivotal Application
Service (PAS)
Application Code & Frameworks
Buildpacks | Spring Boot | Spring Cloud |
Steeltoe
Elastic | Packaged Software | Spark
Pivotal Container
Service (PKS)
>cf push >kubectl run
YOU build the containerWE build the container
vSphere
Azure &
Azure StackGoogle CloudAWSOpenstack
Pivotal
Network
“3Rs”
Github
Concourse
Concourse
Pivotal Services
Marketplace
Pivotal and
Partner Products
Continuous
delivery
Public Cloud
Services
Customer
Managed
Services
OpenServiceBrokerAPI
Repair
— CVEs
Repave Rotate
— Credhub

31. > kubectl
Storage NetworkingCompute
Kubernetes Dashboard
Dev / Apps IT / Ops
App User
Kubernetes is a Runtime for Containerized Workloads

32. Storage NetworkingCompute
Dev / Apps
App User
IT / Ops
> kubectl
Kubernetes Dashboard
Load Balancing / Routing
Container Image
Registry
App Monitoring
App Logging
OS Updates
OS Images
K8S Updates
K8S Images
Log & Monitor
Recover & Restart
Backup & Restore
External
Data Services
Cluster
Provisioning
Provision & Scale
Command Line
/ API
Management
GUI
Monitoring GUI
...but Kubernetes alone is not enough for enterprises

33. Storage NetworkingCompute
Pivotal Container Service (PKS) provides what’s missing
Dev / Apps
App User
IT / Ops
> kubectl
Kubernetes Dashboard
Load Balancing / Routing
Container Image
Registry
OS Updates
OS Images
K8S Updates
K8S Images
Log & Monitor
Recover & Restart
Backup & Restore
External
Data Services
Cluster
Provisioning
Provision & Scale
App Logging
PKS Control Plane
> pks
Operations Manager
vRealize Operations*
*integration
GCP Service Broker

34. Storage NetworkingCompute
Dev / Apps
App User
IT / Ops
> kubectl
Kubernetes Dashboard
Load Balancing / Routing
Container Image
Registry
K8S Updates Log & Monitor Backup & Restore
External
Data Services
Cluster
Provisioning
App Logging
PKS Control Plane
GCP Service Broker
> pks
Operations Manager
vRealize Operations*
*integration
on any Cloud

35. What PKS adds to Kubernetes
PKS
value-added
features
Built into
Kubernetes
Multi-container pods
Stateful Sets of pods
Persistent disks
Single tenant ingress
Pod scaling and high availability
Rolling upgrades to pods
Cluster provisioning and scaling
Embedded, hardened Operating System
Monitoring and recovery of cluster VMs and processes
Rolling upgrades to cluster infrastructure
Secure multi-tenant ingress
Secure container registry

36. PKS Vision
To provide enterprise customers with the ability to
safely and efficiently deliver container services
on their preferred infrastructure so that they can
excel in their market with a cloud native platform

37. PKS does for your Kubernetes
what
Kubernetes does for your apps

38. BOSH
Reliable and consistent operational experience for any cloud.
BOSH
Harbor
NSX-T
Kubernetes
K8s Cluster
K8s Cluster
K8s Cluster
PKS Control Plane
Use the PKS CLI and API to
create, operate, and scale your
clusters.
VMware GCP Azure Openstack AWS
PKSControlPlane
Built with open-source
Kubernetes
Constant compatibility with the
latest stable release of Google
Kubernetes Engine—no
proprietary extensions.
Harbor
An enterprise-class container registry.
Includes vulnerability scanning, identity
management, and more.
NSX-T
Network management, security, and
load balancing out-of-the-box with
VMware NSX-T. Multi-cloud,
multi-hypervisor.
Enterprise-Grade Kubernetes

39. Kubernetes

40. Controllers

41. Controllers make Kubernetes great.

42. Desired State
Actual State

43. Unix Philosophy:
Do one thing. Do it well.

44. A Kubernetes pod is fragile.

45. The ReplicaSet Controller
Adds resiliency to your Pods.

46. The Deployment Controller
Adds upgradability to your ReplicaSets.

47. You can write custom controllers
very easily.

48. Kubernetes is an API
for Operations

49. DNS A record please
Hi, we need a DNS record for bacon.example.com
To point to IP 22.33.44.55. Thanks.

50. external-dns controller
https://github.com/kubernetes-incubator/external-dns

52. cert-manager controller
(/operator)
https://github.com/jetstack/cert-manager

53. Operators extend Kubernetes
to support custom resources

54. https://github.com/operator-framework/awesome-operators

55. Controllers and Operators turn Kubernetes
Into an API for Operations

56. Let’s run Kubernetes

58. “Let somebody else
Do the hard stuff” - Me

59. One Big Cluster or Many Smaller Clusters
One [or two] Big Cluster[s]
● All teams co-located on cluster, “namespaces”
separate them.
● Higher chance of noisy neighbor, other
multi-tenancy issues.
● Better utilization of resources (less servers,
higher bin-packing)
● Cluster Upgrades affect everyone.
● Large Blast radius during cluster issues /
outages.
● Monolithic approach to infrastructure
Many Smaller Clusters
● Each “team” or “business unit” gets its own
cluster.
● Dedicated resources to each cluster reduces
noisy neighbor etc.
● Resources can be customized at the cluster for
the specific use cases.
● Cluster Upgrades only affect one team, easier
to coordinate.
● Cluster issues/outages restricted to one team.
● Microservices approach to Infrastructure

60. One Big Cluster or Many Smaller Clusters
One [or two] Big Cluster[s]
● Pivotal Container Service - Essentials
● Kops
● Some Kubernetes distro from another vendor
● Kubespray
● DIY chef/puppet/ansible
Many Smaller Clusters
● Pivotal Container Service - Enterprise
● Pivotal Container Service - Cloud
● Google Container Engine
● Elastic Container Service
● Azure Container Service

61. https://k8s.af

62. Start with the idea of doing
many smaller clusters.

63. One Big Cluster or Many Smaller Clusters
One [or two] Big Cluster[s] Many Smaller Clusters

64. APP
APP
APP
APP
Production

65. APP
APP
APP
APP

66. APP
APP
APP
APP

67. APP
APP
APP
APP

68. APP
APP
APP
APP
Gitlab

69. APP
APP
APP
APP
Gitlab Concourse

70. APP
APP
APP
APP
Gitlab Concourse Spinnaker

71. APP
APP
APP
APP
Gitlab Concourse Spinnaker

72. APP
APP
APP
APP
Gitlab Concourse Spinnaker

73. APP
APP
APP
APP
Gitlab Concourse Spinnaker

74. A
P
P
A
P
P
A
P
P
A
P
P
Gitl
ab
Con
cou
rse
Spi
nna
ker
A
P
P
A
P
P
A
P
P
A
P
P
Gitl
ab
Con
cou
rse
Spi
nna
ker
A
P
P
A
P
P
A
P
P
A
P
P
Gitl
ab
Con
cou
rse
Spi
nna
ker
A
P
P
A
P
P
A
P
P
A
P
P
Gitl
ab
Con
cou
rse
Spi
nna
ker

75. We’ve spent 10 years figuring
out how to do agile operations

77. Helm Chart
Repo Kube Env
#1 Repo
Kube Env
#2 Repo
A
P
P
A
P
P
A
P
P
A
P
P
Gitl
ab
Con
cou
rse
Spi
nna
ker
A
P
P
A
P
P
A
P
P
A
P
P
Gitl
ab
Con
cou
rse
Spi
nna
ker
Platform Operations

78. App Source
Code App Env
Repo #2
App Env
Repo #1
Developer Workflow
Concourse Spinnaker

79. DEMO TIME

80. Transforming How The World Builds Software
© Copyright 2019 Pivotal Software, Inc. All rights Reserved.