Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
DevOps 개발자가 되기 위한 쿠버
네티스 핵심 활용 예제 알아보기
정영준
솔루션즈 아키텍트
AWS
이상호
선임
삼성전자 Health서비스팀
Agenda
• Container Service Overview
• EKS Tenets and Core Concept
• Samsung Health Use Case
• EKS Best Practices & Sample ...
We’re making AWS the best place to run contain
ers and Kubernetes
AWS container services landscape
Management
Deployment, Scheduling,
Scaling & Management of
containerized applications
Hos...
Balancing flexibility and simplicity:
Workload-by-workload
Flexibility focused
Low level of opinion
Low level of abstracti...
We give you the power to choose
Amazon ECS Amazon EKS
Amazon
EC2
AWS
Fargate
Amazon
EC2
AWS
Fargate
1. Choose your
orchest...
EKS tenets
1. Amazon EKS is a platform to run production-grade workloads. Security and reliability are
our first priority....
How are customer using Amazon EKS?
Microservices
PaaS
Platform as a service Enterprise App
Migration
Machine Learning
[mycluster].eks.amazonaws.com
Availability
Zone 1
Availability
Zone 2
Availability
Zone 3
Kubectl
Your AWS account
VPC
Kubernetes control plane
Highly available and single
tenant infrastructure
All “native AWS” components
Fronted by a Ne...
Kubernetes architecture
etcd
eksctl - a CLI for Amazon EKS
• Single command cluster creation
eksctl create cluster --nodes=4
• Open source and on GitHu...
Standard EC2 compute instance types
P and G type accelerated instances
i3 bare metal
Spot Instances
Bring your own instanc...
Bring your own OS
Amazon EKS AMI build scripts
https://github.com/awslabs/amazon-eks-ami
Amazon
Amazon
Amazon
Windows containers
Run Windows containers and Windows Server nodes with Amazon EKS
Supports heterogeneous (mixed) clusters...
Provisioning worker nodes
AWS CloudFormation eksctl Partners
…more
Terraform
Pulumi
Rancher
Now supporting P3dn.24xlarge instances
CUDA 10 with NVIDIA v410 coming soon!
Amazon EKS-optimized GPU AMI
Support Kubernetes version
Latest Kubernetes: 1.14
Amazon EKS will support up to three versions of Kubernetes at once
Depr...
Amazon EKS platform version
Platform version revisions represent API server configuration
changes or Kubernetes patches
Pl...
Amazon EKS update life cycle
May 21, 2019
Blog: https://aws.amazon.com/blogs/compute/updates-to-amazon-eks-version-lifecyc...
Amazon EKS support sophisticated and scalable infrastructure
[mycluster].eks.amazonaws.com
Availability
Zone 1
Availabilit...
Copyright © 2019 Samsung Electronics, Co., Ltd. All rights reserved. 25
Lee Sang Ho, 삼성전자
AWS EKS on Samsung Health Server
Copyright © 2019 Samsung Electronics, Co., Ltd. All rights reserved. 26
Contents
1.Why Samsung Health choose AWS EKS
2.AWS...
Copyright © 2019 Samsung Electronics, Co., Ltd. All rights reserved. 27
Who am I
 Lee Sang Ho
 삼성전자, 무선 사업부, Health Serv...
Copyright © 2019 Samsung Electronics, Co., Ltd. All rights reserved. 28
Why Samsung Health choose AWS EKS
 Inefficiency o...
Copyright © 2019 Samsung Electronics, Co., Ltd. All rights reserved. 29
Migration with Kubernetes
Internal
NLB-1
Internal
...
Copyright © 2019 Samsung Electronics, Co., Ltd. All rights reserved. 30
What else
 Kube2Iam
 HPA
 Cluster Autoscaler
te...
Copyright © 2019 Samsung Electronics, Co., Ltd. All rights reserved. 31
PrometheusPrometheus
CloudwatchCloudwatch
In-House...
Copyright © 2019 Samsung Electronics, Co., Ltd. All rights reserved. 32
Log Monitoring
 EFK + AWS Cloudwatch
Cloudwatch
(...
Reverse Proxy with ES
 in case of vpc access
NGINXNGINX
KIBANAKIBANA
COGNITOCOGNITO
② if no authentication then 302 redir...
Copyright © 2019 Samsung Electronics, Co., Ltd. All rights reserved. 34
AWS Xray
 APM tool
 Analyze and debug production...
Copyright © 2019 Samsung Electronics, Co., Ltd. All rights reserved. 35
Spinnaker
 Continuous Delivery Platform
 Open so...
Copyright © 2019 Samsung Electronics, Co., Ltd. All rights reserved. 36
AWS ECR
Overall architecture
Node-01Node-01
Pod-01...
Copyright © 2019 Samsung Electronics, Co., Ltd. All rights reserved. 37
 When deploy
 Before
 Now
What’s better? #1
Jen...
Copyright © 2019 Samsung Electronics, Co., Ltd. All rights reserved. 38
 When operation
 Self-healing, HPA (Horizontal P...
Copyright © 2019 Samsung Electronics, Co., Ltd. All rights reserved. 39
Little things that need to be improved
 NLB Multi...
Challenges in Using & Deploying Containers
As cloud native technologies change the way companies are designing an
d buildi...
Container security onion model: Defense in depth
• full blown distro (Ubuntu, AL) vs. minimal
environment (container-optim...
Security tooling
Amazon
Inspector
AWS KMS AWS
Secrets Manager
AWS WAF
AWS IAM
Amazon
GuardDuty
Amazon
Macie
AWS
Security H...
PKI configuration
Kubelet
Generates
public/private keys
Kubelet installs
server cert
Kubelet issues CSR
Certificate rotati...
API-server endpoint access control
Worker VPC (your account)
Kubectl
Master VPC (AWS account)
etcd
AZ 1
API Server
etcd
AP...
API-server endpoint access control
Worker VPC (your account)
Kubectl
Master VPC (AWS account)
etcd
AZ 1
AZ 2
API Server
et...
API-server endpoint access control
Worker VPC (your account)
Kubectl
Master VPC (AWS account)
etcd
AZ 1 AZ 2
API Server
et...
AWS Identity and Access Management (IAM) Aut
hentication
Kubectl
3) Authorizes AWS identity with RBAC
K8s API
1) Passes AW...
Great integration responsibility – IAM roles
Frontend pod
Node
Backend pod
Amazon S3
UI customization
bucket
Billing repor...
Great integration responsibility – IAM roles
Frontend pod
Node
Backend pod
Amazon S3
UI customization
bucket
Billing repor...
Service type LoadBalancer
Service
(LoadBalancer)
ELB Users
Pod selector
(app = frontend)
Frontend pod 1
Node
NodePort
Fron...
Service type LoadBalancer
Service
(LoadBalancer)
ELB Users
Pod selector
(type = nodejs)
Frontend pod 1
Node
NodePort
Front...
Great responsibility – Service type LoadBalancer
Service
(LoadBalancer)
ELB Users
Pod selector
(app = frontend)
Frontend p...
Amazon CloudWatch Container Insights
Gives you complete visibility into your cloud resources and applications
so you can m...
 Collects, aggregates, and summarizes
 Reliable, secure metrics and logs collection
 Automated dashboards and analysis
...
Images on DockerHub
Performance Metrics – CloudWatch Agent: https://hub.docker.com/r/
amazon/cloudwatch-agent
• Tag: lates...
Container Insights available now
1. Fully managed, AWS native observability service providing autom
ated summary and analy...
Container storage interface (CSI)
A flexible standard for orchestration
and storage provider connections
We support the CS...
Storage volume lifecycle
Provisioning Binding Using Reclaiming
• Static
• Dynamic*
• Control loop watches
for PVC requests...
What if I need specific volume type?
StorageClass
gp2 io1 sc1 encrypted
io1
st1
1) Admin pre-provisions
StorageClass based...
Amazon VPC CNI plugin
Elastic
network
interface
Secondary IPs:
10.0.0.1
10.0.0.2
10.0.0.1
10.0.0.2
Elastic
network
interfa...
Load balancing
All three Elastic Load Balancing products are supported
NLB and CLB supported by Kubernetes Service
type=Lo...
• Exposes the service externally using a cloud
provider’s load balancer
• NodePort and ClusterIP services (to which LB
wil...
Load balancing
Want to use an Internal Load Balancer? Use annotation:
service.beta.kubernetes.io/aws-load-balancer-interna...
Service load balancer: Network Load Balancer
apiVersion: v1
kind: Service
metadata:
name: nginx
namespace: default
labels:...
Service load balancer: Network Load Balancer (NLB)
• NLB supports forwarding the client’s IP through to the node
.spec.ext...
• exposes HTTP/HTTPS routes
to services within the cluster
• Many implementations: ALB,
NGINX, F5, HAProxy etc.
• Default ...
ALB Ingress controller
AWS Resources
Kubernetes Cluster
Node Node
Kubernetes
API Server ALB Ingress
Controller
Node
HTTP L...
Amazon EKS logging
EKS managedCustomer account
Internet
Amazon
CloudWatch
AWS
CloudTrail
Implementing logging with EFK
fluentd is an open source
data collector providing a
unified logging layer
elasticsearch is ...
Implementing logging with EFK
EKS Worker
pod
fluentd
daemonset
Logging with FluentBit
• New AWS FluentBit container
plugin
• Optimize costs. Route logs fr
om Amazon EKS and Amazon
ECS c...
Logging performance compare
Log Lines Per second Data Out Fluentd CPU Fluent Bit CPU Fluentd Memory Fluent Bit Memory
100 ...
Why AWS App Mesh?
http/tcp
Service
team A
Service
team B
Common need: Manage interservice traffic
• How to observe (logs, ...
App Mesh uses Envoy proxy
OSS community project
Wide community support, numerous integrations
Stable and production-proven...
Why App Mesh?
HTTP / TCP
Service
team A
Service
team B
Control plane
Translates logical intent to proxy config
Distributes...
App Mesh: App-level communication across AWS
Amazon ECS
AWS Fargate
Amazon EKS
Amazon EC2
AWS App Mesh
Kubernetes on EC2
App Mesh: Application observability
Logging
HTTP access logging
Amazon CloudWatch Logs
Available as container logs on
Amaz...
App Mesh: Client-side traffic management
Traffic shaping
Load balancing
Weight targets
Service discovery (DNS + AWS Cloud ...
Identify
performance
bottlenecks
How does X-Ray help?
Pinpoint specific
service issues
Identify
errors
Identify impact to
...
X-Ray concepts
user
Frontend API
Amazon
DynamoDB table
Amazon Simple
Queue Service
(Amazon SQS)
App instrumentation (Node.js)
//Add aws-xray-sdk package to package.json
const AWSXRay = require('aws-xray-sdk');
AWSXRay....
Adding business data (Node.js)
//Example showing how to add business data to traces
app.use(function(req, res, next){
if (...
What it should look like
• Trace List
• Trace Detail
Links
Amazon EKS: https://aws.amazon.com/eks/
AWS X-Ray: https://aws.amazon.com/xray/
Amazon CloudWatch: https://aws.amazo...
To conclude…
Clearly understand managed systems’ ownership boundaries and your sys
tems to build the own K8s Platform.
“Wi...
여러분의 피드백을 기다립니다!
#AWSDEVDAYSEOUL
[AWS Dev Day] 앱 현대화 | DevOps 개발자가 되기 위한 쿠버네티스 핵심 활용 예제 알아보기 - 정영준 AWS 솔루션즈 아키텍트, 이상호 삼성전자 Health서비스팀 선임
[AWS Dev Day] 앱 현대화 | DevOps 개발자가 되기 위한 쿠버네티스 핵심 활용 예제 알아보기 - 정영준 AWS 솔루션즈 아키텍트, 이상호 삼성전자 Health서비스팀 선임
[AWS Dev Day] 앱 현대화 | DevOps 개발자가 되기 위한 쿠버네티스 핵심 활용 예제 알아보기 - 정영준 AWS 솔루션즈 아키텍트, 이상호 삼성전자 Health서비스팀 선임
[AWS Dev Day] 앱 현대화 | DevOps 개발자가 되기 위한 쿠버네티스 핵심 활용 예제 알아보기 - 정영준 AWS 솔루션즈 아키텍트, 이상호 삼성전자 Health서비스팀 선임
[AWS Dev Day] 앱 현대화 | DevOps 개발자가 되기 위한 쿠버네티스 핵심 활용 예제 알아보기 - 정영준 AWS 솔루션즈 아키텍트, 이상호 삼성전자 Health서비스팀 선임
[AWS Dev Day] 앱 현대화 | DevOps 개발자가 되기 위한 쿠버네티스 핵심 활용 예제 알아보기 - 정영준 AWS 솔루션즈 아키텍트, 이상호 삼성전자 Health서비스팀 선임
[AWS Dev Day] 앱 현대화 | DevOps 개발자가 되기 위한 쿠버네티스 핵심 활용 예제 알아보기 - 정영준 AWS 솔루션즈 아키텍트, 이상호 삼성전자 Health서비스팀 선임
[AWS Dev Day] 앱 현대화 | DevOps 개발자가 되기 위한 쿠버네티스 핵심 활용 예제 알아보기 - 정영준 AWS 솔루션즈 아키텍트, 이상호 삼성전자 Health서비스팀 선임
[AWS Dev Day] 앱 현대화 | DevOps 개발자가 되기 위한 쿠버네티스 핵심 활용 예제 알아보기 - 정영준 AWS 솔루션즈 아키텍트, 이상호 삼성전자 Health서비스팀 선임
[AWS Dev Day] 앱 현대화 | DevOps 개발자가 되기 위한 쿠버네티스 핵심 활용 예제 알아보기 - 정영준 AWS 솔루션즈 아키텍트, 이상호 삼성전자 Health서비스팀 선임
[AWS Dev Day] 앱 현대화 | DevOps 개발자가 되기 위한 쿠버네티스 핵심 활용 예제 알아보기 - 정영준 AWS 솔루션즈 아키텍트, 이상호 삼성전자 Health서비스팀 선임
[AWS Dev Day] 앱 현대화 | DevOps 개발자가 되기 위한 쿠버네티스 핵심 활용 예제 알아보기 - 정영준 AWS 솔루션즈 아키텍트, 이상호 삼성전자 Health서비스팀 선임
Upcoming SlideShare
Loading in …5
×
Upcoming SlideShare
What to Upload to SlideShare
Next
Download to read offline and view in fullscreen.

Share

[AWS Dev Day] 앱 현대화 | DevOps 개발자가 되기 위한 쿠버네티스 핵심 활용 예제 알아보기 - 정영준 AWS 솔루션즈 아키텍트, 이상호 삼성전자 Health서비스팀 선임

Download to read offline


쿠버네티스에 어플리케이션을 손쉽게 배포하는 방법은 무엇일까요? 복잡하게 배포된 어플리케이션의 파드들은 어떻게 디버깅하고 로깅해야 할까요? 또한 요즘 자주 이야기 되는 클라우드 네이티브 아키텍처로 설계된 어플리케이션은 어떻게 만들고 배포해야하는 걸까요?삼성전자 무선사업부에서 삼성헬스를 EKS 에 배포한 사례를 살펴보며, 이러한 문제를 어떻게 해결했는지 알아봅니다. 또한 복잡하게만 느껴졌던 쿠버네티스의 어플리케이션 배포와 클라우드 네이티브 아키텍처의 베스트 프렉티스를 EKS 에 어플리케이션을 배포하고, 관리하는 예제를 통하여 간편하게 이해할 수 있게 도와드립니다.

Related Books

Free with a 30 day trial from Scribd

See all

Related Audiobooks

Free with a 30 day trial from Scribd

See all

[AWS Dev Day] 앱 현대화 | DevOps 개발자가 되기 위한 쿠버네티스 핵심 활용 예제 알아보기 - 정영준 AWS 솔루션즈 아키텍트, 이상호 삼성전자 Health서비스팀 선임

  1. 1. DevOps 개발자가 되기 위한 쿠버 네티스 핵심 활용 예제 알아보기 정영준 솔루션즈 아키텍트 AWS 이상호 선임 삼성전자 Health서비스팀
  2. 2. Agenda • Container Service Overview • EKS Tenets and Core Concept • Samsung Health Use Case • EKS Best Practices & Sample Deploy Architecture
  3. 3. We’re making AWS the best place to run contain ers and Kubernetes
  4. 4. AWS container services landscape Management Deployment, Scheduling, Scaling & Management of containerized applications Hosting Where the containers run Amazon Elastic Container Service Amazon Elastic Kubernetes Service Amazon EC2 AWS Fargate Image Registry Container Image Repository Amazon Elastic Container Registry
  5. 5. Balancing flexibility and simplicity: Workload-by-workload Flexibility focused Low level of opinion Low level of abstraction Focus on infrastructure and configuration Installing, configuring, and managing my compute environment is critical to achieving my goals Value simplicity High level of opinion High level of abstraction Focus only on app and primitive Having a standardized and on-demand compute environment is critical to achieving my goals
  6. 6. We give you the power to choose Amazon ECS Amazon EKS Amazon EC2 AWS Fargate Amazon EC2 AWS Fargate 1. Choose your orchestration tool 2. Choose your launch type We’re working on it #32
  7. 7. EKS tenets 1. Amazon EKS is a platform to run production-grade workloads. Security and reliability are our first priority. After that we focus on doing the heavy lifting for you in the control plane, including life cycle-related things like version upgrades. 2. Amazon EKS provides a native and upstream Kubernetes experience. Amazon EKS provides vanilla, un-forked Kubernetes. In keeping with our first tenant, we ensure the Kubernetes versions we run have security-related patches, even for older, supported versions as quickly as possible. But there’s no special sauce and no lock in. 3. If you want to use additional AWS services, integrations are as seamless as possible. 4. The Amazon EKS team in AWS actively contributes to the upstream Kubernetes project and the wider CNCF activities, both on the technical level as well as community, from communicating good practices to participation in SIGs and working groups.
  8. 8. How are customer using Amazon EKS? Microservices PaaS Platform as a service Enterprise App Migration Machine Learning
  9. 9. [mycluster].eks.amazonaws.com Availability Zone 1 Availability Zone 2 Availability Zone 3 Kubectl Your AWS account
  10. 10. VPC Kubernetes control plane Highly available and single tenant infrastructure All “native AWS” components Fronted by a Network Load Balancer NLB Amazon EKS Availability Zone 1 Availability Zone 2 Availability Zone 3 Etcd API Servers
  11. 11. Kubernetes architecture etcd
  12. 12. eksctl - a CLI for Amazon EKS • Single command cluster creation eksctl create cluster --nodes=4 • Open source and on GitHub • Built by Weave and AWS • Official Amazon EKS CLI
  13. 13. Standard EC2 compute instance types P and G type accelerated instances i3 bare metal Spot Instances Bring your own instances Instance flexibility
  14. 14. Bring your own OS Amazon EKS AMI build scripts https://github.com/awslabs/amazon-eks-ami Amazon Amazon Amazon
  15. 15. Windows containers Run Windows containers and Windows Server nodes with Amazon EKS Supports heterogeneous (mixed) clusters. Kubernetes version 1.11+ Available in all Amazon EKS Regions Developer preview: https://github.com/aws/containers-roadmap
  16. 16. Provisioning worker nodes AWS CloudFormation eksctl Partners …more Terraform Pulumi Rancher
  17. 17. Now supporting P3dn.24xlarge instances CUDA 10 with NVIDIA v410 coming soon! Amazon EKS-optimized GPU AMI
  18. 18. Support Kubernetes version Latest Kubernetes: 1.14 Amazon EKS will support up to three versions of Kubernetes at once Deprecation in line with the community stopping support for older versions Version 1.11 deprecation on Nov 4th, 2019
  19. 19. Amazon EKS platform version Platform version revisions represent API server configuration changes or Kubernetes patches Platform versions increment within a Kubernetes version only
  20. 20. Amazon EKS update life cycle May 21, 2019 Blog: https://aws.amazon.com/blogs/compute/updates-to-amazon-eks-version-lifecycle/
  21. 21. Amazon EKS support sophisticated and scalable infrastructure [mycluster].eks.amazonaws.com Availability Zone 1 Availability Zone 2 Availability Zone 3 Kubectl VPC Instance Auto Scaling group for m4.large Spot Instances Auto Scaling group for t2.medium Spot Instances Auto Scaling group for On-Demand Instances Cluster Autoscaler Daemonset Spot Interruption handler Daemonset https://eksworkshop.com/spot/ managespot/deployhandler
  22. 22. Copyright © 2019 Samsung Electronics, Co., Ltd. All rights reserved. 25 Lee Sang Ho, 삼성전자 AWS EKS on Samsung Health Server
  23. 23. Copyright © 2019 Samsung Electronics, Co., Ltd. All rights reserved. 26 Contents 1.Why Samsung Health choose AWS EKS 2.AWS EKS on Samsung Health Server 3.Little things that need to be improved
  24. 24. Copyright © 2019 Samsung Electronics, Co., Ltd. All rights reserved. 27 Who am I  Lee Sang Ho  삼성전자, 무선 사업부, Health Service Team  2016 ~ Present : Health Service Team  2014 ~ 2016 : Big Data Center  randomday222@gmail.com  Server Developer  Samsung Health Server Development  Health Care B2B Project Server Development  AWS Cloud System Architecture  DevOps Engineering
  25. 25. Copyright © 2019 Samsung Electronics, Co., Ltd. All rights reserved. 28 Why Samsung Health choose AWS EKS  Inefficiency of Infra CI/CD  Difficulties in Communication with Oversea Research Center (in charge or service operation)  Frequent operational issues due to human error  Docker!  Now is the time  Too much cost when we get huge legacy infra migrated.  So, we started using Kubernetes on new B2B Project.  Multi-tier support  Dev, Stg, Stg-2,Pre-Prod, Prod  Too many tier, too much cost
  26. 26. Copyright © 2019 Samsung Electronics, Co., Ltd. All rights reserved. 29 Migration with Kubernetes Internal NLB-1 Internal NLB-2 aws vpc link aws vpc link Node Port : 30700Node Port : 30700Target Group : 8070Target Group : 8070 Target Port : 8080 - H ServerTarget Port : 8080 - H Server Node Port : 30800Node Port : 30800Target Group : 8080Target Group : 8080 Target Port : 8080 - I ServerTarget Port : 8080 - I Server aws vpc link aws vpc link Node Port : 30000Node Port : 30000Target Group : 8000Target Group : 8000 Target Port : 8080 - A ServerTarget Port : 8080 - A Server Node Port : 30100Node Port : 30100Target Group : 8010Target Group : 8010 Target Port : 8080 - B ServerTarget Port : 8080 - B Server Node Port : 30200Node Port : 30200Target Group : 8020Target Group : 8020 Target Port : 8080 - C ServerTarget Port : 8080 - C Server Node Port : 30300Node Port : 30300Target Group : 8030Target Group : 8030 Target Port : 8080 - D ServerTarget Port : 8080 - D Server Node Port : 30400Node Port : 30400Target Group : 8040Target Group : 8040 Target Port : 8080 - E ServerTarget Port : 8080 - E Server Node Port : 30500Node Port : 30500Target Group : 8050Target Group : 8050 Target Port : 8080 - F ServerTarget Port : 8080 - F Server Node Port : 30600Node Port : 30600Target Group : 8060Target Group : 8060 Target Port : 8080 - G ServerTarget Port : 8080 - G Server core dns . . . Admin ALB Node Port : 30000Node Port : 30000Target Group : 443Target Group : 443 Target Port : 8080 - J ServerTarget Port : 8080 - J Server Kubernetes DomainAws Domain Amazon API Gateway
  27. 27. Copyright © 2019 Samsung Electronics, Co., Ltd. All rights reserved. 30 What else  Kube2Iam  HPA  Cluster Autoscaler template: metadata: annotations: iam.amazonaws.com/role: data-ec2-role
  28. 28. Copyright © 2019 Samsung Electronics, Co., Ltd. All rights reserved. 31 PrometheusPrometheus CloudwatchCloudwatch In-House Secure Agent In-House Secure Agent GrafanaGrafana ElasticsearchElasticsearch KibanaKibana SlackSlack AWS SESAWS SES Samsung Health Cloud Samsung Health Cloud AWS Infra Metric (API GW, RDS, DynamoDB) K8S Metric Integrated Dashboard Logging Log Monitoring Notification Monitoring Flow K8S Metric Infra Metric
  29. 29. Copyright © 2019 Samsung Electronics, Co., Ltd. All rights reserved. 32 Log Monitoring  EFK + AWS Cloudwatch Cloudwatch (Aggregator) Cloudwatch (Aggregator) Set ES Index Name by Service name, date & add Some Fields Node-01Node-01 Pod-01Pod-01 fluentd agent (Daemonset) fluentd agent (Daemonset) Pod-02Pod-02 Std-OutStd-Out ElasticsearchElasticsearch KibanaKibana Node-02Node-02 Pod-03Pod-03 fluentd agent (Daemonset) fluentd agent (Daemonset) Pod-04Pod-04 Std-OutStd-Out elasticsearch stream configure & set up fluentd by kubernetes ex) specific error pattern Amazon CloudWatch Logs Alarm Amazon Cognito
  30. 30. Reverse Proxy with ES  in case of vpc access NGINXNGINX KIBANAKIBANA COGNITOCOGNITO ② if no authentication then 302 redirect_to_cognito ?redirect_uri={vpc_kibana_domain} ① kibana access via nginx ③ sign up then redirect to kibana with redirect_uri proxy_redirect https://{cognito_host} https://$host; proxy_redirect https://{kibana_host} https://$host; user
  31. 31. Copyright © 2019 Samsung Electronics, Co., Ltd. All rights reserved. 34 AWS Xray  APM tool  Analyze and debug production, distributed applications like MSA  Review request behavior  Discover application issues  Improve performance
  32. 32. Copyright © 2019 Samsung Electronics, Co., Ltd. All rights reserved. 35 Spinnaker  Continuous Delivery Platform  Open source Powered by Netflix  Support Multi-cloud (AWS, Google Cloud, MS Azure)  Orchestration Pipeline
  33. 33. Copyright © 2019 Samsung Electronics, Co., Ltd. All rights reserved. 36 AWS ECR Overall architecture Node-01Node-01 Pod-01Pod-01 Pod-02Pod-02 Node-02Node-02 Pod-01Pod-01 Pod-02Pod-02 Node-03Node-03 Pod-01Pod-01 Pod-02Pod-02 Node-04Node-04 Pod-01Pod-01 Pod-02Pod-02 Node-05Node-05 Pod-01Pod-01 Pod-02Pod-02 Node-06Node-06 Pod-01Pod-01 Pod-02Pod-02 Node-07Node-07 Pod-01Pod-01 Pod-02Pod-02 Node-08Node-08 Pod-01Pod-01 Pod-02Pod-02 SpinnakerSpinnaker Aws CloudWatch Jenkins ① build and upload docker image to aws ecr repository ② trigger k8s deployment pipeline ③ deploy on k8s cluster ④ log monitoring ⑤ k8s cluster monitoring K8S Control Plane
  34. 34. Copyright © 2019 Samsung Electronics, Co., Ltd. All rights reserved. 37  When deploy  Before  Now What’s better? #1 Jenkins OSOS javajava tomcattomcat external libraryexternal library OSOS javajava tomcattomcat external libraryexternal library applicationapplication deploy war(jar) & configure setenv OSOS configuration OSOS deploy docker image to cluster OSOS javajava tomc at tomc at external libraryexternal library appapp config map config map it took so long and caused many human errors.
  35. 35. Copyright © 2019 Samsung Electronics, Co., Ltd. All rights reserved. 38  When operation  Self-healing, HPA (Horizontal Pod Autoscaler) What’s better? #2 K8S MasterK8S Master SchedulerScheduler ControllerController Node-01Node-01 App AApp A App BApp B App AApp A Node-02Node-02 App AApp A App BApp B Node-03Node-03 App AApp A App BApp B App BApp B K8S MasterK8S Master SchedulerScheduler ControllerController Node-01Node-01 App AApp A App BApp B App AApp A Node-02Node-02 App AApp A App BApp B Node-03Node-03 App AApp A App BApp BApp BApp B When certain app breaks down, k8s scheduler notices situation. Then, restarts and replaces the app.
  36. 36. Copyright © 2019 Samsung Electronics, Co., Ltd. All rights reserved. 39 Little things that need to be improved  NLB Multi Port Support  Fargate + EKS  Totally Managed Kubernetes Node  When??  Parameter Store + EKS  Like ECS + SSM.ParameterStore
  37. 37. Challenges in Using & Deploying Containers As cloud native technologies change the way companies are designing an d building applications, challenges are inevitable. The top challenges that respondents face are: • Cultural Changes with Development Team (41%) • Complexity (40% up from 35%) • Lack of Training (40%) • Security (38% down from 43%) • Monitoring (34% down from 38%) • Storage (30% down from 41%) • Networking (30% down from 38%) https://www.cncf.io/blog/2018/08/29/cncf-survey-use-of-cloud-native-technologies-in-production-has-grown-over-200-percent/
  38. 38. Container security onion model: Defense in depth • full blown distro (Ubuntu, AL) vs. minimal environment (container-optimized distribution) • multi-tenancy requirements • gotchas: Linux packages/CVEs, leaks, GDPR (in Europe) • runtime/standards (OCI) • immutability of images • all containers share a kernel (mitigation: Firecracker) • gotchas: unnecessary privileged users, no scans, trust • code analysis • source available? • gotchas: big surface, many languages {} } • sanitizing user input • static code analysis • gotchas: log-leaking } • sensitive config (passwords, API keys, etc.) • gotchas: commits-to-source, non-separated access (dev has cleartext password) { • business core data • Personal Identifiable Information (PII) • gotchas: leaks, GDPR (in Europe) { host container dependencies code config user data
  39. 39. Security tooling Amazon Inspector AWS KMS AWS Secrets Manager AWS WAF AWS IAM Amazon GuardDuty Amazon Macie AWS Security Hub AWS CloudHSM AWS Certificate Manager AWS CloudTrail host container dependencies code config user data
  40. 40. PKI configuration Kubelet Generates public/private keys Kubelet installs server cert Kubelet issues CSR Certificate rotation Amazon EKS API serverEKS worker Each Amazon EKS cluster is a unique CA
  41. 41. API-server endpoint access control Worker VPC (your account) Kubectl Master VPC (AWS account) etcd AZ 1 API Server etcd API Server prod-cluster-123.eks.amazonaws.com EKS-owned ENI Kubelet AZ 1 Worker node EKS-owned ENI Kubelet AZ 2 Worker node Public == true AZ 2 Kube-proxy Kube-proxy
  42. 42. API-server endpoint access control Worker VPC (your account) Kubectl Master VPC (AWS account) etcd AZ 1 AZ 2 API Server etcd API Server prod-cluster-123.eks.amazonaws.com EKS-owned ENIs Public == true Private == true prod-cluster-123.eks.amazonaws.com Private hosted zone Kubelet AZ 1 Worker node Kube-proxy Kubelet AZ 2 Worker node Kube-proxy
  43. 43. API-server endpoint access control Worker VPC (your account) Kubectl Master VPC (AWS account) etcd AZ 1 AZ 2 API Server etcd API Server EKS-owned ENIs Public == false Private == true prod-cluster-123.eks.amazonaws.com Private hosted zone Kubelet AZ 1 Worker node Kube-proxy Kubelet AZ 2 Worker node Kube-proxy
  44. 44. AWS Identity and Access Management (IAM) Aut hentication Kubectl 3) Authorizes AWS identity with RBAC K8s API 1) Passes AWS identity 2) Verifies AWS identity 4) K8s action allowed/denied
  45. 45. Great integration responsibility – IAM roles Frontend pod Node Backend pod Amazon S3 UI customization bucket Billing reports bucket Role IAM Log DaemonSet Kinesis Kinesis Data Streams ElastiCache ElastiCache for Redis
  46. 46. Great integration responsibility – IAM roles Frontend pod Node Backend pod Amazon S3 UI customization bucket Billing reports bucket IAM Kinesis Kinesis Data Streams Log DaemonSet ElastiCache ElastiCache for RedisCredential Credential Credential
  47. 47. Service type LoadBalancer Service (LoadBalancer) ELB Users Pod selector (app = frontend) Frontend pod 1 Node NodePort Frontend pod 2 Node NodePort Frontend pod 3 Node NodePort
  48. 48. Service type LoadBalancer Service (LoadBalancer) ELB Users Pod selector (type = nodejs) Frontend pod 1 Node NodePort Frontend pod 2 Node NodePort Frontend pod 3 Node NodePort Backend pod 1 Node NodePort Generic pod selector
  49. 49. Great responsibility – Service type LoadBalancer Service (LoadBalancer) ELB Users Pod selector (app = frontend) Frontend pod 1 Node NodePort Frontend pod 2 Node NodePort Frontend pod 3 Node NodePort Backend pod 1 Node NodePortService (LoadBalancer) Pod selector (app = backend) ELB Users
  50. 50. Amazon CloudWatch Container Insights Gives you complete visibility into your cloud resources and applications so you can monitor, troubleshoot, and remediate issues Collect logs & metrics Monitor, troubleshoot & set alarms Act AnalyzeAmazon CloudWatch
  51. 51.  Collects, aggregates, and summarizes  Reliable, secure metrics and logs collection  Automated dashboards and analysis  Observability experience across metrics, logs, traces  Ad hoc analytics CloudWatch Container Insights A fully managed observability service for monitoring, troubleshooting , and alarming on your containerized applications and microservices
  52. 52. Images on DockerHub Performance Metrics – CloudWatch Agent: https://hub.docker.com/r/ amazon/cloudwatch-agent • Tag: latest Logs – FluentBit: https://hub.docker.com/r/amazon/aws-for-fluent-bit • Tag: latest Logs – Fluentd: https://hub.docker.com/r/fluent/fluentd-kubernetes-daemonset • Tag: v1.3.3-debian-cloudwatch-1.4
  53. 53. Container Insights available now 1. Fully managed, AWS native observability service providing autom ated summary and analysis of compute capacity 2. Reliable and secure collection of application logs with built-in an alytics capabilities 3. Prebuilt visualization to summarize cluster and node errors 4. Application & microservice tracing - Troubleshoot and debug ap plication & microservice
  54. 54. Container storage interface (CSI) A flexible standard for orchestration and storage provider connections We support the CSI standard through following drivers: Amazon Elastic Block Store: Amazon EBS CSI Driver Amazon Elastic File System: Amazon EFS CSI Driver Amazon FSx for Lustre: Amazon FSx CSI Driver
  55. 55. Storage volume lifecycle Provisioning Binding Using Reclaiming • Static • Dynamic* • Control loop watches for PVC requests and satisfies if PV is available. • For Dynamic, PVC will provision PV • PVC to PV binding is one-to-one mapping • Cluster mounts volume based on PVC • Retain (default) • Recycle • Delete
  56. 56. What if I need specific volume type? StorageClass gp2 io1 sc1 encrypted io1 st1 1) Admin pre-provisions StorageClass based on workload needs 2) End user requests for specific volume types (e.g., encrypted io1 volume) 3) Control loop watches PVC request and allocates volume if PV exists MySQL Pods 4) User creates stateful workload
  57. 57. Amazon VPC CNI plugin Elastic network interface Secondary IPs: 10.0.0.1 10.0.0.2 10.0.0.1 10.0.0.2 Elastic network interface 10.0.0.20 10.0.0.22 Secondary IPs: 10.0.0.20 10.0.0.22 ec2.associateaddress() VPC Subnet – 10.0.0.0/24 Instance 1 Instance 2 VPC https://github.com/aws/amazon-vpc-cni-k8s
  58. 58. Load balancing All three Elastic Load Balancing products are supported NLB and CLB supported by Kubernetes Service type=LoadBalancer Internal and External Load Balancer support https://aws.amazon.com/blogs/opensource/network-load-balancer-nginx-ingress-controller-eks/
  59. 59. • Exposes the service externally using a cloud provider’s load balancer • NodePort and ClusterIP services (to which LB will route) automatically created • Each service exposed with a LoadBalancer (ELB or NLB) will get its own IP address • Exposes L4 (TCP) or L7 (HTTP) services Kubernetes ServiceType: LoadBalancer
  60. 60. Load balancing Want to use an Internal Load Balancer? Use annotation: service.beta.kubernetes.io/aws-load-balancer-internal: 0.0.0.0/0 Want to use an NLB? Use annotation: service.beta.kubernetes.io/aws-load-balancer-type: nlb
  61. 61. Service load balancer: Network Load Balancer apiVersion: v1 kind: Service metadata: name: nginx namespace: default labels: app: nginx annotations: service.beta.kubernetes.io/aws-load-balancer-type: "nlb" spec: externalTrafficPolicy: Local ports: - name: http port: 80 protocol: TCP targetPort: 80 selector: app: nginx type: LoadBalancer
  62. 62. Service load balancer: Network Load Balancer (NLB) • NLB supports forwarding the client’s IP through to the node .spec.externalTrafficPolicy = Local  client ip passed to pod • Nodes with no matching pods will be removed by specified NLB’s health check .spec.healthCheckNodePort • Use DaemonSet or pod anti-affinity to verify even traffic split
  63. 63. • exposes HTTP/HTTPS routes to services within the cluster • Many implementations: ALB, NGINX, F5, HAProxy etc. • Default service type: ClusterIP Kubernetes Ingress object
  64. 64. ALB Ingress controller AWS Resources Kubernetes Cluster Node Node Kubernetes API Server ALB Ingress Controller Node HTTP ListenerHTTPS Listener Rule: /cheesesRule: /charcuterie TargetGroup: Green (IP Mode) TargetGroup: Blue (Instance Mode) NodePort NodePort https://docs.aws.amazon.com/en_pv/eks/latest/userguide/alb-ingress.html
  65. 65. Amazon EKS logging EKS managedCustomer account Internet Amazon CloudWatch AWS CloudTrail
  66. 66. Implementing logging with EFK fluentd is an open source data collector providing a unified logging layer elasticsearch is a distributed, RESTful search and analytics engine kibana lets you visualize your Elasticsearch data
  67. 67. Implementing logging with EFK EKS Worker pod fluentd daemonset
  68. 68. Logging with FluentBit • New AWS FluentBit container plugin • Optimize costs. Route logs fr om Amazon EKS and Amazon ECS clusters directly to Amaz on S3 and query with Amazo n Athena • Open source • More resource-efficient than Fluentd. Tests show Fluentd u ses 4x more CPU and 6x more memory https://aws.amazon.com/blogs/opensource/centra lized-container-logging-fluent-bit/
  69. 69. Logging performance compare Log Lines Per second Data Out Fluentd CPU Fluent Bit CPU Fluentd Memory Fluent Bit Memory 100 25 KB/s 0.013 vCPU 0.003 vCPU 146 MB 27 MB 1000 250 KB/s 0.103 vCPU 0.03 vCPU 303 MB 44 MB 10000 2.5 MB/s 1.03 vCPU 0.19 vCPU 376 MB 65 MB Log Lines Per second Data Out Fluentd CPU Fluent Bit CPU Fluentd Memory Fluent Bit Memory 100 25 KB/s 0.006 vCPU 0.003 vCPU 84 MB 27 MB 1000 250 KB/s 0.073 vCPU 0.033 vCPU 102 MB 37 MB 10000 2.5 MB/s 0.86 vCPU 0.13 vCPU 438 MB 55 MB
  70. 70. Why AWS App Mesh? http/tcp Service team A Service team B Common need: Manage interservice traffic • How to observe (logs, metrics, traces) • How to load balance E/W traffic • How to shift traffic between deployments • How to decouple service teams • How to minimize impact to app code
  71. 71. App Mesh uses Envoy proxy OSS community project Wide community support, numerous integrations Stable and production-proven Graduated Project in Cloud Native Computing Foundation Started at Lyft in 2016
  72. 72. Why App Mesh? HTTP / TCP Service team A Service team B Control plane Translates logical intent to proxy config Distributes proxy config Proxy Sits between all services Manages and observes traffic Control plane
  73. 73. App Mesh: App-level communication across AWS Amazon ECS AWS Fargate Amazon EKS Amazon EC2 AWS App Mesh Kubernetes on EC2
  74. 74. App Mesh: Application observability Logging HTTP access logging Amazon CloudWatch Logs Available as container logs on Amazon ECS, Amazon EKS, AWS Fargate Metrics CloudWatch metrics StatsD (with tags) Prometheus Tracing AWS X-Ray Other Envoy tracing drivers
  75. 75. App Mesh: Client-side traffic management Traffic shaping Load balancing Weight targets Service discovery (DNS + AWS Cloud Map) Health checks Retries* Timeouts* Circuit breakers* Routing controls Protocols support (HTTP, TCP, gRPC*) Path-based Header-based* Cookie-based* Host-based* *coming soon
  76. 76. Identify performance bottlenecks How does X-Ray help? Pinpoint specific service issues Identify errors Identify impact to users
  77. 77. X-Ray concepts user Frontend API Amazon DynamoDB table Amazon Simple Queue Service (Amazon SQS)
  78. 78. App instrumentation (Node.js) //Add aws-xray-sdk package to package.json const AWSXRay = require('aws-xray-sdk'); AWSXRay.config([AWSXRay.plugins.EC2Plugin,AWSXRay.plugins.ECSPlugin]); const xrayExpress = require('aws-xray-sdk-express’); app.use(xrayExpress.openSegment('Frontend’)); app.get('/', function(req, res) … app.get(‘/static', function(req, res) app.use(xrayExpress.closeSegment()); //Add aws-xray-sdk package to package.json const AWSXRay = require('aws-xray-sdk'); AWSXRay.config([AWSXRay.plugins.EC2Plugin,AWSXRay.plugins.ECSPlugin]); const xrayExpress = require('aws-xray-sdk-express’); app.use(xrayExpress.openSegment('Frontend’)); app.get('/', function(req, res) … app.get(‘/static', function(req, res) app.use(xrayExpress.closeSegment());
  79. 79. Adding business data (Node.js) //Example showing how to add business data to traces app.use(function(req, res, next){ if (req.session !== undefined) { let segment = AWSXRay.getSegment() // User sessionID as userID segment.addAnnotation(‘userID', req.sessionID); } next(); }) //Example showing how to add business data to traces app.use(function(req, res, next){ if (req.session !== undefined) { let segment = AWSXRay.getSegment() // User sessionID as userID segment.addAnnotation(‘userID', req.sessionID); } next(); })
  80. 80. What it should look like
  81. 81. • Trace List
  82. 82. • Trace Detail
  83. 83. Links Amazon EKS: https://aws.amazon.com/eks/ AWS X-Ray: https://aws.amazon.com/xray/ Amazon CloudWatch: https://aws.amazon.com/cloudwatch/
  84. 84. To conclude… Clearly understand managed systems’ ownership boundaries and your sys tems to build the own K8s Platform. “With great power comes great responsibility.”
  85. 85. 여러분의 피드백을 기다립니다! #AWSDEVDAYSEOUL
  • freeslur

    Apr. 23, 2020
  • WonJinDo

    Apr. 16, 2020
  • JiaxinShan

    Nov. 10, 2019
  • bajutae

    Oct. 7, 2019
  • SANGHOLEE76

    Oct. 1, 2019

쿠버네티스에 어플리케이션을 손쉽게 배포하는 방법은 무엇일까요? 복잡하게 배포된 어플리케이션의 파드들은 어떻게 디버깅하고 로깅해야 할까요? 또한 요즘 자주 이야기 되는 클라우드 네이티브 아키텍처로 설계된 어플리케이션은 어떻게 만들고 배포해야하는 걸까요?삼성전자 무선사업부에서 삼성헬스를 EKS 에 배포한 사례를 살펴보며, 이러한 문제를 어떻게 해결했는지 알아봅니다. 또한 복잡하게만 느껴졌던 쿠버네티스의 어플리케이션 배포와 클라우드 네이티브 아키텍처의 베스트 프렉티스를 EKS 에 어플리케이션을 배포하고, 관리하는 예제를 통하여 간편하게 이해할 수 있게 도와드립니다.

Views

Total views

1,674

On Slideshare

0

From embeds

0

Number of embeds

0

Actions

Downloads

32

Shares

0

Comments

0

Likes

5

×