SlideShare une entreprise Scribd logo
1  sur  53
Télécharger pour lire hors ligne
Hello I am Sanath Kumar Ramesh. I am a software engineer in AWS
Serverless organization where I lead developer tools initiatives. I develop tools
to simplify the process of building serverless applications. I built AWS SAM
and SAM CLI, open source toolchains to help developers be more productive.
1
A brief background about me.. I started designing computer processors, then
built parts of Windows kernel, and graduated up the stack to build web
services at startups followed by AWS. In 2016 I joined AWS Lambda team, few
months after the service was generally available. I worked on the inner guts of
the service but quickly found my passion with developer tools. I have been a
huge supporter and evangelist of open source since college. I started the first
OSS club in undergrad which is still running successfully. I open sourced the
underlying implementation of AWS SAM, built SAM CLI, and work on 100%
open source software at AWS.
In this talk, I will introduce you to the amazing open source community of
serverless developer tools which I have been a part of since 2016. For the
longest time, I wasn’t conscious of the scale and potential of this
community, until I embarked on my own journey of open source two strategic
products at AWS. We will talk about the broader open source AWS community
first before diving deep into serverless.
2
Happy Birthday Open Source !! Twenty years! How amazing is that ??
3
At AWS, we have been a huge believer and supporter of Open Source
software. Did you know that some of our foundational services are built on top
popular OSS projects - EC2 uses Xen, RDS supports MySQL, EKS supports
Kubernetes, to name a few. We go beyond just support and use software, we
help build communities.
Grow communities: Communities of developers - AWS joined open source
foundations like CNCF, OSI, Apache Software Foundation, Linux Foundation
etc. We have supported some of our customers like Netflix and Capital One to
help build their open source programs.
Improve code: We have hunderds of open source projects on our Github
repositories, contribute to projects like Kubernetes, Apache MX Net etc.
Increase Contributions: Internally, we have several initiatives that make it
easier for employees to contribute open source software. For example, using
Standard Licenses (Apache or MIT for most part), AWS credits for academic
and non-profit projects, Encouraging external communications etc
100s of open source repos on Github: Where we have open sourced
everything from SDKs, client side tools, IDE plugins, full blown apps, to core
components of some of our services such as s2n.
4
• We are motivated to collaborate by many of our partners and customers
• Open Source projects helps us to innovate
• Scaling AWS services around open source helps us meet customer
needs
• Scaling for open data help customers build services they need
(Customers!
Customers = community. )
5
6
2016 projects, new contributions to apache mxnet
Contibution volume is growing every year
7
2017 projects so far, new ones in orange, notably our first contributions to
Kubernetes
8
9
That was the broader AWS landscape. Let's drill down to the focus of today's
talk - Serverless!
By show of hands, how many of you have heard of the term Serverless? How
many have built serverless apps - hobby, for work, anything? How many of you
build & maintain serverless apps in production today?
Fantastic! What's all the buzz about serverless? Why do we care? And, Why is
Open Source in serverless such a big deal? This is going to be the focus of
our talk.
10
No servers to mange, never pay for idle, apps you build scale with usage
automatically, and highly available & fault tolerant. AWS Lambda offers a
serverless computing platform, API Gateway offers an hosted API service,
DynamoDB offers a database etc. For any of you that have built apps on the
cloud, serverless offers several system properties that you otherwise have to
build yourself.
11
Secret sauce? The things you usually write code for, are now expressed as
simple, static, configurations. For example, in a standard cloud application,
you would write code for your business logic, API interfaces, and all plumbing
between them. Resources like databases, storage, permissions etc are
expressed as configurations. I have written the code for user authentication so
many times now that I almost decided to quit programming when I wrote user
auth for my hobby project recently.
In a serverless app, a lot of the undifferentiated heavy lifting is handled by
services for you. You write a static configuration file to configure the API
endpoints, user auth, workflows etc and the services takes care of running
them for you. This means you focus primarily on the business logic and write
some plumbing here and there. This also means, all your code-first developer
tools such as IDEs, test harnesses, build systems, deployment engines, etc
need to support expressing the configuration.
12
This was an opportunity and a whole slew of developer tools emerged in the
community. Guess what? Nearly every one of them is open source!
Serverless Framework is the biggest and most popular one by far, going by
open source community stats. 26k stars on Github, 80k downloads on NPM, >
450 contributors, and has a rich plugin ecosystem.
Architect provides a simple DSL to express your entire app with. Paired with
their Javascript library, you can write regular NodeJS apps that will be
deployed as serverless apps.
Apex provides is a simple and opinionated tool to manage your Lambda
functions. It can be combined with infrastructure provisioning tools such as
Terraform to manage your entire app.
Sparta allows you to write Go code that, when deployed, will automatically
provision & configure underlying resources for you.
Pulumi similarly supports automatic resource provisionng from code, but they
support a variety of programing languages, and non-serverless resources too.
Claudia.JS is a javascript library to create serverless apps using Nodejs.
13
Zappa is similar to ClaudiaJS but for Python.
All of these tools provide varying levels of convenience, target certain
programming languages, or specialize in certain tasks.
13
AWS SAM is a language agnostic way of creating, testing, deploying, and
managing serverless apps. Shameless plug: I built SAM and the
accompanying CLI. We are going to talk a lot about SAM in today's talk.
<<talk about other AWS dev tools as much as time fits>>
14
15
16
17
18
19
20
Each of these projects have their own vibrant ecosystem of hundreds of
developers working alongside with AWS.
How did this all happen? I can’t speak for other projects, but here is a case
study of how I created a community around SAM.
21
22
23
Shameless Plug
SAM is fully open source and includes a companion CLI tool to locally test &
run serverless aplications
24
25
26
27
28
- Open Specification means: No code but just one spec file on Github. Think
of this as essentially our docs are on Github.
- Excellent traction on Github. Great community of developers for the first few
months.
29
Month or two after launch, someone filed an issue asking us to Open Source
the implementation.
30
31
32
33
34
35
36
37
38
Kubernetes: Amazon EKS
Hadoop: Amazon EMR
Key difference with SAM:
• We didn’t open source EKS, but instead support Kubernetes as part of the
EKS service. This is different from SAM where we will open source the
entire service
• We are the producer of this product and we run it as a service too..
39
40
41
These questions themselves focus on the “software”, but once I answered
these questions, it became apparent that I was “building a community” and not
just another piece of software. The process of running an open source project
naturally drove me to build communities.
Goal is to allow the community to build software with full assistance from AWS.
Combined the agility of open source community and quality assurance
practices of AWS, we believe this model can produce amazing software.
42
When I proposed open source SAM, my manager asked me how many ppl do
I want in this team. I said 100! Only a handful of engineers directly come from
AWS, but a large majority are from the community. To build this team, we
needed open communication where everyone feels connected and a part of
same team. Hence Slack with ~1000 ppl including customers, devs, etc.
Builds Bonds:
• Customers feel like a part of the product, talk directly to product engineers
without several layers of support/management to go thru. They feel more
empowered to use our product
43
Because development is open:
• Builds Trust
• Reduces possibility of malicious code
• Commercial friendly Apache 2.0 license invites the community to build
innovative solutions on top of SAM. Like for example what Stackery did.
Community Protects against:
• MALICIOUS code
• Poor quality code
44
We have a team, we have a product that is open. Next is to keep everyone on
same page of what to work on. In an internal team, product & project
managers come up with this list. We use Github Labels & Gitub Issues to keep
everyone informed of the priority.
• Community is on same page on what to work on next
• Visibility into top-issues
• Provides customers a clear visibility into how we think about this problem.
They can obviously provide feedback to change our mind. This is refreshing
instead of the opaque “we have this in our backlog somewhere”
45
Running a Service:
* We take care of deployments on behalf of the community. At AWS,
everything is a service and we have deep knowledge in deploying services.
• Automation – all AWS regions
• Safe Guards – Beta, Gamma, Auto-rollback, Alarms, Canaries
• Bleeding edge vs Stability: We guarantee stability of code (back
compatibility etc) by doing deploymsents with several safe guards. Bleeding
edge customers can consume source directly
• Open communication on Github Releases to mark global availability
46
* Oncall: To make sure Github Issues & PRs don’t’ go stale, we have a oncall
rotation where on person is in-charge per week to manage the repository.
Obviously any team member can operate the repo at anytime, but having this
person ensures the community is always engaged.
• Operational Review: After every week, we have an internal meeting to
discuss highlights of the oncall. This provides the entire team visibility into
product, community health, emergent issues or anything that could change
priorities.
• We bring our DevOps experience from AWS into an open source
product to make sure both the community & the product stays
healthy.
• Information from each week’s operational review is visible to broader
AWS Serverless organization.
• We often trade notes with other popular OSS projects like AWS CLI,
SDKs to make sure we don’t repeat same mistakes and continue to
raise the bar.
47
We support the community by automatically providing them with the best
practices we have learnt at AWS over several years.
48
49
Operational Practices
50
51
52

Contenu connexe

Tendances

2021 free aws study material beginner guide
2021 free aws study material   beginner guide2021 free aws study material   beginner guide
2021 free aws study material beginner guidessuseradb53f
 
Serverless Geospatial Mobile Apps with AWS
Serverless Geospatial Mobile Apps with AWSServerless Geospatial Mobile Apps with AWS
Serverless Geospatial Mobile Apps with AWSAmazon Web Services
 
Design Patterns for Developers - Technical 201
Design Patterns for Developers - Technical 201Design Patterns for Developers - Technical 201
Design Patterns for Developers - Technical 201Amazon Web Services
 
Intro to AWS: Amazon EC2 and Compute Services
Intro to AWS: Amazon EC2 and Compute ServicesIntro to AWS: Amazon EC2 and Compute Services
Intro to AWS: Amazon EC2 and Compute ServicesAmazon Web Services
 
An Introduction to AWS - AWS Summit Tel Aviv 2017
An Introduction to AWS - AWS Summit Tel Aviv 2017An Introduction to AWS - AWS Summit Tel Aviv 2017
An Introduction to AWS - AWS Summit Tel Aviv 2017Amazon Web Services
 
DevOps on AWS: Deep Dive on Continuous Delivery and the AWS Developer Tools
DevOps on AWS: Deep Dive on Continuous Delivery and the AWS Developer ToolsDevOps on AWS: Deep Dive on Continuous Delivery and the AWS Developer Tools
DevOps on AWS: Deep Dive on Continuous Delivery and the AWS Developer ToolsAmazon Web Services
 
Getting Started with AWS Security
Getting Started with AWS SecurityGetting Started with AWS Security
Getting Started with AWS SecurityAmazon Web Services
 
AWSome Day Bangkok Opening Keynote
AWSome Day Bangkok Opening KeynoteAWSome Day Bangkok Opening Keynote
AWSome Day Bangkok Opening KeynoteAmazon Web Services
 
Security Innovations in the Cloud
Security Innovations in the CloudSecurity Innovations in the Cloud
Security Innovations in the CloudAmazon Web Services
 
Come costruire un'architettura Serverless nel Cloud AWS
Come costruire un'architettura Serverless nel Cloud AWSCome costruire un'architettura Serverless nel Cloud AWS
Come costruire un'architettura Serverless nel Cloud AWSAmazon Web Services
 
I servizi AWS per le applicazioni mobili: sviluppo, test e produzione
I servizi AWS per le applicazioni mobili: sviluppo, test e produzioneI servizi AWS per le applicazioni mobili: sviluppo, test e produzione
I servizi AWS per le applicazioni mobili: sviluppo, test e produzioneAmazon Web Services
 
Continuous Integration e Delivery per (r)innovare lo sviluppo software e la g...
Continuous Integration e Delivery per (r)innovare lo sviluppo software e la g...Continuous Integration e Delivery per (r)innovare lo sviluppo software e la g...
Continuous Integration e Delivery per (r)innovare lo sviluppo software e la g...Amazon Web Services
 
Securing Your Desktops with Amazon WorkSpaces - AWS Online Tech Talks
Securing Your Desktops with Amazon WorkSpaces - AWS Online Tech Talks Securing Your Desktops with Amazon WorkSpaces - AWS Online Tech Talks
Securing Your Desktops with Amazon WorkSpaces - AWS Online Tech Talks Amazon Web Services
 
AWS FSI Symposium 2017 NYC - Moving at the Speed of Serverless ft Broadridge
AWS FSI Symposium 2017 NYC - Moving at the Speed of Serverless ft BroadridgeAWS FSI Symposium 2017 NYC - Moving at the Speed of Serverless ft Broadridge
AWS FSI Symposium 2017 NYC - Moving at the Speed of Serverless ft BroadridgeAmazon Web Services
 
HSBC and AWS Day - AWS foundations
HSBC and AWS Day - AWS foundationsHSBC and AWS Day - AWS foundations
HSBC and AWS Day - AWS foundationsAmazon Web Services
 

Tendances (20)

2021 free aws study material beginner guide
2021 free aws study material   beginner guide2021 free aws study material   beginner guide
2021 free aws study material beginner guide
 
Enterprise Applications on AWS
Enterprise Applications on AWSEnterprise Applications on AWS
Enterprise Applications on AWS
 
Getting started with AWS
Getting started with AWSGetting started with AWS
Getting started with AWS
 
Networking and Security
Networking and SecurityNetworking and Security
Networking and Security
 
Serverless Geospatial Mobile Apps with AWS
Serverless Geospatial Mobile Apps with AWSServerless Geospatial Mobile Apps with AWS
Serverless Geospatial Mobile Apps with AWS
 
Design Patterns for Developers - Technical 201
Design Patterns for Developers - Technical 201Design Patterns for Developers - Technical 201
Design Patterns for Developers - Technical 201
 
Intro to AWS: Amazon EC2 and Compute Services
Intro to AWS: Amazon EC2 and Compute ServicesIntro to AWS: Amazon EC2 and Compute Services
Intro to AWS: Amazon EC2 and Compute Services
 
AWS Training and Certification
AWS Training and CertificationAWS Training and Certification
AWS Training and Certification
 
An Introduction to AWS - AWS Summit Tel Aviv 2017
An Introduction to AWS - AWS Summit Tel Aviv 2017An Introduction to AWS - AWS Summit Tel Aviv 2017
An Introduction to AWS - AWS Summit Tel Aviv 2017
 
Getting to MVP on AWS
Getting to MVP on AWSGetting to MVP on AWS
Getting to MVP on AWS
 
DevOps on AWS: Deep Dive on Continuous Delivery and the AWS Developer Tools
DevOps on AWS: Deep Dive on Continuous Delivery and the AWS Developer ToolsDevOps on AWS: Deep Dive on Continuous Delivery and the AWS Developer Tools
DevOps on AWS: Deep Dive on Continuous Delivery and the AWS Developer Tools
 
Getting Started with AWS Security
Getting Started with AWS SecurityGetting Started with AWS Security
Getting Started with AWS Security
 
AWSome Day Bangkok Opening Keynote
AWSome Day Bangkok Opening KeynoteAWSome Day Bangkok Opening Keynote
AWSome Day Bangkok Opening Keynote
 
Security Innovations in the Cloud
Security Innovations in the CloudSecurity Innovations in the Cloud
Security Innovations in the Cloud
 
Come costruire un'architettura Serverless nel Cloud AWS
Come costruire un'architettura Serverless nel Cloud AWSCome costruire un'architettura Serverless nel Cloud AWS
Come costruire un'architettura Serverless nel Cloud AWS
 
I servizi AWS per le applicazioni mobili: sviluppo, test e produzione
I servizi AWS per le applicazioni mobili: sviluppo, test e produzioneI servizi AWS per le applicazioni mobili: sviluppo, test e produzione
I servizi AWS per le applicazioni mobili: sviluppo, test e produzione
 
Continuous Integration e Delivery per (r)innovare lo sviluppo software e la g...
Continuous Integration e Delivery per (r)innovare lo sviluppo software e la g...Continuous Integration e Delivery per (r)innovare lo sviluppo software e la g...
Continuous Integration e Delivery per (r)innovare lo sviluppo software e la g...
 
Securing Your Desktops with Amazon WorkSpaces - AWS Online Tech Talks
Securing Your Desktops with Amazon WorkSpaces - AWS Online Tech Talks Securing Your Desktops with Amazon WorkSpaces - AWS Online Tech Talks
Securing Your Desktops with Amazon WorkSpaces - AWS Online Tech Talks
 
AWS FSI Symposium 2017 NYC - Moving at the Speed of Serverless ft Broadridge
AWS FSI Symposium 2017 NYC - Moving at the Speed of Serverless ft BroadridgeAWS FSI Symposium 2017 NYC - Moving at the Speed of Serverless ft Broadridge
AWS FSI Symposium 2017 NYC - Moving at the Speed of Serverless ft Broadridge
 
HSBC and AWS Day - AWS foundations
HSBC and AWS Day - AWS foundationsHSBC and AWS Day - AWS foundations
HSBC and AWS Day - AWS foundations
 

Similaire à Building Open Source Communities for AWS Serverless Developer Tools

Montréal AWS Users United: Let's go Serverless!
Montréal AWS Users United: Let's go Serverless!Montréal AWS Users United: Let's go Serverless!
Montréal AWS Users United: Let's go Serverless!Daniel Zivkovic
 
Jfokus Workshop: Code in the Cloud for the Cloud
Jfokus Workshop: Code in the Cloud for the CloudJfokus Workshop: Code in the Cloud for the Cloud
Jfokus Workshop: Code in the Cloud for the CloudLauren Hayward Schaefer
 
Platform engineering 101
Platform engineering 101Platform engineering 101
Platform engineering 101Sander Knape
 
33 Software Development Tools that Drive Dialexa’s Success
33 Software Development Tools that Drive Dialexa’s Success33 Software Development Tools that Drive Dialexa’s Success
33 Software Development Tools that Drive Dialexa’s SuccessDialexa
 
Workshop Trends In Open Source Tech 1 20 10
Workshop Trends In Open Source Tech 1 20 10Workshop Trends In Open Source Tech 1 20 10
Workshop Trends In Open Source Tech 1 20 10NuRelm
 
Open Stack Cloud Services
Open Stack Cloud ServicesOpen Stack Cloud Services
Open Stack Cloud ServicesSaurabh Gupta
 
Serverless architectures-with-aws-lambda
Serverless architectures-with-aws-lambdaServerless architectures-with-aws-lambda
Serverless architectures-with-aws-lambdasaifam
 
Building CI/CD Pipelines for Serverless Applications
Building CI/CD Pipelines for Serverless ApplicationsBuilding CI/CD Pipelines for Serverless Applications
Building CI/CD Pipelines for Serverless ApplicationsDaniel Zivkovic
 
Building Scalable Web Applications Using Microservices Architecture and NodeJ...
Building Scalable Web Applications Using Microservices Architecture and NodeJ...Building Scalable Web Applications Using Microservices Architecture and NodeJ...
Building Scalable Web Applications Using Microservices Architecture and NodeJ...NodejsFoundation
 
Building CI/CD Pipelines for Serverless Applications - SRV302 - re:Invent 2017
Building CI/CD Pipelines for Serverless Applications - SRV302 - re:Invent 2017Building CI/CD Pipelines for Serverless Applications - SRV302 - re:Invent 2017
Building CI/CD Pipelines for Serverless Applications - SRV302 - re:Invent 2017Amazon Web Services
 
Building Scalable Web Applications using Microservices Architecture and NodeJ...
Building Scalable Web Applications using Microservices Architecture and NodeJ...Building Scalable Web Applications using Microservices Architecture and NodeJ...
Building Scalable Web Applications using Microservices Architecture and NodeJ...Mitoc Group
 
DevOps at Amazon: A Look at Our Tools and Processes
DevOps at Amazon: A Look at Our Tools and ProcessesDevOps at Amazon: A Look at Our Tools and Processes
DevOps at Amazon: A Look at Our Tools and ProcessesAmazon Web Services
 
AWS re:Invent 2016: Open Source at AWS—Contributions, Support, and Engagement...
AWS re:Invent 2016: Open Source at AWS—Contributions, Support, and Engagement...AWS re:Invent 2016: Open Source at AWS—Contributions, Support, and Engagement...
AWS re:Invent 2016: Open Source at AWS—Contributions, Support, and Engagement...Amazon Web Services
 
20180111 we bde-bs - serverless url shortener
20180111   we bde-bs - serverless url shortener20180111   we bde-bs - serverless url shortener
20180111 we bde-bs - serverless url shortenerLuca Bianchi
 
Aws vs azure bakeoff
Aws vs azure bakeoffAws vs azure bakeoff
Aws vs azure bakeoffSoHo Dragon
 
DevOps Friendly Doc Publishing for APIs & Microservices
DevOps Friendly Doc Publishing for APIs & MicroservicesDevOps Friendly Doc Publishing for APIs & Microservices
DevOps Friendly Doc Publishing for APIs & MicroservicesSonatype
 
Architecting Cloud Computing Solutions with Java [1.1]
Architecting Cloud Computing Solutions with Java [1.1]Architecting Cloud Computing Solutions with Java [1.1]
Architecting Cloud Computing Solutions with Java [1.1]Otávio Santana
 
DevOps on AWS: Deep Dive on Continuous Delivery and the AWS Developer Tools
DevOps on AWS: Deep Dive on Continuous Delivery and the AWS Developer ToolsDevOps on AWS: Deep Dive on Continuous Delivery and the AWS Developer Tools
DevOps on AWS: Deep Dive on Continuous Delivery and the AWS Developer ToolsAmazon Web Services
 
AWS re:Invent 2020 Serverless Recap
AWS re:Invent 2020 Serverless RecapAWS re:Invent 2020 Serverless Recap
AWS re:Invent 2020 Serverless RecapDaniel Zivkovic
 
Introduction to AWS CodeStar: Quickly develop, build, and deploy applications...
Introduction to AWS CodeStar: Quickly develop, build, and deploy applications...Introduction to AWS CodeStar: Quickly develop, build, and deploy applications...
Introduction to AWS CodeStar: Quickly develop, build, and deploy applications...Amazon Web Services
 

Similaire à Building Open Source Communities for AWS Serverless Developer Tools (20)

Montréal AWS Users United: Let's go Serverless!
Montréal AWS Users United: Let's go Serverless!Montréal AWS Users United: Let's go Serverless!
Montréal AWS Users United: Let's go Serverless!
 
Jfokus Workshop: Code in the Cloud for the Cloud
Jfokus Workshop: Code in the Cloud for the CloudJfokus Workshop: Code in the Cloud for the Cloud
Jfokus Workshop: Code in the Cloud for the Cloud
 
Platform engineering 101
Platform engineering 101Platform engineering 101
Platform engineering 101
 
33 Software Development Tools that Drive Dialexa’s Success
33 Software Development Tools that Drive Dialexa’s Success33 Software Development Tools that Drive Dialexa’s Success
33 Software Development Tools that Drive Dialexa’s Success
 
Workshop Trends In Open Source Tech 1 20 10
Workshop Trends In Open Source Tech 1 20 10Workshop Trends In Open Source Tech 1 20 10
Workshop Trends In Open Source Tech 1 20 10
 
Open Stack Cloud Services
Open Stack Cloud ServicesOpen Stack Cloud Services
Open Stack Cloud Services
 
Serverless architectures-with-aws-lambda
Serverless architectures-with-aws-lambdaServerless architectures-with-aws-lambda
Serverless architectures-with-aws-lambda
 
Building CI/CD Pipelines for Serverless Applications
Building CI/CD Pipelines for Serverless ApplicationsBuilding CI/CD Pipelines for Serverless Applications
Building CI/CD Pipelines for Serverless Applications
 
Building Scalable Web Applications Using Microservices Architecture and NodeJ...
Building Scalable Web Applications Using Microservices Architecture and NodeJ...Building Scalable Web Applications Using Microservices Architecture and NodeJ...
Building Scalable Web Applications Using Microservices Architecture and NodeJ...
 
Building CI/CD Pipelines for Serverless Applications - SRV302 - re:Invent 2017
Building CI/CD Pipelines for Serverless Applications - SRV302 - re:Invent 2017Building CI/CD Pipelines for Serverless Applications - SRV302 - re:Invent 2017
Building CI/CD Pipelines for Serverless Applications - SRV302 - re:Invent 2017
 
Building Scalable Web Applications using Microservices Architecture and NodeJ...
Building Scalable Web Applications using Microservices Architecture and NodeJ...Building Scalable Web Applications using Microservices Architecture and NodeJ...
Building Scalable Web Applications using Microservices Architecture and NodeJ...
 
DevOps at Amazon: A Look at Our Tools and Processes
DevOps at Amazon: A Look at Our Tools and ProcessesDevOps at Amazon: A Look at Our Tools and Processes
DevOps at Amazon: A Look at Our Tools and Processes
 
AWS re:Invent 2016: Open Source at AWS—Contributions, Support, and Engagement...
AWS re:Invent 2016: Open Source at AWS—Contributions, Support, and Engagement...AWS re:Invent 2016: Open Source at AWS—Contributions, Support, and Engagement...
AWS re:Invent 2016: Open Source at AWS—Contributions, Support, and Engagement...
 
20180111 we bde-bs - serverless url shortener
20180111   we bde-bs - serverless url shortener20180111   we bde-bs - serverless url shortener
20180111 we bde-bs - serverless url shortener
 
Aws vs azure bakeoff
Aws vs azure bakeoffAws vs azure bakeoff
Aws vs azure bakeoff
 
DevOps Friendly Doc Publishing for APIs & Microservices
DevOps Friendly Doc Publishing for APIs & MicroservicesDevOps Friendly Doc Publishing for APIs & Microservices
DevOps Friendly Doc Publishing for APIs & Microservices
 
Architecting Cloud Computing Solutions with Java [1.1]
Architecting Cloud Computing Solutions with Java [1.1]Architecting Cloud Computing Solutions with Java [1.1]
Architecting Cloud Computing Solutions with Java [1.1]
 
DevOps on AWS: Deep Dive on Continuous Delivery and the AWS Developer Tools
DevOps on AWS: Deep Dive on Continuous Delivery and the AWS Developer ToolsDevOps on AWS: Deep Dive on Continuous Delivery and the AWS Developer Tools
DevOps on AWS: Deep Dive on Continuous Delivery and the AWS Developer Tools
 
AWS re:Invent 2020 Serverless Recap
AWS re:Invent 2020 Serverless RecapAWS re:Invent 2020 Serverless Recap
AWS re:Invent 2020 Serverless Recap
 
Introduction to AWS CodeStar: Quickly develop, build, and deploy applications...
Introduction to AWS CodeStar: Quickly develop, build, and deploy applications...Introduction to AWS CodeStar: Quickly develop, build, and deploy applications...
Introduction to AWS CodeStar: Quickly develop, build, and deploy applications...
 

Plus de Amazon Web Services

Come costruire servizi di Forecasting sfruttando algoritmi di ML e deep learn...
Come costruire servizi di Forecasting sfruttando algoritmi di ML e deep learn...Come costruire servizi di Forecasting sfruttando algoritmi di ML e deep learn...
Come costruire servizi di Forecasting sfruttando algoritmi di ML e deep learn...Amazon Web Services
 
Big Data per le Startup: come creare applicazioni Big Data in modalità Server...
Big Data per le Startup: come creare applicazioni Big Data in modalità Server...Big Data per le Startup: come creare applicazioni Big Data in modalità Server...
Big Data per le Startup: come creare applicazioni Big Data in modalità Server...Amazon Web Services
 
Esegui pod serverless con Amazon EKS e AWS Fargate
Esegui pod serverless con Amazon EKS e AWS FargateEsegui pod serverless con Amazon EKS e AWS Fargate
Esegui pod serverless con Amazon EKS e AWS FargateAmazon Web Services
 
Costruire Applicazioni Moderne con AWS
Costruire Applicazioni Moderne con AWSCostruire Applicazioni Moderne con AWS
Costruire Applicazioni Moderne con AWSAmazon Web Services
 
Come spendere fino al 90% in meno con i container e le istanze spot
Come spendere fino al 90% in meno con i container e le istanze spot Come spendere fino al 90% in meno con i container e le istanze spot
Come spendere fino al 90% in meno con i container e le istanze spot Amazon Web Services
 
Rendi unica l’offerta della tua startup sul mercato con i servizi Machine Lea...
Rendi unica l’offerta della tua startup sul mercato con i servizi Machine Lea...Rendi unica l’offerta della tua startup sul mercato con i servizi Machine Lea...
Rendi unica l’offerta della tua startup sul mercato con i servizi Machine Lea...Amazon Web Services
 
OpsWorks Configuration Management: automatizza la gestione e i deployment del...
OpsWorks Configuration Management: automatizza la gestione e i deployment del...OpsWorks Configuration Management: automatizza la gestione e i deployment del...
OpsWorks Configuration Management: automatizza la gestione e i deployment del...Amazon Web Services
 
Microsoft Active Directory su AWS per supportare i tuoi Windows Workloads
Microsoft Active Directory su AWS per supportare i tuoi Windows WorkloadsMicrosoft Active Directory su AWS per supportare i tuoi Windows Workloads
Microsoft Active Directory su AWS per supportare i tuoi Windows WorkloadsAmazon Web Services
 
Database Oracle e VMware Cloud on AWS i miti da sfatare
Database Oracle e VMware Cloud on AWS i miti da sfatareDatabase Oracle e VMware Cloud on AWS i miti da sfatare
Database Oracle e VMware Cloud on AWS i miti da sfatareAmazon Web Services
 
Crea la tua prima serverless ledger-based app con QLDB e NodeJS
Crea la tua prima serverless ledger-based app con QLDB e NodeJSCrea la tua prima serverless ledger-based app con QLDB e NodeJS
Crea la tua prima serverless ledger-based app con QLDB e NodeJSAmazon Web Services
 
API moderne real-time per applicazioni mobili e web
API moderne real-time per applicazioni mobili e webAPI moderne real-time per applicazioni mobili e web
API moderne real-time per applicazioni mobili e webAmazon Web Services
 
Database Oracle e VMware Cloud™ on AWS: i miti da sfatare
Database Oracle e VMware Cloud™ on AWS: i miti da sfatareDatabase Oracle e VMware Cloud™ on AWS: i miti da sfatare
Database Oracle e VMware Cloud™ on AWS: i miti da sfatareAmazon Web Services
 
Tools for building your MVP on AWS
Tools for building your MVP on AWSTools for building your MVP on AWS
Tools for building your MVP on AWSAmazon Web Services
 
How to Build a Winning Pitch Deck
How to Build a Winning Pitch DeckHow to Build a Winning Pitch Deck
How to Build a Winning Pitch DeckAmazon Web Services
 
Building a web application without servers
Building a web application without serversBuilding a web application without servers
Building a web application without serversAmazon Web Services
 
AWS_HK_StartupDay_Building Interactive websites while automating for efficien...
AWS_HK_StartupDay_Building Interactive websites while automating for efficien...AWS_HK_StartupDay_Building Interactive websites while automating for efficien...
AWS_HK_StartupDay_Building Interactive websites while automating for efficien...Amazon Web Services
 
Introduzione a Amazon Elastic Container Service
Introduzione a Amazon Elastic Container ServiceIntroduzione a Amazon Elastic Container Service
Introduzione a Amazon Elastic Container ServiceAmazon Web Services
 

Plus de Amazon Web Services (20)

Come costruire servizi di Forecasting sfruttando algoritmi di ML e deep learn...
Come costruire servizi di Forecasting sfruttando algoritmi di ML e deep learn...Come costruire servizi di Forecasting sfruttando algoritmi di ML e deep learn...
Come costruire servizi di Forecasting sfruttando algoritmi di ML e deep learn...
 
Big Data per le Startup: come creare applicazioni Big Data in modalità Server...
Big Data per le Startup: come creare applicazioni Big Data in modalità Server...Big Data per le Startup: come creare applicazioni Big Data in modalità Server...
Big Data per le Startup: come creare applicazioni Big Data in modalità Server...
 
Esegui pod serverless con Amazon EKS e AWS Fargate
Esegui pod serverless con Amazon EKS e AWS FargateEsegui pod serverless con Amazon EKS e AWS Fargate
Esegui pod serverless con Amazon EKS e AWS Fargate
 
Costruire Applicazioni Moderne con AWS
Costruire Applicazioni Moderne con AWSCostruire Applicazioni Moderne con AWS
Costruire Applicazioni Moderne con AWS
 
Come spendere fino al 90% in meno con i container e le istanze spot
Come spendere fino al 90% in meno con i container e le istanze spot Come spendere fino al 90% in meno con i container e le istanze spot
Come spendere fino al 90% in meno con i container e le istanze spot
 
Open banking as a service
Open banking as a serviceOpen banking as a service
Open banking as a service
 
Rendi unica l’offerta della tua startup sul mercato con i servizi Machine Lea...
Rendi unica l’offerta della tua startup sul mercato con i servizi Machine Lea...Rendi unica l’offerta della tua startup sul mercato con i servizi Machine Lea...
Rendi unica l’offerta della tua startup sul mercato con i servizi Machine Lea...
 
OpsWorks Configuration Management: automatizza la gestione e i deployment del...
OpsWorks Configuration Management: automatizza la gestione e i deployment del...OpsWorks Configuration Management: automatizza la gestione e i deployment del...
OpsWorks Configuration Management: automatizza la gestione e i deployment del...
 
Microsoft Active Directory su AWS per supportare i tuoi Windows Workloads
Microsoft Active Directory su AWS per supportare i tuoi Windows WorkloadsMicrosoft Active Directory su AWS per supportare i tuoi Windows Workloads
Microsoft Active Directory su AWS per supportare i tuoi Windows Workloads
 
Computer Vision con AWS
Computer Vision con AWSComputer Vision con AWS
Computer Vision con AWS
 
Database Oracle e VMware Cloud on AWS i miti da sfatare
Database Oracle e VMware Cloud on AWS i miti da sfatareDatabase Oracle e VMware Cloud on AWS i miti da sfatare
Database Oracle e VMware Cloud on AWS i miti da sfatare
 
Crea la tua prima serverless ledger-based app con QLDB e NodeJS
Crea la tua prima serverless ledger-based app con QLDB e NodeJSCrea la tua prima serverless ledger-based app con QLDB e NodeJS
Crea la tua prima serverless ledger-based app con QLDB e NodeJS
 
API moderne real-time per applicazioni mobili e web
API moderne real-time per applicazioni mobili e webAPI moderne real-time per applicazioni mobili e web
API moderne real-time per applicazioni mobili e web
 
Database Oracle e VMware Cloud™ on AWS: i miti da sfatare
Database Oracle e VMware Cloud™ on AWS: i miti da sfatareDatabase Oracle e VMware Cloud™ on AWS: i miti da sfatare
Database Oracle e VMware Cloud™ on AWS: i miti da sfatare
 
Tools for building your MVP on AWS
Tools for building your MVP on AWSTools for building your MVP on AWS
Tools for building your MVP on AWS
 
How to Build a Winning Pitch Deck
How to Build a Winning Pitch DeckHow to Build a Winning Pitch Deck
How to Build a Winning Pitch Deck
 
Building a web application without servers
Building a web application without serversBuilding a web application without servers
Building a web application without servers
 
Fundraising Essentials
Fundraising EssentialsFundraising Essentials
Fundraising Essentials
 
AWS_HK_StartupDay_Building Interactive websites while automating for efficien...
AWS_HK_StartupDay_Building Interactive websites while automating for efficien...AWS_HK_StartupDay_Building Interactive websites while automating for efficien...
AWS_HK_StartupDay_Building Interactive websites while automating for efficien...
 
Introduzione a Amazon Elastic Container Service
Introduzione a Amazon Elastic Container ServiceIntroduzione a Amazon Elastic Container Service
Introduzione a Amazon Elastic Container Service
 

Building Open Source Communities for AWS Serverless Developer Tools

  • 1. Hello I am Sanath Kumar Ramesh. I am a software engineer in AWS Serverless organization where I lead developer tools initiatives. I develop tools to simplify the process of building serverless applications. I built AWS SAM and SAM CLI, open source toolchains to help developers be more productive. 1
  • 2. A brief background about me.. I started designing computer processors, then built parts of Windows kernel, and graduated up the stack to build web services at startups followed by AWS. In 2016 I joined AWS Lambda team, few months after the service was generally available. I worked on the inner guts of the service but quickly found my passion with developer tools. I have been a huge supporter and evangelist of open source since college. I started the first OSS club in undergrad which is still running successfully. I open sourced the underlying implementation of AWS SAM, built SAM CLI, and work on 100% open source software at AWS. In this talk, I will introduce you to the amazing open source community of serverless developer tools which I have been a part of since 2016. For the longest time, I wasn’t conscious of the scale and potential of this community, until I embarked on my own journey of open source two strategic products at AWS. We will talk about the broader open source AWS community first before diving deep into serverless. 2
  • 3. Happy Birthday Open Source !! Twenty years! How amazing is that ?? 3
  • 4. At AWS, we have been a huge believer and supporter of Open Source software. Did you know that some of our foundational services are built on top popular OSS projects - EC2 uses Xen, RDS supports MySQL, EKS supports Kubernetes, to name a few. We go beyond just support and use software, we help build communities. Grow communities: Communities of developers - AWS joined open source foundations like CNCF, OSI, Apache Software Foundation, Linux Foundation etc. We have supported some of our customers like Netflix and Capital One to help build their open source programs. Improve code: We have hunderds of open source projects on our Github repositories, contribute to projects like Kubernetes, Apache MX Net etc. Increase Contributions: Internally, we have several initiatives that make it easier for employees to contribute open source software. For example, using Standard Licenses (Apache or MIT for most part), AWS credits for academic and non-profit projects, Encouraging external communications etc 100s of open source repos on Github: Where we have open sourced everything from SDKs, client side tools, IDE plugins, full blown apps, to core components of some of our services such as s2n. 4
  • 5. • We are motivated to collaborate by many of our partners and customers • Open Source projects helps us to innovate • Scaling AWS services around open source helps us meet customer needs • Scaling for open data help customers build services they need (Customers! Customers = community. ) 5
  • 6. 6
  • 7. 2016 projects, new contributions to apache mxnet Contibution volume is growing every year 7
  • 8. 2017 projects so far, new ones in orange, notably our first contributions to Kubernetes 8
  • 9. 9
  • 10. That was the broader AWS landscape. Let's drill down to the focus of today's talk - Serverless! By show of hands, how many of you have heard of the term Serverless? How many have built serverless apps - hobby, for work, anything? How many of you build & maintain serverless apps in production today? Fantastic! What's all the buzz about serverless? Why do we care? And, Why is Open Source in serverless such a big deal? This is going to be the focus of our talk. 10
  • 11. No servers to mange, never pay for idle, apps you build scale with usage automatically, and highly available & fault tolerant. AWS Lambda offers a serverless computing platform, API Gateway offers an hosted API service, DynamoDB offers a database etc. For any of you that have built apps on the cloud, serverless offers several system properties that you otherwise have to build yourself. 11
  • 12. Secret sauce? The things you usually write code for, are now expressed as simple, static, configurations. For example, in a standard cloud application, you would write code for your business logic, API interfaces, and all plumbing between them. Resources like databases, storage, permissions etc are expressed as configurations. I have written the code for user authentication so many times now that I almost decided to quit programming when I wrote user auth for my hobby project recently. In a serverless app, a lot of the undifferentiated heavy lifting is handled by services for you. You write a static configuration file to configure the API endpoints, user auth, workflows etc and the services takes care of running them for you. This means you focus primarily on the business logic and write some plumbing here and there. This also means, all your code-first developer tools such as IDEs, test harnesses, build systems, deployment engines, etc need to support expressing the configuration. 12
  • 13. This was an opportunity and a whole slew of developer tools emerged in the community. Guess what? Nearly every one of them is open source! Serverless Framework is the biggest and most popular one by far, going by open source community stats. 26k stars on Github, 80k downloads on NPM, > 450 contributors, and has a rich plugin ecosystem. Architect provides a simple DSL to express your entire app with. Paired with their Javascript library, you can write regular NodeJS apps that will be deployed as serverless apps. Apex provides is a simple and opinionated tool to manage your Lambda functions. It can be combined with infrastructure provisioning tools such as Terraform to manage your entire app. Sparta allows you to write Go code that, when deployed, will automatically provision & configure underlying resources for you. Pulumi similarly supports automatic resource provisionng from code, but they support a variety of programing languages, and non-serverless resources too. Claudia.JS is a javascript library to create serverless apps using Nodejs. 13
  • 14. Zappa is similar to ClaudiaJS but for Python. All of these tools provide varying levels of convenience, target certain programming languages, or specialize in certain tasks. 13
  • 15. AWS SAM is a language agnostic way of creating, testing, deploying, and managing serverless apps. Shameless plug: I built SAM and the accompanying CLI. We are going to talk a lot about SAM in today's talk. <<talk about other AWS dev tools as much as time fits>> 14
  • 16. 15
  • 17. 16
  • 18. 17
  • 19. 18
  • 20. 19
  • 21. 20
  • 22. Each of these projects have their own vibrant ecosystem of hundreds of developers working alongside with AWS. How did this all happen? I can’t speak for other projects, but here is a case study of how I created a community around SAM. 21
  • 23. 22
  • 24. 23
  • 25. Shameless Plug SAM is fully open source and includes a companion CLI tool to locally test & run serverless aplications 24
  • 26. 25
  • 27. 26
  • 28. 27
  • 29. 28
  • 30. - Open Specification means: No code but just one spec file on Github. Think of this as essentially our docs are on Github. - Excellent traction on Github. Great community of developers for the first few months. 29
  • 31. Month or two after launch, someone filed an issue asking us to Open Source the implementation. 30
  • 32. 31
  • 33. 32
  • 34. 33
  • 35. 34
  • 36. 35
  • 37. 36
  • 38. 37
  • 39. 38
  • 40. Kubernetes: Amazon EKS Hadoop: Amazon EMR Key difference with SAM: • We didn’t open source EKS, but instead support Kubernetes as part of the EKS service. This is different from SAM where we will open source the entire service • We are the producer of this product and we run it as a service too.. 39
  • 41. 40
  • 42. 41
  • 43. These questions themselves focus on the “software”, but once I answered these questions, it became apparent that I was “building a community” and not just another piece of software. The process of running an open source project naturally drove me to build communities. Goal is to allow the community to build software with full assistance from AWS. Combined the agility of open source community and quality assurance practices of AWS, we believe this model can produce amazing software. 42
  • 44. When I proposed open source SAM, my manager asked me how many ppl do I want in this team. I said 100! Only a handful of engineers directly come from AWS, but a large majority are from the community. To build this team, we needed open communication where everyone feels connected and a part of same team. Hence Slack with ~1000 ppl including customers, devs, etc. Builds Bonds: • Customers feel like a part of the product, talk directly to product engineers without several layers of support/management to go thru. They feel more empowered to use our product 43
  • 45. Because development is open: • Builds Trust • Reduces possibility of malicious code • Commercial friendly Apache 2.0 license invites the community to build innovative solutions on top of SAM. Like for example what Stackery did. Community Protects against: • MALICIOUS code • Poor quality code 44
  • 46. We have a team, we have a product that is open. Next is to keep everyone on same page of what to work on. In an internal team, product & project managers come up with this list. We use Github Labels & Gitub Issues to keep everyone informed of the priority. • Community is on same page on what to work on next • Visibility into top-issues • Provides customers a clear visibility into how we think about this problem. They can obviously provide feedback to change our mind. This is refreshing instead of the opaque “we have this in our backlog somewhere” 45
  • 47. Running a Service: * We take care of deployments on behalf of the community. At AWS, everything is a service and we have deep knowledge in deploying services. • Automation – all AWS regions • Safe Guards – Beta, Gamma, Auto-rollback, Alarms, Canaries • Bleeding edge vs Stability: We guarantee stability of code (back compatibility etc) by doing deploymsents with several safe guards. Bleeding edge customers can consume source directly • Open communication on Github Releases to mark global availability 46
  • 48. * Oncall: To make sure Github Issues & PRs don’t’ go stale, we have a oncall rotation where on person is in-charge per week to manage the repository. Obviously any team member can operate the repo at anytime, but having this person ensures the community is always engaged. • Operational Review: After every week, we have an internal meeting to discuss highlights of the oncall. This provides the entire team visibility into product, community health, emergent issues or anything that could change priorities. • We bring our DevOps experience from AWS into an open source product to make sure both the community & the product stays healthy. • Information from each week’s operational review is visible to broader AWS Serverless organization. • We often trade notes with other popular OSS projects like AWS CLI, SDKs to make sure we don’t repeat same mistakes and continue to raise the bar. 47
  • 49. We support the community by automatically providing them with the best practices we have learnt at AWS over several years. 48
  • 50. 49
  • 52. 51
  • 53. 52