Nflx ds meetup-5_dec2012

•

2 j'aime•1,279 vues

Jason Brown

2013 Roadmap of Cassandra development at Netflix. Presented to the SV Cassandra User Group.

Technologie

2013
Roadmap:

Cassandra
at
Ne4lix

5
December
2012

Jason
Brown

Senior
So@ware
Engineer
@
Ne4lix

@jasobrown

jasedbrown@gmail.com

hGp://www.linkedin.com/in/jasedbrown

Cassandra
Deployment
Numbers

> 50 Production clusters
> 800 Total nodes
15 Number of multi-region clusters
4 Max regions for one cluster
144 Largest Cassandra cluster
4 Size of Cassandra dev-ops team

Record
reads/writes

1.2
million
writes/second

300k
reads/second

72
node
cluster
–
m2.4xlarge

2013
Cassandra
Roadmap

• Priam

• Explorers

• Compliance/security

• Data
as
a
Service

Priam
updates

• C*
1.2

• Virtual
nodes
integraSon

Vnodes
info

• @samoverton
(Acunu)

• hGp://www.acunu.com/2/post/2012/07/
virtual-‐nodes-‐strategies.html
(arScle)

• (images
borrowed
from
this
great
blog
post)

• hGp://tweets.acunu.com/tweets/
240082932524523520
(video)

Priam
updates

• C*
1.2

• Virtual
nodes
integraSon

• Grow
a
cluster
without
doubling

• Auto-‐scale
c*
cluster
with
load

• Commit
log
backups
(CASSANDRA-‐3690)

• Accelerated
BI
integraSon

Explorers

• Web-‐based
UI

• Cassandra
plugin
for
admin/query

• Coming
soon
to
a
github
repo
near
you

Compliance

Everyone
to
the
cloud

…
even
billing!

Security,
encrypSon,
audiSng,
logging

Lots
of
details,
lots
of
quesSons
…

Data
as
a
Service

• “Gimme
my
data”

• RESTful
interface
to
get/set
data

• More
than
just
C*

• Cluster
management
(consolidaSon)

New
Year’s
ResoluSons

• Learn
CQL3

• Make
vnodes
rock
in
EC2

• More
open
source
contribuSons

• Watch
more
Ne4lix
..
Rather
than
just
code
it

Recommandé

JOSA TechTalks - Downgrade your CostsJordan Open Source Association

AWS to Bare Metal: Motivation, Pitfalls, and ResultsMongoDB

Cloud computing fundamentals with Microsoft AzureRadoslav Gatev

How to reduce hosting costs for Redis based applications on JavaNikita Koksharov

HPC Cloud - SURF Research Boot CampNuno Ferreira

[AWSKRUG&JAWS-UG Meetup #1] 70% Cost Reduction with On-demand resizingAWSKRUG - AWS한국사용자모임

Automating Cloud Cluster Deployment: Beyond the BookBill Havanki

beSharp a serverless approach to big data on awsClaudio Pontili

Recommandé

JOSA TechTalks - Downgrade your CostsJordan Open Source Association

AWS to Bare Metal: Motivation, Pitfalls, and ResultsMongoDB

Cloud computing fundamentals with Microsoft AzureRadoslav Gatev

How to reduce hosting costs for Redis based applications on JavaNikita Koksharov

HPC Cloud - SURF Research Boot CampNuno Ferreira

[AWSKRUG&JAWS-UG Meetup #1] 70% Cost Reduction with On-demand resizingAWSKRUG - AWS한국사용자모임

Automating Cloud Cluster Deployment: Beyond the BookBill Havanki

beSharp a serverless approach to big data on awsClaudio Pontili

Icinga Director and vSphereDB - how they play together - Icinga Camp Zurich 2019Icinga

Apache Cassandra ManagementInstaclustr

AWS re:Invent 2016 Recap: What Happened, What It MeansRightScale

Current State of Icinga - Icinga Camp Zurich 2019Icinga

Kubernetes at Spreadshirt - First steps to productionJens Hadlich

Cnam azure 2014 web sites et integration continueAymeric Weinbach

Production Ready Docker Stack for Alfresco DeploymentsRoxana Angheluta

MongoDB World 2016: Get MEAN and Lean with MongoDB and KubernetesMongoDB

On CloudStack, Docker, Kubernetes, and Big Data…Oh my ! By Sebastien Goasguen...Radhika Puthiyetath

Scaling MongoDB in the cloud with Microsoft AzureIvan Fioravanti

[GS네오텍] Google Kubernetes Engine GS Neotek

The Cloud Native StackQAware GmbH

Ali AlShehab Insight Demo LetsHangAli AlShehab

Containers and CloudStackShapeBlue

Jenkins, jclouds, CloudStack, and CentOS by David Nalleybuildacloud

Taking Cloud to Extremes: Scaled-down, Highly Available, and Mission-critical...Altoros

Cnam azure 2014 storageAymeric Weinbach

Rancher + Kubernetes; Stories from the trenchesGreg Cockburn

Cloud data center and openstackAndrew Yongjoon Kong

2012 re:Invent Netflix: embracing the cloud finalYuryIzrailevsky

NPR Presentation at Wolfram Data Summit 2010Daniel Jacobson

Cloud Application Security: Lessons LearnedJason Chan

Contenu connexe

Tendances

Icinga Director and vSphereDB - how they play together - Icinga Camp Zurich 2019Icinga

Apache Cassandra ManagementInstaclustr

AWS re:Invent 2016 Recap: What Happened, What It MeansRightScale

Current State of Icinga - Icinga Camp Zurich 2019Icinga

Kubernetes at Spreadshirt - First steps to productionJens Hadlich

Cnam azure 2014 web sites et integration continueAymeric Weinbach

Production Ready Docker Stack for Alfresco DeploymentsRoxana Angheluta

MongoDB World 2016: Get MEAN and Lean with MongoDB and KubernetesMongoDB

On CloudStack, Docker, Kubernetes, and Big Data…Oh my ! By Sebastien Goasguen...Radhika Puthiyetath

Scaling MongoDB in the cloud with Microsoft AzureIvan Fioravanti

[GS네오텍] Google Kubernetes Engine GS Neotek

The Cloud Native StackQAware GmbH

Ali AlShehab Insight Demo LetsHangAli AlShehab

Containers and CloudStackShapeBlue

Jenkins, jclouds, CloudStack, and CentOS by David Nalleybuildacloud

Taking Cloud to Extremes: Scaled-down, Highly Available, and Mission-critical...Altoros

Cnam azure 2014 storageAymeric Weinbach

Rancher + Kubernetes; Stories from the trenchesGreg Cockburn

Cloud data center and openstackAndrew Yongjoon Kong

Tendances (19)

Icinga Director and vSphereDB - how they play together - Icinga Camp Zurich 2019

Apache Cassandra Management

AWS re:Invent 2016 Recap: What Happened, What It Means

Current State of Icinga - Icinga Camp Zurich 2019

Kubernetes at Spreadshirt - First steps to production

Cnam azure 2014 web sites et integration continue

Production Ready Docker Stack for Alfresco Deployments

MongoDB World 2016: Get MEAN and Lean with MongoDB and Kubernetes

On CloudStack, Docker, Kubernetes, and Big Data…Oh my ! By Sebastien Goasguen...

Scaling MongoDB in the cloud with Microsoft Azure

[GS네오텍] Google Kubernetes Engine

The Cloud Native Stack

Ali AlShehab Insight Demo LetsHang

Containers and CloudStack

Jenkins, jclouds, CloudStack, and CentOS by David Nalley

Taking Cloud to Extremes: Scaled-down, Highly Available, and Mission-critical...

Cnam azure 2014 storage

Rancher + Kubernetes; Stories from the trenches

Cloud data center and openstack

En vedette

2012 re:Invent Netflix: embracing the cloud finalYuryIzrailevsky

NPR Presentation at Wolfram Data Summit 2010Daniel Jacobson

Cloud Application Security: Lessons LearnedJason Chan

Maintaining reliability in an unreliable worldJeremy Edberg

Virtualization: Security and IT Audit PerspectivesJason Chan

The intersection of culture and OperationsJeremy Edberg

SSL Certificate Expiration and Howler Monkey's Inceptionroyrapoport

APIs for Internal Audiences - Netflix - App Dev ConferenceDaniel Jacobson

Netflix API : BAPI 2011 Presentation : SFDaniel Jacobson

Surge 2013: Maximizing Scalability, Resiliency, and Engineering Velocity in t...Coburn Watson

Asgard: Using Grails to Deploy Netflix to AWS (Extended Slides)Joe Sondow

Fault Tolerance in a High Volume, Distributed SystemBen Christensen

Cassandra from the trenches: migrating Netflix (update)Jason Brown

Performance and Fault Tolerance for the Netflix API Ben Christensen

Why API? - Business of APIs ConferenceDaniel Jacobson

Scaling the Netflix API - OSCONDaniel Jacobson

Resilience and Security @ Scale: Lessons LearnedJason Chan

Techniques for Scaling the Netflix API - QCon SFDaniel Jacobson

Gluecon 2013 - Netflix Cloud Native Tutorial Details (part 2)Adrian Cockcroft

En vedette (20)

2012 re:Invent Netflix: embracing the cloud final

NPR Presentation at Wolfram Data Summit 2010

Cloud Application Security: Lessons Learned

Maintaining reliability in an unreliable world

Virtualization: Security and IT Audit Perspectives

The intersection of culture and Operations

SSL Certificate Expiration and Howler Monkey's Inception

APIs for Internal Audiences - Netflix - App Dev Conference

Netflix API : BAPI 2011 Presentation : SF

Surge 2013: Maximizing Scalability, Resiliency, and Engineering Velocity in t...

Asgard: Using Grails to Deploy Netflix to AWS (Extended Slides)

Fault Tolerance in a High Volume, Distributed System

Cassandra from the trenches: migrating Netflix (update)

Performance and Fault Tolerance for the Netflix API

Why API? - Business of APIs Conference

Scaling the Netflix API - OSCON

Resilience and Security @ Scale: Lessons Learned

Techniques for Scaling the Netflix API - QCon SF

Gluecon 2013 - Netflix Cloud Native Tutorial Details (part 2)

Similaire à Nflx ds meetup-5_dec2012

Apache Cassandra in the CloudInstaclustr

Yow Conference Dec 2013 Netflix Workshop Slides with NotesAdrian Cockcroft

Apache Cassandra in the Real WorldJeremy Hanna

Amazon RDS for Microsoft SQL: Performance, Security, Best Practices (DAT303) ...Amazon Web Services

C* Summit 2013: Netflix Open Source Tools and Benchmarks for Cassandra by Adr...DataStax Academy

Cassandra 2.0 (Introduction)bigdatagurus_meetup

BigData Developers MeetUpChristian Johannsen

Svc 202-netflix-open-sourceRuslan Meshenberg

Netflix and Open SourceAdrian Cockcroft

How Netflix’s Tools Can Help Accelerate Your Start-up (SVC202) | AWS re:Inven...Amazon Web Services

Running SQL Server on AWS | John McCormack | DataGrillen 2019John McCormack

Dsdt meetup 2017 11-21JDA Labs MTL

DSDT Meetup Nov 2017DSDT_MTL

mParticle's Journey to Scylla from CassandraScyllaDB

Deploying Big-Data-as-a-Service (BDaaS) in the EnterpriseBig-Data-as-a-Service (BDaaS) Meetup

AWS Black Belt TipsAmazon Web Services

Developing for Your Target Market - Social, Games & Mobile - AWS India Summit...Amazon Web Services

Why Kubernetes as a container orchestrator is a right choice for running spar...DataWorks Summit

Cloud and Windows AzureRadu Vunvulea

Getting started with amazon redshift - TorontoAmazon Web Services

Similaire à Nflx ds meetup-5_dec2012 (20)

Apache Cassandra in the Cloud

Yow Conference Dec 2013 Netflix Workshop Slides with Notes

Apache Cassandra in the Real World

Amazon RDS for Microsoft SQL: Performance, Security, Best Practices (DAT303) ...

C* Summit 2013: Netflix Open Source Tools and Benchmarks for Cassandra by Adr...

Cassandra 2.0 (Introduction)

BigData Developers MeetUp

Svc 202-netflix-open-source

Netflix and Open Source

How Netflix’s Tools Can Help Accelerate Your Start-up (SVC202) | AWS re:Inven...

Running SQL Server on AWS | John McCormack | DataGrillen 2019

Dsdt meetup 2017 11-21

DSDT Meetup Nov 2017

mParticle's Journey to Scylla from Cassandra

Deploying Big-Data-as-a-Service (BDaaS) in the Enterprise

AWS Black Belt Tips

Developing for Your Target Market - Social, Games & Mobile - AWS India Summit...

Why Kubernetes as a container orchestrator is a right choice for running spar...

Cloud and Windows Azure

Getting started with amazon redshift - Toronto

Dernier

How AI, OpenAI, and ChatGPT impact business and software.Curtis Poe

Story boards and shot lists for my a level piececharlottematthew16

Developer Data Modeling Mistakes: From Postgres to NoSQLScyllaDB

TrustArc Webinar - How to Build Consumer Trust Through Data PrivacyTrustArc

Designing IA for AI - Information Architecture Conference 2024Enterprise Knowledge

Advanced Test Driven-Development @ php[tek] 2024Scott Keck-Warren

"Debugging python applications inside k8s environment", Andrii SoldatenkoFwdays

Ensuring Technical Readiness For Copilot in Microsoft 3652toLead Limited

New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024BookNet Canada

Human Factors of XR: Using Human Factors to Design XR SystemsMark Billinghurst

TeamStation AI System Report LATAM IT Salaries 2024Lonnie McRorey

Hyperautomation and AI/ML: A Strategy for Digital Transformation Success.pdfPrecisely

Nell’iperspazio con Rocket: il Framework Web di Rust!Commit University

Powerpoint exploring the locations used in television show Time Clashcharlottematthew16

Unraveling Multimodality with Large Language Models.pdfAlex Barbosa Coqueiro

"LLMs for Python Engineers: Advanced Data Analysis and Semantic Kernel",Oleks...Fwdays

Artificial intelligence in cctv survelliance.pptxhariprasad279825

DevoxxFR 2024 Reproducible Builds with Apache MavenHervé Boutemy

Dev Dives: Streamline document processing with UiPath Studio WebUiPathCommunity

Are Multi-Cloud and Serverless Good or Bad?Mattias Andersson

Dernier (20)

How AI, OpenAI, and ChatGPT impact business and software.

Story boards and shot lists for my a level piece

Developer Data Modeling Mistakes: From Postgres to NoSQL

TrustArc Webinar - How to Build Consumer Trust Through Data Privacy

Designing IA for AI - Information Architecture Conference 2024

Advanced Test Driven-Development @ php[tek] 2024

"Debugging python applications inside k8s environment", Andrii Soldatenko

Ensuring Technical Readiness For Copilot in Microsoft 365

New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024

Human Factors of XR: Using Human Factors to Design XR Systems

TeamStation AI System Report LATAM IT Salaries 2024

Hyperautomation and AI/ML: A Strategy for Digital Transformation Success.pdf

Nell’iperspazio con Rocket: il Framework Web di Rust!

Powerpoint exploring the locations used in television show Time Clash

Unraveling Multimodality with Large Language Models.pdf

"LLMs for Python Engineers: Advanced Data Analysis and Semantic Kernel",Oleks...

Artificial intelligence in cctv survelliance.pptx

DevoxxFR 2024 Reproducible Builds with Apache Maven

Dev Dives: Streamline document processing with UiPath Studio Web

Are Multi-Cloud and Serverless Good or Bad?

Nflx ds meetup-5_dec2012

1. 2013 Roadmap: Cassandra at Ne4lix 5 December 2012 Jason Brown Senior So@ware Engineer @ Ne4lix @jasobrown jasedbrown@gmail.com hGp://www.linkedin.com/in/jasedbrown

2. blah blah blah

3. Cassandra Deployment Numbers > 50 Production clusters > 800 Total nodes 15 Number of multi-region clusters 4 Max regions for one cluster 144 Largest Cassandra cluster 4 Size of Cassandra dev-ops team

4. Record reads/writes 1.2 million writes/second 300k reads/second 72 node cluster – m2.4xlarge

5. 2013 Cassandra Roadmap • Priam • Explorers • Compliance/security • Data as a Service

6. Priam updates • C* 1.2 • Virtual nodes integraSon

7. Token distribuSon -‐ now

8. Token distribuSon -‐ vnodes

9. Vnodes info • @samoverton (Acunu) • hGp://www.acunu.com/2/post/2012/07/ virtual-‐nodes-‐strategies.html (arScle) • (images borrowed from this great blog post) • hGp://tweets.acunu.com/tweets/ 240082932524523520 (video)

10. Priam updates • C* 1.2 • Virtual nodes integraSon • Grow a cluster without doubling • Auto-‐scale c* cluster with load • Commit log backups (CASSANDRA-‐3690) • Accelerated BI integraSon

11. Explorers • Web-‐based UI • Cassandra plugin for admin/query • Coming soon to a github repo near you

12. Compliance Everyone to the cloud … even billing! Security, encrypSon, audiSng, logging Lots of details, lots of quesSons …

13. Data as a Service • “Gimme my data” • RESTful interface to get/set data • More than just C* • Cluster management (consolidaSon)

14. New Year’s ResoluSons • Learn CQL3 • Make vnodes rock in EC2 • More open source contribuSons • Watch more Ne4lix .. Rather than just code it

15. Thanks!