Soumettre la recherche
Mettre en ligne
How to Scale Your Architecture and DevOps Practices for Big Data Applications
•
3 j'aime
•
1,067 vues
Amazon Web Services
Suivre
by Manoj Chaudhary, CTO & VP of Engineering at Loggly
Lire moins
Lire la suite
Présentations et discours publics
Signaler
Partager
Signaler
Partager
1 sur 55
Recommandé
Big Data Architectural Patterns and Best Practices on AWS
Big Data Architectural Patterns and Best Practices on AWS
Amazon Web Services
Big Data adoption success using AWS Big Data Services - Pop-up Loft TLV 2017
Big Data adoption success using AWS Big Data Services - Pop-up Loft TLV 2017
Amazon Web Services
AWS re:Invent 2016: Big Data Mini Con State of the Union (BDM205)
AWS re:Invent 2016: Big Data Mini Con State of the Union (BDM205)
Amazon Web Services
BDA307 Real-time Streaming Applications on AWS, Patterns and Use Cases
BDA307 Real-time Streaming Applications on AWS, Patterns and Use Cases
Amazon Web Services
Getting Started With AWS Security
Getting Started With AWS Security
Amazon Web Services
Active Archiving with Amazon S3 and Tiering to Amazon Glacier - March 2017 AW...
Active Archiving with Amazon S3 and Tiering to Amazon Glacier - March 2017 AW...
Amazon Web Services
Big Data Architectural Patterns and Best Practices
Big Data Architectural Patterns and Best Practices
Amazon Web Services
ENT316 Keeping Pace With The Cloud: Managing and Optimizing as You Scale
ENT316 Keeping Pace With The Cloud: Managing and Optimizing as You Scale
Amazon Web Services
Recommandé
Big Data Architectural Patterns and Best Practices on AWS
Big Data Architectural Patterns and Best Practices on AWS
Amazon Web Services
Big Data adoption success using AWS Big Data Services - Pop-up Loft TLV 2017
Big Data adoption success using AWS Big Data Services - Pop-up Loft TLV 2017
Amazon Web Services
AWS re:Invent 2016: Big Data Mini Con State of the Union (BDM205)
AWS re:Invent 2016: Big Data Mini Con State of the Union (BDM205)
Amazon Web Services
BDA307 Real-time Streaming Applications on AWS, Patterns and Use Cases
BDA307 Real-time Streaming Applications on AWS, Patterns and Use Cases
Amazon Web Services
Getting Started With AWS Security
Getting Started With AWS Security
Amazon Web Services
Active Archiving with Amazon S3 and Tiering to Amazon Glacier - March 2017 AW...
Active Archiving with Amazon S3 and Tiering to Amazon Glacier - March 2017 AW...
Amazon Web Services
Big Data Architectural Patterns and Best Practices
Big Data Architectural Patterns and Best Practices
Amazon Web Services
ENT316 Keeping Pace With The Cloud: Managing and Optimizing as You Scale
ENT316 Keeping Pace With The Cloud: Managing and Optimizing as You Scale
Amazon Web Services
Deep Dive on Amazon S3 - AWS Online Tech Talks
Deep Dive on Amazon S3 - AWS Online Tech Talks
Amazon Web Services
Build an App on AWS for Your First 10 Million Users
Build an App on AWS for Your First 10 Million Users
Amazon Web Services
Amazon Kinesis Firehose - Pop-up Loft TLV 2017
Amazon Kinesis Firehose - Pop-up Loft TLV 2017
Amazon Web Services
Building your First Big Data Application on AWS
Building your First Big Data Application on AWS
Amazon Web Services
Data Storage for the Long Haul: Compliance and Archive
Data Storage for the Long Haul: Compliance and Archive
Amazon Web Services
AWS re:Invent 2016: Deep Dive on Amazon Glacier (STG302)
AWS re:Invent 2016: Deep Dive on Amazon Glacier (STG302)
Amazon Web Services
AWS re:Invent 2016: Case Study: How Startups like Mapbox, Ring, Hudl, and Oth...
AWS re:Invent 2016: Case Study: How Startups like Mapbox, Ring, Hudl, and Oth...
Amazon Web Services
BDA403 How Netflix Monitors Applications in Real-time with Amazon Kinesis
BDA403 How Netflix Monitors Applications in Real-time with Amazon Kinesis
Amazon Web Services
Modern Data Architectures for Real Time Analytics & Engagement
Modern Data Architectures for Real Time Analytics & Engagement
Amazon Web Services
BDA402 Deep Dive: Log analytics with Amazon Elasticsearch Service
BDA402 Deep Dive: Log analytics with Amazon Elasticsearch Service
Amazon Web Services
ENT306 Migrating Large Scale Data Sets to the Cloud
ENT306 Migrating Large Scale Data Sets to the Cloud
Amazon Web Services
Policy Ninja
Policy Ninja
Amazon Web Services
Sec301 Security @ (Cloud) Scale
Sec301 Security @ (Cloud) Scale
Amazon Web Services
AWS re:Invent 2016: Building Big Data Applications with the AWS Big Data Plat...
AWS re:Invent 2016: Building Big Data Applications with the AWS Big Data Plat...
Amazon Web Services
AWS October Webinar Series - Introducing Amazon Kinesis Firehose
AWS October Webinar Series - Introducing Amazon Kinesis Firehose
Amazon Web Services
Migrate your Data Warehouse to Amazon Redshift - September Webinar Series
Migrate your Data Warehouse to Amazon Redshift - September Webinar Series
Amazon Web Services
AWS re:Invent 2016: Workshop: AWS S3 Deep-Dive Hands-On Workshop: Deploying a...
AWS re:Invent 2016: Workshop: AWS S3 Deep-Dive Hands-On Workshop: Deploying a...
Amazon Web Services
ENT202 Creating Your Virtual Data Center: VPC Fundamentals and Connectivity O...
ENT202 Creating Your Virtual Data Center: VPC Fundamentals and Connectivity O...
Amazon Web Services
Migrating Large Scale Datasets
Migrating Large Scale Datasets
Amazon Web Services
ENT314 Automate Best Practices and Operational Health for Your AWS Resources
ENT314 Automate Best Practices and Operational Health for Your AWS Resources
Amazon Web Services
WTF is Sensu and Monitoring
WTF is Sensu and Monitoring
Toby Jackson
Whats new in IBM MQ; V9 LTS, V9.0.1 CD and V9.0.2 CD
Whats new in IBM MQ; V9 LTS, V9.0.1 CD and V9.0.2 CD
David Ware
Contenu connexe
Tendances
Deep Dive on Amazon S3 - AWS Online Tech Talks
Deep Dive on Amazon S3 - AWS Online Tech Talks
Amazon Web Services
Build an App on AWS for Your First 10 Million Users
Build an App on AWS for Your First 10 Million Users
Amazon Web Services
Amazon Kinesis Firehose - Pop-up Loft TLV 2017
Amazon Kinesis Firehose - Pop-up Loft TLV 2017
Amazon Web Services
Building your First Big Data Application on AWS
Building your First Big Data Application on AWS
Amazon Web Services
Data Storage for the Long Haul: Compliance and Archive
Data Storage for the Long Haul: Compliance and Archive
Amazon Web Services
AWS re:Invent 2016: Deep Dive on Amazon Glacier (STG302)
AWS re:Invent 2016: Deep Dive on Amazon Glacier (STG302)
Amazon Web Services
AWS re:Invent 2016: Case Study: How Startups like Mapbox, Ring, Hudl, and Oth...
AWS re:Invent 2016: Case Study: How Startups like Mapbox, Ring, Hudl, and Oth...
Amazon Web Services
BDA403 How Netflix Monitors Applications in Real-time with Amazon Kinesis
BDA403 How Netflix Monitors Applications in Real-time with Amazon Kinesis
Amazon Web Services
Modern Data Architectures for Real Time Analytics & Engagement
Modern Data Architectures for Real Time Analytics & Engagement
Amazon Web Services
BDA402 Deep Dive: Log analytics with Amazon Elasticsearch Service
BDA402 Deep Dive: Log analytics with Amazon Elasticsearch Service
Amazon Web Services
ENT306 Migrating Large Scale Data Sets to the Cloud
ENT306 Migrating Large Scale Data Sets to the Cloud
Amazon Web Services
Policy Ninja
Policy Ninja
Amazon Web Services
Sec301 Security @ (Cloud) Scale
Sec301 Security @ (Cloud) Scale
Amazon Web Services
AWS re:Invent 2016: Building Big Data Applications with the AWS Big Data Plat...
AWS re:Invent 2016: Building Big Data Applications with the AWS Big Data Plat...
Amazon Web Services
AWS October Webinar Series - Introducing Amazon Kinesis Firehose
AWS October Webinar Series - Introducing Amazon Kinesis Firehose
Amazon Web Services
Migrate your Data Warehouse to Amazon Redshift - September Webinar Series
Migrate your Data Warehouse to Amazon Redshift - September Webinar Series
Amazon Web Services
AWS re:Invent 2016: Workshop: AWS S3 Deep-Dive Hands-On Workshop: Deploying a...
AWS re:Invent 2016: Workshop: AWS S3 Deep-Dive Hands-On Workshop: Deploying a...
Amazon Web Services
ENT202 Creating Your Virtual Data Center: VPC Fundamentals and Connectivity O...
ENT202 Creating Your Virtual Data Center: VPC Fundamentals and Connectivity O...
Amazon Web Services
Migrating Large Scale Datasets
Migrating Large Scale Datasets
Amazon Web Services
ENT314 Automate Best Practices and Operational Health for Your AWS Resources
ENT314 Automate Best Practices and Operational Health for Your AWS Resources
Amazon Web Services
Tendances
(20)
Deep Dive on Amazon S3 - AWS Online Tech Talks
Deep Dive on Amazon S3 - AWS Online Tech Talks
Build an App on AWS for Your First 10 Million Users
Build an App on AWS for Your First 10 Million Users
Amazon Kinesis Firehose - Pop-up Loft TLV 2017
Amazon Kinesis Firehose - Pop-up Loft TLV 2017
Building your First Big Data Application on AWS
Building your First Big Data Application on AWS
Data Storage for the Long Haul: Compliance and Archive
Data Storage for the Long Haul: Compliance and Archive
AWS re:Invent 2016: Deep Dive on Amazon Glacier (STG302)
AWS re:Invent 2016: Deep Dive on Amazon Glacier (STG302)
AWS re:Invent 2016: Case Study: How Startups like Mapbox, Ring, Hudl, and Oth...
AWS re:Invent 2016: Case Study: How Startups like Mapbox, Ring, Hudl, and Oth...
BDA403 How Netflix Monitors Applications in Real-time with Amazon Kinesis
BDA403 How Netflix Monitors Applications in Real-time with Amazon Kinesis
Modern Data Architectures for Real Time Analytics & Engagement
Modern Data Architectures for Real Time Analytics & Engagement
BDA402 Deep Dive: Log analytics with Amazon Elasticsearch Service
BDA402 Deep Dive: Log analytics with Amazon Elasticsearch Service
ENT306 Migrating Large Scale Data Sets to the Cloud
ENT306 Migrating Large Scale Data Sets to the Cloud
Policy Ninja
Policy Ninja
Sec301 Security @ (Cloud) Scale
Sec301 Security @ (Cloud) Scale
AWS re:Invent 2016: Building Big Data Applications with the AWS Big Data Plat...
AWS re:Invent 2016: Building Big Data Applications with the AWS Big Data Plat...
AWS October Webinar Series - Introducing Amazon Kinesis Firehose
AWS October Webinar Series - Introducing Amazon Kinesis Firehose
Migrate your Data Warehouse to Amazon Redshift - September Webinar Series
Migrate your Data Warehouse to Amazon Redshift - September Webinar Series
AWS re:Invent 2016: Workshop: AWS S3 Deep-Dive Hands-On Workshop: Deploying a...
AWS re:Invent 2016: Workshop: AWS S3 Deep-Dive Hands-On Workshop: Deploying a...
ENT202 Creating Your Virtual Data Center: VPC Fundamentals and Connectivity O...
ENT202 Creating Your Virtual Data Center: VPC Fundamentals and Connectivity O...
Migrating Large Scale Datasets
Migrating Large Scale Datasets
ENT314 Automate Best Practices and Operational Health for Your AWS Resources
ENT314 Automate Best Practices and Operational Health for Your AWS Resources
En vedette
WTF is Sensu and Monitoring
WTF is Sensu and Monitoring
Toby Jackson
Whats new in IBM MQ; V9 LTS, V9.0.1 CD and V9.0.2 CD
Whats new in IBM MQ; V9 LTS, V9.0.1 CD and V9.0.2 CD
David Ware
Internationale clusters in vergelijkend perpsectief
Internationale clusters in vergelijkend perpsectief
Anika Snel
Microservices Tracing with Spring Cloud and Zipkin
Microservices Tracing with Spring Cloud and Zipkin
Marcin Grzejszczak
Setex Brochure by Matrax Bulgaria
Setex Brochure by Matrax Bulgaria
Веселка Дамянова
Amazon Elastic Block Store for Application Storage
Amazon Elastic Block Store for Application Storage
Amazon Web Services
Application Development on Metapod
Application Development on Metapod
Cisco DevNet
Monitoring & Analysis 101 - N00b to Ninja in 60 Minutes at ISSW on April 9, 2016
Monitoring & Analysis 101 - N00b to Ninja in 60 Minutes at ISSW on April 9, 2016
grecsl
Logs Don't Lie Or Do They?
Logs Don't Lie Or Do They?
Alan K'necht
How to Keep Students Motivated During Winter
How to Keep Students Motivated During Winter
Robert Peters, Ed.D
Respond to and troubleshoot production incidents like an sa
Respond to and troubleshoot production incidents like an sa
Tom Cudd
FDA's Brian Bradley Case Study and Process Review of the Veterans Review and ...
FDA's Brian Bradley Case Study and Process Review of the Veterans Review and ...
Foundation for Democratic Advancement
Dashboards: Using data to find out what's really going on
Dashboards: Using data to find out what's really going on
rouanw
Cloud adoption patterns
Cloud adoption patterns
Kyle Brown
LJC Mashup "Building Java Microservices for the Cloud && Chuck Norris Doesn't...
LJC Mashup "Building Java Microservices for the Cloud && Chuck Norris Doesn't...
Daniel Bryant
Delphi XE2, door André Mussche op de 4DotNet Developers Day
Delphi XE2, door André Mussche op de 4DotNet Developers Day
Hanneke Dotnet
vanEngelen 360 Inspiratieborrel - Trends Update 2014
vanEngelen 360 Inspiratieborrel - Trends Update 2014
Van Engelen
Bsides threat hunting
Bsides threat hunting
Rodrigo Montoro
Image (PNG) Forensic Analysis
Image (PNG) Forensic Analysis
Cysinfo Cyber Security Community
Can you handle The TRUTH ,..? Missing page history of JESUS and Hidden TRUTH
Can you handle The TRUTH ,..? Missing page history of JESUS and Hidden TRUTH
Heri kusrianto
En vedette
(20)
WTF is Sensu and Monitoring
WTF is Sensu and Monitoring
Whats new in IBM MQ; V9 LTS, V9.0.1 CD and V9.0.2 CD
Whats new in IBM MQ; V9 LTS, V9.0.1 CD and V9.0.2 CD
Internationale clusters in vergelijkend perpsectief
Internationale clusters in vergelijkend perpsectief
Microservices Tracing with Spring Cloud and Zipkin
Microservices Tracing with Spring Cloud and Zipkin
Setex Brochure by Matrax Bulgaria
Setex Brochure by Matrax Bulgaria
Amazon Elastic Block Store for Application Storage
Amazon Elastic Block Store for Application Storage
Application Development on Metapod
Application Development on Metapod
Monitoring & Analysis 101 - N00b to Ninja in 60 Minutes at ISSW on April 9, 2016
Monitoring & Analysis 101 - N00b to Ninja in 60 Minutes at ISSW on April 9, 2016
Logs Don't Lie Or Do They?
Logs Don't Lie Or Do They?
How to Keep Students Motivated During Winter
How to Keep Students Motivated During Winter
Respond to and troubleshoot production incidents like an sa
Respond to and troubleshoot production incidents like an sa
FDA's Brian Bradley Case Study and Process Review of the Veterans Review and ...
FDA's Brian Bradley Case Study and Process Review of the Veterans Review and ...
Dashboards: Using data to find out what's really going on
Dashboards: Using data to find out what's really going on
Cloud adoption patterns
Cloud adoption patterns
LJC Mashup "Building Java Microservices for the Cloud && Chuck Norris Doesn't...
LJC Mashup "Building Java Microservices for the Cloud && Chuck Norris Doesn't...
Delphi XE2, door André Mussche op de 4DotNet Developers Day
Delphi XE2, door André Mussche op de 4DotNet Developers Day
vanEngelen 360 Inspiratieborrel - Trends Update 2014
vanEngelen 360 Inspiratieborrel - Trends Update 2014
Bsides threat hunting
Bsides threat hunting
Image (PNG) Forensic Analysis
Image (PNG) Forensic Analysis
Can you handle The TRUTH ,..? Missing page history of JESUS and Hidden TRUTH
Can you handle The TRUTH ,..? Missing page history of JESUS and Hidden TRUTH
Similaire à How to Scale Your Architecture and DevOps Practices for Big Data Applications
Loggly - How to Scale Your Architecture and DevOps Practices for Big Data App...
Loggly - How to Scale Your Architecture and DevOps Practices for Big Data App...
SolarWinds Loggly
Loggly - Tools and Techniques For Logging Microservices
Loggly - Tools and Techniques For Logging Microservices
SolarWinds Loggly
Loggly - Case Study - Datami Keeps Developer Productivity High with Loggly
Loggly - Case Study - Datami Keeps Developer Productivity High with Loggly
SolarWinds Loggly
Using AWS CloudTrail to Enhance Governance and Compliance of Amazon S3 - DEV3...
Using AWS CloudTrail to Enhance Governance and Compliance of Amazon S3 - DEV3...
Amazon Web Services
How Trek10 Uses Datadog's Distributed Tracing to Improve AWS Lambda Projects ...
How Trek10 Uses Datadog's Distributed Tracing to Improve AWS Lambda Projects ...
Amazon Web Services
STG301_Deep Dive on Amazon S3 and Glacier Architecture
STG301_Deep Dive on Amazon S3 and Glacier Architecture
Amazon Web Services
Storage Data Management: Tools and Templates to Seamlessly Automate and Optim...
Storage Data Management: Tools and Templates to Seamlessly Automate and Optim...
Amazon Web Services
Visualization with Amazon QuickSight
Visualization with Amazon QuickSight
Amazon Web Services
TiVo: How to Scale New Products with a Data Lake on AWS and Qubole
TiVo: How to Scale New Products with a Data Lake on AWS and Qubole
Amazon Web Services
TiVo: How to Scale New Products with a Data Lake on AWS and Qubole
TiVo: How to Scale New Products with a Data Lake on AWS and Qubole
Amazon Web Services
ENT324-Automating and Auditing Cloud Governance and Compliance in Multi-Accou...
ENT324-Automating and Auditing Cloud Governance and Compliance in Multi-Accou...
Amazon Web Services
Visualization with Amazon QuickSight
Visualization with Amazon QuickSight
Amazon Web Services
Amazon QuickSight First Call Deck
Amazon QuickSight First Call Deck
Amazon Web Services
Amazon QuickSight First Call Deck
Amazon QuickSight First Call Deck
Amazon Web Services
How Nextdoor Built a Scalable, Serverless Data Pipeline for Billions of Event...
How Nextdoor Built a Scalable, Serverless Data Pipeline for Billions of Event...
Amazon Web Services
Analyzing application activities with KSQL and Elasticsearch
Analyzing application activities with KSQL and Elasticsearch
Katherine Golovinova
Cloudlytics: In Depth S3 & CloudFront Log Analysis - Featuring Reports
Cloudlytics: In Depth S3 & CloudFront Log Analysis - Featuring Reports
Blazeclan Technologies Private Limited
Visualization with Amazon QuickSight
Visualization with Amazon QuickSight
Amazon Web Services
[DataCon.TW 2017] Data Lake: centralize in on-prem vs. decentralize on cloud
[DataCon.TW 2017] Data Lake: centralize in on-prem vs. decentralize on cloud
Jeff Hung
Analytics Web Day | Query your Data in S3 with SQL and optimize for Cost and ...
Analytics Web Day | Query your Data in S3 with SQL and optimize for Cost and ...
AWS Germany
Similaire à How to Scale Your Architecture and DevOps Practices for Big Data Applications
(20)
Loggly - How to Scale Your Architecture and DevOps Practices for Big Data App...
Loggly - How to Scale Your Architecture and DevOps Practices for Big Data App...
Loggly - Tools and Techniques For Logging Microservices
Loggly - Tools and Techniques For Logging Microservices
Loggly - Case Study - Datami Keeps Developer Productivity High with Loggly
Loggly - Case Study - Datami Keeps Developer Productivity High with Loggly
Using AWS CloudTrail to Enhance Governance and Compliance of Amazon S3 - DEV3...
Using AWS CloudTrail to Enhance Governance and Compliance of Amazon S3 - DEV3...
How Trek10 Uses Datadog's Distributed Tracing to Improve AWS Lambda Projects ...
How Trek10 Uses Datadog's Distributed Tracing to Improve AWS Lambda Projects ...
STG301_Deep Dive on Amazon S3 and Glacier Architecture
STG301_Deep Dive on Amazon S3 and Glacier Architecture
Storage Data Management: Tools and Templates to Seamlessly Automate and Optim...
Storage Data Management: Tools and Templates to Seamlessly Automate and Optim...
Visualization with Amazon QuickSight
Visualization with Amazon QuickSight
TiVo: How to Scale New Products with a Data Lake on AWS and Qubole
TiVo: How to Scale New Products with a Data Lake on AWS and Qubole
TiVo: How to Scale New Products with a Data Lake on AWS and Qubole
TiVo: How to Scale New Products with a Data Lake on AWS and Qubole
ENT324-Automating and Auditing Cloud Governance and Compliance in Multi-Accou...
ENT324-Automating and Auditing Cloud Governance and Compliance in Multi-Accou...
Visualization with Amazon QuickSight
Visualization with Amazon QuickSight
Amazon QuickSight First Call Deck
Amazon QuickSight First Call Deck
Amazon QuickSight First Call Deck
Amazon QuickSight First Call Deck
How Nextdoor Built a Scalable, Serverless Data Pipeline for Billions of Event...
How Nextdoor Built a Scalable, Serverless Data Pipeline for Billions of Event...
Analyzing application activities with KSQL and Elasticsearch
Analyzing application activities with KSQL and Elasticsearch
Cloudlytics: In Depth S3 & CloudFront Log Analysis - Featuring Reports
Cloudlytics: In Depth S3 & CloudFront Log Analysis - Featuring Reports
Visualization with Amazon QuickSight
Visualization with Amazon QuickSight
[DataCon.TW 2017] Data Lake: centralize in on-prem vs. decentralize on cloud
[DataCon.TW 2017] Data Lake: centralize in on-prem vs. decentralize on cloud
Analytics Web Day | Query your Data in S3 with SQL and optimize for Cost and ...
Analytics Web Day | Query your Data in S3 with SQL and optimize for Cost and ...
Plus de Amazon Web Services
Come costruire servizi di Forecasting sfruttando algoritmi di ML e deep learn...
Come costruire servizi di Forecasting sfruttando algoritmi di ML e deep learn...
Amazon Web Services
Big Data per le Startup: come creare applicazioni Big Data in modalità Server...
Big Data per le Startup: come creare applicazioni Big Data in modalità Server...
Amazon Web Services
Esegui pod serverless con Amazon EKS e AWS Fargate
Esegui pod serverless con Amazon EKS e AWS Fargate
Amazon Web Services
Costruire Applicazioni Moderne con AWS
Costruire Applicazioni Moderne con AWS
Amazon Web Services
Come spendere fino al 90% in meno con i container e le istanze spot
Come spendere fino al 90% in meno con i container e le istanze spot
Amazon Web Services
Open banking as a service
Open banking as a service
Amazon Web Services
Rendi unica l’offerta della tua startup sul mercato con i servizi Machine Lea...
Rendi unica l’offerta della tua startup sul mercato con i servizi Machine Lea...
Amazon Web Services
OpsWorks Configuration Management: automatizza la gestione e i deployment del...
OpsWorks Configuration Management: automatizza la gestione e i deployment del...
Amazon Web Services
Microsoft Active Directory su AWS per supportare i tuoi Windows Workloads
Microsoft Active Directory su AWS per supportare i tuoi Windows Workloads
Amazon Web Services
Computer Vision con AWS
Computer Vision con AWS
Amazon Web Services
Database Oracle e VMware Cloud on AWS i miti da sfatare
Database Oracle e VMware Cloud on AWS i miti da sfatare
Amazon Web Services
Crea la tua prima serverless ledger-based app con QLDB e NodeJS
Crea la tua prima serverless ledger-based app con QLDB e NodeJS
Amazon Web Services
API moderne real-time per applicazioni mobili e web
API moderne real-time per applicazioni mobili e web
Amazon Web Services
Database Oracle e VMware Cloud™ on AWS: i miti da sfatare
Database Oracle e VMware Cloud™ on AWS: i miti da sfatare
Amazon Web Services
Tools for building your MVP on AWS
Tools for building your MVP on AWS
Amazon Web Services
How to Build a Winning Pitch Deck
How to Build a Winning Pitch Deck
Amazon Web Services
Building a web application without servers
Building a web application without servers
Amazon Web Services
Fundraising Essentials
Fundraising Essentials
Amazon Web Services
AWS_HK_StartupDay_Building Interactive websites while automating for efficien...
AWS_HK_StartupDay_Building Interactive websites while automating for efficien...
Amazon Web Services
Introduzione a Amazon Elastic Container Service
Introduzione a Amazon Elastic Container Service
Amazon Web Services
Plus de Amazon Web Services
(20)
Come costruire servizi di Forecasting sfruttando algoritmi di ML e deep learn...
Come costruire servizi di Forecasting sfruttando algoritmi di ML e deep learn...
Big Data per le Startup: come creare applicazioni Big Data in modalità Server...
Big Data per le Startup: come creare applicazioni Big Data in modalità Server...
Esegui pod serverless con Amazon EKS e AWS Fargate
Esegui pod serverless con Amazon EKS e AWS Fargate
Costruire Applicazioni Moderne con AWS
Costruire Applicazioni Moderne con AWS
Come spendere fino al 90% in meno con i container e le istanze spot
Come spendere fino al 90% in meno con i container e le istanze spot
Open banking as a service
Open banking as a service
Rendi unica l’offerta della tua startup sul mercato con i servizi Machine Lea...
Rendi unica l’offerta della tua startup sul mercato con i servizi Machine Lea...
OpsWorks Configuration Management: automatizza la gestione e i deployment del...
OpsWorks Configuration Management: automatizza la gestione e i deployment del...
Microsoft Active Directory su AWS per supportare i tuoi Windows Workloads
Microsoft Active Directory su AWS per supportare i tuoi Windows Workloads
Computer Vision con AWS
Computer Vision con AWS
Database Oracle e VMware Cloud on AWS i miti da sfatare
Database Oracle e VMware Cloud on AWS i miti da sfatare
Crea la tua prima serverless ledger-based app con QLDB e NodeJS
Crea la tua prima serverless ledger-based app con QLDB e NodeJS
API moderne real-time per applicazioni mobili e web
API moderne real-time per applicazioni mobili e web
Database Oracle e VMware Cloud™ on AWS: i miti da sfatare
Database Oracle e VMware Cloud™ on AWS: i miti da sfatare
Tools for building your MVP on AWS
Tools for building your MVP on AWS
How to Build a Winning Pitch Deck
How to Build a Winning Pitch Deck
Building a web application without servers
Building a web application without servers
Fundraising Essentials
Fundraising Essentials
AWS_HK_StartupDay_Building Interactive websites while automating for efficien...
AWS_HK_StartupDay_Building Interactive websites while automating for efficien...
Introduzione a Amazon Elastic Container Service
Introduzione a Amazon Elastic Container Service
Dernier
Testing and Development Challenges for Complex Cyber-Physical Systems: Insigh...
Testing and Development Challenges for Complex Cyber-Physical Systems: Insigh...
Sebastiano Panichella
Engaging Eid Ul Fitr Presentation for Kindergartners.pptx
Engaging Eid Ul Fitr Presentation for Kindergartners.pptx
AsifArshad8
Sunlight Spectacle 2024 Practical Action Launch Event 2024-04-08
Sunlight Spectacle 2024 Practical Action Launch Event 2024-04-08
LloydHelferty
cse-csp batch4 review-1.1.pptx cyber security
cse-csp batch4 review-1.1.pptx cyber security
sandeepnani2260
Understanding Post Production changes (PPC) in Clinical Data Management (CDM)...
Understanding Post Production changes (PPC) in Clinical Data Management (CDM)...
soumyapottola
Don't Miss Out: Strategies for Making the Most of the Ethena DigitalOpportunity
Don't Miss Out: Strategies for Making the Most of the Ethena DigitalOpportunity
App Ethena
GESCO SE Press and Analyst Conference on Financial Results 2024
GESCO SE Press and Analyst Conference on Financial Results 2024
GESCO SE
General Elections Final Press Noteas per M
General Elections Final Press Noteas per M
VidyaAdsule1
Testing with Fewer Resources: Toward Adaptive Approaches for Cost-effective ...
Testing with Fewer Resources: Toward Adaptive Approaches for Cost-effective ...
Sebastiano Panichella
Scootsy Overview Deck - Pan City Delivery
Scootsy Overview Deck - Pan City Delivery
rishi338139
INDIAN GCP GUIDELINE. for Regulatory affair 1st sem CRR
INDIAN GCP GUIDELINE. for Regulatory affair 1st sem CRR
sarwankumar4524
Application of GIS in Landslide Disaster Response.pptx
Application of GIS in Landslide Disaster Response.pptx
Roquia Salam
05.02 MMC - Assignment 4 - Image Attribution Lovepreet.pptx
05.02 MMC - Assignment 4 - Image Attribution Lovepreet.pptx
erickamwana1
RACHEL-ANN M. TENIBRO PRODUCT RESEARCH PRESENTATION
RACHEL-ANN M. TENIBRO PRODUCT RESEARCH PRESENTATION
RachelAnnTenibroAmaz
Dernier
(14)
Testing and Development Challenges for Complex Cyber-Physical Systems: Insigh...
Testing and Development Challenges for Complex Cyber-Physical Systems: Insigh...
Engaging Eid Ul Fitr Presentation for Kindergartners.pptx
Engaging Eid Ul Fitr Presentation for Kindergartners.pptx
Sunlight Spectacle 2024 Practical Action Launch Event 2024-04-08
Sunlight Spectacle 2024 Practical Action Launch Event 2024-04-08
cse-csp batch4 review-1.1.pptx cyber security
cse-csp batch4 review-1.1.pptx cyber security
Understanding Post Production changes (PPC) in Clinical Data Management (CDM)...
Understanding Post Production changes (PPC) in Clinical Data Management (CDM)...
Don't Miss Out: Strategies for Making the Most of the Ethena DigitalOpportunity
Don't Miss Out: Strategies for Making the Most of the Ethena DigitalOpportunity
GESCO SE Press and Analyst Conference on Financial Results 2024
GESCO SE Press and Analyst Conference on Financial Results 2024
General Elections Final Press Noteas per M
General Elections Final Press Noteas per M
Testing with Fewer Resources: Toward Adaptive Approaches for Cost-effective ...
Testing with Fewer Resources: Toward Adaptive Approaches for Cost-effective ...
Scootsy Overview Deck - Pan City Delivery
Scootsy Overview Deck - Pan City Delivery
INDIAN GCP GUIDELINE. for Regulatory affair 1st sem CRR
INDIAN GCP GUIDELINE. for Regulatory affair 1st sem CRR
Application of GIS in Landslide Disaster Response.pptx
Application of GIS in Landslide Disaster Response.pptx
05.02 MMC - Assignment 4 - Image Attribution Lovepreet.pptx
05.02 MMC - Assignment 4 - Image Attribution Lovepreet.pptx
RACHEL-ANN M. TENIBRO PRODUCT RESEARCH PRESENTATION
RACHEL-ANN M. TENIBRO PRODUCT RESEARCH PRESENTATION
How to Scale Your Architecture and DevOps Practices for Big Data Applications
1.
1© 2017 Loggly,
Inc. Confidential & Proprietary. 3/30/17 How to Scale Your Architecture and DevOps Practices for Big Data Applications Manoj Chaudhary CTO and VP of Engineering March 30, 2017
2.
2© 2017 Loggly,
Inc. Confidential & Proprietary. 3/30/17 • Cloud-based log management • Founded in 2009 • Based in San Francisco • 10,000+ customers • Startups to Fortune 500
3.
3© 2017 Loggly,
Inc. Confidential & Proprietary. 3/30/17 How log management starts Management = simple stuff • Rotate files, compress and delete • Scan for specific events (not easy) • Log retention policies evolve over time
4.
4© 2017 Loggly,
Inc. Confidential & Proprietary. 3/30/17 Log volume Self-InflictedPain “…hmmm, our logs are getting a bit bloated” “…let’s spend time managing our log capacity” “…how can I make this someone else’s problem!” As Log Data Grows As log data grows
5.
5© 2017 Loggly,
Inc. Confidential & Proprietary. 3/30/17 Focus on managing application vs. managing logs from the trenches… If you get a disk space alert, first login… % sudo rm –rf /var/log/apache2/* Admit it, we’ve all seen this kind of thing! Do not make this is an either/or proposition! !
6.
6© 2017 Loggly,
Inc. Confidential & Proprietary. 3/30/17 We believe the future is all about increasingly complex systems, and companies need better ways to be able to understand them.
7.
7© 2017 Loggly,
Inc. Confidential & Proprietary. 3/30/17 The way we make complex systems understandable is by making it ridiculously easy to reveal the hidden stories in your log data.
8.
8© 2017 Loggly,
Inc. Confidential & Proprietary. 3/30/17 Loggly makes log management easy and simple • Cloud-based SaaS for easy central log collection, aggregation, management • Agent-free, easy setup • Uses open or industry standards and native log transfer capabilities (example: syslog) • No dependency on / maintenance effort from proprietary agents • 54 logging technologies supported out of the box, list growing • Setup Wizard or optionally fully manual configuration with full control and transparency
9.
9© 2017 Loggly,
Inc. Confidential & Proprietary. 3/30/17 Dynamic parsing • Parses data in real-time at ingestion time, auto-recognizes common log formats, but also supports custom parsing rules • Automatically breaks data down into meaningful groups, fields, values that can be mouse-navigated in a menu- style structure (Dynamic Field Explorer™) • JSON support / extraction • Custom parsing and tagging rules allow to segregate dynamically • Self-documenting data inventory
10.
10© 2017 Loggly,
Inc. Confidential & Proprietary. 3/30/17 Gamut Search™ • Provides instant results over massive amounts of data and long time periods • Rather than waiting minutes, hours or days for a query to complete, the new capabilities enable users to begin analyzing the entire range, or the “gamut” of their log data, immediately, in a highly interactive fashion. • No special query language to learn, common Regular Expressions syntax • Surround Search shows events context with one single click
11.
11© 2017 Loggly,
Inc. Confidential & Proprietary. 3/30/17 Other key features • In-depth analytics • Dashboards, pre-configured and customizable, shareable • Anomaly Detection • Alerts that can be sent to HipChat, Slack, PagerDuty, HTTP endpoints, others • JIRA Software integration, point-and- click ticket creation without leaving Loggly Log management is part of a bigger process! DevOps, Agile, CI/CD, …
12.
12© 2017 Loggly,
Inc. Confidential & Proprietary. 3/30/17 Log management is a big data problem Massive incoming event stream Fundamentally multi-tenant Scalable framework for analysis Near real-time indexing Near real-time search Time-series index management Logs are TLDR; Traffic is unpredictable
13.
13© 2017 Loggly,
Inc. Confidential & Proprietary. 3/30/17 But can be viewed as simple as Ingest Process Index Search and other Services
14.
14© 2017 Loggly,
Inc. Confidential & Proprietary. 3/30/17 Let’s look under the covers of simplicity Amazon Route 53 Kafka broker Kafka broker Search and analytics engine Ingestion pod Indexing pod S3 ingestion Services on processing pod Services on indexing Index management • Convert unstructured data to structured • Data processing happens here • Checkpoint processed data to Kafka • Can have multiple indexing clusters • Time-series index management • Index and store data in search and analytics engine • Entry point for logs • Collect and validate • Checkpoint logs to Kafka Amazon S3 & AWS CloudTrail Amazon RDS MySQL S3 archiving Amazon S3
15.
15© 2017 Loggly,
Inc. Confidential & Proprietary. 3/30/17 Scalability means not just the ability to operate, but to operate efficiently and with adequate quality of service, over the given range of configurations. Build for scalability developing service Scalability is a feature “ ” + running service governing service+ SaaS engineering =
16.
16© 2017 Loggly,
Inc. Confidential & Proprietary. 3/30/17 All customers don’t behave as good citizens at all times “Noisy neighbors” with spikes in log volumes • Application on fire • Log management configuration problem • Other human error • Spikes can last a long time
17.
17© 2017 Loggly,
Inc. Confidential & Proprietary. 3/30/17 How to mitigate the effects of unpredictable load patterns • Have governors • Hawk for your infrastructure • Processes for managing out-of-policy activity • Set up metrics and alerts that let you know about unexpected behavior
18.
18© 2017 Loggly,
Inc. Confidential & Proprietary. 3/30/17 Sits on top of platform and “watches” what’s going on – across tenants Governors to segregate tenants Identify out-of- policy behavior Segregate misbehavior Inform the right people
19.
19© 2017 Loggly,
Inc. Confidential & Proprietary. 3/30/17 Search and analytics engine Index management Trusted platform for all customers Amazon Route53 Kafka broker Kafka broker Services on indexing pod Search and analytics engine Index management Kafka broker Ingestion pod Indexing pod Services on ingestion pod like S3 archiving Services on processing pod Governors Amazon ElastiCache Amazon RDS MySQL RDS RDS
20.
20© 2017 Loggly,
Inc. Confidential & Proprietary. 3/30/17 Case study
21.
21© 2017 Loggly,
Inc. Confidential & Proprietary. 3/30/17 Troubleshooting use case Problem • Running our business involves integrating with more than 100 web-based services and billions of API calls every month. • There are many places where things can go awry. Segment depends on its log data to troubleshoot operational issues • Solve operational issues with partner integrations • Isolate relevant log data from billions of API calls per month Competing solutions • ELK stack • Homegrown log management system
22.
22© 2017 Loggly,
Inc. Confidential & Proprietary. 3/30/17 Troubleshooting use case Solution • Use UUID to trace API calls to and from partner services • Monitor API volume from specific customers • Error monitoring with code releases Results • Faster MTTR • DevOps team no longer involved in most troubleshooting = Devops efficiency • Pro-active alerts = Devops efficiency and increase customer satisfaction. • Integrated with bug tracking system = Devops and engineering team efficiency
23.
23© 2017 Loggly,
Inc. Confidential & Proprietary. 3/30/17 Troubleshooting demo
24.
24© 2017 Loggly,
Inc. Confidential & Proprietary. 3/30/17
25.
25© 2017 Loggly,
Inc. Confidential & Proprietary. 3/30/17
26.
26© 2017 Loggly,
Inc. Confidential & Proprietary. 3/30/17
27.
27© 2017 Loggly,
Inc. Confidential & Proprietary. 3/30/17
28.
28© 2017 Loggly,
Inc. Confidential & Proprietary. 3/30/17
29.
29© 2017 Loggly,
Inc. Confidential & Proprietary. 3/30/17
30.
30© 2017 Loggly,
Inc. Confidential & Proprietary. 3/30/17
31.
31© 2017 Loggly,
Inc. Confidential & Proprietary. 3/30/17
32.
32© 2017 Loggly,
Inc. Confidential & Proprietary. 3/30/17
33.
33© 2017 Loggly,
Inc. Confidential & Proprietary. 3/30/17
34.
34© 2017 Loggly,
Inc. Confidential & Proprietary. 3/30/17
35.
35© 2017 Loggly,
Inc. Confidential & Proprietary. 3/30/17
36.
36© 2017 Loggly,
Inc. Confidential & Proprietary. 3/30/17
37.
37© 2017 Loggly,
Inc. Confidential & Proprietary. 3/30/17
38.
38© 2017 Loggly,
Inc. Confidential & Proprietary. 3/30/17
39.
39© 2017 Loggly,
Inc. Confidential & Proprietary. 3/30/17
40.
40© 2017 Loggly,
Inc. Confidential & Proprietary. 3/30/17
41.
41© 2017 Loggly,
Inc. Confidential & Proprietary. 3/30/17
42.
42© 2017 Loggly,
Inc. Confidential & Proprietary. 3/30/17 Monitoring for errors
43.
43© 2017 Loggly,
Inc. Confidential & Proprietary. 3/30/17
44.
44© 2017 Loggly,
Inc. Confidential & Proprietary. 3/30/17
45.
45© 2017 Loggly,
Inc. Confidential & Proprietary. 3/30/17
46.
46© 2017 Loggly,
Inc. Confidential & Proprietary. 3/30/17
47.
47© 2017 Loggly,
Inc. Confidential & Proprietary. 3/30/17
48.
48© 2017 Loggly,
Inc. Confidential & Proprietary. 3/30/17
49.
49© 2017 Loggly,
Inc. Confidential & Proprietary. 3/30/17
50.
50© 2017 Loggly,
Inc. Confidential & Proprietary. 3/30/17
51.
51© 2017 Loggly,
Inc. Confidential & Proprietary. 3/30/17
52.
52© 2017 Loggly,
Inc. Confidential & Proprietary. 3/30/17
53.
53© 2017 Loggly,
Inc. Confidential & Proprietary. 3/30/17 Other common Loggly use cases Proactive log monitoring and alerting Data analysis and optimization Team collaboration
54.
54© 2017 Loggly,
Inc. Confidential & Proprietary. 3/30/17 “Logfooding” at Loggly
55.
55© 2017 Loggly,
Inc. Confidential & Proprietary. 3/30/17 Manoj Chaudhary email: manoj@loggly.com twittter:twitt_2_manoj Visit us at loggly.com or follow @loggly on Twitter. Try Loggly for Free! → https://www.loggly.com/