SlideShare une entreprise Scribd logo
1  sur  55
Télécharger pour lire hors ligne
Cloud Computing
Agenda

 Design Considerations in moving to Cloud
 Introduction to Windows Azure
 Demo
Azure and cloud design patterns
Business Benefits of Cloud Computing

 Almost zero upfront infrastructure investment.
 Just-in-time infrastructure.
 More efficient resource utilization.
 Usage based costing.
 Reduced time to Market.
Cloud Computing
Cloud Computing
Challenges Faced by Apps in the
Cloud
 Application Scalability
     Cloud promises rapid (de)provisioning of resources.
     How do you tap into that to create scalable systems?
 Application Availability
     Underlying resource failures happen
            … usually more frequently than in
            traditional data centers.
     How do you overcome that to create highly available systems?
The Scalability Challenge

 Two different components to scale:
     State (inputs, data store, output)
     Behavior (business logic)
 Any non-trivial application has both.
 Scaling one component means scaling the other, too.
Scalability Considerations

                          Performance vs Scalability
                          Latency vs Throughput
                          Availability vs Consistency




How do you manage overload ?
Scalable Service

 A scalable architecture is critical to take advantage of scalable
  infrastructure.
 Characteristic of Scalable Service:
     Increasing resources results in a proportional increase in performance
     A scalable service is capable of handling heterogeneity
     A scalable service is operationally efficient
     A scalable service is resilient
     A scalable service becomes more cost effective when it grows.
1. Design for Failure
     and nothing will really fail

 Avoid single points of failure
 Assume everything fails and design backwards.
 Applications should continue to function even if the physical hardware fails
  or removed or replaced.
Design for Failure contd..

 Unit of failure is a single host
 Where possible, choose services and infrastructure that assume host failures
  happen.
 By building simple services composed of a single host, rather then multiple
  dependent hosts, one can create replicated service instances that can
  survive host failures.
 Make your services small and stateless.
 Relax consistency requirements.
2. Build Loosely Coupled Systems
The looser they’re coupled, the bigger they scale

 Loosely Coupled Dependencies
 Avoid complex design and interactions.
 Best Practices:
     Tiered Architecture
     Scale out units
     Single role
3. Implement Elasticity
Elasticity is fundamental property of cloud

 Ability to add and remove capacity as and when it is required.
 Use Elastic Load Balancing.
 Use Auto-Scaling (free)
4. Build Security in every layer
Design with security in mind


 Encrypt data at rest.
 Encrypt data at transit (SSL)
 Consider encrypted file system for sensitive data.
 Rotate your credentials, Pass in arguments encrypted.
 Use MultiFactor authentication.
 Restrict external access to specific IP ranges.
5. Don’t Fear Constraints
Re-think architectural constraints


 More RAM? – Distribute load across machines. Shared distributed cache.
 Better IOPS on database – Multiple read-only / Sharding.
 Performance – Caching at different levels
6. Think Parallel
Serial and Sequential is now history


 Experiment different architectures in parallel
 Multi-threading and Concurrent requests to cloud services.
 Run parallel Map Reduce Jobs
 Use Elastic Load balancing to distribute load across multiple servers.
 Decompose job into its simplest form – and with “Shared Nothing”
7. Leverage many storage options
One Size does not fit all


 Amazon S3 / Azure Blob – Large Static Objects
 Amazon Simple DB / Azure Tables – Data indexing and Querying
 Amazon RDS / SQL Azure – RDMBS Service – Automated and Managed
  MySQL/Azure
 Amazon Cloud Front / Azure CDN – Content Distribution
Cloud Architecture Lessons

 Design for failure and nothing fails.
 Loose coupling sets you free.
 Implement Elasticity.
 Build Security in every layer.
 Don’t fear constraints.
 Think Parallel.
 Leverage many storage options.
Windows
Why Windows Azure?
Azure Overview
Execution Models
Application Scenarios
Data Management
Storage: What are our options?
Blob Storage Concepts
Networking
Business Analytics
Messaging
Caching
Stages of Service Deployment
Packaging & Deployment
Questions
References
http://WindowsAzure.com
MISC SLIDES
App Scalability Patterns for State

 Data Grids                           CAP theorem: Data Consistency
 Distributed Caching                      Eventually Consistent

 HTTP Caching                             Atomic Data

     Reverse Proxy                    DB Strategies
     CDN                                  RDBMS
                                               Denormalization
 Concurrency
                                               Sharding
     Message-Passing
                                           NOSQL
     Dataflow
                                               Key-Value store
     Software Transactional Memory
                                               Document store
     Shared-State                             Data Structure store
 Partitioning                                 Graph database
App Scalability Patterns for Behavior

 Compute Grids                        Load Balancing
                                          Round-robin
 Event-Driven Architecture
                                          Random
    Messaging
                                          Weighted
    Actors
                                          Dynamic
    Enterprise Service Bus
                                       Parallel Computing
    Domain Events                        Master/Worker
    Event Stream Processing              Fork/Join
    Event Sourcing                       MapReduce
    Command & Query Responsibility       SPMD
     Segregation (CQRS)                   Loop Parallelism
The Availability Challenge

 Availability: Tolerate failures
 Traditional IT focuses on increasing MTTF
     Mean Time to Failure
 Cloud IT focuses on reducing MTTR
     Mean Time to Recovery
Data modelling

 Classic distributed systems focused on ACID semantics
     Atomicity: either the operation (e.g., write) is performed on all
    replicas or is not performed on any of them
     Consistency: after each operation all replicas reach the same state
     Isolation: no operation (e.g., read) can see the data from another
    operation (e.g., write) in an intermediate state
     Durability: once a write has been successful, that write will persist
    indefinitely
 Modern Internet Systems – focused on BASE
     Basically Available
     Soft-state (or scalable)
     Eventually consistent
CAP Theorem

Any distributed system has three properties – CAP
 Strong Consistency: all clients see the same view, even in the presence of
  updates
 High Availability: all clients can find some replica of the data, even in the
  presence of failures
 Partition-tolerance: the system properties hold even when the system is
  partitioned


As per CAP theorem you can only have two of these three properties. Choice
of which feature to discard determines the nature of your system.
Map Reduce

 Model for processing large data sets.
 Many tasks composed of processing lots of data to produce lots of other
  data
 Want to use hundreds or thousands of CPUs... but this needs to be easy!
 Contains Map and Reduce functions.
Programming model

 Input & Output: each a set of key/value pairs
 Programmer specifies two functions:
 map (in_key, in_value) -> list(out_key, intermediate_value)
 Processes input key/value pair
 Produces set of intermediate pairs
 reduce (out_key, list(intermediate_value)) -> list(out_value)
 Combines all intermediate values for a particular key
 Produces a set of merged output values (usually just one)
So How Does It Work?
So How Does It Work?
Azure and cloud design patterns
Example

 Page 1: the weather is good
 Page 2: today is good
 Page 3: good weather is good.
Map output

 Worker 1:
    (the 1), (weather 1), (is 1), (good 1).
 Worker 2:
    (today 1), (is 1), (good 1).
 Worker 3:
    (good 1), (weather 1), (is 1), (good 1).
Reduce Input

 Worker 1:
   (the 1)
 Worker 2:
   (is 1), (is 1), (is 1)
 Worker 3:
   (weather 1), (weather 1)
 Worker 4:
   (today 1)
 Worker 5:
   (good 1), (good 1), (good 1), (good 1)
Reduce Output

 Worker 1:
   (the 1)
 Worker 2:
   (is 3)
 Worker 3:
   (weather 2)
 Worker 4:
   (today 1)
 Worker 5:
   (good 4)
Conclusion – Map Reduce

 MapReduce has proven to be a useful abstraction
 Greatly simplifies large-scale computations
 Fun to use: focus on problem, let library deal w/ messy details
Azure and cloud design patterns
Azure and cloud design patterns
Azure and cloud design patterns

Contenu connexe

Tendances

Coherence Overview - OFM Canberra July 2014
Coherence Overview - OFM Canberra July 2014Coherence Overview - OFM Canberra July 2014
Coherence Overview - OFM Canberra July 2014Joelith
 
IMCSummit 2015 - Day 2 General Session - Flash-Extending In-Memory Computing
IMCSummit 2015 - Day 2 General Session - Flash-Extending In-Memory ComputingIMCSummit 2015 - Day 2 General Session - Flash-Extending In-Memory Computing
IMCSummit 2015 - Day 2 General Session - Flash-Extending In-Memory ComputingIn-Memory Computing Summit
 
Nordic infrastructure Conference 2017 - SQL Server on Linux Overview
Nordic infrastructure Conference 2017 - SQL Server on Linux OverviewNordic infrastructure Conference 2017 - SQL Server on Linux Overview
Nordic infrastructure Conference 2017 - SQL Server on Linux OverviewTravis Wright
 
A Tour of Azure SQL Databases (NOVA SQL UG 2020)
A Tour of Azure SQL Databases  (NOVA SQL UG 2020)A Tour of Azure SQL Databases  (NOVA SQL UG 2020)
A Tour of Azure SQL Databases (NOVA SQL UG 2020)Timothy McAliley
 
An Engineer's Intro to Oracle Coherence
An Engineer's Intro to Oracle CoherenceAn Engineer's Intro to Oracle Coherence
An Engineer's Intro to Oracle CoherenceOracle
 
Azure SQL Database & Azure SQL Data Warehouse
Azure SQL Database & Azure SQL Data WarehouseAzure SQL Database & Azure SQL Data Warehouse
Azure SQL Database & Azure SQL Data WarehouseMohamed Tawfik
 
Getting Started with MariaDB with Docker
Getting Started with MariaDB with DockerGetting Started with MariaDB with Docker
Getting Started with MariaDB with DockerMariaDB plc
 
Lessons from Large-Scale Cloud Software at Databricks
Lessons from Large-Scale Cloud Software at DatabricksLessons from Large-Scale Cloud Software at Databricks
Lessons from Large-Scale Cloud Software at DatabricksMatei Zaharia
 
Proactive Threat Detection and Safeguarding of Data for Enhanced Cyber resili...
Proactive Threat Detection and Safeguarding of Data for Enhanced Cyber resili...Proactive Threat Detection and Safeguarding of Data for Enhanced Cyber resili...
Proactive Threat Detection and Safeguarding of Data for Enhanced Cyber resili...Sandeep Patil
 
Implement SQL Server on an Azure VM
Implement SQL Server on an Azure VMImplement SQL Server on an Azure VM
Implement SQL Server on an Azure VMJames Serra
 
Development of concurrent services using In-Memory Data Grids
Development of concurrent services using In-Memory Data GridsDevelopment of concurrent services using In-Memory Data Grids
Development of concurrent services using In-Memory Data Gridsjlorenzocima
 
Introducing Azure SQL Database
Introducing Azure SQL DatabaseIntroducing Azure SQL Database
Introducing Azure SQL DatabaseJames Serra
 
The Evolution of SQL Server as a Service - SQL Azure Managed Instance
The Evolution of SQL Server as a Service - SQL Azure Managed InstanceThe Evolution of SQL Server as a Service - SQL Azure Managed Instance
The Evolution of SQL Server as a Service - SQL Azure Managed InstanceJavier Villegas
 
Understanding the IBM Power Systems Advantage
Understanding the IBM Power Systems AdvantageUnderstanding the IBM Power Systems Advantage
Understanding the IBM Power Systems AdvantageIBM Power Systems
 
Which Change Data Capture Strategy is Right for You?
Which Change Data Capture Strategy is Right for You?Which Change Data Capture Strategy is Right for You?
Which Change Data Capture Strategy is Right for You?Precisely
 

Tendances (20)

Oracle Coherence
Oracle CoherenceOracle Coherence
Oracle Coherence
 
Coherence Overview - OFM Canberra July 2014
Coherence Overview - OFM Canberra July 2014Coherence Overview - OFM Canberra July 2014
Coherence Overview - OFM Canberra July 2014
 
IMCSummit 2015 - Day 2 General Session - Flash-Extending In-Memory Computing
IMCSummit 2015 - Day 2 General Session - Flash-Extending In-Memory ComputingIMCSummit 2015 - Day 2 General Session - Flash-Extending In-Memory Computing
IMCSummit 2015 - Day 2 General Session - Flash-Extending In-Memory Computing
 
Nordic infrastructure Conference 2017 - SQL Server on Linux Overview
Nordic infrastructure Conference 2017 - SQL Server on Linux OverviewNordic infrastructure Conference 2017 - SQL Server on Linux Overview
Nordic infrastructure Conference 2017 - SQL Server on Linux Overview
 
A Tour of Azure SQL Databases (NOVA SQL UG 2020)
A Tour of Azure SQL Databases  (NOVA SQL UG 2020)A Tour of Azure SQL Databases  (NOVA SQL UG 2020)
A Tour of Azure SQL Databases (NOVA SQL UG 2020)
 
An Engineer's Intro to Oracle Coherence
An Engineer's Intro to Oracle CoherenceAn Engineer's Intro to Oracle Coherence
An Engineer's Intro to Oracle Coherence
 
Azure SQL Database & Azure SQL Data Warehouse
Azure SQL Database & Azure SQL Data WarehouseAzure SQL Database & Azure SQL Data Warehouse
Azure SQL Database & Azure SQL Data Warehouse
 
Getting Started with MariaDB with Docker
Getting Started with MariaDB with DockerGetting Started with MariaDB with Docker
Getting Started with MariaDB with Docker
 
Lessons from Large-Scale Cloud Software at Databricks
Lessons from Large-Scale Cloud Software at DatabricksLessons from Large-Scale Cloud Software at Databricks
Lessons from Large-Scale Cloud Software at Databricks
 
Oracle Coherence
Oracle CoherenceOracle Coherence
Oracle Coherence
 
Proactive Threat Detection and Safeguarding of Data for Enhanced Cyber resili...
Proactive Threat Detection and Safeguarding of Data for Enhanced Cyber resili...Proactive Threat Detection and Safeguarding of Data for Enhanced Cyber resili...
Proactive Threat Detection and Safeguarding of Data for Enhanced Cyber resili...
 
Data Management
Data ManagementData Management
Data Management
 
Implement SQL Server on an Azure VM
Implement SQL Server on an Azure VMImplement SQL Server on an Azure VM
Implement SQL Server on an Azure VM
 
Multi-Tenancy
Multi-TenancyMulti-Tenancy
Multi-Tenancy
 
Development of concurrent services using In-Memory Data Grids
Development of concurrent services using In-Memory Data GridsDevelopment of concurrent services using In-Memory Data Grids
Development of concurrent services using In-Memory Data Grids
 
Introducing Azure SQL Database
Introducing Azure SQL DatabaseIntroducing Azure SQL Database
Introducing Azure SQL Database
 
The Evolution of SQL Server as a Service - SQL Azure Managed Instance
The Evolution of SQL Server as a Service - SQL Azure Managed InstanceThe Evolution of SQL Server as a Service - SQL Azure Managed Instance
The Evolution of SQL Server as a Service - SQL Azure Managed Instance
 
Understanding the IBM Power Systems Advantage
Understanding the IBM Power Systems AdvantageUnderstanding the IBM Power Systems Advantage
Understanding the IBM Power Systems Advantage
 
Which Change Data Capture Strategy is Right for You?
Which Change Data Capture Strategy is Right for You?Which Change Data Capture Strategy is Right for You?
Which Change Data Capture Strategy is Right for You?
 
1200x630 1
1200x630 11200x630 1
1200x630 1
 

Similaire à Azure and cloud design patterns

Application architecture for cloud
Application architecture for cloudApplication architecture for cloud
Application architecture for cloudMarco Parenzan
 
SQL and NoSQL in SQL Server
SQL and NoSQL in SQL ServerSQL and NoSQL in SQL Server
SQL and NoSQL in SQL ServerMichael Rys
 
20141021 AWS Cloud Taekwon - Startup Best Practices on AWS
20141021 AWS Cloud Taekwon - Startup Best Practices on AWS20141021 AWS Cloud Taekwon - Startup Best Practices on AWS
20141021 AWS Cloud Taekwon - Startup Best Practices on AWSAmazon Web Services Korea
 
AWS Summit 2011: Architecting in the cloud
AWS Summit 2011: Architecting in the cloudAWS Summit 2011: Architecting in the cloud
AWS Summit 2011: Architecting in the cloudAmazon Web Services
 
Application architecture for the rest of us - php xperts devcon 2012
Application architecture for the rest of us -  php xperts devcon 2012Application architecture for the rest of us -  php xperts devcon 2012
Application architecture for the rest of us - php xperts devcon 2012M N Islam Shihan
 
Microsoft Azure Cloud Basics Tutorial
Microsoft Azure Cloud Basics TutorialMicrosoft Azure Cloud Basics Tutorial
Microsoft Azure Cloud Basics TutorialIIMSE Edu
 
Handling Data in Mega Scale Systems
Handling Data in Mega Scale SystemsHandling Data in Mega Scale Systems
Handling Data in Mega Scale SystemsDirecti Group
 
Designing distributed systems
Designing distributed systemsDesigning distributed systems
Designing distributed systemsMalisa Ncube
 
NoSQL Introduction, Theory, Implementations
NoSQL Introduction, Theory, ImplementationsNoSQL Introduction, Theory, Implementations
NoSQL Introduction, Theory, ImplementationsFirat Atagun
 
Cloud computing skepticism - But i'm sure
Cloud computing skepticism - But i'm sureCloud computing skepticism - But i'm sure
Cloud computing skepticism - But i'm sureNguyen Duong
 
AWS Summit 2011: Big Data Analytics in the AWS cloud
AWS Summit 2011: Big Data Analytics in the AWS cloudAWS Summit 2011: Big Data Analytics in the AWS cloud
AWS Summit 2011: Big Data Analytics in the AWS cloudAmazon Web Services
 
Everything comes in 3's
Everything comes in 3'sEverything comes in 3's
Everything comes in 3'sdelagoya
 
2014.11.14 Data Opportunities with Azure
2014.11.14 Data Opportunities with Azure2014.11.14 Data Opportunities with Azure
2014.11.14 Data Opportunities with AzureMarco Parenzan
 
ARC205 Building Web-scale Applications Architectures with AWS - AWS re: Inven...
ARC205 Building Web-scale Applications Architectures with AWS - AWS re: Inven...ARC205 Building Web-scale Applications Architectures with AWS - AWS re: Inven...
ARC205 Building Web-scale Applications Architectures with AWS - AWS re: Inven...Amazon Web Services
 
AI&BigData Lab 2016. Сарапин Виктор: Размер имеет значение: анализ по требова...
AI&BigData Lab 2016. Сарапин Виктор: Размер имеет значение: анализ по требова...AI&BigData Lab 2016. Сарапин Виктор: Размер имеет значение: анализ по требова...
AI&BigData Lab 2016. Сарапин Виктор: Размер имеет значение: анализ по требова...GeeksLab Odessa
 
Compare Clustering Methods for MS SQL Server
Compare Clustering Methods for MS SQL ServerCompare Clustering Methods for MS SQL Server
Compare Clustering Methods for MS SQL ServerAlexDepo
 
AWS Webcast - Best Practices in Architecting for the Cloud
AWS Webcast - Best Practices in Architecting for the CloudAWS Webcast - Best Practices in Architecting for the Cloud
AWS Webcast - Best Practices in Architecting for the CloudAmazon Web Services
 

Similaire à Azure and cloud design patterns (20)

Application architecture for cloud
Application architecture for cloudApplication architecture for cloud
Application architecture for cloud
 
SQL and NoSQL in SQL Server
SQL and NoSQL in SQL ServerSQL and NoSQL in SQL Server
SQL and NoSQL in SQL Server
 
20141021 AWS Cloud Taekwon - Startup Best Practices on AWS
20141021 AWS Cloud Taekwon - Startup Best Practices on AWS20141021 AWS Cloud Taekwon - Startup Best Practices on AWS
20141021 AWS Cloud Taekwon - Startup Best Practices on AWS
 
AWS Summit 2011: Architecting in the cloud
AWS Summit 2011: Architecting in the cloudAWS Summit 2011: Architecting in the cloud
AWS Summit 2011: Architecting in the cloud
 
Application architecture for the rest of us - php xperts devcon 2012
Application architecture for the rest of us -  php xperts devcon 2012Application architecture for the rest of us -  php xperts devcon 2012
Application architecture for the rest of us - php xperts devcon 2012
 
Microsoft Azure Cloud Basics Tutorial
Microsoft Azure Cloud Basics TutorialMicrosoft Azure Cloud Basics Tutorial
Microsoft Azure Cloud Basics Tutorial
 
Handling Data in Mega Scale Systems
Handling Data in Mega Scale SystemsHandling Data in Mega Scale Systems
Handling Data in Mega Scale Systems
 
Designing distributed systems
Designing distributed systemsDesigning distributed systems
Designing distributed systems
 
NoSQL Introduction, Theory, Implementations
NoSQL Introduction, Theory, ImplementationsNoSQL Introduction, Theory, Implementations
NoSQL Introduction, Theory, Implementations
 
Cloud computing skepticism - But i'm sure
Cloud computing skepticism - But i'm sureCloud computing skepticism - But i'm sure
Cloud computing skepticism - But i'm sure
 
Clustering van IT-componenten
Clustering van IT-componentenClustering van IT-componenten
Clustering van IT-componenten
 
AWS Summit 2011: Big Data Analytics in the AWS cloud
AWS Summit 2011: Big Data Analytics in the AWS cloudAWS Summit 2011: Big Data Analytics in the AWS cloud
AWS Summit 2011: Big Data Analytics in the AWS cloud
 
Everything comes in 3's
Everything comes in 3'sEverything comes in 3's
Everything comes in 3's
 
Software Engineering 101
Software Engineering 101Software Engineering 101
Software Engineering 101
 
2014.11.14 Data Opportunities with Azure
2014.11.14 Data Opportunities with Azure2014.11.14 Data Opportunities with Azure
2014.11.14 Data Opportunities with Azure
 
ARC205 Building Web-scale Applications Architectures with AWS - AWS re: Inven...
ARC205 Building Web-scale Applications Architectures with AWS - AWS re: Inven...ARC205 Building Web-scale Applications Architectures with AWS - AWS re: Inven...
ARC205 Building Web-scale Applications Architectures with AWS - AWS re: Inven...
 
AI&BigData Lab 2016. Сарапин Виктор: Размер имеет значение: анализ по требова...
AI&BigData Lab 2016. Сарапин Виктор: Размер имеет значение: анализ по требова...AI&BigData Lab 2016. Сарапин Виктор: Размер имеет значение: анализ по требова...
AI&BigData Lab 2016. Сарапин Виктор: Размер имеет значение: анализ по требова...
 
Compare Clustering Methods for MS SQL Server
Compare Clustering Methods for MS SQL ServerCompare Clustering Methods for MS SQL Server
Compare Clustering Methods for MS SQL Server
 
Database as a Service - Tutorial @ICDE 2010
Database as a Service - Tutorial @ICDE 2010Database as a Service - Tutorial @ICDE 2010
Database as a Service - Tutorial @ICDE 2010
 
AWS Webcast - Best Practices in Architecting for the Cloud
AWS Webcast - Best Practices in Architecting for the CloudAWS Webcast - Best Practices in Architecting for the Cloud
AWS Webcast - Best Practices in Architecting for the Cloud
 

Plus de Venkatesh Narayanan

Plus de Venkatesh Narayanan (9)

Azure ML Training - Deep Dive
Azure ML Training - Deep DiveAzure ML Training - Deep Dive
Azure ML Training - Deep Dive
 
Azure Functions - Introduction
Azure Functions - IntroductionAzure Functions - Introduction
Azure Functions - Introduction
 
Azure Active Directory - An Introduction
Azure Active Directory  - An IntroductionAzure Active Directory  - An Introduction
Azure Active Directory - An Introduction
 
Angular js 1.0-fundamentals
Angular js 1.0-fundamentalsAngular js 1.0-fundamentals
Angular js 1.0-fundamentals
 
Big data in Azure
Big data in AzureBig data in Azure
Big data in Azure
 
Markdown – An Introduction
Markdown – An IntroductionMarkdown – An Introduction
Markdown – An Introduction
 
Introduction to facebook platform
Introduction to facebook platformIntroduction to facebook platform
Introduction to facebook platform
 
Introduction to o data
Introduction to o dataIntroduction to o data
Introduction to o data
 
Threading net 4.5
Threading net 4.5Threading net 4.5
Threading net 4.5
 

Dernier

Crea il tuo assistente AI con lo Stregatto (open source python framework)
Crea il tuo assistente AI con lo Stregatto (open source python framework)Crea il tuo assistente AI con lo Stregatto (open source python framework)
Crea il tuo assistente AI con lo Stregatto (open source python framework)Commit University
 
Nanopower In Semiconductor Industry.pdf
Nanopower  In Semiconductor Industry.pdfNanopower  In Semiconductor Industry.pdf
Nanopower In Semiconductor Industry.pdfPedro Manuel
 
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDE
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDEADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDE
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDELiveplex
 
Linked Data in Production: Moving Beyond Ontologies
Linked Data in Production: Moving Beyond OntologiesLinked Data in Production: Moving Beyond Ontologies
Linked Data in Production: Moving Beyond OntologiesDavid Newbury
 
Empowering Africa's Next Generation: The AI Leadership Blueprint
Empowering Africa's Next Generation: The AI Leadership BlueprintEmpowering Africa's Next Generation: The AI Leadership Blueprint
Empowering Africa's Next Generation: The AI Leadership BlueprintMahmoud Rabie
 
Salesforce Miami User Group Event - 1st Quarter 2024
Salesforce Miami User Group Event - 1st Quarter 2024Salesforce Miami User Group Event - 1st Quarter 2024
Salesforce Miami User Group Event - 1st Quarter 2024SkyPlanner
 
Comparing Sidecar-less Service Mesh from Cilium and Istio
Comparing Sidecar-less Service Mesh from Cilium and IstioComparing Sidecar-less Service Mesh from Cilium and Istio
Comparing Sidecar-less Service Mesh from Cilium and IstioChristian Posta
 
Meet the new FSP 3000 M-Flex800™
Meet the new FSP 3000 M-Flex800™Meet the new FSP 3000 M-Flex800™
Meet the new FSP 3000 M-Flex800™Adtran
 
COMPUTER 10 Lesson 8 - Building a Website
COMPUTER 10 Lesson 8 - Building a WebsiteCOMPUTER 10 Lesson 8 - Building a Website
COMPUTER 10 Lesson 8 - Building a Websitedgelyza
 
UiPath Platform: The Backend Engine Powering Your Automation - Session 1
UiPath Platform: The Backend Engine Powering Your Automation - Session 1UiPath Platform: The Backend Engine Powering Your Automation - Session 1
UiPath Platform: The Backend Engine Powering Your Automation - Session 1DianaGray10
 
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdf
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdfIaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdf
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdfDaniel Santiago Silva Capera
 
Building AI-Driven Apps Using Semantic Kernel.pptx
Building AI-Driven Apps Using Semantic Kernel.pptxBuilding AI-Driven Apps Using Semantic Kernel.pptx
Building AI-Driven Apps Using Semantic Kernel.pptxUdaiappa Ramachandran
 
Building Your Own AI Instance (TBLC AI )
Building Your Own AI Instance (TBLC AI )Building Your Own AI Instance (TBLC AI )
Building Your Own AI Instance (TBLC AI )Brian Pichman
 
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...DianaGray10
 
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...Aggregage
 
Videogame localization & technology_ how to enhance the power of translation.pdf
Videogame localization & technology_ how to enhance the power of translation.pdfVideogame localization & technology_ how to enhance the power of translation.pdf
Videogame localization & technology_ how to enhance the power of translation.pdfinfogdgmi
 
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019IES VE
 

Dernier (20)

Crea il tuo assistente AI con lo Stregatto (open source python framework)
Crea il tuo assistente AI con lo Stregatto (open source python framework)Crea il tuo assistente AI con lo Stregatto (open source python framework)
Crea il tuo assistente AI con lo Stregatto (open source python framework)
 
Nanopower In Semiconductor Industry.pdf
Nanopower  In Semiconductor Industry.pdfNanopower  In Semiconductor Industry.pdf
Nanopower In Semiconductor Industry.pdf
 
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDE
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDEADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDE
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDE
 
201610817 - edge part1
201610817 - edge part1201610817 - edge part1
201610817 - edge part1
 
Linked Data in Production: Moving Beyond Ontologies
Linked Data in Production: Moving Beyond OntologiesLinked Data in Production: Moving Beyond Ontologies
Linked Data in Production: Moving Beyond Ontologies
 
Empowering Africa's Next Generation: The AI Leadership Blueprint
Empowering Africa's Next Generation: The AI Leadership BlueprintEmpowering Africa's Next Generation: The AI Leadership Blueprint
Empowering Africa's Next Generation: The AI Leadership Blueprint
 
Salesforce Miami User Group Event - 1st Quarter 2024
Salesforce Miami User Group Event - 1st Quarter 2024Salesforce Miami User Group Event - 1st Quarter 2024
Salesforce Miami User Group Event - 1st Quarter 2024
 
20230104 - machine vision
20230104 - machine vision20230104 - machine vision
20230104 - machine vision
 
Comparing Sidecar-less Service Mesh from Cilium and Istio
Comparing Sidecar-less Service Mesh from Cilium and IstioComparing Sidecar-less Service Mesh from Cilium and Istio
Comparing Sidecar-less Service Mesh from Cilium and Istio
 
20150722 - AGV
20150722 - AGV20150722 - AGV
20150722 - AGV
 
Meet the new FSP 3000 M-Flex800™
Meet the new FSP 3000 M-Flex800™Meet the new FSP 3000 M-Flex800™
Meet the new FSP 3000 M-Flex800™
 
COMPUTER 10 Lesson 8 - Building a Website
COMPUTER 10 Lesson 8 - Building a WebsiteCOMPUTER 10 Lesson 8 - Building a Website
COMPUTER 10 Lesson 8 - Building a Website
 
UiPath Platform: The Backend Engine Powering Your Automation - Session 1
UiPath Platform: The Backend Engine Powering Your Automation - Session 1UiPath Platform: The Backend Engine Powering Your Automation - Session 1
UiPath Platform: The Backend Engine Powering Your Automation - Session 1
 
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdf
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdfIaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdf
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdf
 
Building AI-Driven Apps Using Semantic Kernel.pptx
Building AI-Driven Apps Using Semantic Kernel.pptxBuilding AI-Driven Apps Using Semantic Kernel.pptx
Building AI-Driven Apps Using Semantic Kernel.pptx
 
Building Your Own AI Instance (TBLC AI )
Building Your Own AI Instance (TBLC AI )Building Your Own AI Instance (TBLC AI )
Building Your Own AI Instance (TBLC AI )
 
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...
 
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...
 
Videogame localization & technology_ how to enhance the power of translation.pdf
Videogame localization & technology_ how to enhance the power of translation.pdfVideogame localization & technology_ how to enhance the power of translation.pdf
Videogame localization & technology_ how to enhance the power of translation.pdf
 
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019
 

Azure and cloud design patterns

  • 2. Agenda  Design Considerations in moving to Cloud  Introduction to Windows Azure  Demo
  • 4. Business Benefits of Cloud Computing  Almost zero upfront infrastructure investment.  Just-in-time infrastructure.  More efficient resource utilization.  Usage based costing.  Reduced time to Market.
  • 7. Challenges Faced by Apps in the Cloud  Application Scalability  Cloud promises rapid (de)provisioning of resources.  How do you tap into that to create scalable systems?  Application Availability  Underlying resource failures happen … usually more frequently than in traditional data centers.  How do you overcome that to create highly available systems?
  • 8. The Scalability Challenge  Two different components to scale:  State (inputs, data store, output)  Behavior (business logic)  Any non-trivial application has both.  Scaling one component means scaling the other, too.
  • 9. Scalability Considerations  Performance vs Scalability  Latency vs Throughput  Availability vs Consistency How do you manage overload ?
  • 10. Scalable Service  A scalable architecture is critical to take advantage of scalable infrastructure.  Characteristic of Scalable Service:  Increasing resources results in a proportional increase in performance  A scalable service is capable of handling heterogeneity  A scalable service is operationally efficient  A scalable service is resilient  A scalable service becomes more cost effective when it grows.
  • 11. 1. Design for Failure and nothing will really fail  Avoid single points of failure  Assume everything fails and design backwards.  Applications should continue to function even if the physical hardware fails or removed or replaced.
  • 12. Design for Failure contd..  Unit of failure is a single host  Where possible, choose services and infrastructure that assume host failures happen.  By building simple services composed of a single host, rather then multiple dependent hosts, one can create replicated service instances that can survive host failures.  Make your services small and stateless.  Relax consistency requirements.
  • 13. 2. Build Loosely Coupled Systems The looser they’re coupled, the bigger they scale  Loosely Coupled Dependencies  Avoid complex design and interactions.  Best Practices:  Tiered Architecture  Scale out units  Single role
  • 14. 3. Implement Elasticity Elasticity is fundamental property of cloud  Ability to add and remove capacity as and when it is required.  Use Elastic Load Balancing.  Use Auto-Scaling (free)
  • 15. 4. Build Security in every layer Design with security in mind  Encrypt data at rest.  Encrypt data at transit (SSL)  Consider encrypted file system for sensitive data.  Rotate your credentials, Pass in arguments encrypted.  Use MultiFactor authentication.  Restrict external access to specific IP ranges.
  • 16. 5. Don’t Fear Constraints Re-think architectural constraints  More RAM? – Distribute load across machines. Shared distributed cache.  Better IOPS on database – Multiple read-only / Sharding.  Performance – Caching at different levels
  • 17. 6. Think Parallel Serial and Sequential is now history  Experiment different architectures in parallel  Multi-threading and Concurrent requests to cloud services.  Run parallel Map Reduce Jobs  Use Elastic Load balancing to distribute load across multiple servers.  Decompose job into its simplest form – and with “Shared Nothing”
  • 18. 7. Leverage many storage options One Size does not fit all  Amazon S3 / Azure Blob – Large Static Objects  Amazon Simple DB / Azure Tables – Data indexing and Querying  Amazon RDS / SQL Azure – RDMBS Service – Automated and Managed MySQL/Azure  Amazon Cloud Front / Azure CDN – Content Distribution
  • 19. Cloud Architecture Lessons  Design for failure and nothing fails.  Loose coupling sets you free.  Implement Elasticity.  Build Security in every layer.  Don’t fear constraints.  Think Parallel.  Leverage many storage options.
  • 26. Storage: What are our options?
  • 32. Stages of Service Deployment
  • 38. App Scalability Patterns for State  Data Grids  CAP theorem: Data Consistency  Distributed Caching  Eventually Consistent  HTTP Caching  Atomic Data  Reverse Proxy  DB Strategies  CDN  RDBMS  Denormalization  Concurrency  Sharding  Message-Passing  NOSQL  Dataflow  Key-Value store  Software Transactional Memory  Document store  Shared-State  Data Structure store  Partitioning  Graph database
  • 39. App Scalability Patterns for Behavior  Compute Grids  Load Balancing  Round-robin  Event-Driven Architecture  Random  Messaging  Weighted  Actors  Dynamic  Enterprise Service Bus  Parallel Computing  Domain Events  Master/Worker  Event Stream Processing  Fork/Join  Event Sourcing  MapReduce  Command & Query Responsibility  SPMD Segregation (CQRS)  Loop Parallelism
  • 40. The Availability Challenge  Availability: Tolerate failures  Traditional IT focuses on increasing MTTF  Mean Time to Failure  Cloud IT focuses on reducing MTTR  Mean Time to Recovery
  • 41. Data modelling  Classic distributed systems focused on ACID semantics  Atomicity: either the operation (e.g., write) is performed on all replicas or is not performed on any of them  Consistency: after each operation all replicas reach the same state  Isolation: no operation (e.g., read) can see the data from another operation (e.g., write) in an intermediate state  Durability: once a write has been successful, that write will persist indefinitely  Modern Internet Systems – focused on BASE  Basically Available  Soft-state (or scalable)  Eventually consistent
  • 42. CAP Theorem Any distributed system has three properties – CAP  Strong Consistency: all clients see the same view, even in the presence of updates  High Availability: all clients can find some replica of the data, even in the presence of failures  Partition-tolerance: the system properties hold even when the system is partitioned As per CAP theorem you can only have two of these three properties. Choice of which feature to discard determines the nature of your system.
  • 43. Map Reduce  Model for processing large data sets.  Many tasks composed of processing lots of data to produce lots of other data  Want to use hundreds or thousands of CPUs... but this needs to be easy!  Contains Map and Reduce functions.
  • 44. Programming model  Input & Output: each a set of key/value pairs  Programmer specifies two functions:  map (in_key, in_value) -> list(out_key, intermediate_value)  Processes input key/value pair  Produces set of intermediate pairs  reduce (out_key, list(intermediate_value)) -> list(out_value)  Combines all intermediate values for a particular key  Produces a set of merged output values (usually just one)
  • 45. So How Does It Work?
  • 46. So How Does It Work?
  • 48. Example  Page 1: the weather is good  Page 2: today is good  Page 3: good weather is good.
  • 49. Map output  Worker 1:  (the 1), (weather 1), (is 1), (good 1).  Worker 2:  (today 1), (is 1), (good 1).  Worker 3:  (good 1), (weather 1), (is 1), (good 1).
  • 50. Reduce Input  Worker 1:  (the 1)  Worker 2:  (is 1), (is 1), (is 1)  Worker 3:  (weather 1), (weather 1)  Worker 4:  (today 1)  Worker 5:  (good 1), (good 1), (good 1), (good 1)
  • 51. Reduce Output  Worker 1:  (the 1)  Worker 2:  (is 3)  Worker 3:  (weather 2)  Worker 4:  (today 1)  Worker 5:  (good 4)
  • 52. Conclusion – Map Reduce  MapReduce has proven to be a useful abstraction  Greatly simplifies large-scale computations  Fun to use: focus on problem, let library deal w/ messy details