Jinesh Varia, Technology Evangelist, Discusses AWS architecture best practices and design patterns at the AWS Enterprise Tour - SF - 2010 http://jineshvaria.s3.amazonaws.com/public/cloudbestpractices-jvaria.pdf

1. Jinesh Varia Technology Evangelist jvaria@amazon.com Architectural Design Patterns in Cloud Computing

2. They have asked me to talk But I am here to listen Please Send Feedback jvaria@amazon.com Twitter: @jinman

3. Cloud Best Practices Whitepaper Prescriptive guidance to Cloud Architects Just Google for “Cloud Best Practices” to find the link http://media.amazonwebservices.com/AWS_Cloud_Best_Practices.pdf

4. Cloud Computing Attributes What makes the Cloud so attractive Abstract Resources Focus on your needs, not on hardware specs. As your needs change, so should your resources. On-Demand Provisioning Ask for what you need, exactly when you need it. Get rid of it when you don’t need Scalability in minutes Scale out or in depending on usage needs. Pay per consumption No contracts or long-term commitments. Pay only for what you use. Efficiency of Experts Utilize the skills, knowledge and resources of experts.

5. The “Living and Evolving” Cloud The “Living and Evolving” Cloud AWS services and basic terminology Most Applications Need: Compute Storage Messaging Payment Distribution Scale Analytics Your Application Amazon RDS Amazon CloudFront Amazon SQS Queues Amazon SimpleDB Domains Payment : Amazon FPS/ DevPay Amazon Elastic MapReduceJobFlows Amazon SNS Topics Amazon S3 Objects and Buckets Auto-Scaling Elastic LB Cloud Watch Amazon EC2 Instances(On-Demand, Reserved, Spot) EBS Volumes Snapshots Amazon Virtual Private Cloud Amazon WorldWidePhysical Infrastructure (Geographical Regions, Availability Zones, Edge Locations)

7. Boot from Amazon EBS» Amazon CloudFront Streaming » Amazon VPC enters Unlimited Beta » AWS Region in Northern California » International Support for AWS Import/Export » AWS Multi-Factor Authentication » Virtual Private Cloud » Lower Reserved Instance Pricing » Reserved Instances in EU Region » Elastic MapReduce » SQS in EU Region » Amazon RDS » High-Memory Instances » Lower EC2 Pricing » New SimpleDB Features » FPS General Availability » Amazon SNS » AWS Security Center 2009 Jan 2010 Jan Jul Sep Oct Dec Aug Nov Feb Mar Apr Jun May Feb Mar » Amazon EC2 with Windows » Amazon EC2 in EU Region » AWS Toolkit for Eclipse » Amazon EC2 Reserved Instances » Amazon CloudFront Private Content » SAS70 Type II Audit » AWS SDK for .NET » Amazon Elastic MapReduce in Europe » Amazon EC2 Reserved Instances with Windows, Extra Large High Memory Instances » Amazon S3 Versioning Feature » Consolidated Billing for AWS » Lower pricing for Outbound Data Transfer » AWS Import/Export » New CloudFront Feature » Monitoring, Auto Scaling & Elastic Load Balancing » EBS Shared Snapshots » SimpleDB in EU Region » Monitoring, Auto Scaling & Elastic Load Balancing in EU » Lower pricing tiers for Amazon CloudFront » AWS Management Console

8. Scalability Build Scalable Architecture on AWS A scalable architecture is critical to take advantage of a scalable infrastructure Characteristics of Truly Scalable Service Increasing resources results in a proportional increase in performance A scalable service is capable of handling heterogeneity A scalable service is operationally efficient A scalable service is resilient A scalable service becomes more cost effective when it grows

9. Cloud Architecture Lessons using Amazon Web Services 1. Design for failure and nothing fails 2. Loose coupling sets you free 3. Implement “Elasticity” 4. Build Security in every layer 5. Don't fear constraints 6. Think Parallel 7. Leverage different storage options

10. 1. Design for Failure and nothing will really fail "Everything fails, all the time" Werner Vogels, CTO Amazon.com Avoid single points of failure Assume everything fails, and design backwards Goal: Applications should continue to function even if the underlying physical hardware fails or is removed or replaced.

11. Design for Failure with AWS Tools to make your life easier Use Elastic IP addresses for consistent and re-mappable routes Use multiple Amazon EC2 Availability Zones (AZs) Create multiple database slaves across AZs Use real-time monitoring (Amazon CloudWatch) Use Amazon Elastic Block Store (EBS) for persistent file systems

12. YourWebsite.com EC2 Instance A EC2 Instance B MASTER SLAVE MASTER Replication LOG Volume DATA Volume DATA Volume

13. YourWebTwoDotZeroName.com Availability Zone 2 EC2 Instance B EC2 Instance A Availability Zone 1 MASTER SLAVE MASTER Replication DATA Volume DATA Volume LOG Volume LOG Volume Amazon S3

14. www.YourWebsite.com Elastic IP 183.12.43.11 Production instance App Version 1.1 Database

15. www.YourWebsite.com Staging.YourWebsite.com Elastic IP 183.12.43.11 Dynamic IP 172.0.1.13 Production instance App Version 1.1 Staging instance App Version 1.2 Database

16. www.YourWebsite.com Elastic IP 183.12.43.11 Staging instance App Version 1.2 Database

17. 2. Build Loosely Coupled Systems The looser they're coupled, the bigger they scale Independent components Design everything as a Black Box De-coupling for Hybrid models Load-balance clusters Use Amazon SQS as Buffers Tight Coupling Controller A Controller B Controller C Q Q Q Loose Coupling using Queues Controller A Controller B Controller C

18. MyWebSite.com Exterior Firewall Hardware or Software Solution to open standard Ports (80, 443) Web Load BalancerHardware or Software solution to distribute traffic over web servers LB Web Tier Fleet of machines handling HTTP requests. Web Server Web Server Backend Firewall Limits access to application tier from web tier LB App Load Balancer Hardware or Software solution to spread traffic over app servers App Server Tier Fleet of machines handling Application specific workloads Caching server machines can be implemented at this layer App Server App Server App server Backups on Tapes Periodic backups stored on Tapes usually managed by 3rd party at their site Data Tier Database Server machines with master and local running separately, Network storage for Static objects MySQL Master MySQL(Slave) Tapes

19. MyWebSite.com DNS Elastic Load Balancer ELB to spread traffic to Web Server Auto-scaling groups LB ELB: Web Tier Exterior Firewall no longer needed because EC2 instances are controlled with Security Groups Availability Zone #1 Availability Zone 2 Auto-scaling group : Web Tier Auto-scaling group : Web Tier Availability Zone #n Web Server Web Server Web Server Web Server Auto-scaling Web Tier Group of EC2 instances handling HTTP requests. Edge Caching High Volume Static Content is edge cached using CloudFront Backend Firewall no longer needed SLB App Server Load Balancer Software LB (e.g. HAProxy) on EC2 instance to spread traffic over app server cluster SLB Auto-scaling group : App Tier Auto-scaling group : App Tier Auto-scaling App Tier Group of EC2 instances running the actual app. Instances belong to Auto-scaling group. Caching servers instances can be implemented at this layer App Server App Server App Server App Server Tomcat Tomcat Cloud Front Amazon S3 RDS Slave RDS Master RDS Slave DB Tier MySQL RDS DB Instances (master, local slave, x-AZ slave for failover) , Automated backups to S3 all managed by AWS Backups Amazon S3 used for storing Static Objects and Backups

20. 3. Implement Elasticity Elasticity is fundamental property of the Cloud Don’t assume healthor fixed location of components Use designs that are resilient to reboot and re-launch Bootstrapyour instances: Instances on boot will ask a question “Who am I & what is my role?” Enable dynamic configuration Use Auto-scaling (Free) Use Elastic Load Balancing on multiple layers Use configurations in SimpleDB to bootstrap instance

21. 3. Implement Elasticity Managed Development Environment Automate everything SaaS Paid AMI Web 2.0 Marketing Campaign Dev/Test Apps Prod Managed Development Environment Automated Deployment Environment Cloud-powered Software Lifecycle management AWS Cloud AWS Cloud AWS Cloud ISV Department Enterprise IT

22. 3. Implement Elasticity Standardized Technology Stacks Standardized Application Stacks Apache IIS Apache Tomcat ASP.NET Mongrel Web Server Struts ASP.NET MVC Rails App Server Your Code Your Code Your Code MVC Log4J Log4Net logger Your Code Spring Spring.NET RubyGems Libraries Hibernate nHibernate memcached Packages JEE .NET Ruby Runtime DB Caching Linux Windows Centos Framework OS Java Stack .NET Stack RoR stack

23. 3. Implement Elasticity 3 Approaches to design MDE 3 approaches to designing your AMIs Easier to Setup Inventory of fully baked AMIs (Frozen Pizza Model) “Golden AMIs” with fetch on boot (Take N’ Bake Papa Murphy Model) AMIs with JeOS and “Chef” Agent (Made to Order Pizza Model) More Control Easier to maintain

24. 3. Implement Elasticity 3 Approaches to design MDE 1. Frozen Pizza Model IIS IIS IIS IIS IIS IIS ASP.NET IIS IIS IIS IIS IIS ASP.NET MVC ASP.NET MVC ASP.NET MVC ASP.NET MVC ASP.NET MVC ASP.NET MVC Your Code Your Code Your Code Your Code Your Code Your Code Log4Net Log4Net Log4Net Log4Net Log4Net Log4Net Spring.NET Spring.NET Spring.NET Spring.NET Spring.NET Spring.NET nHibernate nHibernate nHibernate nHibernate nHibernate nHibernate .NET .NET .NET .NET .NET .NET Amazon EC2 Windows Windows Windows Windows Windows Windows .NET AMI .NET Stack

25. 3. Implement Elasticity 3 Approaches to design MDE “Golden AMIs” with fetch on boot 2. Take N Bake Pizza Model IIS IIS Source Control Fetch on boot time Your Code ASP.NET MVC Amazon S3 Your Code ASP.NET MVC Log4Net nHibernate Log4Net Spring.NET Spring.NET IIS IIS IIS IIS IIS nHibernate IIS IIS IIS IIS IIS .NET .NET .NET .NET .NET .NET Windows Amazon EC2 Windows Windows Windows Windows Windows .NET AMI .NET Stack

26. 3. Implement Elasticity 3 Approaches to design MDE 3. Made to Order Pizza Model Apache Mongrel Source Control Rails Your Code Cookbooks Recipes Your Code Amazon S3 ASP.NET MVC Log4Net .NET IIS Chef Server nHibernate logger IIS Spring.NET RubyGems memcached Ruby Runtime CHEF Agent CHEF Agent Centos Windows Windows Amazon EC2 AMI (JeOS) RoR Stack

27. 3. Implement Elasticity 3 Approaches to design MDE 3 approaches to designing your AMIs Easier to Setup Inventory of fully baked AMIs (Frozen/Ready made) “Golden AMIs” with fetch on boot (Take N’ Bake) AMIs with JeOS and “Chef” Agent (Made to Order) More Control Easier to maintain

28. 4. Build Security in every layer Design with Security in mind With cloud, you lose a little bit of physical control but not your ownership Create distinct Security Groups for each Amazon EC2 cluster Use group-based rules for controlling access between layers Restrict external access to specific IP ranges Encrypt data “at-rest” in Amazon S3 Encrypt data “in-transit” (SSL) Consider encrypted file systems in EC2 for sensitive data Rotate your AWS Credentials, Pass in as arguments encrypted Use MultiFactor Authentication

29. 5. Don't fear constraints Re-think architectural constraints More RAM? Distribute load across machines Shared distributed cache Better IOPS on my database? Multiple read-only / sharding / DB clustering Hardware Config does not match? Implement Elasticity Your hardware failed or messed up config? simply throw it away and switch to new hardware with no additional cost Performance Caching at different levels (Page, Render, DB)

30. 6. Think Parallel Serial and Sequential is now history Experiment different architectures in parallel Multi-treading and Concurrent requests to cloud services Run parallel MapReduce Jobs Use Elastic Load Balancing to distribute load across multiple servers Decompose a Job into its simplest form

31. 6. Leverage many storage options One size DOES NOT fit all Amazon S3: large static objects Amazon Cloudfront: content distribution Amazon SimpleDB: simple data indexing/querying Amazon EC2 local disc drive : transient data Amazon EBS: persistent storage for any RDBMS + Snapshots on S3 Amazon RDS: RDBMS service - Automated and Managed MySQL

32. 6. Leverage many storage options Which storage option to use when?

33. Cloud Architecture Lessons Best Practices 1. Design for failure and nothing fails 2. Loose coupling sets you free 3. Implement Elasticity 4. Build Security in every layer 5. Don't fear constraints 6. Think Parallel 7. Leverage many storage options

34. AWS community and Ecosystem Find help, guidance, assistance when you need it AWS Ecosystem AWS Community

35. Migrating a Web Application to AWS Photo: La Pedrera - Casa Milà, Barcelona - Antonio Gaudi

36. Migrating your Web Application Step by Step towards AWS A typical Web App needs: Compute Power Storage capacity Content Distribution Database storage Messaging Load balancing Monitoring

37. Migrating your Web Application - 1/8 Typical Web App Architecture Database Application Server /Business Logic Web Server / Presentation Layer Client Browser

38. Migrating your Web Application - 2/8 Amazon S3 for Storage Store persistent files in Amazon S3 for lower costs, higher reliability Client Browser

39. Migrating your Web Application - 3/8 Use Amazon CloudFront Amazon CloudFront for distribution Amazon CloudFrontis a content delivery network that caches data stored in Amazon S3 across a network of 14 edge locations around the world Client Browser

40. Migrating your Web Application - 4/8 Amazon EC2 for your choice of web servers Configure Amazon EC2 running your choice of web server to handle all incoming web requests. Client Browser

41. Migrating your Web Application - 4/8 Scale out App servers on Amazon EC2 Configure multiple Amazon EC2 instances running your choice of application server to process requests. Use Availability Zones and Elastic IPs for greater reliability and resiliency. Utilize Auto-scaling and Elastic LB service Client Browser

42. Migrating your Web Application - 5/8 Use Amazon EBS for Database EBS for Persistent Storage and S3 for Snapshots Configure an Amazon EBS device to host your existing relational database. Snapshots can be automatically backed up to Amazon S3. Client Browser

43. Migrating your Web Application - 6/8 Use Amazon SQS Amazon SQS for queuing requests SQS Amazon SQS makes it easy to coordinate between the web server and application servers. Client Browser

44. Migrating your Web Application - 7/8 Use Amazon SimpleDB Amazon SimpleDB for log files, metadata SimpleDB SQS Amazon SimpleDBcan be used to store metadata, logfiles, and other information for your site. Client Browser

45. Migrating your Web Application - 8/8 Use Amazon SimpleDB Monitor your Amazon EC2 instances using CloudWatch SimpleDB SQS Amazon CloudWatch to monitoring your Amazon EC2 instances Client Browser

46. Migrating your Web Application Step by Step towards AWS A typical Web App needs: With AWS: Compute Power Storage capacity Content Distribution Database storage Messaging Load balancing Monitoring Amazon EC2 Amazon S3 Amazon CloudFront Amazon EBS Amazon SQS Amazon EC2 Amazon CloudWatch

47. Amazon Web Services tools Things you need Web : AWS Management Console IDE : AWS Toolkit for Eclipse AWS SDK: .NET SDK, Java SDK Tools : 3rd Party tools eg. CA Firefox Plugins : ElasticFox, S3Fox, SDB Tool Several libraries:

48. Identify the right candidate Whiteboard Diagram Dashboard Report Web Search DB logs Service LDAP Auth CRM Engine OLAP ERP List all your IT assets Whiteboard your IT Assets Identify upward and downward dependencies

49. Identify the right candidate Pick one application with lower dependencies to start with Identify the right candidate for the cloud Dashboard CRM Search DB Report CRM Search DB logs Service LDAP Auth Web Engine OLAP ERP Search for under-utilized IT assets Applications that has immediate business need to scale Applications that are running out of capacity Low-hanging fruits (Examples): Web Applications Batch Processing systems Build/QA/Test systems Content Management Systems Digital Asset Management Systems

50. Identify the right candidate Identify the right candidate for the cloud Dashboard Web Search DB Report Web Search DB logs Service LDAP Auth CRM Engine OLAP ERP

51. Conclusions Most Important Lesson From Our Customers: Start small with a well-defined proof of concept Experiment with different architectures; Keep one, throw away others Build Support and Awareness within your organization Once one application is launched others will follow… Photo: Grand Canyon Hopi Point SunSet

52. The day is not too far when applications will cease to be aware of physical hardware. Much like plugging in a microwave in order to power it doesn’t require any knowledge of electricity, one should be able to plug in an application to the cloud in order to receive the power it needs to run, just like a utility. As an architect, you will manage abstract compute, storage and network resources instead of physical servers. Applications will continue to function even if the underlying physical hardware fails or is removed or replaced. Applications will adapt themselves to fluctuating demand patterns by deploying resources instantaneously and automatically, thereby achieving highest utilization levels at all times. Scalability, Security, High availability, Fault-tolerance, Testability and Elasticity will be configurable properties of the application architecture and will be an automated and intrinsic part of the platform on which they are built. The day is not too far…. Scalability, Security, High availability, Fault-tolerance, Testability and Elasticity will be configurable properties of the application architecture and will be an automated and intrinsic part of the platform on which they are built.

53. Thank you! jvaria@amazon.com Twitter:@jinmanPresentation idea and template from @simon

54. http://aws.amazon.com