SlideShare a Scribd company logo
1 of 42
Aviran Mordo
Head of Engineering
@aviranm
linkedin.com/in/aviran
aviransplace.com
Scaling with
Microservices Architecture
and Multi-cloud platforms
Scaling wix with microservices architecture jax london-2015
Wix in Numbers
Over 72M users
Static storage is >2PB of data
3 data centers + 3 clouds (Google, Amazon,Azure)
2B HTTP requests/day
1000 people work atWix
Initial Architecture
Built for fast development
Stateful login (Tomcat session), Ehcache, file uploads
No consideration for performance, scalability and testing
Intended for short-term use
Tomcat, Hibernate, custom web framework
Lighttpd
(file serving) MySQL
DB
Wix
(Tomcat)
The Monolithic Giant
One monolithic server that handled everything
Dependency between features
Changes in unrelated areas of the system caused deployment of
the whole system
Failure in unrelated areas will cause system wide downtime
Breaking the System Apart
Scaling wix with microservices architecture jax london-2015
Concerns and SLA
DataValidation
Security / Authentication
Data consistency
Lots of data
Edit websites
High availability
High performance
Lots of static files
Very high traffic volume
Viewport optimization
Long tail (immutable)
Serving Media
High availability
High performance
High traffic volume
Long tail (mutable)
View sites, created by
Wix editor
Wix Segmentation
1. Editor Segment 3. Public Segment2. Media Segment
Networking
HTML
Editor
Flash
Editor
MSM
Private
Media
Public
Media
Editor Segment Public Segment
Premium
Services
eCommerse
List DB
App
Builder
App
Store
App
Market
Dashboard
Statics/me
dia
Mailer
TimeZone
Public
HTML API
Public API
(Flash)
MSP
Public
Server
HTML
Renderer
HTML SEO
Renderer
Flash
Renderer
Flash SEO
Renderer
Sitemap
Renderer
Robots.txt
Renderer
User
Server
Template
Viewer
ContactsHUB
Activit
y
Site
Members
Provided
Mailing
Service
Comments
Snapshoter
User Pref
Feed Me
Shout-out Hotels
PETRI
Site Pref
Dist LoggerSlicer
eCom
Renderer
eCom Cart
eCom
Checkout
eCom
Catalog
eCom
Orders
Payment
Facade
Account
Info
HTML API
HTML
Embeder
BlogMobile
It is all about
Scaling wix with microservices architecture jax london-2015
Microservices Guidelines
Each service has its own DB schema (if one is needed)
Only one service should write to a specific DB table(s)
There may be additional read-only services that directly
accesses the DB (for performance reasons)
Services are stateless
No DB transactions
Cache is not a building block, but an optimization
Microservices Tradeoffs
Each service has its own DB schema (if one is needed)
Gain - Easy to scale microservices based on service level concerns
Tradeoff – system complexity, performance
Only one service should write to a specific DB table(s)
Gain - Decoupling architecture – faster development
Tradeoff – system complexity / performance
May have additional read-only services that accesses the DB
Gain - Performance gain
Tradeoff - coupling
Services are stateless
Gain - Easy to scale out (just add more servers)
Tradeoff - performance / consistency
No DB transactions
Gain - Better DB performance, easier to scale
Tradeoff - system complexity
1. Editor Segment
Editor Server
Immutable JSON pages (~3M / day)
Site revisions
Active – standby MySQL cross datacenters
Editor Server
MySQL
Active
Sites
MySQL
Archive
Scaling wix with microservices architecture jax london-2015
Protect The Data
DB outage with fast recovery = replication
Data poisoning/corruption = revisions / backup
Make the data available at all times = data distribution to multiple
locations / providers
Browser
Editor
Server
GCS
MySQL
Active
Sites
MySQL
Archive
Saving Editor Data
WixMedia
(Amazon)
WixMedia
(Google)
Save Page(s)
200 OK
Upload
Save Page
DC replication
Notify
MySQL
Archive
MySQL
Active
Sites
S3
WixMedia
(DC-1)
Browser
Editor
Server
GCS
MySQL
Active
Sites
MySQL
Archive
WixMedia
(Amazon)
WixMedia
(Google)
Save Page(s)
200 OK
Upload
Save Page
DC replication
Notify
MySQL
Archive
MySQL
Active
Sites
S3
WixMedia
(DC-1)
Self Healing Process
No DB Transactions
Save each page (JSON) as an atomic operation
Page ID is a content based hash (immutable/idempotent)
Finalize transaction by sending site header (list of pages)
Can generate orphaned pages, not a problem in practice
2. Media Segment (WixMP)
Wix Media Platform (WixMP)
Eventual consistent distributed file system
(2PB user media files)
Dynamic media processing
Multi datacenter aware
Automatic fallback cross DC
Run on commodity servers & cloud
T
Google
Cloud
Prospero – Wix Media Manager
get image.jpg
First
fallback
Second
fallback
If not in
CDN
Amazon
x36
T
x36
T
x32
Austin
CDN
3. Public Segment
Public Segment Roles
Routing (resolve URLs)
Dispatching (to a renderer)
Rendering (HTML,XML,TXT)
Public
Server
HTML
Renderer
HTML SEO
Renderer
Flash
Renderer
Sitemap
Renderer
Robots.txt
Renderer
www.example.com
Flash SEO
Renderer
Public SLA
Our goal: 99% response time <100ms at peak traffic
Publish Site
Publish site header (a map of pages for a site)
Publish routing table
Publish site header / routes (CQRS)
Editor Segment Public Segment
Built For Speed
Minimize out-of-process hops (2 DB, 1 RPC)
Lookup tables are cached in memory, updated every few minutes
Denormalized data – optimize for read by primary key (MySQL)
Minimize business logic
How a Page Gets Rendered
Bootstrap HTML template that contains only data
Only JavaScript imports
JSON data (site-header + dynamic data)
No “real” HTML view
Offload rendering work to the browser
The average Intel Core
i750 can push up to 7
GFLOPS without
overclocking
Why JSON?
Easy to parse in JavaScript and Java/Scala
Fairly compact text format
Highly compressible (5:1 even for small payloads)
Easy to fix rendering bugs and cross browsers issues (just
deploy a new code)
Minimum Number of Public Servers
Needed to Serve 66M Sites
4
Public SLA
Be Available 99.999%
Serving a Site – Sunny Day
Archive
CDN WixMP
Browser
http://example.wix.com
Store HTML
to cache
HTTP
Request
Notify
site view
LB
Public
Renderer
HTML
Resources / Media
HTTP
Request
Serving a Site – DC Lost
Archive
CDN WixMP
Browser
http://example.wix.com
LB
Public
Renderer
LB
Public
Renderer
Change DNS
HTTP
Request
Serving a Site – Public Lost
Archive
Browser
http://example.wix.com
LB
Public
Renderer
Get
Cached HTML
Version
HTML
HTTP
Request
LB
Public
Renderer
Fallback to 2nd
DC
Living in the Browser
CDN WixMP
Browser
http://example.wix.com
LB
Public
Renderer
Editor Pages
Fallback
JSON /
Media
HTML
HTTP
Request Fallback
Summary
Identify concerns and SLA for different parts of the system
Build redundancy in critical path (for availability)
De-normalize data (for performance)
Minimize out-of-process hops (for performance)
Take advantage of client’s CPU power
Scaling wix with microservices architecture jax london-2015
Aviran Mordo
Head of Engineering
@aviranm
linkedin.com/in/aviran
aviransplace.com
http://engineering.wix.com
http://goo.gl/3xhpNW
@WixEng

More Related Content

What's hot

Scaling up to 30M users - The Wix Story
Scaling up to 30M users - The Wix StoryScaling up to 30M users - The Wix Story
Scaling up to 30M users - The Wix StoryAviran Mordo
 
Scaling up to 30 m users
Scaling up to 30 m usersScaling up to 30 m users
Scaling up to 30 m usersYoav Avrahami
 
Scaling wix to over 70 m users
Scaling wix to over 70 m usersScaling wix to over 70 m users
Scaling wix to over 70 m usersYoav Avrahami
 
DV03 Smooth Migration to Windows Azure
DV03 Smooth Migration to Windows AzureDV03 Smooth Migration to Windows Azure
DV03 Smooth Migration to Windows AzureRonald Widha
 
MSDN - Converting an existing ASP.NET application to Windows Azure
MSDN - Converting an existing ASP.NET application to Windows AzureMSDN - Converting an existing ASP.NET application to Windows Azure
MSDN - Converting an existing ASP.NET application to Windows AzureMaarten Balliauw
 
Shopzilla - Performance By Design
Shopzilla - Performance By DesignShopzilla - Performance By Design
Shopzilla - Performance By DesignTim Morrow
 
The Essentials of Building Cloud-Based Web Apps with Azure
The Essentials of Building Cloud-Based Web Apps with AzureThe Essentials of Building Cloud-Based Web Apps with Azure
The Essentials of Building Cloud-Based Web Apps with AzureIdo Flatow
 
Migrating Customers to Microsoft Azure: Lessons Learned From the Field
Migrating Customers to Microsoft Azure: Lessons Learned From the FieldMigrating Customers to Microsoft Azure: Lessons Learned From the Field
Migrating Customers to Microsoft Azure: Lessons Learned From the FieldIdo Flatow
 
Best Practices for couchDB developers on Microsoft Azure
Best Practices for couchDB developers on Microsoft AzureBest Practices for couchDB developers on Microsoft Azure
Best Practices for couchDB developers on Microsoft AzureBrian Benz
 
BizSpark Startup Night Windows Azure March 29, 2011
BizSpark Startup Night Windows Azure March 29, 2011BizSpark Startup Night Windows Azure March 29, 2011
BizSpark Startup Night Windows Azure March 29, 2011Spiffy
 
From VMs to Containers: Introducing Docker Containers for Linux and Windows S...
From VMs to Containers: Introducing Docker Containers for Linux and Windows S...From VMs to Containers: Introducing Docker Containers for Linux and Windows S...
From VMs to Containers: Introducing Docker Containers for Linux and Windows S...Ido Flatow
 
Building & managing wa app wely
Building & managing wa app   welyBuilding & managing wa app   wely
Building & managing wa app welySpiffy
 
Experimenting on Humans - Advanced A/B Tests - QCon SF 2014
Experimenting on Humans - Advanced A/B Tests - QCon SF 2014Experimenting on Humans - Advanced A/B Tests - QCon SF 2014
Experimenting on Humans - Advanced A/B Tests - QCon SF 2014Aviran Mordo
 
Azure in Developer Perspective
Azure in Developer PerspectiveAzure in Developer Perspective
Azure in Developer Perspectiverizaon
 
Java in the Cloud : PaaS Platforms in Comparison
Java in the Cloud : PaaS Platforms in Comparison Java in the Cloud : PaaS Platforms in Comparison
Java in the Cloud : PaaS Platforms in Comparison Eberhard Wolff
 
Azure and web sites hackaton deck
Azure and web sites hackaton deckAzure and web sites hackaton deck
Azure and web sites hackaton deckAlexey Bokov
 
Microsoft Azure: Deploy and Scale Modern Websites
Microsoft Azure: Deploy and Scale Modern WebsitesMicrosoft Azure: Deploy and Scale Modern Websites
Microsoft Azure: Deploy and Scale Modern WebsitesWinWire Technologies Inc
 
Moving to the Cloud: AWS, Zend, RightScale
Moving to the Cloud: AWS, Zend, RightScaleMoving to the Cloud: AWS, Zend, RightScale
Moving to the Cloud: AWS, Zend, RightScalemmoline
 
Azure Storage Revisited
Azure Storage RevisitedAzure Storage Revisited
Azure Storage RevisitedJoel Cochran
 

What's hot (20)

Scaling up to 30M users - The Wix Story
Scaling up to 30M users - The Wix StoryScaling up to 30M users - The Wix Story
Scaling up to 30M users - The Wix Story
 
Scaling up to 30 m users
Scaling up to 30 m usersScaling up to 30 m users
Scaling up to 30 m users
 
Scaling wix to over 70 m users
Scaling wix to over 70 m usersScaling wix to over 70 m users
Scaling wix to over 70 m users
 
DV03 Smooth Migration to Windows Azure
DV03 Smooth Migration to Windows AzureDV03 Smooth Migration to Windows Azure
DV03 Smooth Migration to Windows Azure
 
MSDN - Converting an existing ASP.NET application to Windows Azure
MSDN - Converting an existing ASP.NET application to Windows AzureMSDN - Converting an existing ASP.NET application to Windows Azure
MSDN - Converting an existing ASP.NET application to Windows Azure
 
Shopzilla - Performance By Design
Shopzilla - Performance By DesignShopzilla - Performance By Design
Shopzilla - Performance By Design
 
The Essentials of Building Cloud-Based Web Apps with Azure
The Essentials of Building Cloud-Based Web Apps with AzureThe Essentials of Building Cloud-Based Web Apps with Azure
The Essentials of Building Cloud-Based Web Apps with Azure
 
Azure Data Storage
Azure Data StorageAzure Data Storage
Azure Data Storage
 
Migrating Customers to Microsoft Azure: Lessons Learned From the Field
Migrating Customers to Microsoft Azure: Lessons Learned From the FieldMigrating Customers to Microsoft Azure: Lessons Learned From the Field
Migrating Customers to Microsoft Azure: Lessons Learned From the Field
 
Best Practices for couchDB developers on Microsoft Azure
Best Practices for couchDB developers on Microsoft AzureBest Practices for couchDB developers on Microsoft Azure
Best Practices for couchDB developers on Microsoft Azure
 
BizSpark Startup Night Windows Azure March 29, 2011
BizSpark Startup Night Windows Azure March 29, 2011BizSpark Startup Night Windows Azure March 29, 2011
BizSpark Startup Night Windows Azure March 29, 2011
 
From VMs to Containers: Introducing Docker Containers for Linux and Windows S...
From VMs to Containers: Introducing Docker Containers for Linux and Windows S...From VMs to Containers: Introducing Docker Containers for Linux and Windows S...
From VMs to Containers: Introducing Docker Containers for Linux and Windows S...
 
Building & managing wa app wely
Building & managing wa app   welyBuilding & managing wa app   wely
Building & managing wa app wely
 
Experimenting on Humans - Advanced A/B Tests - QCon SF 2014
Experimenting on Humans - Advanced A/B Tests - QCon SF 2014Experimenting on Humans - Advanced A/B Tests - QCon SF 2014
Experimenting on Humans - Advanced A/B Tests - QCon SF 2014
 
Azure in Developer Perspective
Azure in Developer PerspectiveAzure in Developer Perspective
Azure in Developer Perspective
 
Java in the Cloud : PaaS Platforms in Comparison
Java in the Cloud : PaaS Platforms in Comparison Java in the Cloud : PaaS Platforms in Comparison
Java in the Cloud : PaaS Platforms in Comparison
 
Azure and web sites hackaton deck
Azure and web sites hackaton deckAzure and web sites hackaton deck
Azure and web sites hackaton deck
 
Microsoft Azure: Deploy and Scale Modern Websites
Microsoft Azure: Deploy and Scale Modern WebsitesMicrosoft Azure: Deploy and Scale Modern Websites
Microsoft Azure: Deploy and Scale Modern Websites
 
Moving to the Cloud: AWS, Zend, RightScale
Moving to the Cloud: AWS, Zend, RightScaleMoving to the Cloud: AWS, Zend, RightScale
Moving to the Cloud: AWS, Zend, RightScale
 
Azure Storage Revisited
Azure Storage RevisitedAzure Storage Revisited
Azure Storage Revisited
 

Viewers also liked

Cassandra as event sourced journal for big data analytics
Cassandra as event sourced journal for big data analyticsCassandra as event sourced journal for big data analytics
Cassandra as event sourced journal for big data analyticsAnirvan Chakraborty
 
Advanced A/B Testing - Jax London 2015
Advanced A/B Testing - Jax London 2015Advanced A/B Testing - Jax London 2015
Advanced A/B Testing - Jax London 2015Aviran Mordo
 
GC Tuning Confessions Of A Performance Engineer - Improved :)
GC Tuning Confessions Of A Performance Engineer - Improved :)GC Tuning Confessions Of A Performance Engineer - Improved :)
GC Tuning Confessions Of A Performance Engineer - Improved :)Monica Beckwith
 
Road to Continuous Delivery - Wix.com
Road to Continuous Delivery - Wix.comRoad to Continuous Delivery - Wix.com
Road to Continuous Delivery - Wix.comAviran Mordo
 
Java micro-services
Java micro-servicesJava micro-services
Java micro-servicesJames Lewis
 
Apache Cassandra for Timeseries- and Graph-Data
Apache Cassandra for Timeseries- and Graph-DataApache Cassandra for Timeseries- and Graph-Data
Apache Cassandra for Timeseries- and Graph-DataGuido Schmutz
 
Wix Dev-Centric Culture And Continuous Delivery
Wix Dev-Centric Culture And Continuous DeliveryWix Dev-Centric Culture And Continuous Delivery
Wix Dev-Centric Culture And Continuous DeliveryAviran Mordo
 
Wix.com Back-end Engineering Guild Manifesto
Wix.com Back-end Engineering Guild ManifestoWix.com Back-end Engineering Guild Manifesto
Wix.com Back-end Engineering Guild ManifestoAviran Mordo
 

Viewers also liked (8)

Cassandra as event sourced journal for big data analytics
Cassandra as event sourced journal for big data analyticsCassandra as event sourced journal for big data analytics
Cassandra as event sourced journal for big data analytics
 
Advanced A/B Testing - Jax London 2015
Advanced A/B Testing - Jax London 2015Advanced A/B Testing - Jax London 2015
Advanced A/B Testing - Jax London 2015
 
GC Tuning Confessions Of A Performance Engineer - Improved :)
GC Tuning Confessions Of A Performance Engineer - Improved :)GC Tuning Confessions Of A Performance Engineer - Improved :)
GC Tuning Confessions Of A Performance Engineer - Improved :)
 
Road to Continuous Delivery - Wix.com
Road to Continuous Delivery - Wix.comRoad to Continuous Delivery - Wix.com
Road to Continuous Delivery - Wix.com
 
Java micro-services
Java micro-servicesJava micro-services
Java micro-services
 
Apache Cassandra for Timeseries- and Graph-Data
Apache Cassandra for Timeseries- and Graph-DataApache Cassandra for Timeseries- and Graph-Data
Apache Cassandra for Timeseries- and Graph-Data
 
Wix Dev-Centric Culture And Continuous Delivery
Wix Dev-Centric Culture And Continuous DeliveryWix Dev-Centric Culture And Continuous Delivery
Wix Dev-Centric Culture And Continuous Delivery
 
Wix.com Back-end Engineering Guild Manifesto
Wix.com Back-end Engineering Guild ManifestoWix.com Back-end Engineering Guild Manifesto
Wix.com Back-end Engineering Guild Manifesto
 

Similar to Scaling wix with microservices architecture jax london-2015

Spring 2007 SharePoint Connections Oleson Advanced Administration and Plannin...
Spring 2007 SharePoint Connections Oleson Advanced Administration and Plannin...Spring 2007 SharePoint Connections Oleson Advanced Administration and Plannin...
Spring 2007 SharePoint Connections Oleson Advanced Administration and Plannin...Joel Oleson
 
Dot Net Nuke Presentation
Dot Net Nuke PresentationDot Net Nuke Presentation
Dot Net Nuke PresentationTony Cosentino
 
NZSPC 2013 - Ultimate SharePoint Infrastructure Best Practices Session
NZSPC 2013 - Ultimate SharePoint Infrastructure Best Practices SessionNZSPC 2013 - Ultimate SharePoint Infrastructure Best Practices Session
NZSPC 2013 - Ultimate SharePoint Infrastructure Best Practices SessionMichael Noel
 
Azure Data platform
Azure Data platformAzure Data platform
Azure Data platformMostafa
 
SharePoint Saturday Michigan Keynote - Top 5 Infrastructure Concerns for a Sh...
SharePoint Saturday Michigan Keynote - Top 5 Infrastructure Concerns for a Sh...SharePoint Saturday Michigan Keynote - Top 5 Infrastructure Concerns for a Sh...
SharePoint Saturday Michigan Keynote - Top 5 Infrastructure Concerns for a Sh...Michael Noel
 
Getting Started with Amazon Redshift
Getting Started with Amazon RedshiftGetting Started with Amazon Redshift
Getting Started with Amazon RedshiftAmazon Web Services
 
Building Scalable .NET Web Applications
Building Scalable .NET Web ApplicationsBuilding Scalable .NET Web Applications
Building Scalable .NET Web ApplicationsBuu Nguyen
 
Azure SQL Managed Instance - SqlBits 2019
Azure SQL Managed Instance - SqlBits 2019Azure SQL Managed Instance - SqlBits 2019
Azure SQL Managed Instance - SqlBits 2019Jovan Popovic
 
Ordina SOFTC Presentation - SharePoint 2010 Architecture
Ordina SOFTC Presentation - SharePoint 2010 ArchitectureOrdina SOFTC Presentation - SharePoint 2010 Architecture
Ordina SOFTC Presentation - SharePoint 2010 ArchitectureOrdina Belgium
 
SharePoint Performance Optimization In 10 Steps for the IT Professional
SharePoint Performance Optimization In 10 Steps for the IT ProfessionalSharePoint Performance Optimization In 10 Steps for the IT Professional
SharePoint Performance Optimization In 10 Steps for the IT ProfessionalJoel Oleson
 
Running a Megasite on Microsoft Technologies
Running a Megasite on Microsoft TechnologiesRunning a Megasite on Microsoft Technologies
Running a Megasite on Microsoft Technologiesgoodfriday
 
Taking SharePoint to the Cloud
Taking SharePoint to the CloudTaking SharePoint to the Cloud
Taking SharePoint to the CloudAaron Saikovski
 
TechEd Africa 2011 - OFC307: Architecting a Disaster Tolerant and Highly Avai...
TechEd Africa 2011 - OFC307: Architecting a Disaster Tolerant and Highly Avai...TechEd Africa 2011 - OFC307: Architecting a Disaster Tolerant and Highly Avai...
TechEd Africa 2011 - OFC307: Architecting a Disaster Tolerant and Highly Avai...Michael Noel
 
MOSS 2007 Deployment Fundamentals -Part2
MOSS 2007 Deployment Fundamentals -Part2MOSS 2007 Deployment Fundamentals -Part2
MOSS 2007 Deployment Fundamentals -Part2Information Technology
 
OpenCms Days 2012 - The Dispatch - Running OpenCms 8 on Amazon cloud services
OpenCms Days 2012 - The Dispatch - Running OpenCms 8 on Amazon cloud servicesOpenCms Days 2012 - The Dispatch - Running OpenCms 8 on Amazon cloud services
OpenCms Days 2012 - The Dispatch - Running OpenCms 8 on Amazon cloud servicesAlkacon Software GmbH & Co. KG
 
Ultimate SharePoint Infrastructure Best Practises Session - Isle of Man Share...
Ultimate SharePoint Infrastructure Best Practises Session - Isle of Man Share...Ultimate SharePoint Infrastructure Best Practises Session - Isle of Man Share...
Ultimate SharePoint Infrastructure Best Practises Session - Isle of Man Share...Michael Noel
 
Keynote – When Open Source Meets the Enterprise
Keynote – When Open Source Meets the EnterpriseKeynote – When Open Source Meets the Enterprise
Keynote – When Open Source Meets the EnterpriseMariaDB plc
 
Getting Started with Amazon Redshift
Getting Started with Amazon RedshiftGetting Started with Amazon Redshift
Getting Started with Amazon RedshiftAmazon Web Services
 
Building the Perfect SharePoint 2010 Farm - SharePoint Saturday NYC 2011
Building the Perfect SharePoint 2010 Farm - SharePoint Saturday NYC 2011Building the Perfect SharePoint 2010 Farm - SharePoint Saturday NYC 2011
Building the Perfect SharePoint 2010 Farm - SharePoint Saturday NYC 2011Michael Noel
 

Similar to Scaling wix with microservices architecture jax london-2015 (20)

Spring 2007 SharePoint Connections Oleson Advanced Administration and Plannin...
Spring 2007 SharePoint Connections Oleson Advanced Administration and Plannin...Spring 2007 SharePoint Connections Oleson Advanced Administration and Plannin...
Spring 2007 SharePoint Connections Oleson Advanced Administration and Plannin...
 
Dot Net Nuke Presentation
Dot Net Nuke PresentationDot Net Nuke Presentation
Dot Net Nuke Presentation
 
NZSPC 2013 - Ultimate SharePoint Infrastructure Best Practices Session
NZSPC 2013 - Ultimate SharePoint Infrastructure Best Practices SessionNZSPC 2013 - Ultimate SharePoint Infrastructure Best Practices Session
NZSPC 2013 - Ultimate SharePoint Infrastructure Best Practices Session
 
Azure Data platform
Azure Data platformAzure Data platform
Azure Data platform
 
SharePoint Saturday Michigan Keynote - Top 5 Infrastructure Concerns for a Sh...
SharePoint Saturday Michigan Keynote - Top 5 Infrastructure Concerns for a Sh...SharePoint Saturday Michigan Keynote - Top 5 Infrastructure Concerns for a Sh...
SharePoint Saturday Michigan Keynote - Top 5 Infrastructure Concerns for a Sh...
 
Getting Started with Amazon Redshift
Getting Started with Amazon RedshiftGetting Started with Amazon Redshift
Getting Started with Amazon Redshift
 
Building Scalable .NET Web Applications
Building Scalable .NET Web ApplicationsBuilding Scalable .NET Web Applications
Building Scalable .NET Web Applications
 
Azure SQL Managed Instance - SqlBits 2019
Azure SQL Managed Instance - SqlBits 2019Azure SQL Managed Instance - SqlBits 2019
Azure SQL Managed Instance - SqlBits 2019
 
Ordina SOFTC Presentation - SharePoint 2010 Architecture
Ordina SOFTC Presentation - SharePoint 2010 ArchitectureOrdina SOFTC Presentation - SharePoint 2010 Architecture
Ordina SOFTC Presentation - SharePoint 2010 Architecture
 
SharePoint Performance Optimization In 10 Steps for the IT Professional
SharePoint Performance Optimization In 10 Steps for the IT ProfessionalSharePoint Performance Optimization In 10 Steps for the IT Professional
SharePoint Performance Optimization In 10 Steps for the IT Professional
 
Running a Megasite on Microsoft Technologies
Running a Megasite on Microsoft TechnologiesRunning a Megasite on Microsoft Technologies
Running a Megasite on Microsoft Technologies
 
Taking SharePoint to the Cloud
Taking SharePoint to the CloudTaking SharePoint to the Cloud
Taking SharePoint to the Cloud
 
TechEd Africa 2011 - OFC307: Architecting a Disaster Tolerant and Highly Avai...
TechEd Africa 2011 - OFC307: Architecting a Disaster Tolerant and Highly Avai...TechEd Africa 2011 - OFC307: Architecting a Disaster Tolerant and Highly Avai...
TechEd Africa 2011 - OFC307: Architecting a Disaster Tolerant and Highly Avai...
 
MOSS 2007 Deployment Fundamentals -Part2
MOSS 2007 Deployment Fundamentals -Part2MOSS 2007 Deployment Fundamentals -Part2
MOSS 2007 Deployment Fundamentals -Part2
 
OpenCms Days 2012 - The Dispatch - Running OpenCms 8 on Amazon cloud services
OpenCms Days 2012 - The Dispatch - Running OpenCms 8 on Amazon cloud servicesOpenCms Days 2012 - The Dispatch - Running OpenCms 8 on Amazon cloud services
OpenCms Days 2012 - The Dispatch - Running OpenCms 8 on Amazon cloud services
 
Ultimate SharePoint Infrastructure Best Practises Session - Isle of Man Share...
Ultimate SharePoint Infrastructure Best Practises Session - Isle of Man Share...Ultimate SharePoint Infrastructure Best Practises Session - Isle of Man Share...
Ultimate SharePoint Infrastructure Best Practises Session - Isle of Man Share...
 
Keynote – When Open Source Meets the Enterprise
Keynote – When Open Source Meets the EnterpriseKeynote – When Open Source Meets the Enterprise
Keynote – When Open Source Meets the Enterprise
 
Getting Started with Amazon Redshift
Getting Started with Amazon RedshiftGetting Started with Amazon Redshift
Getting Started with Amazon Redshift
 
Introduction To Cloud Computing
Introduction To Cloud ComputingIntroduction To Cloud Computing
Introduction To Cloud Computing
 
Building the Perfect SharePoint 2010 Farm - SharePoint Saturday NYC 2011
Building the Perfect SharePoint 2010 Farm - SharePoint Saturday NYC 2011Building the Perfect SharePoint 2010 Farm - SharePoint Saturday NYC 2011
Building the Perfect SharePoint 2010 Farm - SharePoint Saturday NYC 2011
 

More from Aviran Mordo

Scaling Engineering by Hacking Conway’s Law - Geecon,2022
Scaling Engineering by Hacking Conway’s Law - Geecon,2022Scaling Engineering by Hacking Conway’s Law - Geecon,2022
Scaling Engineering by Hacking Conway’s Law - Geecon,2022Aviran Mordo
 
Arrested by the cap devoxx uk 2018
Arrested by the cap devoxx uk 2018Arrested by the cap devoxx uk 2018
Arrested by the cap devoxx uk 2018Aviran Mordo
 
Scaling Wix engineering
Scaling Wix engineering Scaling Wix engineering
Scaling Wix engineering Aviran Mordo
 
The Art of A/B Testing
The Art of A/B TestingThe Art of A/B Testing
The Art of A/B TestingAviran Mordo
 
Scaling r&d org while maintaining quality
Scaling r&d org while maintaining qualityScaling r&d org while maintaining quality
Scaling r&d org while maintaining qualityAviran Mordo
 
Introduction to HTTP protocol
Introduction to HTTP protocolIntroduction to HTTP protocol
Introduction to HTTP protocolAviran Mordo
 
Lessons Learned Monitoring Production
Lessons Learned Monitoring ProductionLessons Learned Monitoring Production
Lessons Learned Monitoring ProductionAviran Mordo
 
Strategies in continuous delivery
Strategies in continuous deliveryStrategies in continuous delivery
Strategies in continuous deliveryAviran Mordo
 

More from Aviran Mordo (8)

Scaling Engineering by Hacking Conway’s Law - Geecon,2022
Scaling Engineering by Hacking Conway’s Law - Geecon,2022Scaling Engineering by Hacking Conway’s Law - Geecon,2022
Scaling Engineering by Hacking Conway’s Law - Geecon,2022
 
Arrested by the cap devoxx uk 2018
Arrested by the cap devoxx uk 2018Arrested by the cap devoxx uk 2018
Arrested by the cap devoxx uk 2018
 
Scaling Wix engineering
Scaling Wix engineering Scaling Wix engineering
Scaling Wix engineering
 
The Art of A/B Testing
The Art of A/B TestingThe Art of A/B Testing
The Art of A/B Testing
 
Scaling r&d org while maintaining quality
Scaling r&d org while maintaining qualityScaling r&d org while maintaining quality
Scaling r&d org while maintaining quality
 
Introduction to HTTP protocol
Introduction to HTTP protocolIntroduction to HTTP protocol
Introduction to HTTP protocol
 
Lessons Learned Monitoring Production
Lessons Learned Monitoring ProductionLessons Learned Monitoring Production
Lessons Learned Monitoring Production
 
Strategies in continuous delivery
Strategies in continuous deliveryStrategies in continuous delivery
Strategies in continuous delivery
 

Recently uploaded

Your Vision, Our Expertise: TECUNIQUE's Tailored Software Teams
Your Vision, Our Expertise: TECUNIQUE's Tailored Software TeamsYour Vision, Our Expertise: TECUNIQUE's Tailored Software Teams
Your Vision, Our Expertise: TECUNIQUE's Tailored Software TeamsJaydeep Chhasatia
 
Webinar_050417_LeClair12345666777889.ppt
Webinar_050417_LeClair12345666777889.pptWebinar_050417_LeClair12345666777889.ppt
Webinar_050417_LeClair12345666777889.pptkinjal48
 
Watermarking in Source Code: Applications and Security Challenges
Watermarking in Source Code: Applications and Security ChallengesWatermarking in Source Code: Applications and Security Challenges
Watermarking in Source Code: Applications and Security ChallengesShyamsundar Das
 
online pdf editor software solutions.pdf
online pdf editor software solutions.pdfonline pdf editor software solutions.pdf
online pdf editor software solutions.pdfMeon Technology
 
Streamlining Your Application Builds with Cloud Native Buildpacks
Streamlining Your Application Builds  with Cloud Native BuildpacksStreamlining Your Application Builds  with Cloud Native Buildpacks
Streamlining Your Application Builds with Cloud Native BuildpacksVish Abrams
 
Growing Oxen: channel operators and retries
Growing Oxen: channel operators and retriesGrowing Oxen: channel operators and retries
Growing Oxen: channel operators and retriesSoftwareMill
 
Enterprise Document Management System - Qualityze Inc
Enterprise Document Management System - Qualityze IncEnterprise Document Management System - Qualityze Inc
Enterprise Document Management System - Qualityze Incrobinwilliams8624
 
JS-Experts - Cybersecurity for Generative AI
JS-Experts - Cybersecurity for Generative AIJS-Experts - Cybersecurity for Generative AI
JS-Experts - Cybersecurity for Generative AIIvo Andreev
 
Fields in Java and Kotlin and what to expect.pptx
Fields in Java and Kotlin and what to expect.pptxFields in Java and Kotlin and what to expect.pptx
Fields in Java and Kotlin and what to expect.pptxJoão Esperancinha
 
Top Software Development Trends in 2024
Top Software Development Trends in  2024Top Software Development Trends in  2024
Top Software Development Trends in 2024Mind IT Systems
 
20240319 Car Simulator Plan.pptx . Plan for a JavaScript Car Driving Simulator.
20240319 Car Simulator Plan.pptx . Plan for a JavaScript Car Driving Simulator.20240319 Car Simulator Plan.pptx . Plan for a JavaScript Car Driving Simulator.
20240319 Car Simulator Plan.pptx . Plan for a JavaScript Car Driving Simulator.Sharon Liu
 
Kawika Technologies pvt ltd Software Development Company in Trivandrum
Kawika Technologies pvt ltd Software Development Company in TrivandrumKawika Technologies pvt ltd Software Development Company in Trivandrum
Kawika Technologies pvt ltd Software Development Company in TrivandrumKawika Technologies
 
ERP For Electrical and Electronics manufecturing.pptx
ERP For Electrical and Electronics manufecturing.pptxERP For Electrical and Electronics manufecturing.pptx
ERP For Electrical and Electronics manufecturing.pptxAutus Cyber Tech
 
Optimizing Business Potential: A Guide to Outsourcing Engineering Services in...
Optimizing Business Potential: A Guide to Outsourcing Engineering Services in...Optimizing Business Potential: A Guide to Outsourcing Engineering Services in...
Optimizing Business Potential: A Guide to Outsourcing Engineering Services in...Jaydeep Chhasatia
 
Introduction-to-Software-Development-Outsourcing.pptx
Introduction-to-Software-Development-Outsourcing.pptxIntroduction-to-Software-Development-Outsourcing.pptx
Introduction-to-Software-Development-Outsourcing.pptxIntelliSource Technologies
 
How Does the Epitome of Spyware Differ from Other Malicious Software?
How Does the Epitome of Spyware Differ from Other Malicious Software?How Does the Epitome of Spyware Differ from Other Malicious Software?
How Does the Epitome of Spyware Differ from Other Malicious Software?AmeliaSmith90
 
IA Generativa y Grafos de Neo4j: RAG time
IA Generativa y Grafos de Neo4j: RAG timeIA Generativa y Grafos de Neo4j: RAG time
IA Generativa y Grafos de Neo4j: RAG timeNeo4j
 
ARM Talk @ Rejekts - Will ARM be the new Mainstream in our Data Centers_.pdf
ARM Talk @ Rejekts - Will ARM be the new Mainstream in our Data Centers_.pdfARM Talk @ Rejekts - Will ARM be the new Mainstream in our Data Centers_.pdf
ARM Talk @ Rejekts - Will ARM be the new Mainstream in our Data Centers_.pdfTobias Schneck
 
Big Data Bellevue Meetup | Enhancing Python Data Loading in the Cloud for AI/ML
Big Data Bellevue Meetup | Enhancing Python Data Loading in the Cloud for AI/MLBig Data Bellevue Meetup | Enhancing Python Data Loading in the Cloud for AI/ML
Big Data Bellevue Meetup | Enhancing Python Data Loading in the Cloud for AI/MLAlluxio, Inc.
 

Recently uploaded (20)

Your Vision, Our Expertise: TECUNIQUE's Tailored Software Teams
Your Vision, Our Expertise: TECUNIQUE's Tailored Software TeamsYour Vision, Our Expertise: TECUNIQUE's Tailored Software Teams
Your Vision, Our Expertise: TECUNIQUE's Tailored Software Teams
 
Webinar_050417_LeClair12345666777889.ppt
Webinar_050417_LeClair12345666777889.pptWebinar_050417_LeClair12345666777889.ppt
Webinar_050417_LeClair12345666777889.ppt
 
Salesforce AI Associate Certification.pptx
Salesforce AI Associate Certification.pptxSalesforce AI Associate Certification.pptx
Salesforce AI Associate Certification.pptx
 
Watermarking in Source Code: Applications and Security Challenges
Watermarking in Source Code: Applications and Security ChallengesWatermarking in Source Code: Applications and Security Challenges
Watermarking in Source Code: Applications and Security Challenges
 
online pdf editor software solutions.pdf
online pdf editor software solutions.pdfonline pdf editor software solutions.pdf
online pdf editor software solutions.pdf
 
Streamlining Your Application Builds with Cloud Native Buildpacks
Streamlining Your Application Builds  with Cloud Native BuildpacksStreamlining Your Application Builds  with Cloud Native Buildpacks
Streamlining Your Application Builds with Cloud Native Buildpacks
 
Growing Oxen: channel operators and retries
Growing Oxen: channel operators and retriesGrowing Oxen: channel operators and retries
Growing Oxen: channel operators and retries
 
Enterprise Document Management System - Qualityze Inc
Enterprise Document Management System - Qualityze IncEnterprise Document Management System - Qualityze Inc
Enterprise Document Management System - Qualityze Inc
 
JS-Experts - Cybersecurity for Generative AI
JS-Experts - Cybersecurity for Generative AIJS-Experts - Cybersecurity for Generative AI
JS-Experts - Cybersecurity for Generative AI
 
Fields in Java and Kotlin and what to expect.pptx
Fields in Java and Kotlin and what to expect.pptxFields in Java and Kotlin and what to expect.pptx
Fields in Java and Kotlin and what to expect.pptx
 
Top Software Development Trends in 2024
Top Software Development Trends in  2024Top Software Development Trends in  2024
Top Software Development Trends in 2024
 
20240319 Car Simulator Plan.pptx . Plan for a JavaScript Car Driving Simulator.
20240319 Car Simulator Plan.pptx . Plan for a JavaScript Car Driving Simulator.20240319 Car Simulator Plan.pptx . Plan for a JavaScript Car Driving Simulator.
20240319 Car Simulator Plan.pptx . Plan for a JavaScript Car Driving Simulator.
 
Kawika Technologies pvt ltd Software Development Company in Trivandrum
Kawika Technologies pvt ltd Software Development Company in TrivandrumKawika Technologies pvt ltd Software Development Company in Trivandrum
Kawika Technologies pvt ltd Software Development Company in Trivandrum
 
ERP For Electrical and Electronics manufecturing.pptx
ERP For Electrical and Electronics manufecturing.pptxERP For Electrical and Electronics manufecturing.pptx
ERP For Electrical and Electronics manufecturing.pptx
 
Optimizing Business Potential: A Guide to Outsourcing Engineering Services in...
Optimizing Business Potential: A Guide to Outsourcing Engineering Services in...Optimizing Business Potential: A Guide to Outsourcing Engineering Services in...
Optimizing Business Potential: A Guide to Outsourcing Engineering Services in...
 
Introduction-to-Software-Development-Outsourcing.pptx
Introduction-to-Software-Development-Outsourcing.pptxIntroduction-to-Software-Development-Outsourcing.pptx
Introduction-to-Software-Development-Outsourcing.pptx
 
How Does the Epitome of Spyware Differ from Other Malicious Software?
How Does the Epitome of Spyware Differ from Other Malicious Software?How Does the Epitome of Spyware Differ from Other Malicious Software?
How Does the Epitome of Spyware Differ from Other Malicious Software?
 
IA Generativa y Grafos de Neo4j: RAG time
IA Generativa y Grafos de Neo4j: RAG timeIA Generativa y Grafos de Neo4j: RAG time
IA Generativa y Grafos de Neo4j: RAG time
 
ARM Talk @ Rejekts - Will ARM be the new Mainstream in our Data Centers_.pdf
ARM Talk @ Rejekts - Will ARM be the new Mainstream in our Data Centers_.pdfARM Talk @ Rejekts - Will ARM be the new Mainstream in our Data Centers_.pdf
ARM Talk @ Rejekts - Will ARM be the new Mainstream in our Data Centers_.pdf
 
Big Data Bellevue Meetup | Enhancing Python Data Loading in the Cloud for AI/ML
Big Data Bellevue Meetup | Enhancing Python Data Loading in the Cloud for AI/MLBig Data Bellevue Meetup | Enhancing Python Data Loading in the Cloud for AI/ML
Big Data Bellevue Meetup | Enhancing Python Data Loading in the Cloud for AI/ML
 

Scaling wix with microservices architecture jax london-2015

  • 1. Aviran Mordo Head of Engineering @aviranm linkedin.com/in/aviran aviransplace.com Scaling with Microservices Architecture and Multi-cloud platforms
  • 3. Wix in Numbers Over 72M users Static storage is >2PB of data 3 data centers + 3 clouds (Google, Amazon,Azure) 2B HTTP requests/day 1000 people work atWix
  • 4. Initial Architecture Built for fast development Stateful login (Tomcat session), Ehcache, file uploads No consideration for performance, scalability and testing Intended for short-term use Tomcat, Hibernate, custom web framework Lighttpd (file serving) MySQL DB Wix (Tomcat)
  • 5. The Monolithic Giant One monolithic server that handled everything Dependency between features Changes in unrelated areas of the system caused deployment of the whole system Failure in unrelated areas will cause system wide downtime
  • 8. Concerns and SLA DataValidation Security / Authentication Data consistency Lots of data Edit websites High availability High performance Lots of static files Very high traffic volume Viewport optimization Long tail (immutable) Serving Media High availability High performance High traffic volume Long tail (mutable) View sites, created by Wix editor
  • 9. Wix Segmentation 1. Editor Segment 3. Public Segment2. Media Segment Networking
  • 10. HTML Editor Flash Editor MSM Private Media Public Media Editor Segment Public Segment Premium Services eCommerse List DB App Builder App Store App Market Dashboard Statics/me dia Mailer TimeZone Public HTML API Public API (Flash) MSP Public Server HTML Renderer HTML SEO Renderer Flash Renderer Flash SEO Renderer Sitemap Renderer Robots.txt Renderer User Server Template Viewer ContactsHUB Activit y Site Members Provided Mailing Service Comments Snapshoter User Pref Feed Me Shout-out Hotels PETRI Site Pref Dist LoggerSlicer eCom Renderer eCom Cart eCom Checkout eCom Catalog eCom Orders Payment Facade Account Info HTML API HTML Embeder BlogMobile
  • 11. It is all about
  • 13. Microservices Guidelines Each service has its own DB schema (if one is needed) Only one service should write to a specific DB table(s) There may be additional read-only services that directly accesses the DB (for performance reasons) Services are stateless No DB transactions Cache is not a building block, but an optimization
  • 14. Microservices Tradeoffs Each service has its own DB schema (if one is needed) Gain - Easy to scale microservices based on service level concerns Tradeoff – system complexity, performance Only one service should write to a specific DB table(s) Gain - Decoupling architecture – faster development Tradeoff – system complexity / performance May have additional read-only services that accesses the DB Gain - Performance gain Tradeoff - coupling Services are stateless Gain - Easy to scale out (just add more servers) Tradeoff - performance / consistency No DB transactions Gain - Better DB performance, easier to scale Tradeoff - system complexity
  • 16. Editor Server Immutable JSON pages (~3M / day) Site revisions Active – standby MySQL cross datacenters Editor Server MySQL Active Sites MySQL Archive
  • 18. Protect The Data DB outage with fast recovery = replication Data poisoning/corruption = revisions / backup Make the data available at all times = data distribution to multiple locations / providers
  • 19. Browser Editor Server GCS MySQL Active Sites MySQL Archive Saving Editor Data WixMedia (Amazon) WixMedia (Google) Save Page(s) 200 OK Upload Save Page DC replication Notify MySQL Archive MySQL Active Sites S3 WixMedia (DC-1)
  • 20. Browser Editor Server GCS MySQL Active Sites MySQL Archive WixMedia (Amazon) WixMedia (Google) Save Page(s) 200 OK Upload Save Page DC replication Notify MySQL Archive MySQL Active Sites S3 WixMedia (DC-1) Self Healing Process
  • 21. No DB Transactions Save each page (JSON) as an atomic operation Page ID is a content based hash (immutable/idempotent) Finalize transaction by sending site header (list of pages) Can generate orphaned pages, not a problem in practice
  • 22. 2. Media Segment (WixMP)
  • 23. Wix Media Platform (WixMP) Eventual consistent distributed file system (2PB user media files) Dynamic media processing Multi datacenter aware Automatic fallback cross DC Run on commodity servers & cloud
  • 24. T Google Cloud Prospero – Wix Media Manager get image.jpg First fallback Second fallback If not in CDN Amazon x36 T x36 T x32 Austin CDN
  • 26. Public Segment Roles Routing (resolve URLs) Dispatching (to a renderer) Rendering (HTML,XML,TXT) Public Server HTML Renderer HTML SEO Renderer Flash Renderer Sitemap Renderer Robots.txt Renderer www.example.com Flash SEO Renderer
  • 27. Public SLA Our goal: 99% response time <100ms at peak traffic
  • 28. Publish Site Publish site header (a map of pages for a site) Publish routing table Publish site header / routes (CQRS) Editor Segment Public Segment
  • 29. Built For Speed Minimize out-of-process hops (2 DB, 1 RPC) Lookup tables are cached in memory, updated every few minutes Denormalized data – optimize for read by primary key (MySQL) Minimize business logic
  • 30. How a Page Gets Rendered Bootstrap HTML template that contains only data Only JavaScript imports JSON data (site-header + dynamic data) No “real” HTML view
  • 31. Offload rendering work to the browser
  • 32. The average Intel Core i750 can push up to 7 GFLOPS without overclocking
  • 33. Why JSON? Easy to parse in JavaScript and Java/Scala Fairly compact text format Highly compressible (5:1 even for small payloads) Easy to fix rendering bugs and cross browsers issues (just deploy a new code)
  • 34. Minimum Number of Public Servers Needed to Serve 66M Sites 4
  • 36. Serving a Site – Sunny Day Archive CDN WixMP Browser http://example.wix.com Store HTML to cache HTTP Request Notify site view LB Public Renderer HTML Resources / Media HTTP Request
  • 37. Serving a Site – DC Lost Archive CDN WixMP Browser http://example.wix.com LB Public Renderer LB Public Renderer Change DNS HTTP Request
  • 38. Serving a Site – Public Lost Archive Browser http://example.wix.com LB Public Renderer Get Cached HTML Version HTML HTTP Request LB Public Renderer Fallback to 2nd DC
  • 39. Living in the Browser CDN WixMP Browser http://example.wix.com LB Public Renderer Editor Pages Fallback JSON / Media HTML HTTP Request Fallback
  • 40. Summary Identify concerns and SLA for different parts of the system Build redundancy in critical path (for availability) De-normalize data (for performance) Minimize out-of-process hops (for performance) Take advantage of client’s CPU power
  • 42. Aviran Mordo Head of Engineering @aviranm linkedin.com/in/aviran aviransplace.com http://engineering.wix.com http://goo.gl/3xhpNW @WixEng

Editor's Notes

  1. Editor – Read immediately after write – Small working set Viewer optimize for reads We fight for every ms. Page view = many resource downloading
  2. Read-only services only if it is part of the same business functionality
  3. Read-only services only if it is part of the same business functionality
  4. Immutable data helps handle eventual consistency MySql is a great key-value store Not all data is equal (only 6% of websites are edited 3 months after creation)
  5. Revision keep data safe from poisoning Pay in storage and management
  6. We can change the arrows as we want Tech vendor lock is a myth,  easy to change the api (small dev effort).  Invest in data distribution. Evaluation of new platform starts by putting the data.
  7. Save pages on JSON Upload to static storage
  8. Explain what is JSON and what is HTML
  9. UPS dies, secondary power source connected to the same UPS
  10. Due to error or bad configuration