In this talk, Aviran will describe how http://Wix.com is pushing this trend even further to build its own Platform as a Runtime (PaaR) infrastructure that allows developers to develop faster, better with higher quality. By allowing nano deployments of different modules into a “SingleRuntime” inside a robust internal platform that handles many of the non-functional concerns developers are facing on a daily basis.

1. When DevOps Runs Its Course
We Need Platform as a Runtime
Aviran Mordo - VP Engineering
Wix.com
Qcon London 2024
www.aviransplace.com Twitter: @aviranm linkedin/aviran

2. Monolith Microservices Serverless ?
Which is better?
When DevOps Runs Its Course – We Need Platform as a Runtime @aviranm

3. Monolith Microservices Serverless
Where each shines
When DevOps Runs Its Course – We Need Platform as a Runtime @aviranm
Cost*
Cross service
latency
Ownership
(single responsibility)
Coding
Deployment
Scaling

4. When DevOps Runs Its Course – We Need Platform as a Runtime @aviranm
Complexity !!!

5. When DevOps Runs Its Course – We Need Platform as a Runtime @aviranm
3 Pillars of system’s design tradeoffs
Why is it complicated ?
Code Run
Deploy
Dev velocity
Performance
Resilience
Scalability
Cost
Code complexity
Maintainability
Org scale

6. When DevOps Runs Its Course – We Need Platform as a Runtime @aviranm
Hello
Aviran Mordo,
VP of Engineering,
Wix.com

8. When DevOps Runs Its Course – We Need Platform as a Runtime @aviranm
Wix In Numbers (2023)
±250M
Registered users
(website builders)
From
190
Countries
±7%
of all Internet web
sites running on
Wix
±150M
Websites
1B
Human visitors
every month
±4000
Microservices
clusters
3
Data Centers

9. When DevOps Runs Its Course – We Need Platform as a Runtime @aviranm
WixEng Locations (2023) EU Ukraine Israel ROW
Vilnius Kyiv Tel-Aviv USA
Krakow Dnipro Be’er Sheva Canada
Berlin Lviv Haifa
Amsterdam

10. When DevOps Runs Its Course – We Need Platform as a Runtime @aviranm
Complexity !!!

11. When DevOps Runs Its Course – We Need Platform as a Runtime @aviranm
The responsibility of
engineering is to
deliver business value
Successful
companies deliver
high quality code
FAST

12. When DevOps Runs Its Course – We Need Platform as a Runtime @aviranm
Monolith Microservices Serverless ?
Why is it complicated?

13. When DevOps Runs Its Course – We Need Platform as a Runtime @aviranm
A single product is usually
simple & fast to build as a monolith
Building a Service

14. Run
Deploy
Code
When DevOps Runs Its Course – We Need Platform as a Runtime @aviranm
Monolith
Coding is easy
(at the beginning)
Mixing domain
(spaghetti)
Everything is accessible
to all developers
Hard to scale to multiple
teams
Easy to test*
Simple topology
Good performance (between models)
Hard to scale and keep SLO
(multiple endpoints)
Multiple teams need to
coordinate the release
No service owner?
Easy to debug
Many changes in each
release = higher risk
Easy to break APIs between
components

15. When DevOps Runs Its Course – We Need Platform as a Runtime @aviranm
Monolith Microservices Serverless ?
Why is it complicated?

16. When DevOps Runs Its Course – We Need Platform as a Runtime @aviranm
Direct Integrations & Dependencies
Not so simple

17. Not just direct dependencies - Call chains

18. Wix has Over 4000 microservices
Starship

19. Run
Deploy
Code
When DevOps Runs Its Course – We Need Platform as a Runtime @aviranm
Microservices / Serverless
Coding is easy
– Single domain
Complex integration
Complex testing
Independent release cycle
Complex topology
Backwards compatibility
Performance
Hard to monitor/debug
Easy to deploy
Easy to scale
Clear service owner
Cross cutting concerns

20. Run
Deploy
Code
When DevOps Runs Its Course – We Need Platform as a Runtime @aviranm
Let’s talk about code complexity
Independent release cycle
Complex topology
Backwards compatibility
Performance
Hard to monitor/debug
Easy to deploy
Easy to scale
Clear service owner
Coding is easy
– Single domain
Complex integration
Complex testing
Cross cutting concerns

21. When DevOps Runs Its Course – We Need Platform as a Runtime @aviranm
Act II
Let’s Code

22. When DevOps Runs Its Course – We Need Platform as a Runtime @aviranm
Meet Dana !
A new developer
Write task management system
Write a new microservice, and….
Job
Action:

23. When DevOps Runs Its Course – We Need Platform as a Runtime @aviranm
Domain modeling
API Design
Authentication
Authorization
Input Validation
Object Mapping
RPC
Secrets
Data Access
Domain Events
Error Handling
GDPR
PII
Plat. Query
Advanced Search
Caching
Webhooks
Scheduled Tasks
BI
Logging
Testing Strategy & Tools
Scaffold
Build
Deploy
A/B Testing Support
Gradual Rollout
Production Ownership
Monitoring
Debugging
Best Practices
A new microservice needs ...

24. Framework
Dana reads the documentation

25. Framework Identity

26. Framework
Visibility
Web Hooks GDPR
Identity

27. Framework Input validation
Visibility
Web Hooks GDPR
Identity
Testing
API Design
Brokers & Events Error Handling Database

28. Framework Events
Visibility Web Hooks GDPR
Identity
Testing
API Design
Brokers & Events Error Handling Database
Data integrity
gRPC
Webhooks

29. Framework Events
Visibility Web Hooks GDPR
Identity
Testing
API Design
Brokers & Events Error Handling Database
Data integrity
gRPC
Webhooks

30. When DevOps Runs Its Course – We Need Platform as a Runtime @aviranm
The line of code that’s the
fastest to write, the line of code that
never brakes, that
doesn’t need maintenance is the line
you never had to write.
The goal here is to
eliminate 80% of the code that you
have to write for your app.
If we can develop faster
we can give more focus
to great engineering practices.
Eliminating code

31. When DevOps Runs Its Course – We Need Platform as a Runtime @aviranm
How can we CODE
faster in a complex
environment?

32. NILE
Streamlines platformized
microservice development
* Highly opinionated
Introducing:
Codify and automate the guidelines into the platform.
When DevOps Runs Its Course – We Need Platform as a Runtime @aviranm

33. - Framework
- Schema Creation
- Messaging
- Async Task Queue
- gRPC
- Data Access
- Search
- SPI
- Logging
- Metrics
When DevOps Runs Its Course – We Need Platform as a Runtime @aviranm
Nile layers and tools
- Data Migration
- Multi-Lingual
- API Migration
- Caching
- GDPR
- …..

34. Run
Deploy
Code
When DevOps Runs Its Course – We Need Platform as a Runtime @aviranm
Platform
Let’s talk about deployment
Complex topology
Performance
Hard to monitor/debug
Easy to scale
Clear service owner
Independent release cycle
Backwards compatibility
Easy to deploy
Deploy

35. When DevOps Runs Its Course – We Need Platform as a Runtime @aviranm
Building Serverless Platform

36. When DevOps Runs Its Course – We Need Platform as a Runtime @aviranm
Business
Logic
Trusted Env Framework
(contracts)
Microservice framework
(Spring)
Containers (K8)
VM
Typical Software stack

37. When DevOps Runs Its Course – We Need Platform as a Runtime @aviranm
Business
Logic
Trusted Env Framework
(contracts)
Microservice framework
(Spring)
Package and deploy a service
90% of the code is not yours

38. When DevOps Runs Its Course – We Need Platform as a Runtime @aviranm
Microservices shortcoming at scale
Business Logic
Trusted Env Framework
(contracts)
Microservice framework
(Spring)
● Deployment size has a large footprint -
framework is compiled with each
microservice.
● Frameworks/libraries versions are tied
to the microservice deploy lifecycle
● Urgent security fixes of thousands of
services (aka Log4J)
● Supporting multiple languages need to
duplicate the whole framework

39. When DevOps Runs Its Course – We Need Platform as a Runtime @aviranm
Microservice
95%-99% of the code that runs a typical Microservice,
is code that you DIDN’T write

40. Run
Deploy
Code
When DevOps Runs Its Course – We Need Platform as a Runtime @aviranm
In a perfect world
No integration wiring
overhead
Minimal testing
Zero boilerplate code
Fast deployment
Easy development with
minimal cognitive load
Good performance
No or little operations
overhead
Low cost
Easy updates
Scalable

41. When DevOps Runs Its Course – We Need Platform as a Runtime @aviranm
Why not take the
best of all worlds
Microservices
Serverless
Monolith
Managed
Platform

42. When DevOps Runs Its Course – We Need Platform as a Runtime @aviranm
Monolith Microservices Serverless ?
Which is better?

43. PaaR
Platform as a Runtime =
Reality
+
Vision

44. When DevOps Runs Its Course – We Need Platform as a Runtime @aviranm
• Easy development with minimal
cognitive load
• No integration wiring overhead
• Good performance
• Fast deployment
• Low cost
• No or little operations overhead
• Minimal testing
• Zero boilerplate code
Wix Velo
(Managed platform)

45. When DevOps Runs Its Course – We Need Platform as a Runtime @aviranm
Wix Platform as a Runtime V1 (PaaR)
Business
Logic
Data services
Service integration layer
(gRPC/REST/Kafka)
Application framework
Node.js

46. When DevOps Runs Its Course – We Need Platform as a Runtime
Business
Logic
Data services
Service integration layer
(gRPC/REST/Kafka)
Application framework
Node.js
When DevOps Runs Its Course – We Need Platform as a Runtime @aviranm
Node.js
• Lightweight
• Dynamic code loading
• Easy to learn
Node.js

47. When DevOps Runs Its Course – We Need Platform as a Runtime
Business
Logic
Data services
Service integration layer
(gRPC/REST/Kafka)
Application framework
Node.js
When DevOps Runs Its Course – We Need Platform as a Runtime @aviranm
Wix Microservices framework
• Based on express
• Handles trusted environment contracts
○ HTTP headers
○ Authorization
○ Authentication
○ Monitoring
○ Logging / BI
○ Experiments (A/B Testing)
Application framework

48. When DevOps Runs Its Course – We Need Platform as a Runtime
Business
Logic
Data services
Service integration layer
(gRPC/REST/Kafka)
Application framework
Node.js
When DevOps Runs Its Course – We Need Platform as a Runtime @aviranm
Service Integration Layer
• Repository of all Wix GRPC / REST clients
○ Based on Protobuf
• Kafka client integrations
Service integration layer
(gRPC/REST/Kafka)

49. When DevOps Runs Its Course – We Need Platform as a Runtime
Business
Logic
Data services
Service integration layer
(gRPC/REST/Kafka)
Application framework
Node.js
When DevOps Runs Its Course – We Need Platform as a Runtime @aviranm
Data Services
• Integration to simple Key/Value store
(DynamoDB)
Data services

50. When DevOps Runs Its Course – We Need Platform as a Runtime @aviranm
PaaR Serverless
Package the platform as a serverless container
Data services
Service integration layer
(gRPC/REST/Kafka)
Application framework
Node.js

51. When DevOps Runs Its Course – We Need Platform as a Runtime @aviranm
PaaR Serverless
Your Code
Your Code
Dynamically loaded into the runtime

52. When DevOps Runs Its Course – We Need Platform as a Runtime @aviranm
PaaR Serverless
F1
F2
F3
Trusted environment –
Can run multiple functions
in a single container

53. When DevOps Runs Its Course – We Need Platform as a Runtime
Business
Logic
Data services
Service integration layer
(gRPC/REST/Kafka)
Application framework
Node.js
When DevOps Runs Its Course – We Need Platform as a Runtime @aviranm
Write your function with just business logic
● No integrations
○ Just declare which services you want to use
● Less testing
○ No integration tests needed (don’t test the
platform)
● Think very small functions
○ Just your business logic.
● Fast deployment
○ Small code, you don’t package the framework
● Zero boilerplate
○ All integrations are provided and
preconfigured
○ No boilerplate or wiring needed

54. When DevOps Runs Its Course – We Need Platform as a Runtime @aviranm
Developer Experience
Code Example

55. When DevOps Runs Its Course – We Need Platform as a Runtime @aviranm
1. Add an API to:
• Retrieve a task details from the tasks
microservice
• Retrieve details of the person assigned to
this task from the contacts microservice.
We have a task management system
2. Every time a task is created
write an audit log to a database
Task

56. When DevOps Runs Its Course – We Need Platform as a Runtime @aviranm
Just the code you need...

57. When DevOps Runs Its Course – We Need Platform as a Runtime @aviranm
Just the code you need...

58. When DevOps Runs Its Course – We Need Platform as a Runtime @aviranm
Just the code you need...

59. When DevOps Runs Its Course – We Need Platform as a Runtime @aviranm
Just the code you need...

60. When DevOps Runs Its Course – We Need Platform as a Runtime @aviranm
Just the code you need...

61. When DevOps Runs Its Course – We Need Platform as a Runtime @aviranm
Quick Pipeline
Nano Deployments = Stream Development
• Minimal code
• No framework packaging
• Zero startup time
• Deployable artifact size <10kb
• Separation of source structure
from deployment packaging
1-2 minutes
Run
Deploy
Build
Push

62. Run
Deploy
Code
When DevOps Runs Its Course – We Need Platform as a Runtime @aviranm
Let’s optimize the Runtime
Platform as
a Service
Platform
Complex topology
Performance
Hard to monitor/debug
Easy to scale
Clear service owner

63. When DevOps Runs Its Course – We Need Platform as a Runtime @aviranm
PaaR
Flexible, Dynamic, Optimized,
Cost-Effective Scalability
This is the “R(untime)” in PaaR.

64. Node
Serverless Runtime
Ecom
Node
Serverless Runtime
Payment
When DevOps Runs Its Course – We Need Platform as a Runtime @aviranm
F1 F2
F4
F6 F3
F5
Container strategy
For regulations (PCI)
Noisy neighbors
Team ownership
Flexible Topology - Isolated Environment

65. Node
Serverless Runtime
Ecom
Node
Serverless Runtime
Payment
When DevOps Runs Its Course – We Need Platform as a Runtime @aviranm
F1 F2
F4
F6 F3
F5
In process calls = RPC calls

66. Node
Serverless Runtime
Ecom
Node
Serverless Runtime
Payment
When DevOps Runs Its Course – We Need Platform as a Runtime @aviranm
Lets Scale out F4 F4
F2
F4
F6 F3
F5 F4
F1
Auto scale on a function level

67. Node
Serverless Runtime
Ecom
Node
Serverless Runtime
Payment
When DevOps Runs Its Course – We Need Platform as a Runtime @aviranm
F2 F5
F2 and F5 frequently interact
F1 F2
F4
F6 F3
F5
F5
JIT Performance optimization
Reduce network calls
Increase throughput
Optimized Function Deployment Affinity

68. When DevOps Runs Its Course – We Need Platform as a Runtime @aviranm
Microservices challenges at scale
● Deployment size has a large footprint -
framework is compiled with each microservice.
● Frameworks/libraries versions are tied to the
microservice build/deploy lifecycle
● Urgent security fixes of thousands of services
(aka Log4J)
● Supporting multiple languages need to duplicate
the whole framework

69. But Wait…We are missing something (WIP)
Additional languages support
Without rewriting the framework multiple times
To support more use cases that node.js is not the right tool for the job
To quickly integrate acquired companies

70. When DevOps Runs Its Course – We Need Platform as a Runtime @aviranm
Wix Single
Runtime (WIP)
V2

71. When DevOps Runs Its Course – We Need Platform as a Runtime @aviranm
• One framework
development
• Can support any
language for
business logic
• Not just functions,
but Microservices
Host
Guest
Wix Single Runtime – Support any Language
Business
Logic
Guest SDK
Data Services
Service integration Layer
(GRPC/REST/Kafka)
Single Runtime framework

72. When DevOps Runs Its Course – We Need Platform as a Runtime @aviranm
Ingress and Egress communication via “Host”, i.e
Platform as a Service (Daemon set)
VM
PaaR Host
(Daemon set)
PaaR Guest
(K8s pod)
Srv 4
PaaR Guest
(K8s pod)
Srv 5
PaaR Guest
(K8s pod)
Srv 2
VM
PaaR Host
(Daemon set)
PaaR Guest
(K8s pod)
Srv 1
PaaR Guest
(K8s pod)
Srv 3
PaaR Guest
(K8s pod)
Srv 2

73. When DevOps Runs Its Course – We Need Platform as a Runtime @aviranm
Decisions and tradeoffs
VM
PaaR Host
(Daemon set)
PaaR Guest
(K8s pod)
Srv 4
PaaR Guest
(K8s pod)
Srv 5
PaaR Guest
(K8s pod)
Srv 2
• Tradeoff in-process to localhost
communication.
• Taking advantage of K8s
ecosystem for deployment
management.
• Larger guest footprint
(still ±50% smaller than a regular
microservice)
• Up to 2ms overhead from typical
microservice

74. When DevOps Runs Its Course – We Need Platform as a Runtime @aviranm
Microservices challenges at scale [SOLVED]
Host
Guest
Business
Logic
Guest SDK
Data Services
Service integration Layer
(GRPC/REST/Kafka)
Single Runtime framework
● Cost effective, smaller footprint of guest than
microservice
(no framework is compiled with each service)
● Single framework version,
with its own lifecycle
● Urgent security fixes of thousands of services
(aka Log4J)
● Multiple languages
(just write a different guest SDK instead of
duplicating the whole framework)

75. When DevOps Runs Its Course – We Need Platform as a Runtime @aviranm
Wix
Managed
Data Layer
Message
Bus
Platform
Services
(logs…)
Single Runtime is the Platform (PaaR)
Single Runtime
Microservices
Microservices
Microservices
Microservices
Microservices
Microservices
DB
DB
DB
DB
API GW / Networking

76. Run
Deploy
Code
When DevOps Runs Its Course – We Need Platform as a Runtime @aviranm
Complexity solved*
Platform as
a Service
Platform PaaR

77. When DevOps Runs Its Course – We Need Platform as a Runtime @aviranm
Code Run
Deploy
PaaR
Like a
Microservice
Like a
Function
Like a
(virtual)
Monolith
When DevOps Runs Its Course – We Need Platform as a Runtime @aviranm
Brings business value FAST
Improved dev velocity by 50%-80%
Up to 50% cost reduction on compute*

78. Thank you.
Let’s talk!
www.aviransplace.com
Twitter: @aviranm
linkedin/aviran
Aviran Mordo
VP Engineering Wix.com Qcon London 2024
Blog post: https://wix.to/WYA8sUw

79. Remember to vote and share feedback on the InfoQ App or online.
Please vote and leave feedback!
Any questions?