Today, there are several trends that are forcing application architectures to evolve. Users expect a rich, interactive and dynamic user experience on a wide variety of clients including mobile devices. Applications must be highly scalable, highly available and run on cloud environments. Organizations often want to frequently roll out updates, even multiple times a day. Consequently, it’s no longer adequate to develop simple, monolithic web applications that serve up HTML to desktop browsers. In this talk we describe the limitations of a monolithic architecture. You will learn how to use the scale cube to decompose your application into a set of narrowly focused, independently deployable back-end services and an HTML 5 client. We will also discuss the role of technologies such as NodeJS and AMQP brokers. You will learn how a modern PaaS such as Cloud Foundry simplifies the development and deployment of this style of application.

1. Decomposing applications for
deployability and scalability
Chris Richardson
Author of POJOs in Action
Founder of the original CloudFoundry.com
@crichardson
crichardson@vmware.com
http://plainoldobjects.com/
1

2. Presentation goal
How decomposing
applications improves
deployability and scalability
and
How Cloud Foundry helps
2

3. About Chris
3

4. (About Chris)
4

5. About Chris()
5

6. About Chris
6

7. About Chris
http://www.theregister.co.uk/2009/08/19/springsource_cloud_foundry/
7

8. vmc push About-Chris
Developer Advocate for
CloudFoundry.com
Signup at http://cloudfoundry.com
8

9. Agenda
§The (sometimes evil) monolith
§Decomposing applications into services
§How do services communicate?
§Presentation layer design
§How Cloud Foundry helps
9

10. Let’s imagine you are building
an e-commerce application
10

11. Traditional web application architecture
WAR
StoreFrontUI
Accounting
Service
MySQL
Browser Apache
InventoryService Database
Shipping
Service
Simple to Tomcat
develop
test
deploy
scale 11

12. But there are problems with
a monolithic architecture
12

13. Users expect a rich, dynamic
and interactive experience
h
oug
en
ood
’tg
HTTP Request
isn
ure
ect
Java Web
Browser
it
HTML/Javascript Application
Ia rch
ty le U
s
Old
Real-time web ≅ NodeJS
13

14. Intimidates developers
14

15. Obstacle to frequent deployments
§Need to redeploy everything to change one component
§Interrupts long running background (e.g. Quartz) jobs
§Increases risk of failure
Fear of change
§Updates will happen less often
§e.g. Makes A/B testing UI really difficult 15

16. Overloads your IDE and container
Slows down development 16

17. Obstacle to scaling development
Accounting team
E-commerce
Engineering
application
Shipping team
17

18. Obstacle to scaling development
WAR
UI team StoreFrontUI
Accounting team Accounting
Inventory team InventoryService
Shipping team Shipping
18

19. Obstacle to scaling development
Lots of coordination and
communication required 19

20. Requires long-
term commitment
to a technology
stack
20

21. Agenda
§The (sometimes evil) monolith
§Decomposing applications into services
§How do services communicate?
§Presentation layer design
§How Cloud Foundry helps
21

22. 22

23. The scale cube
Y axis -
functional
decomposition
Scale by
im ing
splitting
g s on
r
ila
tin iti
different things
lit art
p
gs y ata
th ale - d
sp
i s
ax
in b
Z
X axis - horizontal
Sc
duplication
23

24. Y-axis scaling - application level
WAR
StoreFrontUI
Accounting
Service
InventoryService
Shipping
Service
24

25. Y-axis scaling - application level
accounting web application
Accounting
Service
Store front web application inventory web application
InventoryService
StoreFrontUI
shipping web application
Shipping
Service
Apply X axis cloning and/or Z axis partitioning to each service 25

26. Partitioning strategies
§Partition by verb, e.g. shipping service
§Partition by noun, e.g. inventory service
§Single Responsibility Principle
§Unix utilities - do one focussed thing well
Something of an art
26

27. Real world examples
http://techblog.netflix.com/
Between 100-150 services are accessed to build a
page.
http://highscalability.com/amazon-architecture
http://www.addsimplicity.com/downloads/
eBaySDForum2006-11-29.pdf
http://queue.acm.org/detail.cfm?id=1394128
27

28. There are drawbacks
28

29. Complexity
See Steve Yegge’s Google
Platforms Rant re Amazon.com
29

30. Multiple databases
=
Transaction management
challenges
30

31. When to use it?
In the beginning:
•You don’t need it
•It will slow you down
Later on:
•You need it
•Refactoring is painful 31

32. But there are many benefits
§Scales development: develop, deploy and scale
each service independently
§Update UI independently
§Improves fault isolation
§Eliminates long-term commitment to a single
technology stack
Modular, polyglot, multi-
framework applications
32

33. Two levels of architecture
System-level
Services
Inter-service glue: interfaces and communication mechanisms
Slow changing
Service-level
Internal architecture of each service
Each service could use a different technology stack
Pick the best tool for the job
Rapidly evolving
33

34. If services are small...
§Regularly rewrite using a better technology stack
§Adapt system to changing requirements and
better technology without a total rewrite
§Pick the best developers rather than best <pick
a language> developers polyglot culture
Fred George
“Developer Anarchy”
34

35. The human body as a system
35

36. 50 to 70 billion of your cells die each
day
36

37. Yet you (the system) remain you
37

38. Can we build software systems
with these characteristics?
http://dreamsongs.com/Files/
DesignBeyondHumanAbilitiesSimp.pdf
http://dreamsongs.com/Files/WhitherSoftware.pdf
38

39. Agenda
§The (sometimes evil) monolith
§Decomposing applications into services
§How do services communicate?
§Presentation layer design
§How Cloud Foundry helps
39

40. Inter-service communication options
§Synchronous HTTP asynchronous AMQP
§Formats: JSON, XML, Protocol Buffers, Thrift, ...
§Even via the database
Asynchronous is preferred
JSON is fashionable but binary
format is more efficient
40

41. Asynchronous message-based communication
wgrus-billing.war
Accounting
Service
wgrus-store.war wgrus-inventory.war
RabbitMQ
StoreFrontUI (Message InventoryService MySQL
Broker)
wgrus-shipping.war
ShippingService
41

42. Benefits
§Decouples caller from server
§Caller unaware of server’s coordinates (URL)
§Message broker buffers message when server is
down/slow
42

43. Drawbacks
§Additional complexity of message broker
§RPC using messaging is more complex
43

44. Writing code that calls
services
44

45. The need for parallelism
Service B
b = serviceB()
Call in parallel
c = serviceC()
Service A Service C
d = serviceD(b, c)
Service D
45

46. Java Futures are a great
concurrency abstraction
http://en.wikipedia.org/wiki/Futures_and_promises
46

47. Akka’s composable
futures are even better
47

48. Using Akka futures
def callB() : Future[...] = ...
def callC() : Future[...] = ...
def callD() : Future[...] = ... Two calls execute in parallel
val future = for {
(b, c) <- callB() zip callC();
d <- callD(b, c)
And then invokes D
} yield d
val result = Await.result(future, 1 second)
http://doc.akka.io/docs/akka/2.0.1/scala/futures.html Get the result of D
48

49. Spring Integration
§Provides the building blocks for
a pipes and filters architecture
§Enables development of
application components that are
•loosely coupled
•insulated from messaging
infrastructure
§Messaging defined declaratively
49

50. Handling failure
Service A Service B
Errors happen
in distributed systems
50

51. About Netflix
> 1B API calls/day
1 API call average 6 service calls
Fault tolerance is essential
http://techblog.netflix.com/2012/02/fault-tolerance-in-high-volume.html
51

52. Use timeouts and retries
Never wait forever
Errors can be transient retry
http://techblog.netflix.com/2012/02/fault-tolerance-in-high-volume.html
52

53. Use per-dependency bounded thread pool
Service A
Runnable 1 Task 1
Runnable 2 Task 2
Service B
Runnable ... Task ...
bounded queue bounded thread pool
Fails fast if Limits number of
service is slow or down outstanding requests
http://techblog.netflix.com/2012/02/fault-tolerance-in-high-volume.html 53

54. Use a circuit breaker
High error rate stop calling temporarily
Down wait for it to come back up
Slow gives it a chance to recover
http://techblog.netflix.com/2012/02/fault-tolerance-in-high-volume.html 54

55. On failure
Return cached data
Avoid
Failing
Return default data
Fail fast
http://techblog.netflix.com/2012/02/fault-tolerance-in-high-volume.html 55

56. Aspects + Actors
Dependency
Caller Invoker Implements
Aspect circuit
breaker state
machine
CircuitBreaker
Equivalent of Actor
thread pool
Worker Dependency
Worker
Actor Stub
Actor
56

57. @DependencyProxy
trait RestaurantService {
def findNearbyRestaurants(location: Location) :
Future[FindNearbyRestaurantResponse]
}
@Service
@DependencyProxy(circuitBreaker = "factualCircuitBreaker",
timeoutInMilliseconds=750)
class FactualRestaurantService extends RestaurantService {
...
}
57

58. Aspect-based Async
@Aspect
class DependencyInvokingAspect {
@Pointcut("execution(* (@DependencyProxy *).*(..))")
def dependencyProxyMethods {}
@Around("dependencyProxyMethods()")
def invoke(jp: ProceedingJoinPoint) = {
val a = AnnotationUtils.findAnnotation(jp.getTarget.getClass, classOf[DependencyProxy])
val actor = findActor(a)
val timeout = Timeout(a.timeoutInMilliseconds milliseconds)
actor.ask(InvokeDependency( () => jp.proceed())) (timeout)
}
}
Ask actor to invoke jp.proceed() and return Future
58

59. See
https://github.com/cer/votemeeteat
for the Actor code
59

60. Agenda
§The (sometimes evil) monolith
§Decomposing applications into services
§How do services communicate?
§Presentation layer design
§How Cloud Foundry helps
60

61. Modular application
Choice of presentation layer
technology
61

62. NodeJS is the fashionable technology
62

63. Why NodeJS?
§Familiar Javascript
§High-performance, scalable event-driven, non-blocking
I/O model
§Over 13,000 modules developed by the community
§Many JavaScript client frameworks have a NodeJS
counterpart, e.g. socket.io
63

64. NodeJS example
Handle Handle file
var http = require('http'); HTTP read
var fs = require("fs"); request
http.createServer(function (req, res) {
fs.readFile('somefile.txt', function (err, data) {
if (err) throw err;
res.writeHead(200, {'Content-Type': 'text/plain'});
res.end(data);
});
}).listen(1337, '127.0.0.1');
console.log('Server running at http://127.0.0.1:1337/');
64

65. NodeJS isn’t the only game in town
JVM-based http://vertx.io/
65

66. A modern web application
Service 1
RESTful WS Node JS
Browser
HTML 5 Server Application
Application Service 2
Events
Socket.io Socket.io
client server
...
66

67. NodeJS - using RESTful WS and AMQP
REST
Service
REST
Requests
Node JS
Events
socket.io
AMQP AMQP
RabbitMQ Service
67

68. Agenda
§The (sometimes evil) monolith
§Decomposing applications into services
§How do services communicate?
§Presentation layer design
§How Cloud Foundry helps
68

69. Original architecture
WAR
StoreFrontUI
Accounting
Service
MySQL
Browser Apache
InventoryService Database
Shipping
Service
Tomcat
69

70. Modern architecture
Desktop Browser Native Mobile application HTML5 mobile application
StoreUI StoreUI StoreUI
Asynchronous, NodeJS
NodeJS Javascript
scalable StoreUI
communication
RabbitMQ
Spring/Scala
web Inventory Shipping Standalone
application Inventory
Service Shipping
Service “headless”
Service Service Spring/Java
applications
MySQL Redis
Billing Service Customer Mongo Order
Inventory
Database Database
Database
70

71. Traditional tools: monolithic applications
71

72. Developing modular apps is more difficult
§Many more moving parts to manage
•Platform services: SQL, NoSQL, RabbitMQ
•Application services: your code
§Who is going to setup the environments:
•the developer sandbox?
•...
•QA environments?
But Cloud Foundry
helps...
72

73. Easy polyglot application deployment
and service provisioning
OSS community
vFabric
Postgres
Ap
p
lica
Private
'o
Clouds
n
S
Data Services
erv
ice
In
ter
vFabric
fac
e
RabbitMQTM Public
Clouds
Msg Services
Micro
Other Clouds
Services
Additional partners services
…

74. Creating a platform service instance
$ vmc create-service mysql --name mysql1
Creating Service: OK
$ vmc services
......
=========== Provisioned Services ============
+-------------+---------+
| Name | Service |
+-------------+---------+
| mysql1 | mysql |
+-------------+---------+

75. Multi-application manifest - part 1
---
applications:
inventory/target: Path to application
name: inventory
url: cer-inventory.chrisr.cloudfoundry.me
framework:
name: spring
info:
mem: 512M
description: Java SpringSource Spring Application
exec:
mem: 512M
instances: 1
services: Required platform services
si-rabbit:
type: :rabbitmq
si-mongo:
type: :mongodb
si-redis:
type: :redis
75

76. Multi-application manifest - part 2
store/target:
name: store Path to application
url: cer-store.chrisr.cloudfoundry.me
framework:
name: spring
info:
mem: 512M
description: Java SpringSource Spring Application
exec:
mem: 512M
instances: 1
services:
Required platform services
si-mongo:
type: :mongodb
si-rabbit:
type: :rabbitmq
76

77. One command to create platform services and
deploy application
$ vmc push
Would you like to deploy from the current directory? [Yn]:
Pushing application 'inventory'...
Creating Application: OK
Creating Service [si-rabbit]: OK
Binding Service [si-rabbit]: OK
Creating Service [si-mongo]: OK
Binding Service [si-mongo]: OK
Creating Service [si-redis]: OK
Binding Service [si-redis]: OK
Uploading Application:
Checking for available resources: OK
vmc push:
Processing resources: OK •Reads the manifest file
Packing application: OK
Uploading (12K): OK •Creates the required platform services
Push Status: OK
Staging Application 'inventory': OK
•Deploys all the applications
Starting Application 'inventory': OK
Pushing application 'store'...
Creating Application: OK
Binding Service [si-mongo]: OK
Binding Service [si-rabbit]: OK
Uploading Application:
Checking for available resources: OK
Processing resources: OK
Packing application: OK 77

78. Micro Cloud Foundry: new developer sandbox
App Instances Services
Open source Platform as a Service project
10.04
A PaaS packaged as a VMware Virtual Machine
Use as a developer sandbox
• Use the services from Junit integration tests
• Deploy your application for functional testing
• Remote debugging from STS
78

79. Using Caldecott to tunnel into your services
79

80. Caldecott = TCP over HTTP
native
native
protocol HTTP
Service Caldecott Caldecott protocol
Service
client Port gem application
NNN
Your computer Cloud Foundry
80

81. Using Caldecott…
$ vmc tunnel
1: mysql-135e0
2: mysql1
Which service to tunnel to?: 2
Password: ********
Stopping Application: OK
Redeploying tunnel application 'caldecott'.
Uploading Application:
Checking for available resources: OK
Packing application: OK
Uploading (1K): OK
Push Status: OK
Binding Service [mysql1]: OK
Staging Application: OK
Starting Application: OK
Getting tunnel connection info: OK
Service connection info:
username : uMe6Apgw00AhS
password : pKcD76PcZR7GZ
name : d7cb8afb52f084f3d9bdc269e7d99ab50
Starting tunnel to mysql1 on port 10000.
1: none
2: mysql
Which client would you like to start?: 2

82. …Using Caldecott
Launching 'mysql --protocol=TCP --host=localhost --port=10000 --
user=uMe6Apgw00AhS --password=pKcD76PcZR7GZ
d7cb8afb52f084f3d9bdc269e7d99ab50'
Welcome to the MySQL monitor. Commands end with ; or g.
Your MySQL connection id is 10944342
Server version: 5.1.54-rel12.5 Percona Server with XtraDB (GPL),
Release 12.5, Revision 188
Copyright (c) 2000, 2011, Oracle and/or its affiliates. All rights
reserved.
Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.
Type 'help;' or 'h' for help. Type 'c' to clear the current input
statement.
mysql>

83. Running JUnit test with Caldecott
Configure your test code to use port + connection info
83

84. Summary
84

85. Monolithic applications are
simple to develop and deploy
BUT have significant
drawbacks
85

86. Apply the scale cube
§Modular, polyglot, and
scalable applications
§Services developed,
Y axis -
functional
decomposition
ng
deployed and scaled
independently
i
on
iti
rt
pa
ta
da
is-
ax
Z
X axis - horizontal duplication
86

87. Cloud Foundry helps...
.js
Ap
p lica
Private
'o
Clouds
n
S
Data Services
erv
ice
In
ter
fac
e
fac
e
Public
ter
r
In
Clouds
Msg Services
ide
ov
Pr
ud
Cl o
Micro
Clouds
Other Services

88. @crichardson crichardson@vmware.com
http://slideshare.net/chris.e.richardson/
Questions?
www.cloudfoundry.com