"Any organization that designs a system (defined broadly) - will produce a design whose structure is a copy of the organization's communication structure." - Conway's Law (1967). Centric's Cloud Services Lead Wiliam Klos shares his pointers on simplifying the management of your microservices. More detail on Bill's presentation at the CloudDevelop Conference here: http://centricconsulting.com/events/william-klos-selected-to-speak-at-clouddevelop-2016/
2. INTRODUCTION
ABOUT ME
▸ Senior Architect w/Centric Consulting
▸ National Service Offering Lead for Cloud Computing
▸ @williamklos
▸ bill.klos@centricconsulting.com
3. INTRODUCTION
AGENDA
▸ A Bit of History
▸ Definition of Microservices
▸ Application Simplicity
▸ Application Complexity
▸ Infrastructure Complexity
▸ Summary
▸ Resources
5. “ANY ORGANIZATION THAT DESIGNS A SYSTEM
(DEFINED BROADLY) - WILL PRODUCE A
DESIGN WHOSE STRUCTURE IS A COPY OF
THE ORGANIZATION’S COMMUNICATION
STRUCTURE.”
Conway’s Law (1967)
HISTORY
6. HISTORY
MY STORY
▸ First Set of Applications in 1988
▸ Each PC was a “service”
▸ Running DOS, dBASEII, and support
scripts - modem
▸ 5.25” floppy-based Sneaker-net
▸ Next Major Set of Applications in 1994
▸ 1 AS/400, 1 database, tons of applications
▸ REST Web Services since 200x
7. HISTORY
CLASSIC MONOLITH
▸ One cabinet, maybe more if you’re rich.
▸ One database, shared by all
applications.
▸ One set of shared libraries.
▸ No need for networking between
application modules because they are
all right there in the same address
space.
▸ May have some logical partitions for
other environments.
▸ Probably a monoglot situation.
finance
inventory
report
capacity
ORDER-WOPR
19x9
shop
pick
ship
return
cart
bill
query
DB
8. HISTORY
MODERN MONOLITH
▸ Web services based. Maybe.
▸ Tightly coupled data, tightly coupled
network resources, tightly coupled
business logic, tightly coupled code.
▸ The monolith is in the shape of a
virtual machine.
▸ To put it in the cloud would require a
“lift-and-shift”.
▸ But, we are starting to see external
resources be a big part of the overall
flow.
ship
pick
shop
finance
return
inventory
query
report
notify
bill
review
cart
capacity
ORDER-WOPR
2008
DB
DB
DB
9. HISTORY
WHAT’S CHANGED?
▸ Technology Constraints Removed
▸ Single machines running single core processors,
▸ Slow Networks and Lack of Pervasive Connectivity
▸ Expensive Storage & Expensive RAM
▸ Organizations structured as monoliths
▸ New Technologies
▸ Virtualized Machines, most notably Containers
▸ Wide Variety of Messaging and other Data Movement Options
▸ and of course, the Cloud.
10. HISTORY
THE PROGRESSION TOWARDS THE SMALL
MONOLITH
CLIENT
SERVER
WEB &
INTERNET
CLOUD & WEB
SERVICES
MICRO
SERVICES
2020 - 2030
13. DEFINITION
OPEN FIEFDOM
▸ Self-contained, focused purpose
society - with a healthy importing and
exporting capability.
▸ The ability to open franchises rivals
Starbucks and the ability to close them
rivals Sears.
▸ Automation is such that the feudal lord
can run everything from his throne
without the need of pesky serfs.
▸ If the world goes sideways, they can
pull up the drawbridge and watch
everything outside burn.
18. DEFINITION
RECALL THE LEGACY ORDER MANAGEMENT SYSTEMS
CLASSIC MONOLITH MODERN MONOLITH
finance
inventory
report
capacity
ORDER-WOPR
19x9
shop
pick
ship
return
cart
bill
query
DB
ship
pick
shop
finance
return
inventory
query
report
notify
bill
review
cart
capacity
ORDER-WOPR
2008
DB
DB
DB
22. CHARACTERISTICS
SINGLE RESPONSIBILITY
▸ Goal is for it to do one thing, and to
do it well. It is responsible for its own
behavior and its input contracts.
▸ Has inputs and outputs, and if a data
source or state is necessary, it has its
own.
▸ Not aware of the outside environment
aside from some shared resources in
an API Layer or whatever its input
data tells it.
▸ All other tenets of microservices spin-
off from this core principle.
pick&
ship
shippick
24. CHARACTERISTICS
ISOLATION
▸ Goal is for it to be a completely self-
contained entity with minimal
outside dependencies.
▸ This helps minimize impact on
other parts of the system.
▸ Has inputs and outputs, and if a data
source is necessary, it has its own.
▸ Keeps track of its own state and will
only reveal it of asked.
▸ Independently deployable unit.
rest
ship
pick
query
billcart
return
pubsub
pubsub
pubsub
rest
rest
rest
report
pubsubpubsub
pubsub
finance
pubsub
shop
rest
review
notify
pubsub pubsub
pubsub
pubsub
inventory
rest
rest
rest
rest
25. CHARACTERISTICS
ISOLATED ANATOMY - SIMPLIFIED
▸ At its base level, it only
knows what its specific
business logic tells it -
plus…
▸ …whatever the inputs
bring to it.
micro
service
26. CHARACTERISTICS
ISOLATED ANATOMY - COMPLEX (ILL-ADVISED)
▸ The micro service can include
all of the extras to be a good
application citizen.
▸ Handles tasks that tell it how
to operate, scale,
communicate, control access,
& report its behavior.
▸ Should probably be handled
via an API or mediation layer
(discussed later).
bill
micro
service
security
loggingback
pressure
scaling
config versioning
monitoringdiscovery
cachingtransform
27. CHARACTERISTICS
ISOLATION EXAMPLE
▸ Upgrading the Billing service forces these
things to occur:
▸ New Orders and Returns are collected
in holding queues
▸ Existing Orders are processed until they
reach 0 and the Billing service is
stopped.
▸ When new Billing service launches the
Order and Returns queues are depleted
then processing returns to normal.
▸ No downstream services are affected.
▸ “Safety of Change”
rest
ship
pick
query
billcart
return
pubsub
pubsub
pubsub
rest
rest
rest
report
pubsubpubsub
pubsub
finance
pubsub
shop
rest
review
notify
pubsub pubsub
pubsub
pubsub
inventory
rest
rest
rest
rest
28. CHARACTERISTICS
COMPARTMENTALIZATION
▸ A structure to help limit the
spread of “badness” across a
system.
▸ Because microservices
should be designed to
handle their own issues and
have insurance against
malcontents, cascading
behavior should be
minimized.
29. CHARACTERISTICS
AUTONOMY
▸ Isolation is a pre-requisite for
autonomy. It’s hard to automate
something that has tendrils of
dependency.
▸ When a service can guarantee its own
behavior and is not dependent on
outside interactions - then it has all it
needs to solve its own problems.
▸ Autonomous services allow for more
flexibility in dealing with service
orchestration, choreography,
management, scaling, etc.
shippick reportnotify ship
ship ship
ship ship
ship ship
30. CHARACTERISTICS
CAN BE STATELESS OR STATEFUL
▸ Stateless allows for auto-scaling without
worry since no copy cares about any other.
▸ Strive for immutability
▸ Separate stateful services into their own
containers.
▸ A microservice is responsible for its own
state and the persistence of that state.
There should be no external changes to it.
▸ Don’t share state keeping mechanisms
with other microservices.
?
31. CHARACTERISTICS
ASYNCHRONOUS MESSAGE PASSING
▸ Synchronous operations inside
the web service, async outside
of it.
▸ Aids in de-coupling, isolation,
compartmentalization,
resilience, etc.
▸ Helps change mindset from a
rigid procedural perspective to
a more event-driven one.
▸ Fire and forget. Sometimes.
sync
async
33. CHARACTERISTICS
INTERNAL INTEGRATION - THE INTERSTITIAL
▸ Essentially the spaces in
between the microservices.
▸ Part of the negotiated contract
between left-side and right-
side.
▸ Can be async or sync
mechanisms.
▸ Don’t all have to be the same.
What makes sense for each
interaction?
pick ship
pubsub
amqp
rest
interstitial
34. CHARACTERISTICS
EXTERNAL INTEGRATION
▸ Predicting how external
systems are going to
misbehave is futile. Design
for self-defense.
▸ Construct asynchronous calls
where possible.
▸ Negotiate expected traffic
rates and volume, be able to
handle unforeseen events.
▸ Not really that different over
internal service integration.
my
ms
your
ms
External
API
FORMAT
CSV?
JSON?
XML?
FORM?
METHOD
REST?
SOAP?
PUBSUB?
QUEUE?
RESPONSES
200?
201?
404?
401?
50x?
REACTIONS
PROCESS
QUEUE
RE-SUBMIT
THROTTLE
SLEEP
35. CHARACTERISTICS
MOBILITY
▸ Where possible, a micro service should
be able to operate anywhere in the
enterprise’s operation.
▸ While a microservice should carry what
it needs to execute, certain
environmental dependencies are more
pragmatically shared.
▸ Should be able to run on AWS, Azure,
Joe’s Hosting Shack, or in the corporate
data center.
▸ The service should also always be
addressable across the locational space.
shop
query
query
36. CHARACTERISTICS
SHARED CAPABILITY > SHARED DATA OR SHARED INSTANCES
▸ While there is less of a DRY (Don’t Repeat Yourself) push
in microservices, it’s not a fully WET (Write Everything
Twice) architecture either.
▸ Sharing runtime capabilities like databases and security is
perfectly acceptable given those capabilities are
sufficiently segmented so as to minimize impact of other
applications.
▸ i.e. Don’t negatively impact “change safety”.
37.
38. CHARACTERISTICS
TIME AWARENESS OF DATA
▸ Data in-flight is either in “the
past” (has happened) or in “the
hopeful future” (will happen)
▸ There is no single notion of “the
present”. Only the “local” now.
▸ Inside our microservice we can
control everything within our
boundary.
▸ Outside the boundary, not
our problem.
the local
now
the past
the future
41. API MANAGEMENT
OVERVIEW
▸ Similar to an
orchestration layer like
SOA designs but still
need to abstract direct
visibility to endpoints.
▸ Allows you to change
granularity level without
impacting users.
▸ Includes common
functionality around
routing, security, load-
balancing and
registration services.
ship
pick
query
billcart returnreport finance
shop review notifyinventory
API LAYER
Web App
Mobile
App
Partner
Apps
Client
Apps
Employee
Apps
SECURITYROUTINGDISCOVERYLOGGINGLOAD BALANCEREGISTRATIONAUTHENTICATETRANSFORM
42. API MANAGEMENT
SERVICE DISCOVERY
▸ Upon startup of a micro-
service, it registers itself
with a controlling entity.
▸ Access to service can be
direct if all operating
copies are known or by
redirection.
▸ Assures all nodes get
traffic when autoscaling.
▸ Essentially a load balancer.
billing
billing 172.16.6.22:3000 2016/07/31 13:31:04
billing 172.16.6.22:3001 2016/07/31 13:45:04
API Layer
50ms
40ms
MODULE IP ADDR:PORT TIME STARTED PING
registerService()
POST https://apilayer/order/:ord_id/bill
POST https://172.16.6.22:3001/order/:ord_id/bill
GET https://apilayer/billings
{“hosts”:[“172.16.6.22:3000”,”172.16.6.22:3001”],”count”:2}
Get List of Active Billing Services
Auto Redirect Service
43. API MANAGEMENT
SECURITY
▸ Primary security would typically
be handled by the API Layer.
▸ Individual services have
enough security to know when
they are being called directly
and redirect to API Layer or
reject the request with a 401.
▸ Individual services will also
need to provide authorization
for services that call it (certs,
whitelisted Services, etc.).
shipbillcart
return
shop
47. APPLICATION COMPLEXITY
DETERMINING BOUNDARIES
▸ Define a model based upon a
specific business perspective.
▸ But it doesn’t need, and maybe it
shouldn’t, be based on your
standard organizational makeup.
▸ Distributed > Centralized
▸ Different perspectives of the same
topic can yield different models
▸ e.g. Uber driver vs. Uber
passenger
▸ Domain-Driven Design may help
here.
Back-Office Order ContextCustomer Order Context
rest
ship
pick
query
billcart
return
pubsub
pubsub
pubsub
rest
rest
rest
report
pubsubpubsub
pubsub
finance
pubsub
shop
rest review
notify
pubsub pubsub
pubsub
pubsub
inventory
rest
rest
rest
rest
rest
ORDER PROCESSING DOMAIN
48. APPLICATION COMPLEXITY
DETERMINE GRANULARITY
▸ Need to do one thing very well,
but how do you define “one
thing”?
▸ Signs of being too small include
services being “chatty”, too much
dependence on other services, or
poor-performance.
▸ Signs of being too big include
code changes impacting pseudo-
related functionality, compromises
in data design, or too much
“global cruft”.
shop
shop
bill
cart
order
entry
bill
remove
add2cart
view
cart
save
cart
empty
cart
51. APPLICATION COMPLEXITY
DATA COUPLING
▸ Data consistency requires
coordination.
▸ Coordination can inhibit
scalability, throughput, and
availability.
▸ There is no global single
source of the truth. Truth is
local, within the context of
the service.
bill
cart
shop
review
inventory
DB
billcart
shop review
inventory
LOW
HIGH
56. CHALLENGES
HANDLING FAILURES - BACKPRESSURE
▸ When the next phase
can’t be completed
right now, defer it
yourself or give it
back to the caller.
▸ A way of governing
discrepancy between
slow and fast systems.
▸ Process it when the
issue is cleared.
POST http://apilayer/order/bill billingcart
rest
200
publishBillOrder() > 0
backPressureBillQ billPubSubChannel
NO
YES
50x
Relieve Back Pressure When Possible
Apply Back Pressure When Necessary
OK
billQDepth() > max
billQFillRate() > max
57. CHALLENGES
HANDLING FAILURES - AVAILABILITY
▸ When a service is down, it
can be handled in a similar
way to back-pressure.
▸ Can also look for
additional services of the
same type (API Layer
should handle).
▸ If severe enough, then
stoppages can be
cascaded up in a
controlled fashion to make
sure overall system
integrity is maintained.
POST http://apilayer/order/bill
cart
rest
200
SERVICE IS DOWN
overflowBillQ billPubSubChannel
NO
YES
50x
Pull from Queue When Service Resumes
OK
billing
58. CHALLENGES
HANDLING FAILURES - CIRCUIT BREAKER
▸ As long as the circuit stays
closed, all traffic passes as
normal.
▸ If the service is
experiencing a backlog or
under stress, it can
partially “trip” the circuit
effectively throttling input.
▸ If the circuit is open, then
the service is down and all
calling functions need to
step back behavior until
service resumes.
cart
NO
billing
ADD RETRY DELAY
backPressureBillQ
60. APPLICATION COMPLEXITY
DISTRIBUTED TRANSACTIONS
▸ Traditionally upholds the illusion
that you are the only one using
the data you need.
▸ Distributed transactions can
cause scalability issues and
introduce “brittleness”.
▸ A recommended alternative to
distributed transaction is the
Saga pattern.
▸ Essentially having an “undo”
action along the way
executed in reverse in case of
failure.
cartshop inventorybill
addToCart() billOrder()
removeFromCart()
allocateInv()
compensate()
pick
compensate()
deductInv()
commit()
commit()
commit() commit()
61. APPLICATION COMPLEXITY
MORE DISTRIBUTED TRANSACTIONS
▸ Additional ways of handling would include:
▸ Previously discussed methods of applying “back-
pressure”, deferred processing, or retries.
▸ Back off of any dogmatic approach to distributed effort
in favor of a centralized approach for that particular
piece.
▸ e.g. Inventory in our Order Management model.
63. INDIVIDUAL MICROSERVICES ARE RELATIVELY EASY
TO DESIGN AND DEPLOY.
THE COMPLEXITY COMES WHEN YOU HAVE MORE
THAN A HANDFUL THAT NEED TO BE MANAGED.
64. INFRASTRUCTURE COMPLEXITY
YOU NEED HOW MANY SERVERS?
▸ Our example has 12
microservices consisting
of 36 actual applications
services.
▸ If we need a robust
production environment,
we’ll want 3x servers (36).
▸ If we want additional
environments, then we
will need more even still.
▸ Or…we could use
containers.
rest
ship
pick
query
billcart
return
pubsub
pubsub
pubsub
rest
rest
rest
report
pubsubpubsub
pubsub
finance
pubsub
shop
rest
review
notify
pubsub pubsub
pubsub
pubsub
inventory
rest
rest
rest
rest
65. INFRASTRUCTURE COMPLEXITY
HOW ABOUT CONTAINERS?
▸ The base unit of measure for
a micro service is a container.
▸ Containers are like micro-
VMs.
▸ Lightweight virtual machine
that has all of the
dependencies necessary to
execute the contained
application.
▸ And no more.
MICROSERVICE
pick
shop shop
cart
VM
CONTAINER
CAPACITY
66. INFRASTRUCTURE COMPLEXITY
AUTOMATION IS YOUR FRIEND
pick
shop shop
cart cart
review
ship
query
notify
shop ship shop
bill
bill
return
report▸ Service discovery allows
containers to be brought
into the world and
removed from it in an
automated fashion.
▸ Products like Kubernetes
and Docker Swarm aid in
this ability and make the
management of you
containers fleet much
easier to deal with.
67. INFRASTRUCTURE COMPLEXITY
DATABASE PROLIFERATION
▸ Self-contained means a
private datasource.
▸ Allows for the “right-tool
for the right job”.
▸ Adds management and
monitoring complexity.
▸ Potentially adds
licensing and support
costs.
68. INFRASTRUCTURE COMPLEXITY
TECHNOLOGY PROLIFERATION
▸ It’s not just databases that are
proliferating. Languages,
protocols, connectors - all are up
for consideration.
▸ Allows for the “right-tool for the
right job”.
▸ Builds silos of knowledge, but
they’re small.
▸ Of course, you can “monoglot”
the crap out of your architecture if
you want.
71. CHALLENGES
THE DEV/OPS PLAY
▸ Dev/Ops has a huge
impact on a successful
micro services
implementation.
▸ Monitoring, scaling,
load-balancing,
routing, instantiation,
and overall
robustness require
better levels of
automation.
bill
micro
service
security
logging
back
pressure
scaling
config versioning
monitoringdiscovery
cachingtransform
73. SUMMARY
OPERATING PRINCIPLE EXAMPLES
▸ UNIX
▸ Do One Thing Well
▸ Build Afresh
▸ Expect Output to Become Input
▸ Reduce Human Input
▸ Try/Fail Early
▸ Don’t Hesitate to Throw it Away
▸ Become Toolmakers
▸ Netflix
▸ Antifragility
▸ Immutability
▸ Separation of
Concerns
74. SUMMARY
THINGS TO REMEMBER
▸ Very small scope leads to much easier “micro-applications” that
provide small pieces of value. The trade-off is overall environmental
management.
▸ A solid DevOps practice is vital in managing the seemingly chaotic
environment of a micro services implementation.
▸ You decide the appropriate granularity.
▸ Yes, it’s like SOA, but different. Lighter weight and more nimble.
▸ Once you can automate the management of your microservice fleet,
having 5 or 50 or 500 microservices, the amount of pain in
administering them should be logarithmic rather than linear.
75. SUMMARY
MORE THINGS TO REMEMBER
▸ The ironic thing about systems that deal in 1s and 0s is that the rules that
govern how to implement them is never 1s and 0s.
▸ There is no one true “right way” to do any of this. Everything is a tradeoff
between the needs of the business and the realities of your situation.
▸ That being said, strive towards:
▸ services doing “one thing only”
▸ minimizing coupling (data and functionality)
▸ automation
▸ The function, behavior, and citizenship of your microservice is on you.
76. SUMMARY
THINGS TO TRY
▸ “Antifragility” testing (Netflix does this) will introduce failures
into the system to determine amount of drama created. Try
removing nodes or introducing bad keys into your process to
see how the system reacts. Seriously!
▸ Treat your servers and processes like “cattle” rather than “pets”.
Everything is disposable outside of data.
▸ Assign services to teams. Instead of having people assigned to
support entire applications, assign smaller teams to individual
services where they own the entire stack including design and
support.
77. “ANY ORGANIZATION THAT DESIGNS A SYSTEM
(DEFINED BROADLY) - WILL PRODUCE A
DESIGN WHOSE STRUCTURE IS A COPY OF
THE ORGANIZATION’S COMMUNICATION
STRUCTURE.”
Conway’s Law (1967)
HISTORY
79. RESOURCES
MORE DETAILED INFORMATION
▸ http://www.reactivemanifesto.org/
▸ http://thenewstack.io/ten-commandments-microservices/
▸ http://goo.gl/4qLLbG (Data Outside vs. Data Inside)
▸ http://goo.gl/zMnHtU (Cattle vs. Pets)
▸ https://msdn.microsoft.com/en-us/library/jj591569.aspx
▸ http://martinfowler.com/bliki/CircuitBreaker.html
▸ Microservice Architecture, Nadareishvili, Mitra, et al.
▸ Microservices vs. Service-Oriented Architecture, Mark Richards
▸ Reactive Microservices Architecture, Jonas Bonér