Building Micro-Services with Scala
•
74 j'aime•15,971 vues
"Microservices" is one of the hottest buzzwords and, as usual, everyone wants them, but few know how to build them. In this talk we will offer our interpretation of microservice architecture, and show how we are implementing these ideas: using Scala, Akka, sbt and Docker, we modularized Akka applications, Spark jobs and Play servers. In the talk we will discuss design trade-offs and challenges that we faced in the process, and how we have overcome them. The focus is not on particular features of Scala language or a library, but on building modern applications using the Typesafe stack and other open-source tools.
Recommandé
Contenu connexe
Tendances
Tendances (20)
En vedette
En vedette (20)
Similaire à Building Micro-Services with Scala
Similaire à Building Micro-Services with Scala (20)
Plus de Yardena Meymann
Dernier
Dernier (20)
Building Micro-Services with Scala
- 1. Building Micro-Services with Scala Lior Shapsa, Yardena Meymann September 22, 2014 © 2014 VMware Inc. All rights reserved.
- 4. Problems with Monoliths • Size • Integration (Conway’s law) • Modules Change at Different Rates • Dependencies Collisions • Scalability requirements of modules FEAR OF CHANGE FEAR OF INNOVATION
- 6. Thought Leaders Netflix, Amazon, LinkedIn, Google, ThoughtWorks, Gilt, … Fred George Martin Fowler Kit Colbert Ben Fathi Stefan Tilkov VMware
- 7. What are micro-services? A set of narrowly focused,
- 8. What are micro-services? A set of narrowly focused, independently deployable services,
- 9. What are micro-services? A set of narrowly focused, independently deployable services, talking via uniform interfaces
- 10. What are micro-services? A set of narrowly focused, independently deployable services, talking via uniform interfaces
- 11. Micro-services • Each running in its own process • Communicating with lightweight mechanisms, often an HTTP resource API • Built around business capabilities • Independently deployable – fully automated deployment • May be in a different programming language and use different data storage technologies.
- 12. Micro-services DEPLOYING A CHANGE IS LOW RISK
- 13. #NonBlocking
- 14. Threaded vs. Evented Servers • Monolithic – Most modules communicate in-process • Micro-services – communicate remotely, often over the network – need to support cheap, lightweight remote communication • Threaded servers use thread per connection • Evented servers use non-blocking IO and callbacks – Netty – Node.js – Play – Spray
- 15. services vs. seconds http://www.eecs.berkeley.edu/~rcs/research/interactive_latency.html https://www.youtube.com/watch?v=1-vcErOPofQ
- 16. Threaded servers Service A Service B Service N
- 17. Threaded servers Service A Service B Service N Thread Thread Thread
- 18. Threaded servers Service A Service B Service N Thread Thread Thread Thread Thread Thread
- 19. Threaded servers Service A Service B Service N Thread Thread Thread Thread Thread Thread Thread Thread Thread
- 20. Thread Service A Service B Service N Thread Thread Threaded servers Thread Thread Thread Thread Thread Thread Thread Thread Thread Thread Thread Thread Thread Thread Thread
- 21. Thread Service A Service B Service N Thread Thread Threaded servers Thread Thread Thread Thread Thread Thread Thread Thread Thread Thread Thread Thread Thread Thread Thread
- 22. #NonBlocking – Evented servers Service A Service B Service N Thread Thread Thread callback callback
- 23. #NonBlocking – Play/Spray • Based on Akka using Netty – Non-blocking programming much easier • Easy deployment – Embedded Server • Rich JSON and HTTP support • Plugins – Secure Social – Caching – …. • SCALA!
- 24. #NonBlocking – Play example …
- 25. #AKKA
- 27. Spray Node Dispatch Crawling library Netty Price Engine Head Actor … Pages Level1 Level1 Level2 … Level2 File writer Head Actor … k/v Node Akka Cluster curl http://AkkaCluster/site_dot_com?op=start
- 28. #AKKA • Based on the Actor Model – http://www.reversim.com/2014/04/summit-2014-scale-up-your-thinking.html • Resilience • Location Transparency – Cluster sharding – Cluster clients • Load Balancing • Message Queue Integration
- 29. #Containers
- 30. What is ? • Dockers are like lightweight VM for a single process – Self contained – No hypervisor – Shared kernel, but Isolated and content agnostic – No lib conflict – No OS boot time – Distributing a change is easy, using δ • A Clean, Safe, Isolated and Portable Micro Service
- 31. Docker File • Docker Image is Built From a DockerFile • Consists of Descriptive Set of Instructions – Start from a base image – Run and command – Add a file or directory – Create an environment variable – What process to run on launch
- 32. SBT • Assemble “Dockerized” micro services • We started with sbt-native-packager – Poor Docker functionality • We ended up with sbt-docker plugin – https://github.com/marcuslonnberg/sbt-docker – Still Using sbt-native-packager to package – Ends up with Docker file
- 35. #Containers – Using SBT
- 36. #Containers – Using SBT • Using Sequences to optimize – Reduce build time – Smaller Containers
- 37. #Containers – Orchestration • CoreOS – Linux for massive server deployments – Cluster management with Fleet – Service discovery with etcd • Evaluating – Kubernetes – Flynn – Consul – Ambassadord
- 38. #AKKA Summary #NonBloc king #Containers
- 42. Lior Shapsa lshapsa@gmail.com Twitter: @liorshapsa Yardena Meymann ymeymann@gmail.com Twitter: @ymeymann
Notes de l'éditeur
- Messaging – Kafka Resilience - Handles Failures with Supervision
- We write unit files that tells the CoreOS fleet scheduler how to deploy our Docker containers. We will use the etcd key-value store to communicate between our application containers.
- Logs Monitoring
- Logs Monitoring
- Logs Monitoring