Functional programming finds its roots in mathematics - the pursuit of purity and completeness. We functional programmers look to formalize system behaviors in an algebraic and total manner. Despite this, when it comes time to deploy ones beautiful monadic ivory towers to production, most organizations cast caution to the wind and use a myriad of bash scripts and sticky tape to get the job done. In this talk, the speaker will introduce you to Nelson, an open-source project from Verizon that looks to provide rigor to your large distributed system, whilst offering best-in-class security, runtime traffic shifting and a fully immutable approach to application lifecycle. Nelson itself is entirely composed of free algebras and coproducts, and the speaker will show not only how this has enabled development, but also how it provided a frame with which to reason about solutions to fundamental operational problems.
5. – Dr Tara Smith
“Pragmatism is the opposition to principal, on principle.”
6. Rigorous Systems.
• Theory is the foundation.
• Category Theory, Combinatorics, Set Theory, Group Theory…
• Principles of formal verification becoming more widespread.
• Type systems act as proofs about your program.
• Amazon S3 (among others) have TLA+ models.
• Never fear practical application of rigorous systems.
• Invest now to save your sanity tomorrow (i.e. 2am PagerDuty alert).
12. Problem
• Provisioning applications is still too slow (bare metal or cloud).
• Runtime traffic control systems are medieval at best.
• Coupling CI and CD creates monolithic operational systems.
• These systems do everything. This is a distinct problem.
• Current market solutions limited or hard to adopt.
• Most teams have brittle, painful automation nobody wants to use.
• Many teams attempt CD ignorant of the side-effects.
13. Lessons
• Automate every part of the system.
• Testing a distributed system locally is a fable.
• Emergent properties. Scaling issues etc.
• Uniformity is highly desirable and wildly advantageous.
• Beautiful, unique snowflakes are however, inevitable.
• Automated lifecycle management is required.
14. Goals
• Use the minimally powerful components.
• System elements should be awesome at just one thing.
• Reduce overall platform complexity.
• Increase responsibility of engineering teams. Break it, you bought it.
• Decentralize process gatekeepers.
• No build team. No ticket filing for deployments. No configuration
management.
15. Goals
• All application specifications are checked in.
• Build. Deployment. Alerting etc.
• Reduce deployment time to 2 minutes or less.
• Support multi-DC topologies from the get-go.
• Automatic credential management and secure-introduction
• Transparent, strong encryption for application I/O on the wire.
19. Overview
• Provides “best practices” for your organization; focus on shipping.
• Github driven developer workflow (.com or enterprise).
• Choose whatever build / CI system you want.
• State of the art runtime routing via Envoy.
• Secure introduction for safe distribution of credentials from Vault.
• Target any datacenter running a scheduler (Nomad, Mesos, etc).
38. Graphs.
• Every application declares their dependencies a-priori.
• No routing to disconnected vertices.
• No Vault policy attributes for disconnected vertices.
• Edge ingress points are the “root” of a graph.
• Intersecting graphs with shared or discrete roots.
• Provides anchor-point for lifecycle management.
• Entire world view falls out of a union of all discrete graphs.
54. Security.
• Derivation of security policy, as opposed to proclamation of policy.
• Excellent audibility
• All dependencies are known - internal and external.
• Every single application instance has unique credentials.
• Dynamic certificates, dynamic DB access etc.
• Compromised containers have the minimal attack surface.
• Joined with a service mesh, transparent mTLS for everyone is a breeze.
66. Discovery.
• Discovery protocol written to Consul KV for every stack
• We call this Lighthouse protocol
• Application dependencies are declared a-priori.
• You cannot route to that which you do not tell Nelson about.
• Makes for awesome auditing and security.
• Language implementations need only consume the protocol.
67. Routing.
• Non-prescriptive approach to routing tier implementation.
• Provides a control plane protocol describe routing actions.
• Typically implemented with Envoy, but you can choose.
• Minor application changes required.
• Incentivized these with tracing and context propagation.
• Models traffic shifting as a time vs traffic policy curve.
74. Workflow.
type Op0[A] = Coproduct[DockerOp, ConsulOp, A]
…
type WorkflowOp[A] = Coproduct[SchedulerOp, Op4, A]
type WorkflowF[A] = Free.FreeC[WorkflowOp, A]
75. Workflow.
for {
_ <- status(id, Pending, "workflow about to start")
i <- dockerOps(id, unit, dc.docker.registry)
_ <- status(id, Deploying, s"writing alert definitions to ${dc.name}'s consul")
_ <- writePolicyToVault(cfg = dc.policy, sn = sn, ns = ns.name, rs = rs)
///////////////////////////////
// remove for space limitations
///////////////////////////////
l <- launch(i, dc, ns.name, vunit, p, hash)
_ <- debug(s"response from scheduler $l”)
_ <- status(id, getStatus(unit, p), "======> workflow completed <======")
} yield ()
Free monad
76. Challenges
• Non-trivial level of investment and execution.
• Tight integration with Hashistack is both pro or con.
• Community adding k8s + Istio support %
• Containerizing legacy applications can be “interesting”.
• Migration can be a challenge if not collocated with “the new world”.
• Small organizations better served by existing solutions.
77. Summary
• Provide rigor to your application Death Star.
• Fully automated application lifecycle: no manual housekeeping.
• Choose whatever CI setup best fits your team.
• Secure your deployments.
• Transparent mTLS and rotating credentials.
• Automatic Vault policy management.
79. Related Links.
• Nelson documentation site:
verizon.github.io/nelson/
• Nelson CLI:
https://github.com/Verizon/nelson-cli
• Quiver: Scala port of Haskell FGL
https://github.com/Verizon/quiver
• Slipway: integrate any CI system with Nelson
https://github.com/Verizon/slipway