Verizon Labs is home to one of the largest Scala-based functional programming teams in North America, and in this talk Timothy Perrett provides an insight into the work of his infrastructure engineering team, in driving adoption of pure-functional programming in a fortune 15 company: from language education and proliferation within the team, to the specific positives and negatives of purely functional programming, when applied at massive scale

1. ENTERPRISE ALGEBRAS
Timothy Perrett
Scala World 2016

2. HELLO.
*waves*

3. “Tuesday”
WE’RE
HIRING!

4. WE ❤ FP.
It’s not just for academics!

5. OSS.
github.com/verizon

6. ~1000 REPOS.
~ 85% are Scala projects (yes, built with SBT).

7. 11 SBT PLUGINS.
Doing the nasty so users don't have too.

8. SUCH SIZE. SO WOW.
Requires scaling and automation of all the things.

9. METHOD.
Constraints liberate. Liberties constrain.

10. METHOD.
One uses frameworks. Frameworks use you.

11. Invest in the future, today.
EDUCATION.

12. EDUCATION.
➤ Evangelize functional programming
benefits to team.
➤ Users initially self-directed learning from
texts and online resources.
➤ Dissemination of techniques from staff
(typically the team lead or coworkers)
➤ Practice. Practice. Practice. Pull requests
provide a path to learn every day.
➤ Internal courses and materials for
learning more advanced FP techniques.
➤ Further practice and experience. Rinse
and repeat :-)

13. REAL-WORLD FP
it actually does work in production.

14. HLIST“seems like a good idea”

15. HLIST.
➤ It’s THE gateway drug for advanced Scala.
➤ Easy to understand.
➤ Untold numbers of business uses.
➤ One of the most used third-party
construct across our internal eco-system.

16. FREE &
COYONADA.

17. FREE.Certainly not an exhaustive guide.

18. sealed trait Free[F[_],A]

19. case class Return[F[_],A](a: A)
extends Free[F,A]
case class Suspend[F[_],A](s: F[Free[F, A]])
extends Free[F,A]

20. case class Return[F[_],A](a: A)
extends Free[F,A]
case class Suspend[F[_],A](s: F[Free[F, A]])
extends Free[F,A]

21. case class Return[F[_],A](a: A)
extends Free[F,A]
case class Suspend[F[_],A](s: F[Free[F, A]])
extends Free[F,A]

22. THE YONEDA LEMMA.

23. def apply[B](f: A => B): F[B]

24. abstract class Yoneda[F[_], A]{
def apply[B](f: A => B): F[B]
}

25. def toYoneda[F[_] : Functor, A](fa: F[A]) =
new Yoneda[F,A] {
def apply[B](f: A => B) =
Functor[F].map(fa)(f)
}
def fromYoneda[F[_], A](yo: Yoneda[F, A]) =
yo.apply(identity)

26. SO WHAT?

27. sealed abstract class Coyoneda[F[_], A]{
type I
val fi: F[I]
val k: I => A
}

28. sealed abstract class Coyoneda[F[_], A]{
type I
val fi: F[I]
val k: I => A
}

29. YAY!

30. case class Suspend[F[_],A](s: F[Free[F, A]])
extends Free[F,A]

31. type FreeC[S[_], A] =
Free[({type f[x] = Coyoneda[S, x]})#f, A]

32. NOMAD MONAD.Fun to say. Better to use.

33. sealed abstract class SchedulerOp[A]
extends Product with Serializable

34. final case class Delete(…)
extends SchedulerOp[Unit]
final case class Launch(…)
extends SchedulerOp[Unit]

35. type SchedulerF[A] =
Free.FreeC[SchedulerOp, A]

36. def delete(…): SchedulerF[Unit] =
Free.liftFC(Delete(x,y))

37. SchedulerOp ~> Task
def apply[A](s: SchedulerOp[A]): Task[A]

38. SchedulerOp ~> Task
def apply[A](s: SchedulerOp[A]): Task[A]
TESTABLE

39. CO-PRODUCTS.

40. for {
a <- free1.bar(123)
b <- free2.foo(a)
} yield b

41. for {
a <- free1.bar(123)
b <- free2.foo(a)
} yield b

42. final case class Coproduct[F[_], G[_], A](
run: F[A] / G[A])

43. final case class Coproduct[F[_], G[_], A](
run: F[A] / G[A])
algebra 1
algebra 2

44. type CombOp[A] = Coproduct[QuxOp, FooOp, A]

45. type CombF[A] = Free.FreeC[CombOp, A]

46. def example: CombF[String] =
for {
a <- useQux(1)
b <- useFoo(a)
} yield b

47. def useQux(x: Int): CombF[Int] =
QuxOp.gogogo(x).inject
def useFoo(x: Int): CombF[String] =
FooOp.gogogo(x).inject

48. final implicit class InjectCombF[F[_],A](
val fa: Free.FreeC[F,A])(
implicit I: Inject[F, Comb.CombOp]){
def inject[G[_]] = injectFC.apply(fa)
}

49. def injectFC[F[_], G[_]](implicit I: Inject[F, G]): FreeC[F, ?] ~> FreeC[G, ?] =
new (FreeC[F, ?] ~> FreeC[G, ?]) {
def apply[A](fa: FreeC[F, A]): FreeC[G, A] =
fa.mapSuspension[Coyoneda[G, ?]](
new (Coyoneda[F, ?] ~> Coyoneda[G, ?]) {
def apply[B](fa: Coyoneda[F, B]): Coyoneda[G, B] = fa.trans(I)
}
)
}
sorry, there aren’t really ways to simplify this.

50. def injectFC[F[_], G[_]](implicit I: Inject[F, G]): FreeC[F, ?] ~> FreeC[G, ?] =
new (FreeC[F, ?] ~> FreeC[G, ?]) {
def apply[A](fa: FreeC[F, A]): FreeC[G, A] =
fa.mapSuspension[Coyoneda[G, ?]](
new (Coyoneda[F, ?] ~> Coyoneda[G, ?]) {
def apply[B](fa: Coyoneda[F, B]): Coyoneda[G, B] = fa.trans(I)
}
)
}
transform
free algebras
sorry, there aren’t really ways to simplify this.

51. def injectFC[F[_], G[_]](implicit I: Inject[F, G]): FreeC[F, ?] ~> FreeC[G, ?] =
new (FreeC[F, ?] ~> FreeC[G, ?]) {
def apply[A](fa: FreeC[F, A]): FreeC[G, A] =
fa.mapSuspension[Coyoneda[G, ?]](
new (Coyoneda[F, ?] ~> Coyoneda[G, ?]) {
def apply[B](fa: Coyoneda[F, B]): Coyoneda[G, B] = fa.trans(I)
}
)
}
sorry, there aren’t really ways to simplify this.
transform
coyonedas

52. def injectFC[F[_], G[_]](implicit I: Inject[F, G]): FreeC[F, ?] ~> FreeC[G, ?] =
new (FreeC[F, ?] ~> FreeC[G, ?]) {
def apply[A](fa: FreeC[F, A]): FreeC[G, A] =
fa.mapSuspension[Coyoneda[G, ?]](
new (Coyoneda[F, ?] ~> Coyoneda[G, ?]) {
def apply[B](fa: Coyoneda[F, B]): Coyoneda[G, B] = fa.trans(I)
}
)
}
sorry, there aren’t really ways to simplify this.
F[A] => G[A]

53. APPLICATION DESIGN.

54. short-lived
request
response

55. free algebra

56. coproduct
algebra

57. alternitivly,
use kleisli

58. edge of
the world

59. long-lived
stream

60. edge of
the world

61. case GET -> Root / "v1" / "datacenters" / dcname & IsAuthenticated(session) =>
jsonF(Nelson.fetchDatacenterByName(dcname)){ option =>
option match {
case Some(dc) => Ok(dc.asJson)
case None => NotFound(s"datacenter '$dcname' does not exist")
}
}
sorry, there aren’t really ways to simplify this.
protocol specifics

62. case GET -> Root / "v1" / "datacenters" / dcname & IsAuthenticated(session) =>
jsonF(Nelson.fetchDatacenterByName(dcname)){ option =>
option match {
case Some(dc) => Ok(dc.asJson)
case None => NotFound(s"datacenter '$dcname' does not exist")
}
}
sorry, there aren’t really ways to simplify this.
domain function

63. BlazeBuilder
.bindHttp(8080, 127.0.0.1)
.mountService(service)
.start // Task[Server]
.run // Unit
sorry, there aren’t really ways to simplify this.

64. APPLICATION DESIGN.
➤ All your I/O boundaries - from a systems
perspective - should be a Free algebra.
➤ Database access.
➤ Network access.
➤ Any other random things you can’t
sensibly reason about.
➤ Be extremely selective about the edge of
the world.
➤ Understand the semantics of running
your Task.

65. TECHNICAL DEBT.

66. TECHNICAL DEBT
➤ Accept it will happen. Minimization is
the best you can hope for.
➤ Proactively push debt repositories to the
leaves of your system graph.
➤ Libraries are the key battleground in debt
avoidance. Choose wisely.
➤ Be consistent with different library
APIs; this drives adoption and avoids
accidental debt when users do the
unexpected.

67. CHALLENGES.

68. CHALENGES.
➤ Scala is not a pure FP language.
➤ Ironically a positive and enabling thing
for both beginners, and the business.
➤ Community libraries can be a mixed bag.
➤ Not everyone prioritizes maintaining
binary compatible releases.
➤ Many internal forks to stabilize deps.
➤ GC is a real and present danger.
➤ Ensuring users don’t revert to Java
without the semicolons is a definite and
on-going challenge.
* Both fleshy and technical challenges
* Generally speaking, hiring for Scala
has not been a big issue.

69. WINS.

70. WINS.
➤ Doing FP has allowed us to recruit and
retain some brilliant minds.
➤ The application of Free, Cofree, Fix and
other advanced type-level paradigms
make your software easier to refactor, and
cheaper over time.
➤

71. CONCLUSIONS.

72. CONCLUSIONS.
➤ Free & Coproduct allow us to build modular, testable, systems from discrete
programs.
➤ Your program has programs.
➤ Running Scala at scale is not zero-cost. Far from it.
➤ You need expertise in the JVM to understand corner cases (happens for Java too).
➤ Purely functional stream programming currently struggles at high-load due
to GC pauses.
➤ Team education is a hard problem.
➤ Don’t hire for quantity. Hire for quality and aptitude.

73. SCALA CENTER.
https://github.com/scalacenter/advisoryboard/pulls
we need
the community!

74. EOF
timperrett
github.com/verizon