Systems that run across continents and hemispheres, rather than across racks in a single data center, are becoming increasingly necessary; developers deploy their applications across multiple clouds or locations for redundancy and compliance and to push content closer to their users. Tyler McMullen explains why such systems must have fundamentally different designs than what we’re used to. Without fundamentally different architectures from what we’ve done in the past, these systems become burdens to both their operators and their users. Tyler offers an overview of some of the incredible approaches for designing these systems, including lattices, eventual consistency, CRDTs, and consistency analysis, and discusses the fundamental techniques of such systems: back pressure, randomization, partial availability, and caching. Tyler ends by covering the differences in mental models that must be adopted in order to design and implement globally distributed systems as well as some lessons learned from real-world implementations.

1. Principles of
Globally
Distributed
Systems

2. Hi @tbmcmullen
tyler@fastly.com

3. Practicality

4. Carlos Baquero
Peter Alvaro
Peter Bailis
Joseph Hellerstein
Marc Shapiro
João Leitão
Chris Meiklejohn

7. Use off-the-shelf

8. Partial Availability

10. “Nearly all systems are
probabilistic, whether they
realize it or not.”

15. “With high probability”
is Fine

16. Failure Independence

17. v1 of Fastly Instant Purge

18. E
D
F
C
A
B
Graph
of
Responsibility

19. E
D
F
C
A
B
Graph
of
Responsibility

20. E
D
F
C
A
B
Graph
of
Responsibility

21. E
D
F
C
A
B
PURGE

22. E
D
F
C
A
B
PURGE

23. E
D
F
C
A
B
PURGE

24. E
D
F
C
A
B
PURGE

25. E
D
F
C
A
B
PURGE

26. Failure Dependence

27. E
D
F
C
A
B
PURGE

28. E
D
F
C
A
B
PURGE

29. E
D
F
C
A
B
PURGE

30. E
D
F
C
A
B
PURGE

31. E
D
F
C
A
B
PURGE

32. How does Load flow?

33. The Internet
is both
more and less reliable
than we think

34. Q:
What is a normal amount of jitter
on a North American backbone?

35. A: <250μs

36. Normal packetloss is <0.1%

37. Asymmetric
Network
Splits

38. Paths are not symmetric

39. Q:
How do you see the inbound
path?

40. A:
Trick question. You don’t.

41. Consistency

42. CAP

43. Avoid Coordination

45. AP

46. https://aphyr.com/posts/313-strong-consistency-models

47. Causality

48. CRDTs

51. Further
Reading
Notes on Distributed Systems for Youngbloods
Fallacies of Distributed Computing
Literally everything on aphyr.com
Publications of the SyncFree group

52. Thanks