Video and slides synchronized, mp3 and slide download available at URL https://bit.ly/2x0Fav8. Jose Nino guides the audience through the journey of Mobile APIs at Lyft. He focuses on how the team has reaped the benefits of API generation to experiment with the network transport layer. He also discusses recent developments the team has made with Envoy Mobile and the roadmap ahead. Filmed at qconlondon.com. Jose Nino works as a Software Engineer at Lyft.

1. Jose Nino
@junr03
Next Generation Client APIs
with Envoy Mobile
QCon London - March 2020
Some slides built in collaboration with: @goaway & @rebello95

2. InfoQ.com: News & Community Site
Watch the video with slide
synchronization on InfoQ.com!
https://www.infoq.com/presentations/
api-envoy/
• Over 1,000,000 software developers, architects and CTOs read the site world-
wide every month
• 250,000 senior developers subscribe to our weekly newsletter
• Published in 4 languages (English, Chinese, Japanese and Brazilian
Portuguese)
• Post content from our QCon conferences
• 2 dedicated podcast channels: The InfoQ Podcast, with a focus on
Architecture and The Engineering Culture Podcast, with a focus on building
• 96 deep dives on innovative topics packed as downloadable emags and
minibooks
• Over 40 new content items per week

3. Purpose of QCon
- to empower software development by facilitating the spread of
knowledge and innovation
Strategy
- practitioner-driven conference designed for YOU: influencers of
change and innovation in your teams
- speakers and topics driving the evolution and innovation
- connecting and catalyzing the influencers and innovators
Highlights
- attended by more than 12,000 delegates since 2007
- held in 9 cities worldwide
Presented at QCon London
www.qconlondon.com

4. Server
Networking
Networking
Client
@rebello95

5. Server
Networking
Networking
Client
@junr03
API

6. Server
Networking
Networking
Server
@junr03
Server
Networking
Server
Networking
Server
Networking
Server
Networking Server
Networking
Server
Networking
Server
Networking
Server
Networking

7. Service
Networking
Networking
Server
@junr03
API
Server
Networking
Service
Networking
Server
Networking
Service
Networking Service
Networking
Service
Networking
Server
Networking
Server
Networking
Server
Networking Server
Networking
Server
Networking
Server
Networking
Server
Networking Server
Networking
Server
Networking
Server
Networking
Server
Networking Service
Networking
Server
Networking
Networking
Server
Networking Server
Networking
Server
Networking

8. Service
Networking
Networking
Server
@junr03
API
Server
Networking
Service
Networking
Server
Networking
Service
Networking Service
Networking
Service
Networking
Server
Networking
Server
Networking
Server
Networking Server
Networking
Server
Networking
Server
Networking
Server
Networking Server
Networking
Server
Networking
Server
Networking
Server
Networking Service
Networking
Server
Networking
Networking
Server
Networking Server
Networking
Server
Networking
��
��
��
��
��
��

9. Jose Nino
@junr03
Infrastructure Engineer
~4 years @

10. Service
Networking
Networking
Server
@junr03
API
Server
Networking
Service
Networking
Server
Networking
Service
Networking Service
Networking
Service
Networking
Server
Networking
Server
Networking
Server
Networking Server
Networking
Server
Networking
Server
Networking
Server
Networking Server
Networking
Server
Networking
Server
Networking
Server
Networking Service
Networking
Server
Networking
Networking
Server
Networking Server
Networking
Server
Networking
��
��
��
��
��
��

11. Server
Networking
Networking
Client
@junr03
API

12. We recognized two important dimensions
in API management: what data we send
(shape), and how we send it (transport).
@junr03
~3 years in Server Networking

13. Server-side Interface Definition Language (IDL)
Service A
“server”
@junr03
#service_a.proto
service ServiceA {
rpc UpdateA(ARequest) returns (AResponse) {
option (http.http_options).path = "/service_a";
option (http.http_options).method = "POST";
}
}
Service B
“client”

14. ● Network Proxy driven by Open-source
(OSS) community
● Common substrate for network behavior
What is ?
@junr03
Sidecar
Service B
“client”
Sidecar
Service A
“server”

15. Storage
EdgeMobile Client
Sidecar
Service B
Sidecar
Service A
3rd-party
Strongly-typed APIs and a Universal Network Primitive
@junr03

16. (circa early 2019)
The End
@junr03

17. Storage
EdgeMobile Client
Sidecar
Service B
Sidecar
Service A
3rd-party��
Strongly-typed APIs and a Universal Network Primitive
@junr03

18. Three 9s of reliability at the server-side is
meaningless if the user of a mobile client is only
able to complete the desired product flows a fraction
of the time.
What do we want from our Client APIs?
@junr03

19. Consistency ? ✓
Performance ? ✓
Extensibility ? ✓
Resilience ? ✓
Security ? ✓
Observability ? ✓
Three 9s of reliability at the server-side is meaningless if the user of a mobile
client is only able to complete the desired product flows a fraction of the time.
What do we want from our Client APIs?
@junr03

20. Storage
Edge
Sidecar
Service B
Sidecar
Service A
3rd-party
Clients as another node in the mesh
Mobile Client
@goaway @junr03
API

21. Next generation Client APIs with Envoy Mobile
Mobile Client
@junr03
API
��
We recognized two important dimensions
in API management: what data we send
(shape), and how we send it (transport).
~1 year in Client Networking

22. We recognized two important dimensions
in API management: what data we send
(shape), and how we send it (transport).
@junr03
API Shape

23. Early API workflow at Lyft
Tech spec
handwritten.swift
handwritten.kt
handwritten.py/.go
@rebello95

24. iOS
Server
{
“key_a”: “hello”
}
Android
{
“kye_a”: “hello”
}
Programming errors
@rebello95

25. iOS
Server
{
“key_a”: “hello”
}
Android
{
“kye_a”: “hello”
}
Programming errors
@rebello95

26. iOS
Server
{
“key_a”: “hello”
}
Android
{
“kye_a”: “hello”
}
Programming errors
@rebello95 @junr03
��

27. iOS
Server
{
“key_a”: “hello”
}
Android
{
“kye_a”: “hello”
}
Programming errors
@rebello95 @junr03

28. iOS
Server
{
“key_a”: “hello”
}
Android
{
“kye_a”: “hello”
}
if (android):
else:
Programming errors
@rebello95

29. ● Handwritten code led to mistakes/typos
● No visibility into deserialization failures at compile time
Problems
@rebello95 @junr03

30. YAML schema integration
@rebello95

31. paths:
/foo:
post:
parameters:
- in: body
schema:
$ref: '#/definitions/FooRequest'
responses:
200:
schema:
$ref: '#/definitions/FooResponse'
YAML definitions
@rebello95

32. ● Manual code generation for Android
● No integration on iOS
● No single source of truth
YAML results
@rebello95

33. Consistency
@rebello95 @junr03
We needed a source of truth that
provided guarantees about the shape of
any given API.

34. Envisioned workflow
Tech spec IDL repo
foo_api.swift
foo_api.kt
foo_api.py/.go
foo_api.proto
@rebello95

35. ● Platform-neutral IDL for defining and serializing strongly-typed APIs.
● JSON support (migration)
● Already used server-side (consistency)
Why protobuf?
@rebello95 @junr03

36. IDL pipeline
/api/v1/foo.proto
Generators
+7 others
foo.go
foo.py
foo.kt
foo.swift
@rebello95

37. IDL pipeline
/api/v1/foo.proto
Generators
+7 others
foo.go
foo.py
foo.kt
foo.swift FooV1API library
foo-v1-api library
@rebello95

38. Using generated APIs in Swift
@rebello95

39. // Modules/FooV1API/FooAPI.swift
public struct FooRequest: Codable, Equatable { … }
public struct FooResponse: Codable, Equatable {
public let id: Int64
public let firstName: String
}
Generated models
@rebello95

40. // Modules/FooV1API/FooAPI.swift
import RxSwift
public protocol FooAPI {
func updateFoo(_ request: FooRequest)
-> Single<Result<FooResponse, FooAPIError>>
}
public struct FooClientAPI: FooAPI {
// Abstracted implementation
}
Generated APIs
@rebello95

41. // Modules/FooV1API/FooAPI.swift
import RxSwift
public protocol FooAPI {
func updateFoo(_ request: FooRequest)
-> Single<Result<FooResponse, FooAPIError>>
}
public struct FooClientAPI: FooAPI {
🌟 // Abstracted implementation 🌟 (hint hint wink wink!)
}
Generated APIs
@rebello95 @junr03

42. // Modules/FooV1APIMock/FooAPI.swift
open class FooMockAPI: FooAPI {
open lazy var mockUpdateFoo: (FooRequest)
-> Result<FooResponse, FooAPIError> = { … }
open func updateFoo(_ request: FooRequest)
-> Single<Result<FooResponse, FooAPIError>> { … }
}
Generated mocks
@rebello95

43. ● Single source of truth for all APIs
● Consistent, generated code on all platforms
● Highly testable
● Underlying transport fully abstracted 🌟 565958
Results
@rebello95

44. Performance
@rebello95 @junr03
Experimenting with payload
encoding

45. Client
Service
JSON request
Existing system
JSON response
@rebello95

46. Client
Service
JSON request
Content-Type: json
Accept: x-protobuf,json
Content negotiation
Middleware
JSON > Protobuf
Protobuf request
Content-Type: x-protobuf
Protobuf response
Content-Type: x-protobuf
Middleware
No change
Protobuf response
Content-Type: x-protobuf
@rebello95

47. JSON
Protobuf
@rebello95

48. ● 40%+ reduction in payload sizes
● Faster response times
● Higher success rates
● Transparent to engineers
Wins from protobuf
@rebello95

49. Three 9s of reliability at the server-side is meaningless if the user of a mobile
client is only able to complete the desired product flows a fraction of the time.
What do we want from our Client APIs?
@junr03
Consistency ✓ ✓
Performance ✓ ✓
Extensibility ? ✓
Resilience ? ✓
Security ? ✓
Observability ? ✓

50. Extensibility
@junr03
Using protobuf annotations to declare API
behavior

51. service Foo {
rpc GetFoo(FooRequest) returns (FooResponse) {
option (http.http_options).path = "/foo/receive";
option (http.http_options).method = "GET";
option (http.http_options).client_poll_ms = 5000;
option (http.http_options).gzip_enabled = true;
}
}
Declarative APIs
@rebello95 @junr03

52. Service
Envoy
URLSession
iOS
OkHTTP
Android
@rebello95

53. Three 9s of reliability at the server-side is meaningless if the user of a mobile
client is only able to complete the desired product flows a fraction of the time.
What do we want from our Client APIs?
@junr03
Consistency ✓ ✓
Performance ✓ ✓
Extensibility 🤔 ? ✓
Resilience ? ✓
Security ? ✓
Observability ? ✓

54. We recognized two important dimensions
in API management: what data we send
(shape), and how we send it (transport).
@junr03
API Transport

55. Service
Envoy
Library X
Client
@rebello95

56. Clients as another node in the mesh
Storage
Edge
Sidecar
Service B
Sidecar
Service A
3rd-party
Client
@goaway @junr03

57. Standardizing infrastructure
@goaway @junr03

58. Why is world domination transport standardization useful?
● Write once, deploy everywhere
● Common tooling for common problems
● Reduce cognitive load
@goaway @junr03

59. Envoy Mobile to the rescue!
● v0.1: proof-of-concept
● v0.2: laying the foundation
● v0.3: first production-ready release
@goaway @junr03

60. Envoy as a Library
@junr03
How did we turn a network proxy into a
networking library?

61. Build System: bazel
@goaway @junr03
● Cross-platform support
● Used by Envoy

62. //library/common:c_types
//library/kotlin:android_framework
//library/swift:ios_framework
Layered Architecture
//:ios_dist
//:android_dist
//library/common:main_interface
@goaway @junr03
//library/common/...
@envoy//source/...
Platform (iOS/Android) Bridge (C) Native (C++/Envoy)

63. How to run a process in an app?
picture of an engine (a very fast one)
@goaway @junr03

64. Threading contexts
Application Threads
Envoy Main Thread
Callback Threads
@goaway @junr03

65. Library Matrix
Platform (iOS/Android) Bridge (C types/bindings) Native (C++/Envoy)
Application Threads
Envoy Main Thread
Callback Threads
@goaway @junr03

66. Library Lifecycle - Running Envoy
Application Threads
Envoy Main Thread
Callback Threads
EnvoyEngine
@goaway @junr03
Platform (iOS/Android) Bridge (C types/bindings) Native (C++/Envoy)

67. Library Lifecycle - Running Envoy
Application Threads
Envoy Main Thread
Callback Threads
EnvoyEngine
EnvoyRunner Engine
@goaway @junr03
Platform (iOS/Android) Bridge (C types/bindings) Native (C++/Envoy)

68. Server Envoy
Envoy Main Thread
Worker Threads
Envoy
Dispatcher
Http Manager
Requests
@goaway @junr03

69. Platform (iOS/Android)
Application Threads
Envoy Main Thread
Callback Threads
EnvoyEngine
EnvoyRunner
Dispatcher
Http Manager
@goaway @junr03
Bridge (C types/bindings) Native (C++/Envoy)
Library Lifecycle - using Envoy Constructs

70. Platform (iOS/Android)
Application Threads
Envoy Main Thread
Callback Threads
EnvoyEngine
EnvoyRunner
Dispatcher
Http Manager
HttpStream
@goaway @junr03
Bridge (C types/bindings) Native (C++/Envoy)
Library Lifecycle - starting a stream

71. Library Lifecycle - dispatching a stream
Application Threads
Envoy Main Thread
Callback Threads
EnvoyEngine
EnvoyRunner
Dispatcher
Http Manager
HttpStream
@goaway @junr03
Platform (iOS/Android) Bridge (C types/bindings) Native (C++/Envoy)

72. Http
Manager
Http Filters
@junr03
L7 Filter
L7 FilterHttp
Filter
Dispatcher
● Foundational extension point
● Enact behavior
● Provide the guarantees we want!

73. Callbacks
Library Lifecycle - callbacks
Application Threads
Envoy Main Thread
Callback Threads
EnvoyEngine
EnvoyRunner
Dispatcher
Http Manager
HttpStream
@goaway @junr03
Platform (iOS/Android) Bridge (C types/bindings) Native (C++/Envoy)

74. Library Lifecycle - platform callbacks
Application Threads
Envoy Main Thread
Callback Threads
EnvoyEngine
EnvoyRunner
Dispatcher
Http Manager
HttpStream
Callbacks
Lambdas
Dispatcher
@goaway @junr03
Platform (iOS/Android) Bridge (C types/bindings) Native (C++/Envoy)

75. Platform (iOS/Android)
Overall View
Application Threads
Envoy Main Thread
Callback Threads
EnvoyEngine
EnvoyRunner
Dispatcher
Http Manager
HttpStream
Callbacks
Lambdas
Dispatcher
Canceled
@goaway @junr03
Bridge (C types/bindings) Native (C++/Envoy)
Canceled

76. Platform Code
let envoy = try EnvoyClientBuilder(domain:
"api.envoyproxy.io")
.addLogLevel(.warn)
.addStatsFlushSeconds(60)
.build()
val envoy = EnvoyClientBuilder(
Domain("api.envoyproxy.io"))
.addLogLevel(LogLevel.WARN)
.addStatsFlushSeconds(60)
...
.build()
@goaway @junr03

77. Build an Engine
let envoy = try EnvoyClientBuilder(domain:
"api.envoyproxy.io")
.addLogLevel(.warn)
.addStatsFlushSeconds(60)
.build()
@goaway @junr03

78. Build an Engine
let envoy = try EnvoyClientBuilder(domain:
"api.envoyproxy.io")
.addLogLevel(.warn)
.addStatsFlushSeconds(60)
.build()
@goaway @junr03

79. Build a Request
let request = RequestBuilder(path:
"/pb.api.v1.Foo/GetBar")
.addHeader(name: "x-custom-header", value: "foobar")
.build()
@goaway @junr03

80. Build a Request
let request = RequestBuilder(path:
"/pb.api.v1.Foo/GetBar")
.addHeader(name: "x-custom-header", value: "foobar")
.build()
@goaway @junr03

81. Build a Response Handler
let handler = ResponseHandler()
.onHeaders { headers, status, _ ->
...
}
.onData { data ->
// Deserialize message data here
}
...
@goaway @junr03

82. Build a Response Handler
let handler = ResponseHandler()
.onHeaders { headers, status, _ ->
// Product logic
}
.onData { data ->
// Product logic
}
...
@goaway @junr03

83. Three 9s of reliability at the server-side is meaningless if the user of a mobile
client is only able to complete the desired product flows a fraction of the time.
What do we want from our Client APIs?
@junr03
Consistency ✓ ✓
Performance ✓ ✓
Extensibility 🤔 ? ✓
Resilience ? ✓
Security ? ✓
Observability ? ✓

84. IDL pipeline
/api/v1/foo.proto
Generators
+7 others
foo.go
foo.py
foo.kt
foo.swift FooV1API library
foo-v1-api library
@goaway @junr03
NSUrl
OkHttp

85. IDL pipeline
/api/v1/foo.proto
Generators
+7 others
foo.go
foo.py
foo.kt
foo.swift FooV1API library
foo-v1-api library
Client
@goaway @junr03

86. Ecosystem
@junr03
Shape + Transport = Ecosystem!

87. Three 9s of reliability at the server-side is meaningless if the user of a mobile
client is only able to complete the desired product flows a fraction of the time.
What do we want from our Client APIs?
Consistency ✓ ✓
Performance ✓ ✓
Extensibility ? ✓
Resilience ? ✓
Security ? ✓
Observability ? ✓
@junr03

88. Extensibility
@junr03
Using protobuf annotations to declare API behavior
and Envoy Mobile to enact consistent behavior.

89. service Foo {
rpc GetFoo(FooRequest) returns (FooResponse) {
option (http.http_options).path = "/foo/receive";
option (http.http_options).method = "GET";
option (http.http_options).client_poll_ms = 5000;
option (http.http_options).gzip_enabled = true;
}
}
Declarative APIs
@rebello95

90. Service
Envoy
URLSession
iOS
OkHTTP
Android
@rebello95 @junr03
�� ��
��

91. Extensibility
@junr03
Using protobuf annotations to declare API behavior
and Envoy Mobile to enact consistent behavior.

92. Http
Manager
Http Filters
@junr03
L7 Filter
L7 FilterHttp
Filter
Dispatcher
● Foundational extension point
● Enact behavior
● Provide the guarantees we want!

93. Envoy filters
Compression Filter
gzip request
Mobile
Proxy
Compression Filter
Unzip payload
🥳
EdgeClient
@junr03

94. Advanced transport
@junr03
● Dynamic Forward Proxy ✅
● Compression 🚧
● Deferred Requests 🕒 (Resilience)
● OAuth 🕒 (Security)
● Request Signing 🕒 (Security)
● Network condition 🕒 (Resilience)

95. Deferred Requests
@junr03
Deferred
Request Filter
Mobile
Proxy
Client
❌ Service
❌
Deferred
Request Filter
Mobile
Proxy
Client
✅ Service
✅
🥳
��
��

96. Request Signing
@junr03
Client
Request Signature
Filter
sign request
Mobile
Proxy
Request Signature
Filter
Verify signature
Edge
🕵‍♀

97. Advanced transport
@junr03
● Dynamic Forward Proxy ✅
● Compression 🚧
● Deferred Requests 🕒 (Resilience)
● OAuth 🕒 (Security)
● Request Signing 🕒 (Security)
● Network condition 🕒 (Resilience)

98. Observability
@junr03
True end-to-end insight

99. Observability
ts(envoy_mobile.cluster.api.upstream_rq.count)
ts(envoy_edge.cluster.*.upstream_rq.count)
@goaway @junr03

100. Time-series Metrics
Metrics
Service
@goaway @junr03

101. Dashboards!
ts(envoy_edge...)
@goaway @junr03
ts(envoy_mobile...)

102. Three 9s of reliability at the server-side is meaningless if the user of a mobile
client is only able to complete the desired product flows a fraction of the time.
What do we want from our Client APIs?
@junr03
Consistency ✓ ✓
Extensibility ✓ ✓
Performance ✓ ✓
Resilience ✓ ✓
Security ✓ ✓
Observability ✓ ✓

103. Onwards!
● Additional functionality in filters
● Protocol Experimentation with QUIC
● OSS Code Generators: complete OSS ecosystem for model-based
APIs.
@junr03

104. Next Generation Client APIs
@junr03
Model-based APIs defined in a strongly-typed IDL
so platform engineers can iterate on behavior using
a common transport layer.

105. Alpha Beta Production Experiment @Lyft!
@goaway @junr03

106. envoy-mobile.github.io
@junr03
Join us!

107. Watch the video with slide
synchronization on InfoQ.com!
https://www.infoq.com/presentations/
api-envoy/