SlideShare une entreprise Scribd logo
1  sur  46
1Rest.li: RESTful Service Architecture at Scale
SATURN 2018
14th Annual SEI Architecture Technology User Network Conference
MAY 7–10, 2018 | PLANO, TEXAS
Rest.li: RESTful Service Architecture at Scale
By: Min Chen (LinkedIn)
Email: mnchen@linkedin.com
Profile: www.linkedin.com/in/minchen07
2Rest.li: RESTful Service Architecture at Scale
A Brief History
Monolithic/Layered
Complex Orchestration
Complex Dependency from
Service Explosion
Incompatibilities
Slow Rollout
3Rest.li: RESTful Service Architecture at Scale
Paradigm Shift
Microservices
Fine-grained Service Boundary
Continuous Delivery
Standard Service Access Pattern
Backward-compatible service
evolution
4Rest.li: RESTful Service Architecture at Scale
SATURN 2018
Challenges
 Standard way to describe service in service-oriented architecture.
 Standard access pattern by multi-language diverse clients.
 Asynchronous and scalable to handle heavy service load.
 Fast and easy experience even for non-REST expert developers.
 Support long-term service evolution and growth.
5Rest.li: RESTful Service Architecture at Scale
Monolithic
Data Model
Centric
Microservices
6Rest.li: RESTful Service Architecture at Scale
Rest.li is an open source REST framework for
building robust, scalable RESTful architectures
using asynchronous, non-blocking IO. It fills a
niche for applying RESTful principles at scale with
an end-to-end developer workflow that
promotes clean REST practices.
7Rest.li: RESTful Service Architecture at Scale
8Rest.li: RESTful Service Architecture at Scale
SATURN 2018
Clean REST Practice
 End-to-End Developer Workflow
 Consistent data modeling
 Uniform interface design
 Type-safe client bindings
 Online API Hub
 API documentation
 API service catalog
 Compatibility Checker Enabled API Evolution
9Rest.li: RESTful Service Architecture at Scale
Data Schema
1
Developer Flow
10Rest.li: RESTful Service Architecture at Scale
SATURN 2018
Consistent Data Modeling
 Data schema in PDSC (Pegasus Data Schema)
 PDSC is inspired by Apache Avro specification.
 JSON serialization format
 Rich type support:
 Named: Record, Enum, Fixed, Typeref
 Unnamed: Union, Array, Map
 Custom type
 Custom validator and coercer
11Rest.li: RESTful Service Architecture at Scale
Data Schema Example
12Rest.li: RESTful Service Architecture at Scale
Data Schema
1
Resource Class
Resource Class
Resource Class
3
Record Template
2
Generated
Developer Flow
13Rest.li: RESTful Service Architecture at Scale
SATURN 2018
Uniform Interface Design
 Standardized RESTful resource pattern
 Annotated resource identifier
 Uniform service interface
 CRUD
 Finder
 Action
 BATCH
 REST specification IDL (Interface Description Language)
14Rest.li: RESTful Service Architecture at Scale
Rest.li Resource Patterns
Resource Type Semantics Interface Base Template
Collection
(@RestLiCollection)
Key/value map of
entites
CollectionResource
KeyValueResource
CollectionResourceTemplate
Simple
(@RestLiSimple)
Singleton in scope SimpleResource SimpleResourceTemplate
Association
(@RestLiAssociation)
Relationships
between entities
AssociationResource AssociationResourceTemplate
Sub-Resource Child resources All Above All Above
Action
(@RestliActions)
Flexible for non-
standard behavior
N/A N/A
15Rest.li: RESTful Service Architecture at Scale
Rest.li Resource Method (Basic)
Rest.li Request HTTP Method URI Example Entity Body
(V – Data Model)
CREATE POST /statuses V
GET GET /statuses/<id>
GET_ALL GET /statuses
UPDATE PUT /statuses/<id> V
PARTIAL_UPDATE POST /statuses/<id> Patch of V
DELETE DELETE /statuses/<id>
FINDER GET /statuses?q=search&keywords=li
nkedin
ACTION POST /statuses?action=purge Dynamic Record
16Rest.li: RESTful Service Architecture at Scale
Rest.li Resource Method (Batch)
Rest.li Request HTTP Method URI Example Entity Body
BATCH_CREATE POST /statuses {“elements”: […]}
BATCH_GET GET /statuses?ids=List(1,2,3)
BATCH_UPDATE PUT /statuses?ids=List(1,2,3) {“entities”: {“1”: {…}, “2” :
{…}}
BATCH_PARTIAL_UPDATE POST /statuses?ids=List(1,2,3) {“entities”: {“1”: {“patch”:
{…}}, “2” : {“patch”: {…}}
BATCH_DELETE DELETE /statuses?ids=List(1,2,3)
17Rest.li: RESTful Service Architecture at Scale
What Server Code I Need to Write
18Rest.li: RESTful Service Architecture at Scale
Data Schema
1
Resource Class
Resource Class
Resource Class
3
Record Template
2
Generated
4
REST IDL
Generated
Developer Flow
19Rest.li: RESTful Service Architecture at Scale
REST Spec IDL
Service
Metadata
PDSC
curl -X OPTIONS <ServiceEndpoint>
20Rest.li: RESTful Service Architecture at Scale
Data Schema
1
Server
5
Resource Class
Resource Class
Resource Class
3
Client
7
Record Template
2
Generated
4
REST IDL
Generated
6
Type-Safe
RequestBuilders
Generated
Developer Flow
21Rest.li: RESTful Service Architecture at Scale
What Client Code I Need to Write
Photo newPhoto = new Photo()
.setTitle("NewPhoto")
.setFormat(PhotoFormats.PNG)
.setExif(new Exif().setLocation(newLocation));
Request<?> createReq = new
PhotosRequestBuilder().create().input(newPhoto).build()build();
POST /photos -H "Content-Type: application/json”
-d {"title":"NewPhoto", "format": "PNG", "exif":{"location":...}}
Strong-typed data model
(auto-generated)
Type-safe request builders
(auto-generated)
22Rest.li: RESTful Service Architecture at Scale
Data Schema
1
Server
5
Resource Class
Resource Class
Resource Class
3
Client
7
Record Template
2
Generated
4
REST IDL
Generated
Human Readable
Documentation
Generated
8
6
Type-Safe
RequestBuilders
Generated
Developer Flow
23Rest.li: RESTful Service Architecture at Scale
Rest.li API Hub
• Lucene Powered full-text search
• Data model and interface
documentation
• Sample request generation
• Live interaction with real REST
endpoint
24Rest.li: RESTful Service Architecture at Scale
SATURN 2018
Backward Compatibility
 Build-time Compatibility Checker
 equivalent - No changes to Resources or Data model.
 backwards - Changes considered backwards.
 ignore - Log the changes but allow changes to pass.
 off - The compatibility checker will not be run at all.
25Rest.li: RESTful Service Architecture at Scale
26Rest.li: RESTful Service Architecture at Scale
Rest.li Server Stack
Rest.li Data Layer and RESTful Operations.li Inversion-of-Control
D2 Dynamic Discovery and Load Balancing Fault tolerance
R2 Request/Response and Network
Communication
Async and
Non-blocking
27Rest.li: RESTful Service Architecture at Scale
Request Response (R2)
Non-Streaming Streaming
Message Communication Abstraction over HTTP
28Rest.li: RESTful Service Architecture at Scale
SATURN 2018
Dynamic Discovery (D2)
 Apache Zookeeper (Announce and Discover)
 Horizontal Scaling
 Services, Servers (ephemeral nodes) and Clusters
 Client side load balancing
 Fault tolerance
 Backup request (tail-latency)
29Rest.li: RESTful Service Architecture at Scale
Rest.li Client
Implement and Interpret Rest.li Protocol
(Pluggable Transport Client)
30Rest.li: RESTful Service Architecture at Scale
Rest.li Client/Server Data Flow
31Rest.li: RESTful Service Architecture at Scale
Open and Extensible
 Filter Chain pipeline
 Polyglot transport client
 Custom Codec
(JSON, PSON, XML, etc)
32Rest.li: RESTful Service Architecture at Scale
Rich Eco-Systems
C#
Java
Android
Java
Objective-C
Swift
Python
33Rest.li: RESTful Service Architecture at Scale
Deco
Describe WHAT data you want, let Deco worry about
HOW to fetch them
posts
profiles
Companies
34Rest.li: RESTful Service Architecture at Scale
Deco Concept
/posts
/companies
/profiles
35Rest.li: RESTful Service Architecture at Scale
Deco Concept
deco://posts/123?projection=(title, content, author~(firstName,
lastName, company~(companyName)))
 Projection
 Decoration
 Resolution
Normalized Domain Modeling
36Rest.li: RESTful Service Architecture at Scale
37Rest.li: RESTful Service Architecture at Scale
SATURN 2018
Asynchronous Challenges
 Callback Hell
 Thread Safety
 Indeterminism
 Debugging
 How to achieve all-async?
38Rest.li: RESTful Service Architecture at Scale
SATURN 2018
ParSeq Concept
Promise<T> Represents state of an asynchronous operation, also
a container for an asynchronous operation result.
Task<T> Main abstraction unit representing an asynchronous
operation.
Task Callable Main ingredient of Task<T>. It starts asynchronous
operation and returns a Promise<T> representing
the operation state.
Engine Harness to run a Task<T>.
39Rest.li: RESTful Service Architecture at Scale
SATURN 2018
ParSeq Programming Model
 Declarative and Functional (What not How)
 Task composition, chaining, transformation and error handling
 Task.async (task creation)
 Task.par (parallel composition)
 task.andThen (sequential composition)
 task.map and task.flatMap (transformation)
 task.recover and task.withTimeout (error handling)
 Ends up with one Task (execution plan) submitted to ParSeq engine to
execute
40Rest.li: RESTful Service Architecture at Scale
SATURN 2018
ParSeq Execution Model
 All task Callables are executed sequentially, such that Callable of a
previous Task in a sequence “happens before” (in Java Memory Model
sense) Callable of a next Task.
 No need to be concerned about:
 Synchronized, locks, volatile, atomic, etc
 Immutability
 3rd party libraries
 Thinking about Java Memory Model
 Thread safety does not mean atomicity
Task<Response<T>> task =
Task.async(name, () -> sendRequest(request, requestContext));
Callable
41Rest.li: RESTful Service Architecture at Scale
SATURN 2018
Seamless Integration with Rest.li
 Rest.li supports resource method returning Task.
 ParSeqRestClient provides API to create Task from a rest.li request
@RestMethod.Get
public Task<Greeting> get(final Long id) {
// rest.li resource implementation
}
public <T> Task<Response<T>> createTask(final Request<T>
request, final RequestContext requestContext) {
}
42Rest.li: RESTful Service Architecture at Scale
ParSeq Trace Visualization
Power Tool for Debugging
43Rest.li: RESTful Service Architecture at Scale
SATURN 2018
Multi-Language Support
C#
Java
Android
Java
Objective-C
Swift
Python
44Rest.li: RESTful Service Architecture at Scale
SATURN 2018
Comparison Matrix
Rest.li Jersey DropWizard Olingo (Odata)
Data Schema yes no no yes
Data Model Auto Manual Manual Manual
Server Coding
URI Pattern Uniform Non-Uniform Non-Uniform Non-Uniform
Batch Support Yes No No Yes
Service Metadata Yes No No Yes
Dynamic
Discovery/Load
balancing
Yes No No No
Documentation Yes No No No
Service Catalog Yes No No No
Graph Query Yes No No Yes
45Rest.li: RESTful Service Architecture at Scale
SATURN 2018
Future of Rest.li
 Object Streaming in Rest.li Layer
 Unstructured data support
 Multi-language support enhancement
 Go
 Cross-language test suites
 Open source Deco
 ParSeq
 Collection API
 ParSeq Trace Analytics
46Rest.li: RESTful Service Architecture at Scale
SATURN 2018
Useful Links
 Rest.li (https://github.com/linkedin/rest.li)
 Rest.li API Hub (https://github.com/linkedin/rest.li-api-hub)
 ParSeq (https://github.com/linkedin/parseq)

Contenu connexe

Tendances

Graphql presentation
Graphql presentationGraphql presentation
Graphql presentationVibhor Grover
 
Hexagonal architecture with Spring Boot
Hexagonal architecture with Spring BootHexagonal architecture with Spring Boot
Hexagonal architecture with Spring BootMikalai Alimenkou
 
WUG Days 2022 Brno - Networking in .NET 7.0 and YARP -- Karel Zikmund
WUG Days 2022 Brno - Networking in .NET 7.0 and YARP -- Karel ZikmundWUG Days 2022 Brno - Networking in .NET 7.0 and YARP -- Karel Zikmund
WUG Days 2022 Brno - Networking in .NET 7.0 and YARP -- Karel ZikmundKarel Zikmund
 
Monoids - Part 1 - with examples using Scalaz and Cats
Monoids - Part 1 - with examples using Scalaz and CatsMonoids - Part 1 - with examples using Scalaz and Cats
Monoids - Part 1 - with examples using Scalaz and CatsPhilip Schwarz
 
Microservice Architecture, over the rainbow
Microservice Architecture, over the rainbowMicroservice Architecture, over the rainbow
Microservice Architecture, over the rainbowVincenzo Carlino
 
React Context API
React Context APIReact Context API
React Context APINodeXperts
 
Hexagonal Design in Django
Hexagonal Design in DjangoHexagonal Design in Django
Hexagonal Design in Djangomvschaik
 
Web API testing : A quick glance
Web API testing : A quick glanceWeb API testing : A quick glance
Web API testing : A quick glanceDhanalaxmi K
 
Introduction to API
Introduction to APIIntroduction to API
Introduction to APIrajnishjha29
 
APIsecure 2023 - Exploring Advanced API Security Techniques and Technologies,...
APIsecure 2023 - Exploring Advanced API Security Techniques and Technologies,...APIsecure 2023 - Exploring Advanced API Security Techniques and Technologies,...
APIsecure 2023 - Exploring Advanced API Security Techniques and Technologies,...apidays
 
Introduction to REST - API
Introduction to REST - APIIntroduction to REST - API
Introduction to REST - APIChetan Gadodia
 
API Maturity Model (Webcast with Accenture)
API Maturity Model (Webcast with Accenture)API Maturity Model (Webcast with Accenture)
API Maturity Model (Webcast with Accenture)Apigee | Google Cloud
 
Spring Framework Petclinic sample application
Spring Framework Petclinic sample applicationSpring Framework Petclinic sample application
Spring Framework Petclinic sample applicationAntoine Rey
 
Introduction to react native
Introduction to react nativeIntroduction to react native
Introduction to react nativeDani Akash
 
Java 8 presentation
Java 8 presentationJava 8 presentation
Java 8 presentationVan Huong
 

Tendances (20)

Rest API
Rest APIRest API
Rest API
 
Graphql presentation
Graphql presentationGraphql presentation
Graphql presentation
 
Hexagonal architecture with Spring Boot
Hexagonal architecture with Spring BootHexagonal architecture with Spring Boot
Hexagonal architecture with Spring Boot
 
WUG Days 2022 Brno - Networking in .NET 7.0 and YARP -- Karel Zikmund
WUG Days 2022 Brno - Networking in .NET 7.0 and YARP -- Karel ZikmundWUG Days 2022 Brno - Networking in .NET 7.0 and YARP -- Karel Zikmund
WUG Days 2022 Brno - Networking in .NET 7.0 and YARP -- Karel Zikmund
 
Monoids - Part 1 - with examples using Scalaz and Cats
Monoids - Part 1 - with examples using Scalaz and CatsMonoids - Part 1 - with examples using Scalaz and Cats
Monoids - Part 1 - with examples using Scalaz and Cats
 
Microservice Architecture, over the rainbow
Microservice Architecture, over the rainbowMicroservice Architecture, over the rainbow
Microservice Architecture, over the rainbow
 
React
React React
React
 
An Introduction To REST API
An Introduction To REST APIAn Introduction To REST API
An Introduction To REST API
 
React Context API
React Context APIReact Context API
React Context API
 
Redux essentials
Redux essentialsRedux essentials
Redux essentials
 
Hexagonal Design in Django
Hexagonal Design in DjangoHexagonal Design in Django
Hexagonal Design in Django
 
React Server Side Rendering with Next.js
React Server Side Rendering with Next.jsReact Server Side Rendering with Next.js
React Server Side Rendering with Next.js
 
Web API testing : A quick glance
Web API testing : A quick glanceWeb API testing : A quick glance
Web API testing : A quick glance
 
Introduction to API
Introduction to APIIntroduction to API
Introduction to API
 
APIsecure 2023 - Exploring Advanced API Security Techniques and Technologies,...
APIsecure 2023 - Exploring Advanced API Security Techniques and Technologies,...APIsecure 2023 - Exploring Advanced API Security Techniques and Technologies,...
APIsecure 2023 - Exploring Advanced API Security Techniques and Technologies,...
 
Introduction to REST - API
Introduction to REST - APIIntroduction to REST - API
Introduction to REST - API
 
API Maturity Model (Webcast with Accenture)
API Maturity Model (Webcast with Accenture)API Maturity Model (Webcast with Accenture)
API Maturity Model (Webcast with Accenture)
 
Spring Framework Petclinic sample application
Spring Framework Petclinic sample applicationSpring Framework Petclinic sample application
Spring Framework Petclinic sample application
 
Introduction to react native
Introduction to react nativeIntroduction to react native
Introduction to react native
 
Java 8 presentation
Java 8 presentationJava 8 presentation
Java 8 presentation
 

Similaire à Saturn 2018 rest.li

A COMPARATIVE STUDY BETWEEN GRAPH-QL& RESTFUL SERVICES IN API MANAGEMENT OF S...
A COMPARATIVE STUDY BETWEEN GRAPH-QL& RESTFUL SERVICES IN API MANAGEMENT OF S...A COMPARATIVE STUDY BETWEEN GRAPH-QL& RESTFUL SERVICES IN API MANAGEMENT OF S...
A COMPARATIVE STUDY BETWEEN GRAPH-QL& RESTFUL SERVICES IN API MANAGEMENT OF S...ijwscjournal
 
A COMPARATIVE STUDY BETWEEN GRAPH-QL RESTFUL SERVICES IN API MANAGEMENT OF S...
A COMPARATIVE STUDY BETWEEN GRAPH-QL  RESTFUL SERVICES IN API MANAGEMENT OF S...A COMPARATIVE STUDY BETWEEN GRAPH-QL  RESTFUL SERVICES IN API MANAGEMENT OF S...
A COMPARATIVE STUDY BETWEEN GRAPH-QL RESTFUL SERVICES IN API MANAGEMENT OF S...Sophia Diaz
 
A COMPARATIVE STUDY BETWEEN GRAPH-QL Amp RESTFUL SERVICES IN API MANAGEMENT ...
A COMPARATIVE STUDY BETWEEN GRAPH-QL Amp  RESTFUL SERVICES IN API MANAGEMENT ...A COMPARATIVE STUDY BETWEEN GRAPH-QL Amp  RESTFUL SERVICES IN API MANAGEMENT ...
A COMPARATIVE STUDY BETWEEN GRAPH-QL Amp RESTFUL SERVICES IN API MANAGEMENT ...Amanda Summers
 
A Comparative Study between Graph-QL& Restful Services in API Management of S...
A Comparative Study between Graph-QL& Restful Services in API Management of S...A Comparative Study between Graph-QL& Restful Services in API Management of S...
A Comparative Study between Graph-QL& Restful Services in API Management of S...ijwscjournal
 
Modeling REST API's Behaviour with Text, Graphics or Both?
Modeling REST API's Behaviour with Text, Graphics or Both?Modeling REST API's Behaviour with Text, Graphics or Both?
Modeling REST API's Behaviour with Text, Graphics or Both?Ana Ivanchikj
 
REST & RESTful APIs: The State of Confusion
REST & RESTful APIs: The State of ConfusionREST & RESTful APIs: The State of Confusion
REST & RESTful APIs: The State of ConfusionGlenn Antoine
 
Rest API Automation with REST Assured
Rest API Automation with REST AssuredRest API Automation with REST Assured
Rest API Automation with REST AssuredTO THE NEW Pvt. Ltd.
 
Transform-to-Smart-ERP-using-Custom-Mobile-Apps.pptx
Transform-to-Smart-ERP-using-Custom-Mobile-Apps.pptxTransform-to-Smart-ERP-using-Custom-Mobile-Apps.pptx
Transform-to-Smart-ERP-using-Custom-Mobile-Apps.pptxkmani5
 
Transform-to-Smart-ERP-using-Custom-Mobile-Apps.pptx (3).ppt
Transform-to-Smart-ERP-using-Custom-Mobile-Apps.pptx (3).pptTransform-to-Smart-ERP-using-Custom-Mobile-Apps.pptx (3).ppt
Transform-to-Smart-ERP-using-Custom-Mobile-Apps.pptx (3).pptHusseinWassof
 
Rest API and Client OM for Developer
Rest API and Client OM for DeveloperRest API and Client OM for Developer
Rest API and Client OM for DeveloperInnoTech
 
ROC for Adaptive Systems
ROC for Adaptive SystemsROC for Adaptive Systems
ROC for Adaptive SystemsTom Mueck
 
Toward data driven programming for RESTful Linked Data
Toward data driven programming for RESTful Linked DataToward data driven programming for RESTful Linked Data
Toward data driven programming for RESTful Linked DataSteffen Stadtmüller
 
Cs 1023 lec 13 web (week 4)
Cs 1023 lec 13 web (week 4)Cs 1023 lec 13 web (week 4)
Cs 1023 lec 13 web (week 4)stanbridge
 
IRJET- Rest API for E-Commerce Site
IRJET- Rest API for E-Commerce SiteIRJET- Rest API for E-Commerce Site
IRJET- Rest API for E-Commerce SiteIRJET Journal
 
apidays Australia 2023 - The Playful Bond Between REST And Data Streams, Warr...
apidays Australia 2023 - The Playful Bond Between REST And Data Streams, Warr...apidays Australia 2023 - The Playful Bond Between REST And Data Streams, Warr...
apidays Australia 2023 - The Playful Bond Between REST And Data Streams, Warr...apidays
 
The Playful Bond Between REST And Data Streams
The Playful Bond Between REST And Data StreamsThe Playful Bond Between REST And Data Streams
The Playful Bond Between REST And Data Streamsconfluent
 
Representational State Transfer (REST) and HATEOAS
Representational State Transfer (REST) and HATEOASRepresentational State Transfer (REST) and HATEOAS
Representational State Transfer (REST) and HATEOASGuy K. Kloss
 
ALM iStack - Application Lifecycle Management using Linked Data
ALM iStack - Application Lifecycle Management using Linked Data ALM iStack - Application Lifecycle Management using Linked Data
ALM iStack - Application Lifecycle Management using Linked Data Nandana Mihindukulasooriya
 

Similaire à Saturn 2018 rest.li (20)

A COMPARATIVE STUDY BETWEEN GRAPH-QL& RESTFUL SERVICES IN API MANAGEMENT OF S...
A COMPARATIVE STUDY BETWEEN GRAPH-QL& RESTFUL SERVICES IN API MANAGEMENT OF S...A COMPARATIVE STUDY BETWEEN GRAPH-QL& RESTFUL SERVICES IN API MANAGEMENT OF S...
A COMPARATIVE STUDY BETWEEN GRAPH-QL& RESTFUL SERVICES IN API MANAGEMENT OF S...
 
A COMPARATIVE STUDY BETWEEN GRAPH-QL RESTFUL SERVICES IN API MANAGEMENT OF S...
A COMPARATIVE STUDY BETWEEN GRAPH-QL  RESTFUL SERVICES IN API MANAGEMENT OF S...A COMPARATIVE STUDY BETWEEN GRAPH-QL  RESTFUL SERVICES IN API MANAGEMENT OF S...
A COMPARATIVE STUDY BETWEEN GRAPH-QL RESTFUL SERVICES IN API MANAGEMENT OF S...
 
A COMPARATIVE STUDY BETWEEN GRAPH-QL Amp RESTFUL SERVICES IN API MANAGEMENT ...
A COMPARATIVE STUDY BETWEEN GRAPH-QL Amp  RESTFUL SERVICES IN API MANAGEMENT ...A COMPARATIVE STUDY BETWEEN GRAPH-QL Amp  RESTFUL SERVICES IN API MANAGEMENT ...
A COMPARATIVE STUDY BETWEEN GRAPH-QL Amp RESTFUL SERVICES IN API MANAGEMENT ...
 
A Comparative Study between Graph-QL& Restful Services in API Management of S...
A Comparative Study between Graph-QL& Restful Services in API Management of S...A Comparative Study between Graph-QL& Restful Services in API Management of S...
A Comparative Study between Graph-QL& Restful Services in API Management of S...
 
Modeling REST API's Behaviour with Text, Graphics or Both?
Modeling REST API's Behaviour with Text, Graphics or Both?Modeling REST API's Behaviour with Text, Graphics or Both?
Modeling REST API's Behaviour with Text, Graphics or Both?
 
REST & RESTful APIs: The State of Confusion
REST & RESTful APIs: The State of ConfusionREST & RESTful APIs: The State of Confusion
REST & RESTful APIs: The State of Confusion
 
Rest API Automation with REST Assured
Rest API Automation with REST AssuredRest API Automation with REST Assured
Rest API Automation with REST Assured
 
Transform-to-Smart-ERP-using-Custom-Mobile-Apps.pptx
Transform-to-Smart-ERP-using-Custom-Mobile-Apps.pptxTransform-to-Smart-ERP-using-Custom-Mobile-Apps.pptx
Transform-to-Smart-ERP-using-Custom-Mobile-Apps.pptx
 
Transform-to-Smart-ERP-using-Custom-Mobile-Apps.pptx (3).ppt
Transform-to-Smart-ERP-using-Custom-Mobile-Apps.pptx (3).pptTransform-to-Smart-ERP-using-Custom-Mobile-Apps.pptx (3).ppt
Transform-to-Smart-ERP-using-Custom-Mobile-Apps.pptx (3).ppt
 
Rest API and Client OM for Developer
Rest API and Client OM for DeveloperRest API and Client OM for Developer
Rest API and Client OM for Developer
 
Unit 2
Unit 2Unit 2
Unit 2
 
ROC for Adaptive Systems
ROC for Adaptive SystemsROC for Adaptive Systems
ROC for Adaptive Systems
 
Lab7 paper
Lab7 paperLab7 paper
Lab7 paper
 
Toward data driven programming for RESTful Linked Data
Toward data driven programming for RESTful Linked DataToward data driven programming for RESTful Linked Data
Toward data driven programming for RESTful Linked Data
 
Cs 1023 lec 13 web (week 4)
Cs 1023 lec 13 web (week 4)Cs 1023 lec 13 web (week 4)
Cs 1023 lec 13 web (week 4)
 
IRJET- Rest API for E-Commerce Site
IRJET- Rest API for E-Commerce SiteIRJET- Rest API for E-Commerce Site
IRJET- Rest API for E-Commerce Site
 
apidays Australia 2023 - The Playful Bond Between REST And Data Streams, Warr...
apidays Australia 2023 - The Playful Bond Between REST And Data Streams, Warr...apidays Australia 2023 - The Playful Bond Between REST And Data Streams, Warr...
apidays Australia 2023 - The Playful Bond Between REST And Data Streams, Warr...
 
The Playful Bond Between REST And Data Streams
The Playful Bond Between REST And Data StreamsThe Playful Bond Between REST And Data Streams
The Playful Bond Between REST And Data Streams
 
Representational State Transfer (REST) and HATEOAS
Representational State Transfer (REST) and HATEOASRepresentational State Transfer (REST) and HATEOAS
Representational State Transfer (REST) and HATEOAS
 
ALM iStack - Application Lifecycle Management using Linked Data
ALM iStack - Application Lifecycle Management using Linked Data ALM iStack - Application Lifecycle Management using Linked Data
ALM iStack - Application Lifecycle Management using Linked Data
 

Dernier

KCD Costa Rica 2024 - Nephio para parvulitos
KCD Costa Rica 2024 - Nephio para parvulitosKCD Costa Rica 2024 - Nephio para parvulitos
KCD Costa Rica 2024 - Nephio para parvulitosVictor Morales
 
CS 3251 Programming in c all unit notes pdf
CS 3251 Programming in c all unit notes pdfCS 3251 Programming in c all unit notes pdf
CS 3251 Programming in c all unit notes pdfBalamuruganV28
 
High Voltage Engineering- OVER VOLTAGES IN ELECTRICAL POWER SYSTEMS
High Voltage Engineering- OVER VOLTAGES IN ELECTRICAL POWER SYSTEMSHigh Voltage Engineering- OVER VOLTAGES IN ELECTRICAL POWER SYSTEMS
High Voltage Engineering- OVER VOLTAGES IN ELECTRICAL POWER SYSTEMSsandhya757531
 
THE SENDAI FRAMEWORK FOR DISASTER RISK REDUCTION
THE SENDAI FRAMEWORK FOR DISASTER RISK REDUCTIONTHE SENDAI FRAMEWORK FOR DISASTER RISK REDUCTION
THE SENDAI FRAMEWORK FOR DISASTER RISK REDUCTIONjhunlian
 
11. Properties of Liquid Fuels in Energy Engineering.pdf
11. Properties of Liquid Fuels in Energy Engineering.pdf11. Properties of Liquid Fuels in Energy Engineering.pdf
11. Properties of Liquid Fuels in Energy Engineering.pdfHafizMudaserAhmad
 
Robotics-Asimov's Laws, Mechanical Subsystems, Robot Kinematics, Robot Dynami...
Robotics-Asimov's Laws, Mechanical Subsystems, Robot Kinematics, Robot Dynami...Robotics-Asimov's Laws, Mechanical Subsystems, Robot Kinematics, Robot Dynami...
Robotics-Asimov's Laws, Mechanical Subsystems, Robot Kinematics, Robot Dynami...Sumanth A
 
Curve setting (Basic Mine Surveying)_MI10412MI.pptx
Curve setting (Basic Mine Surveying)_MI10412MI.pptxCurve setting (Basic Mine Surveying)_MI10412MI.pptx
Curve setting (Basic Mine Surveying)_MI10412MI.pptxRomil Mishra
 
70 POWER PLANT IAE V2500 technical training
70 POWER PLANT IAE V2500 technical training70 POWER PLANT IAE V2500 technical training
70 POWER PLANT IAE V2500 technical trainingGladiatorsKasper
 
Novel 3D-Printed Soft Linear and Bending Actuators
Novel 3D-Printed Soft Linear and Bending ActuatorsNovel 3D-Printed Soft Linear and Bending Actuators
Novel 3D-Printed Soft Linear and Bending ActuatorsResearcher Researcher
 
Mine Environment II Lab_MI10448MI__________.pptx
Mine Environment II Lab_MI10448MI__________.pptxMine Environment II Lab_MI10448MI__________.pptx
Mine Environment II Lab_MI10448MI__________.pptxRomil Mishra
 
priority interrupt computer organization
priority interrupt computer organizationpriority interrupt computer organization
priority interrupt computer organizationchnrketan
 
Main Memory Management in Operating System
Main Memory Management in Operating SystemMain Memory Management in Operating System
Main Memory Management in Operating SystemRashmi Bhat
 
"Exploring the Essential Functions and Design Considerations of Spillways in ...
"Exploring the Essential Functions and Design Considerations of Spillways in ..."Exploring the Essential Functions and Design Considerations of Spillways in ...
"Exploring the Essential Functions and Design Considerations of Spillways in ...Erbil Polytechnic University
 
2022 AWS DNA Hackathon 장애 대응 솔루션 jarvis.
2022 AWS DNA Hackathon 장애 대응 솔루션 jarvis.2022 AWS DNA Hackathon 장애 대응 솔루션 jarvis.
2022 AWS DNA Hackathon 장애 대응 솔루션 jarvis.elesangwon
 
Immutable Image-Based Operating Systems - EW2024.pdf
Immutable Image-Based Operating Systems - EW2024.pdfImmutable Image-Based Operating Systems - EW2024.pdf
Immutable Image-Based Operating Systems - EW2024.pdfDrew Moseley
 
CME 397 - SURFACE ENGINEERING - UNIT 1 FULL NOTES
CME 397 - SURFACE ENGINEERING - UNIT 1 FULL NOTESCME 397 - SURFACE ENGINEERING - UNIT 1 FULL NOTES
CME 397 - SURFACE ENGINEERING - UNIT 1 FULL NOTESkarthi keyan
 
Katarzyna Lipka-Sidor - BIM School Course
Katarzyna Lipka-Sidor - BIM School CourseKatarzyna Lipka-Sidor - BIM School Course
Katarzyna Lipka-Sidor - BIM School Coursebim.edu.pl
 
Input Output Management in Operating System
Input Output Management in Operating SystemInput Output Management in Operating System
Input Output Management in Operating SystemRashmi Bhat
 
List of Accredited Concrete Batching Plant.pdf
List of Accredited Concrete Batching Plant.pdfList of Accredited Concrete Batching Plant.pdf
List of Accredited Concrete Batching Plant.pdfisabel213075
 
DEVICE DRIVERS AND INTERRUPTS SERVICE MECHANISM.pdf
DEVICE DRIVERS AND INTERRUPTS  SERVICE MECHANISM.pdfDEVICE DRIVERS AND INTERRUPTS  SERVICE MECHANISM.pdf
DEVICE DRIVERS AND INTERRUPTS SERVICE MECHANISM.pdfAkritiPradhan2
 

Dernier (20)

KCD Costa Rica 2024 - Nephio para parvulitos
KCD Costa Rica 2024 - Nephio para parvulitosKCD Costa Rica 2024 - Nephio para parvulitos
KCD Costa Rica 2024 - Nephio para parvulitos
 
CS 3251 Programming in c all unit notes pdf
CS 3251 Programming in c all unit notes pdfCS 3251 Programming in c all unit notes pdf
CS 3251 Programming in c all unit notes pdf
 
High Voltage Engineering- OVER VOLTAGES IN ELECTRICAL POWER SYSTEMS
High Voltage Engineering- OVER VOLTAGES IN ELECTRICAL POWER SYSTEMSHigh Voltage Engineering- OVER VOLTAGES IN ELECTRICAL POWER SYSTEMS
High Voltage Engineering- OVER VOLTAGES IN ELECTRICAL POWER SYSTEMS
 
THE SENDAI FRAMEWORK FOR DISASTER RISK REDUCTION
THE SENDAI FRAMEWORK FOR DISASTER RISK REDUCTIONTHE SENDAI FRAMEWORK FOR DISASTER RISK REDUCTION
THE SENDAI FRAMEWORK FOR DISASTER RISK REDUCTION
 
11. Properties of Liquid Fuels in Energy Engineering.pdf
11. Properties of Liquid Fuels in Energy Engineering.pdf11. Properties of Liquid Fuels in Energy Engineering.pdf
11. Properties of Liquid Fuels in Energy Engineering.pdf
 
Robotics-Asimov's Laws, Mechanical Subsystems, Robot Kinematics, Robot Dynami...
Robotics-Asimov's Laws, Mechanical Subsystems, Robot Kinematics, Robot Dynami...Robotics-Asimov's Laws, Mechanical Subsystems, Robot Kinematics, Robot Dynami...
Robotics-Asimov's Laws, Mechanical Subsystems, Robot Kinematics, Robot Dynami...
 
Curve setting (Basic Mine Surveying)_MI10412MI.pptx
Curve setting (Basic Mine Surveying)_MI10412MI.pptxCurve setting (Basic Mine Surveying)_MI10412MI.pptx
Curve setting (Basic Mine Surveying)_MI10412MI.pptx
 
70 POWER PLANT IAE V2500 technical training
70 POWER PLANT IAE V2500 technical training70 POWER PLANT IAE V2500 technical training
70 POWER PLANT IAE V2500 technical training
 
Novel 3D-Printed Soft Linear and Bending Actuators
Novel 3D-Printed Soft Linear and Bending ActuatorsNovel 3D-Printed Soft Linear and Bending Actuators
Novel 3D-Printed Soft Linear and Bending Actuators
 
Mine Environment II Lab_MI10448MI__________.pptx
Mine Environment II Lab_MI10448MI__________.pptxMine Environment II Lab_MI10448MI__________.pptx
Mine Environment II Lab_MI10448MI__________.pptx
 
priority interrupt computer organization
priority interrupt computer organizationpriority interrupt computer organization
priority interrupt computer organization
 
Main Memory Management in Operating System
Main Memory Management in Operating SystemMain Memory Management in Operating System
Main Memory Management in Operating System
 
"Exploring the Essential Functions and Design Considerations of Spillways in ...
"Exploring the Essential Functions and Design Considerations of Spillways in ..."Exploring the Essential Functions and Design Considerations of Spillways in ...
"Exploring the Essential Functions and Design Considerations of Spillways in ...
 
2022 AWS DNA Hackathon 장애 대응 솔루션 jarvis.
2022 AWS DNA Hackathon 장애 대응 솔루션 jarvis.2022 AWS DNA Hackathon 장애 대응 솔루션 jarvis.
2022 AWS DNA Hackathon 장애 대응 솔루션 jarvis.
 
Immutable Image-Based Operating Systems - EW2024.pdf
Immutable Image-Based Operating Systems - EW2024.pdfImmutable Image-Based Operating Systems - EW2024.pdf
Immutable Image-Based Operating Systems - EW2024.pdf
 
CME 397 - SURFACE ENGINEERING - UNIT 1 FULL NOTES
CME 397 - SURFACE ENGINEERING - UNIT 1 FULL NOTESCME 397 - SURFACE ENGINEERING - UNIT 1 FULL NOTES
CME 397 - SURFACE ENGINEERING - UNIT 1 FULL NOTES
 
Katarzyna Lipka-Sidor - BIM School Course
Katarzyna Lipka-Sidor - BIM School CourseKatarzyna Lipka-Sidor - BIM School Course
Katarzyna Lipka-Sidor - BIM School Course
 
Input Output Management in Operating System
Input Output Management in Operating SystemInput Output Management in Operating System
Input Output Management in Operating System
 
List of Accredited Concrete Batching Plant.pdf
List of Accredited Concrete Batching Plant.pdfList of Accredited Concrete Batching Plant.pdf
List of Accredited Concrete Batching Plant.pdf
 
DEVICE DRIVERS AND INTERRUPTS SERVICE MECHANISM.pdf
DEVICE DRIVERS AND INTERRUPTS  SERVICE MECHANISM.pdfDEVICE DRIVERS AND INTERRUPTS  SERVICE MECHANISM.pdf
DEVICE DRIVERS AND INTERRUPTS SERVICE MECHANISM.pdf
 

Saturn 2018 rest.li

  • 1. 1Rest.li: RESTful Service Architecture at Scale SATURN 2018 14th Annual SEI Architecture Technology User Network Conference MAY 7–10, 2018 | PLANO, TEXAS Rest.li: RESTful Service Architecture at Scale By: Min Chen (LinkedIn) Email: mnchen@linkedin.com Profile: www.linkedin.com/in/minchen07
  • 2. 2Rest.li: RESTful Service Architecture at Scale A Brief History Monolithic/Layered Complex Orchestration Complex Dependency from Service Explosion Incompatibilities Slow Rollout
  • 3. 3Rest.li: RESTful Service Architecture at Scale Paradigm Shift Microservices Fine-grained Service Boundary Continuous Delivery Standard Service Access Pattern Backward-compatible service evolution
  • 4. 4Rest.li: RESTful Service Architecture at Scale SATURN 2018 Challenges  Standard way to describe service in service-oriented architecture.  Standard access pattern by multi-language diverse clients.  Asynchronous and scalable to handle heavy service load.  Fast and easy experience even for non-REST expert developers.  Support long-term service evolution and growth.
  • 5. 5Rest.li: RESTful Service Architecture at Scale Monolithic Data Model Centric Microservices
  • 6. 6Rest.li: RESTful Service Architecture at Scale Rest.li is an open source REST framework for building robust, scalable RESTful architectures using asynchronous, non-blocking IO. It fills a niche for applying RESTful principles at scale with an end-to-end developer workflow that promotes clean REST practices.
  • 7. 7Rest.li: RESTful Service Architecture at Scale
  • 8. 8Rest.li: RESTful Service Architecture at Scale SATURN 2018 Clean REST Practice  End-to-End Developer Workflow  Consistent data modeling  Uniform interface design  Type-safe client bindings  Online API Hub  API documentation  API service catalog  Compatibility Checker Enabled API Evolution
  • 9. 9Rest.li: RESTful Service Architecture at Scale Data Schema 1 Developer Flow
  • 10. 10Rest.li: RESTful Service Architecture at Scale SATURN 2018 Consistent Data Modeling  Data schema in PDSC (Pegasus Data Schema)  PDSC is inspired by Apache Avro specification.  JSON serialization format  Rich type support:  Named: Record, Enum, Fixed, Typeref  Unnamed: Union, Array, Map  Custom type  Custom validator and coercer
  • 11. 11Rest.li: RESTful Service Architecture at Scale Data Schema Example
  • 12. 12Rest.li: RESTful Service Architecture at Scale Data Schema 1 Resource Class Resource Class Resource Class 3 Record Template 2 Generated Developer Flow
  • 13. 13Rest.li: RESTful Service Architecture at Scale SATURN 2018 Uniform Interface Design  Standardized RESTful resource pattern  Annotated resource identifier  Uniform service interface  CRUD  Finder  Action  BATCH  REST specification IDL (Interface Description Language)
  • 14. 14Rest.li: RESTful Service Architecture at Scale Rest.li Resource Patterns Resource Type Semantics Interface Base Template Collection (@RestLiCollection) Key/value map of entites CollectionResource KeyValueResource CollectionResourceTemplate Simple (@RestLiSimple) Singleton in scope SimpleResource SimpleResourceTemplate Association (@RestLiAssociation) Relationships between entities AssociationResource AssociationResourceTemplate Sub-Resource Child resources All Above All Above Action (@RestliActions) Flexible for non- standard behavior N/A N/A
  • 15. 15Rest.li: RESTful Service Architecture at Scale Rest.li Resource Method (Basic) Rest.li Request HTTP Method URI Example Entity Body (V – Data Model) CREATE POST /statuses V GET GET /statuses/<id> GET_ALL GET /statuses UPDATE PUT /statuses/<id> V PARTIAL_UPDATE POST /statuses/<id> Patch of V DELETE DELETE /statuses/<id> FINDER GET /statuses?q=search&keywords=li nkedin ACTION POST /statuses?action=purge Dynamic Record
  • 16. 16Rest.li: RESTful Service Architecture at Scale Rest.li Resource Method (Batch) Rest.li Request HTTP Method URI Example Entity Body BATCH_CREATE POST /statuses {“elements”: […]} BATCH_GET GET /statuses?ids=List(1,2,3) BATCH_UPDATE PUT /statuses?ids=List(1,2,3) {“entities”: {“1”: {…}, “2” : {…}} BATCH_PARTIAL_UPDATE POST /statuses?ids=List(1,2,3) {“entities”: {“1”: {“patch”: {…}}, “2” : {“patch”: {…}} BATCH_DELETE DELETE /statuses?ids=List(1,2,3)
  • 17. 17Rest.li: RESTful Service Architecture at Scale What Server Code I Need to Write
  • 18. 18Rest.li: RESTful Service Architecture at Scale Data Schema 1 Resource Class Resource Class Resource Class 3 Record Template 2 Generated 4 REST IDL Generated Developer Flow
  • 19. 19Rest.li: RESTful Service Architecture at Scale REST Spec IDL Service Metadata PDSC curl -X OPTIONS <ServiceEndpoint>
  • 20. 20Rest.li: RESTful Service Architecture at Scale Data Schema 1 Server 5 Resource Class Resource Class Resource Class 3 Client 7 Record Template 2 Generated 4 REST IDL Generated 6 Type-Safe RequestBuilders Generated Developer Flow
  • 21. 21Rest.li: RESTful Service Architecture at Scale What Client Code I Need to Write Photo newPhoto = new Photo() .setTitle("NewPhoto") .setFormat(PhotoFormats.PNG) .setExif(new Exif().setLocation(newLocation)); Request<?> createReq = new PhotosRequestBuilder().create().input(newPhoto).build()build(); POST /photos -H "Content-Type: application/json” -d {"title":"NewPhoto", "format": "PNG", "exif":{"location":...}} Strong-typed data model (auto-generated) Type-safe request builders (auto-generated)
  • 22. 22Rest.li: RESTful Service Architecture at Scale Data Schema 1 Server 5 Resource Class Resource Class Resource Class 3 Client 7 Record Template 2 Generated 4 REST IDL Generated Human Readable Documentation Generated 8 6 Type-Safe RequestBuilders Generated Developer Flow
  • 23. 23Rest.li: RESTful Service Architecture at Scale Rest.li API Hub • Lucene Powered full-text search • Data model and interface documentation • Sample request generation • Live interaction with real REST endpoint
  • 24. 24Rest.li: RESTful Service Architecture at Scale SATURN 2018 Backward Compatibility  Build-time Compatibility Checker  equivalent - No changes to Resources or Data model.  backwards - Changes considered backwards.  ignore - Log the changes but allow changes to pass.  off - The compatibility checker will not be run at all.
  • 25. 25Rest.li: RESTful Service Architecture at Scale
  • 26. 26Rest.li: RESTful Service Architecture at Scale Rest.li Server Stack Rest.li Data Layer and RESTful Operations.li Inversion-of-Control D2 Dynamic Discovery and Load Balancing Fault tolerance R2 Request/Response and Network Communication Async and Non-blocking
  • 27. 27Rest.li: RESTful Service Architecture at Scale Request Response (R2) Non-Streaming Streaming Message Communication Abstraction over HTTP
  • 28. 28Rest.li: RESTful Service Architecture at Scale SATURN 2018 Dynamic Discovery (D2)  Apache Zookeeper (Announce and Discover)  Horizontal Scaling  Services, Servers (ephemeral nodes) and Clusters  Client side load balancing  Fault tolerance  Backup request (tail-latency)
  • 29. 29Rest.li: RESTful Service Architecture at Scale Rest.li Client Implement and Interpret Rest.li Protocol (Pluggable Transport Client)
  • 30. 30Rest.li: RESTful Service Architecture at Scale Rest.li Client/Server Data Flow
  • 31. 31Rest.li: RESTful Service Architecture at Scale Open and Extensible  Filter Chain pipeline  Polyglot transport client  Custom Codec (JSON, PSON, XML, etc)
  • 32. 32Rest.li: RESTful Service Architecture at Scale Rich Eco-Systems C# Java Android Java Objective-C Swift Python
  • 33. 33Rest.li: RESTful Service Architecture at Scale Deco Describe WHAT data you want, let Deco worry about HOW to fetch them posts profiles Companies
  • 34. 34Rest.li: RESTful Service Architecture at Scale Deco Concept /posts /companies /profiles
  • 35. 35Rest.li: RESTful Service Architecture at Scale Deco Concept deco://posts/123?projection=(title, content, author~(firstName, lastName, company~(companyName)))  Projection  Decoration  Resolution Normalized Domain Modeling
  • 36. 36Rest.li: RESTful Service Architecture at Scale
  • 37. 37Rest.li: RESTful Service Architecture at Scale SATURN 2018 Asynchronous Challenges  Callback Hell  Thread Safety  Indeterminism  Debugging  How to achieve all-async?
  • 38. 38Rest.li: RESTful Service Architecture at Scale SATURN 2018 ParSeq Concept Promise<T> Represents state of an asynchronous operation, also a container for an asynchronous operation result. Task<T> Main abstraction unit representing an asynchronous operation. Task Callable Main ingredient of Task<T>. It starts asynchronous operation and returns a Promise<T> representing the operation state. Engine Harness to run a Task<T>.
  • 39. 39Rest.li: RESTful Service Architecture at Scale SATURN 2018 ParSeq Programming Model  Declarative and Functional (What not How)  Task composition, chaining, transformation and error handling  Task.async (task creation)  Task.par (parallel composition)  task.andThen (sequential composition)  task.map and task.flatMap (transformation)  task.recover and task.withTimeout (error handling)  Ends up with one Task (execution plan) submitted to ParSeq engine to execute
  • 40. 40Rest.li: RESTful Service Architecture at Scale SATURN 2018 ParSeq Execution Model  All task Callables are executed sequentially, such that Callable of a previous Task in a sequence “happens before” (in Java Memory Model sense) Callable of a next Task.  No need to be concerned about:  Synchronized, locks, volatile, atomic, etc  Immutability  3rd party libraries  Thinking about Java Memory Model  Thread safety does not mean atomicity Task<Response<T>> task = Task.async(name, () -> sendRequest(request, requestContext)); Callable
  • 41. 41Rest.li: RESTful Service Architecture at Scale SATURN 2018 Seamless Integration with Rest.li  Rest.li supports resource method returning Task.  ParSeqRestClient provides API to create Task from a rest.li request @RestMethod.Get public Task<Greeting> get(final Long id) { // rest.li resource implementation } public <T> Task<Response<T>> createTask(final Request<T> request, final RequestContext requestContext) { }
  • 42. 42Rest.li: RESTful Service Architecture at Scale ParSeq Trace Visualization Power Tool for Debugging
  • 43. 43Rest.li: RESTful Service Architecture at Scale SATURN 2018 Multi-Language Support C# Java Android Java Objective-C Swift Python
  • 44. 44Rest.li: RESTful Service Architecture at Scale SATURN 2018 Comparison Matrix Rest.li Jersey DropWizard Olingo (Odata) Data Schema yes no no yes Data Model Auto Manual Manual Manual Server Coding URI Pattern Uniform Non-Uniform Non-Uniform Non-Uniform Batch Support Yes No No Yes Service Metadata Yes No No Yes Dynamic Discovery/Load balancing Yes No No No Documentation Yes No No No Service Catalog Yes No No No Graph Query Yes No No Yes
  • 45. 45Rest.li: RESTful Service Architecture at Scale SATURN 2018 Future of Rest.li  Object Streaming in Rest.li Layer  Unstructured data support  Multi-language support enhancement  Go  Cross-language test suites  Open source Deco  ParSeq  Collection API  ParSeq Trace Analytics
  • 46. 46Rest.li: RESTful Service Architecture at Scale SATURN 2018 Useful Links  Rest.li (https://github.com/linkedin/rest.li)  Rest.li API Hub (https://github.com/linkedin/rest.li-api-hub)  ParSeq (https://github.com/linkedin/parseq)

Notes de l'éditeur

  1. 5/9/2018
  2. Let’s first brief history of Rest.li. Before rest.li is born, LinkedIn service architecture is Monolithic and Layered. With service expansion and quick iteration, this Monolithic architecture brought too many headaches to us.
  3. To address these painpoints, we are attempting a paradigm shift from this monolithic architecture to modern microservices architecture. We are setting high expectations to our paradigm shift effort.
  4. However, It is quite challenging to meet such high expectations. First, we needed a standard way to describe the resources available in our service-oriented architecture. We wanted to focus on standardizing common API operations, while still allowing developers to create non-standard operations. Secondly, we would like to enable consistent access pattern by diverse clients written in any language Third, we needed our solution to be scalable and asynchronous. Many of our services receive several thousand queries per second per instance, and we needed our solution to work under that sort of load. Fourth, We wanted to ensure that writing APIs would be fast and easy for our developers, even if they are not well-versed with REST. Lastly, We also needed our system to support long-term evolution and growth of our interfaces.
  5. Rest.li is our key secret weapon for us to successfully move from Monolithic architecture at LinkedIn to Data Model centric microservices architecture.
  6. Rest.li is a REST framework that was created by LinkedIn and open sourced in 2012. It allows you to build rich, robust, scalable RESTful services using async, non-blocking I/O. It also works beautifully at scale by promoting CLEAN REST practices. Here, I particularly highlighted several keywords here: Clean REST practices/ Scalable and Open Source. Next, I will elaborate these two important points that differentiate Rest.li from other REST framework on the market.
  7. First, Clean REST practice.
  8. There is a famous Chinese Proverb: nothing can be accomplished without norms or standards. This is particularly true in building a large-scale microservice infrastructure. Rest.li aims to set these norms and standards by promoting clean REST practices from the following 3 aspects: End-to-End developer workflow, Online API hub for service documentation and catalog, and Compatibility checker controlled API evolution.
  9. So what does the end-to-end developer flow look like to build a rest.li application? Step 1. The first step is to define your data schema using Pegasus Data Schemas we call it PDSC. The Pegasus Data Schema format uses a simple Avro-like syntax.
  10. Consistent data modeling plays an important role to achieve data model centric microservices infrastructure.
  11. This is a simple data schema file example.
  12. Step 2, Rest.li code generator will automatically create a language-specific POJO-like class that represent the data model defined in Step 1, we call these POJO-like class RecordTemplate. These RecordTemplate classes serve as language bindings of the data model in both the server and client. Step 3, Developers now need to implement server REST Resource classes and define the operations they support. Rest.li provides a set of annotations and base classes that allow you to map Resource classes to REST endpoints and to specify methods of your Resource classes to respond to REST operations, such as GET or PUT. Your Resource classes are expected to use or return data using instances of the RecordTemplate classes generated in Step 2.
  13. Rest.li is intended to foster Uniform interface design, which makes developing a Restful service like a breeze even for non REST expert developers. Rest.li framework provides some standardized REST resource pattern and a set of annotations for developers to define RESTful endpoints and operations supported. We have categorized typical RESTful operations into CRUD, Finder, Action and BATCH to both standardize common API operations while still allowing developers to create non-standard operations. RESTful interface contract is formally communicated btw client and server through a well-defined IDL format.
  14. Each Rest.li resource endpoint is one of the following types: Collection – used to model key-vale map of entities. Simple – used to model singleton in a certain scope. Association – used to model relationships between entities. Action – used to accommodate some non-standard REST behaviors. Additionally, each resource endpoint may be a sub-resource of any other resource. We have provided out-of-box resource template class and interface with well-defined annotations for developer to customize for their own resource.
  15. Resources are the nouns in the Rest.li world, and Resource Methods are the verbs. Rest.li provides a standard set of Resource Methods that describe what clients may do with the resources. These verbs are different than the HTTP verbs (GET, POST, PUT, DELETE, and so on) but map onto them (CREATE is a POST, UPDATE is a PUT, FINDER is a GET). ACTION are provided for users to define some non-standard functions and actions.
  16. Rest.li also provided batch versioned resource methods to reduce round-trip to servers for performance consideration. Server developers can use BATCH to do some performance optimization like cache, database batch call, etc.
  17. With resource pattern and interface method standardized, server developer’s task to introduce a new rest.li endpoint becomes a piece of cake. You only need to extend a proper ResourceTemplate, and implement those resource methods you are planning to support using those POJO-like RecordTemplate classes.
  18. Step 4, After resources are implemented by developers, Rest.li will automatically generate an interface description (IDL) file that is considered the source of truth for the interface contract between the server resource implemented in Step 3 and its clients. The IDL itself is a language-agnostic JSON format.
  19. After resource class is implemented, our gradle build process will automatically generate a REST spec IDL file for your service which together with your data model PDSC file constitutes your official service metadata and contract to external parties. This service metadata can be dynamically discovered by your external parties using OPTIONS call.
  20. Step 5, Developer now can create your server application, which involves leveraging a few Rest.li classes to instantiate and configure the Rest.li server. Now we are done with server implementation, can move to client side development. Step 6, Based on REST IDL generated in step 4, Rest.li can auto-generate classes known as RequestBuilders that are used by clients to create requests to send to the server. Step 7, Together with the RecordTemplates, RequestBuilders provide convenient and type-safe mechanisms for developers to write client applications to working with the data models supported by the server by issuing rest.li request to rest.li server.
  21. Now let’s look at how client developers need to interact with this service. For example, photos is a resource server developer just exposed and which supports CREATE operation, we need to create a new Photo. To perform this operation, this is the client code you need to write. Auto-generated strong-typed Data model binding and type-safe request builders hide over-the-wire rest.li protocol from client developers and make it quite easy to send a valid rest.li request.
  22. Last but not the least, Rest.li uses this IDL along with the original data schema files to support automatically generating human-readable documentation, which can be requested from a server. This documentation is also used by rest.li framework to check backward compatibility during API evolution. Blue boxes represent code developers need to write, all other components are auto-generated from rest.li framework. Throughout this end-to-end developer flow, we are trying standardizing developer practice through 3 aspects: Consistent data modeling, Uniform interface design, Type-safe client bindings.
  23. To foster cooperation and cross-service communication in a large-scale system, an on-line service catalog becomes an indispensable part to facilitate smooth integration. Rest.li API Hub is a WEB-UI to browse and search a catalog of rest.li APIs along with their on-line documentation. It is empowered with the following features: Lucene powered full-text search. Data model and service interface documentation. Sample request generation and live interaction with real REST endpoint.
  24. Last point in CLEAN REST practice: service backward compatibility. In Restful world, API is not static and needs to evolve based on business needs. This may create many integration headaches if not done properly due to backward-incompatibility issues. Rest.li framework comes with a build-time compatibility checker to provide both data moel and interface compatibility issue at build-time and provide early warnings to API developers before real deployment based on their configured compatibility settings. We support various compatibility settings like equivalent, backwards, ignore, and off.
  25. Now let’s move to second highlight of Rest.li: Scale and Open. This will delve a bit into Rest.li server stack.
  26. Rest.li server stack consists of three layers: First: Rest.li layer. It includes data model layer and RESTful operations exposed through our uniform interface design. Internally it is a Java framework based on inversion-of-control model, which handles most of the data flow and client/server interaction transparently and calls code you supply at the appropriate time. Second: D2 layer. This is our dynamic discovery and client-side load balancing layer. This layer is optional for smaller service, but provide fault tolerance and scalability to large-scale service infrastructure like LinkedIn. Last: R2 layer. This is our transport layer abstraction in Java. It exposes a low-level message style communication inspired by HTTP and a high-performance asynchronous/non-blocking API. We have covered most stuff in Rest.li layer in explaining end-to-end developer flow, which makes Rest.li developer-friendly. Now let’s see how we achieve scalability through R2D2 layer.
  27. R2 stands for Request Response. It is a java abstraction of message-style communication over HTTP. To provide a high-performance asynchronous API with non-blocking back-pressure support, we would like a streaming model illustrated in the right figure, where the body of the request/response could be processed one chuck at a time, instead of fully buffered in the left. This is the EntityStream API embedded in our R2 implementation.
  28. D2 is our dynamic discovery layer built on top of Apache ZooKeeper. By dynamic discovery I mean that clients do not have to hardcode server URLs in their code. Clients simply need to specify what service or resource they want to talk to and D2 converts that name into a URL and directs to the right host. By grouping services into clusters and dynamic service announcement and shutdown with zookeeper ephemeral nodes, we can perform client side load-balancing, and backup request to improve tail-latency, of course, fault tolerance when one server is not healthy. We use ZooKeeper as a registry for information about available services and the hosts that provide them. Clients automatically get the latest information from ZooKeeper and apply load-balancing algorithms on the client side to distribute load evenly across servers and reduce load to overloaded servers. This layer is optional for smaller service, but provide fault tolerance and scalability to large-scale service infrastructure like LinkedIn.
  29. Rest.li defined its own over-the-wire protocol. It provides a client-side framework to shield this low-level wire protocol from users. Apart from type-safe client bindings we have mentioned earlier, the core of Rest.li client framework is RestLiClient interface with its pluggable transport client architecture. This RestLiClient interface will implement and interpret Rest.li protocol and delegate to a pluggable transport client (for example, Netty client, etc) to transport HTTP request/response message to and back from server.
  30. This diagram provides a high-level view of the interaction and data flow between a Rest.li client and server. The yellow arrows indicate the flow of requests out of the client and into the server, while dark blue arrows represent the server’s response.
  31. As an open source project, we are striving to provide great room for customization and extension. It provides several extension points for customization: Filter chain pipeline. Rest.li provides a mechanism to intercept incoming requests and outgoing responses via filters. Each filter contains methods that handle both requests and responses. Filter can be at both Rest.li level (dealing with data model entities) or at R2 level (dealing with Bytes) Polyglot transport client bindings. R2 layer only defined a message communication abstraction. Default out-of-box rest.li supports Netty client, Jetty server, but you are freely plug into their own transport client and servlet container. Custom codec. Out-of-box rest.li supports JSON and PSON content encoding, but it provides pluggable framework for you to easily plugin your custom codec to encode/decode request/response in your preferred format, for example, XML, etc.
  32. Building a large-scale application cannot be limited to just send individual REST requests. Most likely, you may need to deal with heterogenous sub-systems implemented in diverse languages, struggle to build a presentable view on top of our data models, also need to consider executing some downstream calls in some sequential/parallel fashion with complex workflow dependency. This is why our rich eco-systems built around Rest.li come into help, specifically, Deco, ParSeq, and Multi-language support.
  33. What is Deco? The name “Deco” comes from the word ”Decoration”. It is a Java library for projecting fields and fetching interconnected rest.li resources based on a specified query. It was designed to allow users to “Describe WHAT data you want, let Deco worry about HOW to fetch them”. Here shows a perfect real scenario where Deco is used inside LinkedIn to display a post created by a LinkedIn member.
  34. To fetch data for such a Post, we need to make service calls to 3 different services (/posts, /profiles, /companies), and somehow massage data into such a data model view. Deco provides a special query language (deco recipes) to achieve this.
  35. The deco recipe will be like this. Deco supports field projection, field decoration, and FK resolution. All these features enabled us for normalized domain modeling to avoid data redundancy.
  36. Next, I would like to introduce another rest.li eco-system: ParSeq which is also open sourced and well-adopted at LinkedIn in both frontend and midtier services. This is a framework to make asynchronous Java easier. It is well known fact that writing and using asynchronous code is tricky. It often adds accidental complexity that is unrelated to app or business logic code, but rather caused by the asynchronous mechanism.
  37. It is well known fact that writing and using asynchronous code is tricky. It often adds accidental complexity that is unrelated to app or business logic code, but rather caused by the asynchronous mechanism. Here lists many buzz words (Callback hell, Thread safety, Indeterminism, Async debugging) about pain-points of developing all-async code flow. Asynchronous operations can finish in different orders because the underlying operations can take varying amounts of time. This is due to both expected causes like fetching more data and executing more time-consuming operations, and unexpected and problematic causes such as garbage collection or server load. This behavior makes asynchronous operations hard to test and adds elements of non-determinism to asynchronous code. Meaningless stack traces make debugging difficult, and coordinating callbacks on various threads is challenging. Furthermore, developers must always consider the issue of thread safety. These factors, among others, make asynchrony quite complex.
  38. To address these challenges, we built ParSeq to make programming using asynchronous methods in Java easier. Here lists several main concepts in ParSeq. Task: It is the main abstraction unit in ParSeq system to represent an asynchronous operation whose result can be set asynchronously. Promise: represents the state of an asynchronous operation and also a container for the operation result. Task Callable: it is the main ingredient of Task, which is like a Callable starting asynchronous operation and returns a Promise. Finally, ParSeq engine, which is a harness responsible for running a Task.
  39. ParSeq programming model is declarative and functional, focusing on What you want to do, not How you get that. It provides developer-friendly API for coordinating asynchronous tasks, for example, task creation, parallel/sequential composition, transformation, error handling, etc. It ends up with one Task (internally represented as an execution plan) that will be submitted to ParSeq Engine to execute.
  40. ParSeq execution model maintains a very important invariant to relieve developers from various asynchronous programming concerns. That is, All Task Callables are executed sequentially, such that Callable of a previous Task in a sequence “happens before” (in Java Memory Model sense) Callable of a next Task. This is a real example of Task Callable from sending a rest.li request.
  41. Another valuable feature provided by ParSeq is its built-in tracing and visualization. This is very important for service monitoring and observability. Trace visualization provides us a powerful tool for debugging some problematic application flow, also can provide us performance metric on each step in your business logic.
  42. To promote our clean REST practices to heterogenous system and diverse clients, we are striving to expand our multi-language support. So far, we have already had support for Java desktop, Android Java, Objective-C/Swift for iOS, Python, C# support. Go client binding support is working in progress.
  43. Here is an overview matrix of comparing Rest.li framework with other popular RESTful framework on the market. I just picked some representatives, but they are more or less similar.
  44. The rest.li framework has been proven to create robust RESTful service architectures that can work at scale from its wide adoption at LinkedIn and Coursera.  There are still many interesting items on our road map, for example: - Object streaming and unstructured data support in rest.li Multi-language suppprt enhancement including Go and cross-language test suites. Open source our Deco library. In ParSeq, we are planning to enhance it with friendly collection API, and also power ParSeq Tracing with Intelligent data Analytics. We are looking forward to contributions from our audience here.