HyperConnect's 1to1 video matchmaking system is consist of various machine learning techniques to maximize user satisfaction. Our matchmaking system manages large user context containing actions a few seconds ago, and reacts in milliseconds to produce meaningful new results in each user session. It's difficult in traditional way. So, distributed streaming is essential to handle in this cases. Topics include: - Why our team choose Apache Flink in comparison with alternatives - Matchmaking streaming architecture with detail abstraction levels based on Flink operator - Pairwise scoring microservice management with Flink - Stateful matchmaking computation with low latency, fault-tolerance, and scalability - How to manage large-scale events: classifying feature types, collecting with a multi-window stream - Applications: personalization, multi-armed-bandit on stream.

1. Data Driven Matchmaking Streaming
at Hyperconnect
Gihoon Yeom
Jaehyeuk (Jacob) Oh
2020 Flink Forward Virtual Conference

2. Motivation
HyperMatch Architecture
Infra & Operation
Stateful Matchmaking
Contents

3. Video and AI Company
Introduction

4. Scalability
5000 Match Requests
Stability
Stateful matchmaking
Motivation

5. Why Flink?
● Capability for Low-latency iteration
● Loosely coupled architecture
● Intrinsic support for a state management

6. Motivation
HyperMatch Architecture
Infra & Operation
Stateful Matchmaking
Contents

7. Matchmaking Steps
- Collect match requests from specific units of windows
and select the best pairs among them

8. Matchmaking Steps
Input Source
Feature
Engineering
Pair
Generator
Match
Maker
Feedback System
State
Management
Data Source
Leftovers
Scoring Ops
Matchmaking
Results
Multiple Sink
Score
Microservices

9. Matchmaking Steps
Input Source
Feature
Engineering
Pair
Generator
Match
Maker
Feedback System
State
Management
Data Source
Leftovers
Scoring Ops
Matchmaking
Results
Multiple Sink
Score
Microservices

10. Matchmaking Steps
Feature
Engineering
Pair
Generator
Match
Maker
Feedback System
State
Management
Leftovers
Scoring Ops
Matchmaking
Results
Multiple Sink
Input SourceData Source
Score
Microservices

11. Matchmaking Steps
- Option 1. Built-in Kafka source
- To simulate previous matchmaking steps
- Reduce dependencies between applications
- Option 2. Custom Netty Http Source
- For real-time latency
- Apache Bahir extension

12. Matchmaking Steps
Feature
Engineering
Pair
Generator
Match
Maker
Feedback System
State
Management
Leftovers
Scoring Ops
Matchmaking
Results
Multiple Sink
Input SourceData Source
Score
Microservices

13. Matchmaking Steps
- Preprocess data stream before entering match cycle
- Conditional tagging for segmentations (ex) A/B test)
- Vectorize user features for machine learning models
- Save managed state on operator and connect it to feature engineering
operator

14. Matchmaking Steps
- Split match request attributes information
1) Match context infos
- can be changed in matchmaking cycle
- small size
- direct transfer to next operator
2) Match properties
- properties cannot be changed
- large size
- broadcast to only scoring operators
Total Feature memory
Small Mutable Context
Large Immutable Properties

15. Matchmaking Steps
Feature
Engineering
Pair
Generator
Match
Maker
Feedback System
State
Management
Leftovers
Scoring Ops
Matchmaking
Results
Multiple Sink
Input SourceData Source
Score
Microservices

16. Matchmaking Steps
- Aggregate by window
- Global tumbling window
- Keyed tumbling window when segmentation enabled
- Custom trigger & evictor depending on matchmaking function
- Custom pair generating protocol with scoring operators

17. Matchmaking Steps
- Efficient pair generation and bucketizing considering
communication costs
- Total communication costs
= network consume time
+ serialization/deserialization time

18. Matchmaking Steps
Feature
Engineering
Pair
Generator
Match
Maker
Feedback System
State
Management
Leftovers
Scoring Ops
Matchmaking
Results
Multiple Sink
Input SourceData Source
Score
Microservices

19. Matchmaking Steps
Matchmaking logic is divided into several microservices
- Many teams can be involved for scoring logic advancement
- Scoring Operator with broadcasted request data or external in-memory cache
- Prepare data to be provided to each microservices
- Get responses from microservices
- Scoring Microservices can be REST or Event driven
- Use AsyncDataStream

20. Matchmaking Steps
Feature
Engineering
Pair
Generator
Match
Maker
Feedback System
State
Management
Leftovers
Scoring Ops
Matchmaking
Results
Multiple Sink
Input SourceData Source
Score
Microservices

21. Matchmaking Steps
Fault tolerance on scoring microservices
- Custom trigger & evictor
- Distributed sorting algorithm for finding best pairs
- IterativeStream for leftovers

22. Matchmaking Steps

23. Matchmaking Steps
- Trigger condition: Timeout or all candidates arrived
- Timeout dummy token sent by generating operator for fault tolerance
ScorersPair
Generator
Match
Maker
Leftovers
Send Timeout Token
for custom trigger

24. Matchmaking Steps
- Create app level backpressure by buffering some requests when scorer fails
consecutively
ScorersPair
Generator
Match
Maker
Leftovers
Too many
leftovers!
Backpressure:
Buffering
selected requests
Some pair scorer failed
=> Increase requests wasn’t matched
=> Increase next match requests
=> Repeated.. => Service failed

25. Matchmaking Steps
Feature
Engineering
Pair
Generator
Match
Maker
Feedback System
State
Management
Leftovers
Scoring Ops
Matchmaking
Results
Multiple Sink
Input SourceData Source
Score
Microservices

26. Matchmaking Steps
- Matchmaking broker
- Feedback controller: distribution of results
- User state management
- Logging handler
- Metric reporter

27. Motivation
HyperMatch Architecture
Infra & Operation
Stateful Matchmaking
Contents

28. Infra & Operation
- Easy to set up new staging infra
- For performance testing
- Metrics on prometheus/grafana
Match App1
Job manager
Task manager
Match App2
Job manager
Task manager Score App2
Score App1
HA Proxy
ACTIVE
STANDBY
- For zero downtime: Blue green deployment
- Score microservices can be rolling updated

29. Performance Tuning
Network cost & General tips
- Latency vs Throughput trade-off: setBufferTimeout
- Forward vs Rebalance: setParallelism
- For resource isolation: stream.slotSharingGroup

30. Motivation
HyperMatch Architecture
Infra & Operation
Stateful Matchmaking
Contents

31. Stateful MatchMaking
Realtime State + Realtime Inference
Stateful Matchmaking