SlideShare une entreprise Scribd logo
1  sur  55
Asynchronous
Programming
With Spring 4.X, RDBMS in MSA
• Dooray! Mail-API 개발, 운영
• www.pikicast.com 개발, 운영
• Cloud orchestration system 개발, 운영
• Caller-Ring 개발, 운영
“Non-Blocking” or “Blocking”
vs
“Asynchronous” or “Synchronous”
Return 의 여부에 따라…
Thread 상태에 따라서…
“The C10K Problem”
http://www.kegel.com/c10k.html
It’s time for web servers to handle ten thousand clients simultaneously.
The web is a big place now
The C10K Problem
• Scale-out
• Load-balancer
• Haproxy, Nginx
• AWS, Azure, Google Cloud and OpenStack
• NoSQL
• Redis, HazelCast (IMDG)
• Zero-copy, limits on file descriptors
“그러면 프로그래밍의 패러다임
은?”
The C10K Problem
• Event Loop
– Nginx, node.js, Play, Netty, Vert.X
• Event Driven Programming
• Reactor
• Java Executor Service
Event
Loop
Thread PoolEvent Queue
The C10K Problem
• Callback Functions
• Functional Programming vs imperative programming
• Lambda in Java
– Functional programming
– Modern Java
Event
Loop
Thread PoolEvent Queue
Register Callback
Complete CallbackTrigger Callback
The C10K Problem
• 객체 지향형 프로그래밍
• 함수형 프로그래밍
Event
Loop
Thread PoolEvent Queue
Th
#1
Th
#2
Th
#3
Th
#1
Th
#2 VS
얼마나 ‘효율적으로’, ‘효과적’으
로
Thread를
짧게 나눠쓰느냐??”
비동기 회고 발표자료
쓰기 어렵죠?
Asynchronous Programming
with Spring 4.3
• DeferredResult
• @Async
• AsyncRestTemplate & ListenableFuture
• CompletableFuture
@Async & DeferredResult
example
Sample : AsyncController.java
단점은?
Common Architecture using JAVA
“그런데 JDBC 는 blocking IO
라며?”
“왜 비동기 프로그래밍을 할까?”“하지만 많은 기능을 구현할 수
있습니다.”
비동기 회고 발표자료
“휴지통에 있는 메일을 삭제하
는 API”
- 데이터 베이스에 record 삭제
- 메일 파일(Mime) 삭제
- 타 시스템에 메일이 삭제 되었
음을 알려줌.
검색서버, History 서버
그런데…
휴지통에 1만건이 존재한다면?
- DB 작업 : 300ms- 파일 삭제 작업 : 10ms *
10,000 = 100sec
- 타시스템 연동작업 : 30ms *
10,000 * 3 = 900 sec
Total : 약 1000 sec, 16분
@Async
• @Async 를 쓸때는 별도의 ThreadPool 을 사용하자.
• @Async 와 @Transactional 을 쓸때는 주의하자.
– @Transactional 의 원리를 이해하고 사용해야 함.
– 부하
@Transactional
• AOP-Proxy 에서는 public method 로 선언
– @EnableTransactionManagement(proxyTargetClass = true)
– <tx:annotation-driven/>
– AspectJ 에서는 상관없음.
• Bean 내부의 다른 method에서 다른 Transaction을 실행할때
– 선언적 Transaction 을 실행한다.
– Self-autowiring in Spring 4.3+
– @Async 에서 많이 놓치는 실수.
“DB Connection 객체는 비싸
다!”
비동기 프로그래밍에서도 마찬
가지 입니다.
@Transactional(readOnly = false)
public boolean removeMailsByMailFolderId(Long mailFolderId){
List<Long> mailIds = mailRepository.findIds(mailFolderId);
mailRepository.deleteByMailFolderId(mailFolderId);
mailIds.stream.forEach(mailId -> {
storageComponent.deleteByMailId(mailId);
searchComponent.deleteByMailId(mailId);
historyComponent.deleteByMailId(mailId);
);
return true;
}
Th#1
Storage
Search
History
DB
@Transactional(readOnly = false)
public boolean removeMailsByMailFolderId(Long mailFolderId){
List<Long> mailIds = mailRepository.findIds(mailFolderId);
mailRepository.deleteByMailFolderId(mailFolderId);
mailIds.stream.forEach(mailId -> {
storageComponent.deleteByMailId(mailId);
searchComponent.deleteByMailId(mailId);
historyComponent.deleteByMailId(mailId);
);
return true;
}
실행시간 = TX 시간
Long Transaction
End Transaction
Begin Transaction
Th#1
Storage
Search
History
DB
장애유발
가능성
DB Connection을 아끼는 방법들
• @Async
• LazyConnectionDataSourceProxy
• Facade Pattern
• TransactionSynchronizationManager,
TransactionSynchronizationAdaptor
@Async 를 이용한 예제
@Transactional(readOnly = false)
public void removeMailsByMailFolderId(Long mailFolderId){
List<Long> mailIds = mailRepository.findIds(mailFolderId);
Lists.partition(mailIds, 10).stream
.forEach(submailIds -> selfService.removeByAsync(submailIds);
}
@Async
@Transactional(readOnly = false)
public void removeByAsync(List<Long> submailIds){
mailRepository.deleteByMailIdIn(subMailIds)
subMailIds.stream.forEach(mailId -> {
storageComponent.deleteByMailId(mailId);
searchComponent.deleteByMailId(mailId);
historyComponent.deleteByMailId(mailId);
);
}
Th#1
DB
Th#4
Th#3
Th#2
Storage
Search
History
DB
DB
DB
@Transactional(readOnly = false)
public void removeMailsByMailFolderId(Long mailFolderId){
List<Long> mailIds = mailRepository.findIds(mailFolderId);
mailRepository.deleteByMailIdIn(mailIds)
selfService.removeByAsync(mailIds)
}
@Async
public void removeByAsync(List<Long> mailIds){
mailIds.stream.forEach(mailId -> {
storageComponent.deleteByMailId(mailId);
searchComponent.deleteByMailId(mailId);
historyComponent.deleteByMailId(mailId);
);
}
Th
#1
Th#1
DB
Th#4
Storage
Search
History
LazyConnectionDataSource 예
제
@Bean(value = “dataSource”, destoryMethod=“close”)
public DataSource dataSource(){
BasicDataSource ds = new BasicDataSource();
//… more configuration
return ds;
}
@Bean(value = “lazyDataSource”)
public LazyConnectionDataSourceProxy lazyDataSource(){
return new LazyConnectionDataSourceProxy(datasource());
}
@Transactional(readOnly = false)
public boolean removeMailsByMailFolderId(Long mailFolderId){
List<Long> mailIds = mailRepository.findIds(mailFolderId);
mailRepository.deleteByMailFolderId(mailFolderId);
mailIds.stream.forEach(mailId -> {
storageComponent.deleteByMailId(mailId);
searchComponent.deleteByMailId(mailId);
historyComponent.deleteByMailId(mailId);
);
return true;
}
After
What is Difference?
Before
Façade Pattern 을 이용한 예제
public class MailFacade {
public void removeMailsByMailFolderId(Long mailFolderId){
List<Long> removedMailIds = mailService.findIds(mailFolderId);
removedMailIds.stream.forEach(mailId -> {
storageComponent.deleteByMailId(mailId);
searchComponent.deleteByMailId(mailId);
historyComponent.deleteByMailId(mailId);
);
}
}
public class MailServiceImpl implements MailService {
@Transactional(readOnly = false)
public List<Long> removeMailEntities(Long mailFolderId){
List<Long> mailIds = mailRepository.findIds(mailFolderId);
mailRepository.deleteByMailIdIn(mailIds)
return mailIds
}
}
Façade Layer
Service Layer
TransactionSynchronizationMa
nager
TransactionSynchronizationAd
aptor
를 이용한 예제
@Async
@Transactional(readOnly = false)
public void removeByAsync(List<Long> submailIds){
mailRepository.deleteByMailIdIn(subMailIds)
subMailIds.stream.forEach(mailId -> {
storageComponent.deleteByMailId(mailId);
searchComponent.deleteByMailId(mailId);
historyComponent.deleteByMailId(mailId);
);
}
DB Transaction
이 묶일 필요가
있나요?
@Transactional(readOnly = false)
public void removeByAsync(List<Long> submailIds){
mailRepository.deleteByMailIdIn(subMailIds)
TransactionSynchronizationManager.registerSynchronization(new TransactionSynchronizationAdapter() {
@Override
public void afterCommit() {
subMailIds.stream.forEach(mailId -> {
storageComponent.deleteByMailId(mailId);
searchComponent.deleteByMailId(mailId);
historyComponent.deleteByMailId(mailId);
});
}
}
}
afterCommit() – invoke after transaction commit
afterCompletion() - invoke after transaction commit / rollback
beforeCommit() – invoke before trasnaction commit
beforeCompletion() - invoke before transaction commit / rollback
“WAS 스레드, DBPool 갯수 설
정이야기 해볼까요?”
Peek Time 에 비동기로 프로그래
밍된 api 를 호출하면?
DBPool 은 괜찬은가요?
“DataSource 를 분리합니다”
DataSource Configuration
• 비동기용와 API 처리용 DataSource를 분리한다.
– Peek time 때, 비동기 프로세스는 늦게 처리되도 된다.
– 장애는 전파하지 말자.
• AbstractRoutingDataSource class
– 일종의 Proxy DataSource
– setTargetDataSources method - key 에 따른 DataSource 등록
– determineCurrentLookupKey method – key 에 따른 DataSource 획득
@Slf4j
public class ExecutionRoutingDataSource extends AbstractRoutingDataSource {
@Override
protected Object determineCurrentLookupKey() {
return ExecutionContextHolder.getExecutionType();
}
}
public class ExecutionRoutingDataSource extends AbstractRoutingDataSource {
@Override
protected Object determineCurrentLookupKey() {
return ExecutionContextHolder.getExecutionType();
}
}
@Bean("routingDataSource")
public ExecutionRoutingDataSource routingDataSource() {
Map<Object, Object> targetDataSources = new HashMap<>();
targetDataSources.put(ExecutionType.API, apiDataSource());
targetDataSources.put(ExecutionType.BATCH, batchDataSource());
ExecutionRoutingDataSource routingDataSource = new ExecutionRoutingDataSource();
routingDataSource.setTargetDataSources(targetDataSources);
routingDataSource.setDefaultTargetDataSource(apiDataSource());
return routingDataSource;
}
LookUpKey 에 따라 동적으로
DataSource 를 사용
미리 등록된 enum
ExecutionType.API or BATCH
@TransactionalExecutionType(
executionType = ExecutionType.BATCH
)
@Transactional(
readOnly = true,
isolation = Isolation.READ_COMMITTED
)
public List<CompletableFuture<List<Long>>> partition(Long taskQueueId) {
// Business logic and query to DB
}
@Target({ElementType.METHOD})
@Retention(RetentionPolicy.RUNTIME)
public @interface TransactionalExecutionType {
ExecutionType executionType() default ExecutionType.API;
}
AsyncRestTemplate
• ListenableFuture
– Callback Hell
• Factory 종류
– Netty4ClientHttpRequestFactory
– SimpleClientHttpRequestFactory
– OkHttp3ClientHttpRequestFactory
– Pros and Cons
• Backpressure, Throttling
• Circuit Break Pattern
AsyncRestTemplate 은
RestTemplate 과 같다.
Spring 의 PSA
(Portable Service Abstraction)
Sample : AsyncRestTemplateTest.java
getWords(), asyncGetWords(), asyncCombinedProcess()
AsyncRestTemplate
• Callback Hell 을 피하기 위해서는?
– 람다식
– Default Method
– CompletableFuture 객체로 랩핑하는 Util 클래스를 만든다.
– Continue..
public class CompletableFutureBuilder {
public static <T> CompletableFuture<T> build(ListenableFuture<T> listenableFuture) {
CompletableFuture<T> completableFuture = new CompletableFuture<T>() {
@Override
public boolean cancel(boolean mayInterruptIfRunning) {
boolean result = listenableFuture.cancel(mayInterruptIfRunning);
super.cancel(mayInterruptIfRunning);
return result;
}
};
listenableFuture.addCallback(new ListenableFutureCallback<T>() {
@Override
public void onFailure(Throwable ex) {
completableFuture.completeExceptionally(ex);
}
@Override
public void onSuccess(T result) {
completableFuture.complete(result);
}
});
return completableFuture;
}
public class CompletableFutureBuilder {
public static <T> CompletableFuture<T> build(ListenableFuture<T> listenableFuture) {
CompletableFuture<T> completableFuture = new CompletableFuture<T>() {
@Override
public boolean cancel(boolean mayInterruptIfRunning) {
boolean result = listenableFuture.cancel(mayInterruptIfRunning);
super.cancel(mayInterruptIfRunning);
return result;
}
};
listenableFuture.addCallback(new ListenableFutureCallback<T>() {
@Override
public void onFailure(Throwable ex) {
completableFuture.completeExceptionally(ex);
}
@Override
public void onSuccess(T result) {
completableFuture.complete(result);
}
});
return completableFuture;
}
public class CompletableFutureBuilder {
public static <T> CompletableFuture<T> build(ListenableFuture<T> listenableFuture) {
CompletableFuture<T> completableFuture = new CompletableFuture<T>() {
@Override
public boolean cancel(boolean mayInterruptIfRunning) {
boolean result = listenableFuture.cancel(mayInterruptIfRunning);
super.cancel(mayInterruptIfRunning);
return result;
}
};
listenableFuture.addCallback(new ListenableFutureCallback<T>() {
@Override
public void onFailure(Throwable ex) {
completableFuture.completeExceptionally(ex);
}
@Override
public void onSuccess(T result) {
completableFuture.complete(result);
}
});
return completableFuture;
}
CompletableFuture
• 비동기 프로그래밍을 위한 클래스
– NonBlocking 으로 처리 가능하다. (vs Future class)
– Blocking 으로도 처리 가능하다.
– 여러개의 작업들을 엮을 수 있다.
– 여러개의 작업들을 합칠 수 있다.
– 예외 사항을 처리할 수 있다.
• Implements CompletionStage<T>
– 테스크를 포함하는 모델을 의미한다.
– 테스크는 체인패턴의 기본 요소이다.
CompletableFuture
• 접미사를 확인
– thenApply() vs thenApplyAsync()
• 파라미터를 확인
– thenApplyAsync(Function<? super T, ? extends U> fn)
– thenApplyAsync(Function<? super T, ? extends U> fn, Executor executor)
• 비동기 작업 시작 method
– runAsync(), supplyAsync()
• 작업들을 연결하는 method
– thenApplyAsync(), thenComposeAsync(), anyOf(), allOf()
• 예외처리를 하는 method
– exceptionally()
CompletableFuture
Method AsyncMethod Arguments Returns
thenAccept thenAcceptAsync 이전 Stage 의 결과 Noting
thenRun thenRunAsync Noting
thenApply thenApplyAsync 이전 Stage 의 결과 Result of current stage
thenCompose thenComposeAsync 이전 Stage 의 결과 Future result of current
stage
thenCombine thenCombineAsync 이전 두 Stage 의 결과 Result of current stage
whenComplete whenCompleteAsync 이전 두 Stage 의 결과 or Exception Noting
CompletableFuture
• Building a CompletableFuture Chain
– Example. CompletableFutureTest.java chainMethods();
– Example. CompletableFutureTest.java allOfMethods();
– Example. CompletableFutureTest.java allOfAcceptMethods();
• Exception handling
– Example. CompletableFutureTest.java handleMethods();
CompletableFuture
• Thread pool 공유시
– 당연하지만 순서보장이 안될 수 있다.
– Method chaining 시 공정한 처리가 어려울수 있다.
– Starvation 가능성
– executor.setRejectedExecutionHandler();
– AbortPolicy
– DiscardPolicy
– DiscardOldestPolicy
– CallerRunsPolicy
Event
Loop
Thread PoolEvent Queue
Register Callback
Complete CallbackTrigger Callback
Transaction Saving
Programming
- Avoid MySQL Lock wait
- Avoid MySQL Dead Lock
- Query Tuning on JPA…
- 디버깅이 어렵다.
- TC를 만들기 어렵다.
- 항상 RDB 와의 싸움이다.
비동기 회고 발표자료
Q & A

Contenu connexe

Tendances

20141210 rakuten techtalk
20141210 rakuten techtalk20141210 rakuten techtalk
20141210 rakuten techtalkHiroshi SHIBATA
 
How Typepad changed their architecture without taking down the service
How Typepad changed their architecture without taking down the serviceHow Typepad changed their architecture without taking down the service
How Typepad changed their architecture without taking down the serviceroyans
 
Advanced technic for OS upgrading in 3 minutes
Advanced technic for OS upgrading in 3 minutesAdvanced technic for OS upgrading in 3 minutes
Advanced technic for OS upgrading in 3 minutesHiroshi SHIBATA
 
Full stack development with node and NoSQL - All Things Open - October 2017
Full stack development with node and NoSQL - All Things Open - October 2017Full stack development with node and NoSQL - All Things Open - October 2017
Full stack development with node and NoSQL - All Things Open - October 2017Matthew Groves
 
Fluentd and Embulk Game Server 4
Fluentd and Embulk Game Server 4Fluentd and Embulk Game Server 4
Fluentd and Embulk Game Server 4N Masahiro
 
Introduction to Apache Camel
Introduction to Apache CamelIntroduction to Apache Camel
Introduction to Apache CamelClaus Ibsen
 
London devops logging
London devops loggingLondon devops logging
London devops loggingTomas Doran
 
Altitude SF 2017: Advanced VCL: Shielding and Clustering
Altitude SF 2017: Advanced VCL: Shielding and ClusteringAltitude SF 2017: Advanced VCL: Shielding and Clustering
Altitude SF 2017: Advanced VCL: Shielding and ClusteringFastly
 
Async and Non-blocking IO w/ JRuby
Async and Non-blocking IO w/ JRubyAsync and Non-blocking IO w/ JRuby
Async and Non-blocking IO w/ JRubyJoe Kutner
 
DjangoCon 2010 Scaling Disqus
DjangoCon 2010 Scaling DisqusDjangoCon 2010 Scaling Disqus
DjangoCon 2010 Scaling Disquszeeg
 
Как сделать высоконагруженный сервис, не зная количество нагрузки / Олег Обле...
Как сделать высоконагруженный сервис, не зная количество нагрузки / Олег Обле...Как сделать высоконагруженный сервис, не зная количество нагрузки / Олег Обле...
Как сделать высоконагруженный сервис, не зная количество нагрузки / Олег Обле...Ontico
 
Apache Camel K - Copenhagen
Apache Camel K - CopenhagenApache Camel K - Copenhagen
Apache Camel K - CopenhagenClaus Ibsen
 
The details of CI/CD environment for Ruby
The details of CI/CD environment for RubyThe details of CI/CD environment for Ruby
The details of CI/CD environment for RubyHiroshi SHIBATA
 
Empowering developers to deploy their own data stores
Empowering developers to deploy their own data storesEmpowering developers to deploy their own data stores
Empowering developers to deploy their own data storesTomas Doran
 
Spring framework 4.x
Spring framework 4.xSpring framework 4.x
Spring framework 4.xArawn Park
 
Apache Camel K - Copenhagen v2
Apache Camel K - Copenhagen v2Apache Camel K - Copenhagen v2
Apache Camel K - Copenhagen v2Claus Ibsen
 
Apache Camel v3, Camel K and Camel Quarkus
Apache Camel v3, Camel K and Camel QuarkusApache Camel v3, Camel K and Camel Quarkus
Apache Camel v3, Camel K and Camel QuarkusClaus Ibsen
 
Integrated Cache on Netscaler
Integrated Cache on NetscalerIntegrated Cache on Netscaler
Integrated Cache on NetscalerMark Hillick
 

Tendances (20)

20141210 rakuten techtalk
20141210 rakuten techtalk20141210 rakuten techtalk
20141210 rakuten techtalk
 
How Typepad changed their architecture without taking down the service
How Typepad changed their architecture without taking down the serviceHow Typepad changed their architecture without taking down the service
How Typepad changed their architecture without taking down the service
 
Os Webb
Os WebbOs Webb
Os Webb
 
Advanced technic for OS upgrading in 3 minutes
Advanced technic for OS upgrading in 3 minutesAdvanced technic for OS upgrading in 3 minutes
Advanced technic for OS upgrading in 3 minutes
 
Full stack development with node and NoSQL - All Things Open - October 2017
Full stack development with node and NoSQL - All Things Open - October 2017Full stack development with node and NoSQL - All Things Open - October 2017
Full stack development with node and NoSQL - All Things Open - October 2017
 
Fluentd and Embulk Game Server 4
Fluentd and Embulk Game Server 4Fluentd and Embulk Game Server 4
Fluentd and Embulk Game Server 4
 
Introduction to Apache Camel
Introduction to Apache CamelIntroduction to Apache Camel
Introduction to Apache Camel
 
London devops logging
London devops loggingLondon devops logging
London devops logging
 
Altitude SF 2017: Advanced VCL: Shielding and Clustering
Altitude SF 2017: Advanced VCL: Shielding and ClusteringAltitude SF 2017: Advanced VCL: Shielding and Clustering
Altitude SF 2017: Advanced VCL: Shielding and Clustering
 
Async and Non-blocking IO w/ JRuby
Async and Non-blocking IO w/ JRubyAsync and Non-blocking IO w/ JRuby
Async and Non-blocking IO w/ JRuby
 
DjangoCon 2010 Scaling Disqus
DjangoCon 2010 Scaling DisqusDjangoCon 2010 Scaling Disqus
DjangoCon 2010 Scaling Disqus
 
Как сделать высоконагруженный сервис, не зная количество нагрузки / Олег Обле...
Как сделать высоконагруженный сервис, не зная количество нагрузки / Олег Обле...Как сделать высоконагруженный сервис, не зная количество нагрузки / Олег Обле...
Как сделать высоконагруженный сервис, не зная количество нагрузки / Олег Обле...
 
Apache Camel K - Copenhagen
Apache Camel K - CopenhagenApache Camel K - Copenhagen
Apache Camel K - Copenhagen
 
The details of CI/CD environment for Ruby
The details of CI/CD environment for RubyThe details of CI/CD environment for Ruby
The details of CI/CD environment for Ruby
 
Empowering developers to deploy their own data stores
Empowering developers to deploy their own data storesEmpowering developers to deploy their own data stores
Empowering developers to deploy their own data stores
 
Spring framework 4.x
Spring framework 4.xSpring framework 4.x
Spring framework 4.x
 
Apache Camel K - Copenhagen v2
Apache Camel K - Copenhagen v2Apache Camel K - Copenhagen v2
Apache Camel K - Copenhagen v2
 
Apache Camel v3, Camel K and Camel Quarkus
Apache Camel v3, Camel K and Camel QuarkusApache Camel v3, Camel K and Camel Quarkus
Apache Camel v3, Camel K and Camel Quarkus
 
Railsで作るBFFの功罪
Railsで作るBFFの功罪Railsで作るBFFの功罪
Railsで作るBFFの功罪
 
Integrated Cache on Netscaler
Integrated Cache on NetscalerIntegrated Cache on Netscaler
Integrated Cache on Netscaler
 

Similaire à 비동기 회고 발표자료

jclouds High Level Overview by Adrian Cole
jclouds High Level Overview by Adrian Colejclouds High Level Overview by Adrian Cole
jclouds High Level Overview by Adrian ColeEverett Toews
 
StackMate - CloudFormation for CloudStack
StackMate - CloudFormation for CloudStackStackMate - CloudFormation for CloudStack
StackMate - CloudFormation for CloudStackChiradeep Vittal
 
Fighting Against Chaotically Separated Values with Embulk
Fighting Against Chaotically Separated Values with EmbulkFighting Against Chaotically Separated Values with Embulk
Fighting Against Chaotically Separated Values with EmbulkSadayuki Furuhashi
 
Degrading Performance? You Might be Suffering From the Small Files Syndrome
Degrading Performance? You Might be Suffering From the Small Files SyndromeDegrading Performance? You Might be Suffering From the Small Files Syndrome
Degrading Performance? You Might be Suffering From the Small Files SyndromeDatabricks
 
Enterprise Java Web Application Frameworks Sample Stack Implementation
Enterprise Java Web Application Frameworks   Sample Stack ImplementationEnterprise Java Web Application Frameworks   Sample Stack Implementation
Enterprise Java Web Application Frameworks Sample Stack ImplementationMert Çalışkan
 
Asynchronous Web Programming with HTML5 WebSockets and Java
Asynchronous Web Programming with HTML5 WebSockets and JavaAsynchronous Web Programming with HTML5 WebSockets and Java
Asynchronous Web Programming with HTML5 WebSockets and JavaJames Falkner
 
[Coscup 2012] JavascriptMVC
[Coscup 2012] JavascriptMVC[Coscup 2012] JavascriptMVC
[Coscup 2012] JavascriptMVCAlive Kuo
 
Data Modeling and Relational to NoSQL
Data Modeling and Relational to NoSQLData Modeling and Relational to NoSQL
Data Modeling and Relational to NoSQLDATAVERSITY
 
Bluemix paas 기반 saas 개발 사례
Bluemix paas 기반 saas 개발 사례Bluemix paas 기반 saas 개발 사례
Bluemix paas 기반 saas 개발 사례uEngine Solutions
 
Berlin Buzz Words - Apache Drill by Ted Dunning & Michael Hausenblas
Berlin Buzz Words - Apache Drill by Ted Dunning & Michael HausenblasBerlin Buzz Words - Apache Drill by Ted Dunning & Michael Hausenblas
Berlin Buzz Words - Apache Drill by Ted Dunning & Michael HausenblasMapR Technologies
 
Embulk, an open-source plugin-based parallel bulk data loader
Embulk, an open-source plugin-based parallel bulk data loaderEmbulk, an open-source plugin-based parallel bulk data loader
Embulk, an open-source plugin-based parallel bulk data loaderSadayuki Furuhashi
 
Big Data LDN 2017: Look Ma, No Code! Building Streaming Data Pipelines With A...
Big Data LDN 2017: Look Ma, No Code! Building Streaming Data Pipelines With A...Big Data LDN 2017: Look Ma, No Code! Building Streaming Data Pipelines With A...
Big Data LDN 2017: Look Ma, No Code! Building Streaming Data Pipelines With A...Matt Stubbs
 
Solutions for bi-directional integration between Oracle RDBMS & Apache Kafka
Solutions for bi-directional integration between Oracle RDBMS & Apache KafkaSolutions for bi-directional integration between Oracle RDBMS & Apache Kafka
Solutions for bi-directional integration between Oracle RDBMS & Apache KafkaGuido Schmutz
 
Practical Use of MongoDB for Node.js
Practical Use of MongoDB for Node.jsPractical Use of MongoDB for Node.js
Practical Use of MongoDB for Node.jsasync_io
 
Spring 3 - An Introduction
Spring 3 - An IntroductionSpring 3 - An Introduction
Spring 3 - An IntroductionThorsten Kamann
 
Buildingsocialanalyticstoolwithmongodb
BuildingsocialanalyticstoolwithmongodbBuildingsocialanalyticstoolwithmongodb
BuildingsocialanalyticstoolwithmongodbMongoDB APAC
 

Similaire à 비동기 회고 발표자료 (20)

jclouds High Level Overview by Adrian Cole
jclouds High Level Overview by Adrian Colejclouds High Level Overview by Adrian Cole
jclouds High Level Overview by Adrian Cole
 
StackMate - CloudFormation for CloudStack
StackMate - CloudFormation for CloudStackStackMate - CloudFormation for CloudStack
StackMate - CloudFormation for CloudStack
 
Fighting Against Chaotically Separated Values with Embulk
Fighting Against Chaotically Separated Values with EmbulkFighting Against Chaotically Separated Values with Embulk
Fighting Against Chaotically Separated Values with Embulk
 
Azure cosmosdb
Azure cosmosdbAzure cosmosdb
Azure cosmosdb
 
Degrading Performance? You Might be Suffering From the Small Files Syndrome
Degrading Performance? You Might be Suffering From the Small Files SyndromeDegrading Performance? You Might be Suffering From the Small Files Syndrome
Degrading Performance? You Might be Suffering From the Small Files Syndrome
 
Enterprise Java Web Application Frameworks Sample Stack Implementation
Enterprise Java Web Application Frameworks   Sample Stack ImplementationEnterprise Java Web Application Frameworks   Sample Stack Implementation
Enterprise Java Web Application Frameworks Sample Stack Implementation
 
Guillotina
GuillotinaGuillotina
Guillotina
 
MeteorJS Introduction
MeteorJS IntroductionMeteorJS Introduction
MeteorJS Introduction
 
Asynchronous Web Programming with HTML5 WebSockets and Java
Asynchronous Web Programming with HTML5 WebSockets and JavaAsynchronous Web Programming with HTML5 WebSockets and Java
Asynchronous Web Programming with HTML5 WebSockets and Java
 
[Coscup 2012] JavascriptMVC
[Coscup 2012] JavascriptMVC[Coscup 2012] JavascriptMVC
[Coscup 2012] JavascriptMVC
 
Data Modeling and Relational to NoSQL
Data Modeling and Relational to NoSQLData Modeling and Relational to NoSQL
Data Modeling and Relational to NoSQL
 
Bluemix paas 기반 saas 개발 사례
Bluemix paas 기반 saas 개발 사례Bluemix paas 기반 saas 개발 사례
Bluemix paas 기반 saas 개발 사례
 
Berlin Buzz Words - Apache Drill by Ted Dunning & Michael Hausenblas
Berlin Buzz Words - Apache Drill by Ted Dunning & Michael HausenblasBerlin Buzz Words - Apache Drill by Ted Dunning & Michael Hausenblas
Berlin Buzz Words - Apache Drill by Ted Dunning & Michael Hausenblas
 
Embulk, an open-source plugin-based parallel bulk data loader
Embulk, an open-source plugin-based parallel bulk data loaderEmbulk, an open-source plugin-based parallel bulk data loader
Embulk, an open-source plugin-based parallel bulk data loader
 
Big Data LDN 2017: Look Ma, No Code! Building Streaming Data Pipelines With A...
Big Data LDN 2017: Look Ma, No Code! Building Streaming Data Pipelines With A...Big Data LDN 2017: Look Ma, No Code! Building Streaming Data Pipelines With A...
Big Data LDN 2017: Look Ma, No Code! Building Streaming Data Pipelines With A...
 
Solutions for bi-directional integration between Oracle RDBMS & Apache Kafka
Solutions for bi-directional integration between Oracle RDBMS & Apache KafkaSolutions for bi-directional integration between Oracle RDBMS & Apache Kafka
Solutions for bi-directional integration between Oracle RDBMS & Apache Kafka
 
Top ten-list
Top ten-listTop ten-list
Top ten-list
 
Practical Use of MongoDB for Node.js
Practical Use of MongoDB for Node.jsPractical Use of MongoDB for Node.js
Practical Use of MongoDB for Node.js
 
Spring 3 - An Introduction
Spring 3 - An IntroductionSpring 3 - An Introduction
Spring 3 - An Introduction
 
Buildingsocialanalyticstoolwithmongodb
BuildingsocialanalyticstoolwithmongodbBuildingsocialanalyticstoolwithmongodb
Buildingsocialanalyticstoolwithmongodb
 

Dernier

Quasi-Stochastic Approximation: Algorithm Design Principles with Applications...
Quasi-Stochastic Approximation: Algorithm Design Principles with Applications...Quasi-Stochastic Approximation: Algorithm Design Principles with Applications...
Quasi-Stochastic Approximation: Algorithm Design Principles with Applications...Sean Meyn
 
Technology Features of Apollo HDD Machine, Its Technical Specification with C...
Technology Features of Apollo HDD Machine, Its Technical Specification with C...Technology Features of Apollo HDD Machine, Its Technical Specification with C...
Technology Features of Apollo HDD Machine, Its Technical Specification with C...Apollo Techno Industries Pvt Ltd
 
A Seminar on Electric Vehicle Software Simulation
A Seminar on Electric Vehicle Software SimulationA Seminar on Electric Vehicle Software Simulation
A Seminar on Electric Vehicle Software SimulationMohsinKhanA
 
solar wireless electric vechicle charging system
solar wireless electric vechicle charging systemsolar wireless electric vechicle charging system
solar wireless electric vechicle charging systemgokuldongala
 
nvidia AI-gtc 2024 partial slide deck.pptx
nvidia AI-gtc 2024 partial slide deck.pptxnvidia AI-gtc 2024 partial slide deck.pptx
nvidia AI-gtc 2024 partial slide deck.pptxjasonsedano2
 
EPE3163_Hydro power stations_Unit2_Lect2.pptx
EPE3163_Hydro power stations_Unit2_Lect2.pptxEPE3163_Hydro power stations_Unit2_Lect2.pptx
EPE3163_Hydro power stations_Unit2_Lect2.pptxJoseeMusabyimana
 
Summer training report on BUILDING CONSTRUCTION for DIPLOMA Students.pdf
Summer training report on BUILDING CONSTRUCTION for DIPLOMA Students.pdfSummer training report on BUILDING CONSTRUCTION for DIPLOMA Students.pdf
Summer training report on BUILDING CONSTRUCTION for DIPLOMA Students.pdfNaveenVerma126
 
sdfsadopkjpiosufoiasdoifjasldkjfl a asldkjflaskdjflkjsdsdf
sdfsadopkjpiosufoiasdoifjasldkjfl a asldkjflaskdjflkjsdsdfsdfsadopkjpiosufoiasdoifjasldkjfl a asldkjflaskdjflkjsdsdf
sdfsadopkjpiosufoiasdoifjasldkjfl a asldkjflaskdjflkjsdsdfJulia Kaye
 
IT3401-WEB ESSENTIALS PRESENTATIONS.pptx
IT3401-WEB ESSENTIALS PRESENTATIONS.pptxIT3401-WEB ESSENTIALS PRESENTATIONS.pptx
IT3401-WEB ESSENTIALS PRESENTATIONS.pptxSAJITHABANUS
 
GENERAL CONDITIONS FOR CONTRACTS OF CIVIL ENGINEERING WORKS
GENERAL CONDITIONS  FOR  CONTRACTS OF CIVIL ENGINEERING WORKS GENERAL CONDITIONS  FOR  CONTRACTS OF CIVIL ENGINEERING WORKS
GENERAL CONDITIONS FOR CONTRACTS OF CIVIL ENGINEERING WORKS Bahzad5
 
SATELITE COMMUNICATION UNIT 1 CEC352 REGULATION 2021 PPT BASICS OF SATELITE ....
SATELITE COMMUNICATION UNIT 1 CEC352 REGULATION 2021 PPT BASICS OF SATELITE ....SATELITE COMMUNICATION UNIT 1 CEC352 REGULATION 2021 PPT BASICS OF SATELITE ....
SATELITE COMMUNICATION UNIT 1 CEC352 REGULATION 2021 PPT BASICS OF SATELITE ....santhyamuthu1
 
Phase noise transfer functions.pptx
Phase noise transfer      functions.pptxPhase noise transfer      functions.pptx
Phase noise transfer functions.pptxSaiGouthamSunkara
 
Strategies of Urban Morphologyfor Improving Outdoor Thermal Comfort and Susta...
Strategies of Urban Morphologyfor Improving Outdoor Thermal Comfort and Susta...Strategies of Urban Morphologyfor Improving Outdoor Thermal Comfort and Susta...
Strategies of Urban Morphologyfor Improving Outdoor Thermal Comfort and Susta...amrabdallah9
 
Basic Principle of Electrochemical Sensor
Basic Principle of  Electrochemical SensorBasic Principle of  Electrochemical Sensor
Basic Principle of Electrochemical SensorTanvir Moin
 
دليل تجارب الاسفلت المختبرية - Asphalt Experiments Guide Laboratory
دليل تجارب الاسفلت المختبرية - Asphalt Experiments Guide Laboratoryدليل تجارب الاسفلت المختبرية - Asphalt Experiments Guide Laboratory
دليل تجارب الاسفلت المختبرية - Asphalt Experiments Guide LaboratoryBahzad5
 
UNIT4_ESD_wfffffggggggggggggith_ARM.pptx
UNIT4_ESD_wfffffggggggggggggith_ARM.pptxUNIT4_ESD_wfffffggggggggggggith_ARM.pptx
UNIT4_ESD_wfffffggggggggggggith_ARM.pptxrealme6igamerr
 
Guardians and Glitches: Navigating the Duality of Gen AI in AppSec
Guardians and Glitches: Navigating the Duality of Gen AI in AppSecGuardians and Glitches: Navigating the Duality of Gen AI in AppSec
Guardians and Glitches: Navigating the Duality of Gen AI in AppSecTrupti Shiralkar, CISSP
 

Dernier (20)

Présentation IIRB 2024 Marine Cordonnier.pdf
Présentation IIRB 2024 Marine Cordonnier.pdfPrésentation IIRB 2024 Marine Cordonnier.pdf
Présentation IIRB 2024 Marine Cordonnier.pdf
 
Quasi-Stochastic Approximation: Algorithm Design Principles with Applications...
Quasi-Stochastic Approximation: Algorithm Design Principles with Applications...Quasi-Stochastic Approximation: Algorithm Design Principles with Applications...
Quasi-Stochastic Approximation: Algorithm Design Principles with Applications...
 
Technology Features of Apollo HDD Machine, Its Technical Specification with C...
Technology Features of Apollo HDD Machine, Its Technical Specification with C...Technology Features of Apollo HDD Machine, Its Technical Specification with C...
Technology Features of Apollo HDD Machine, Its Technical Specification with C...
 
Lecture 4 .pdf
Lecture 4                              .pdfLecture 4                              .pdf
Lecture 4 .pdf
 
A Seminar on Electric Vehicle Software Simulation
A Seminar on Electric Vehicle Software SimulationA Seminar on Electric Vehicle Software Simulation
A Seminar on Electric Vehicle Software Simulation
 
solar wireless electric vechicle charging system
solar wireless electric vechicle charging systemsolar wireless electric vechicle charging system
solar wireless electric vechicle charging system
 
nvidia AI-gtc 2024 partial slide deck.pptx
nvidia AI-gtc 2024 partial slide deck.pptxnvidia AI-gtc 2024 partial slide deck.pptx
nvidia AI-gtc 2024 partial slide deck.pptx
 
EPE3163_Hydro power stations_Unit2_Lect2.pptx
EPE3163_Hydro power stations_Unit2_Lect2.pptxEPE3163_Hydro power stations_Unit2_Lect2.pptx
EPE3163_Hydro power stations_Unit2_Lect2.pptx
 
Summer training report on BUILDING CONSTRUCTION for DIPLOMA Students.pdf
Summer training report on BUILDING CONSTRUCTION for DIPLOMA Students.pdfSummer training report on BUILDING CONSTRUCTION for DIPLOMA Students.pdf
Summer training report on BUILDING CONSTRUCTION for DIPLOMA Students.pdf
 
Lecture 2 .pdf
Lecture 2                           .pdfLecture 2                           .pdf
Lecture 2 .pdf
 
sdfsadopkjpiosufoiasdoifjasldkjfl a asldkjflaskdjflkjsdsdf
sdfsadopkjpiosufoiasdoifjasldkjfl a asldkjflaskdjflkjsdsdfsdfsadopkjpiosufoiasdoifjasldkjfl a asldkjflaskdjflkjsdsdf
sdfsadopkjpiosufoiasdoifjasldkjfl a asldkjflaskdjflkjsdsdf
 
IT3401-WEB ESSENTIALS PRESENTATIONS.pptx
IT3401-WEB ESSENTIALS PRESENTATIONS.pptxIT3401-WEB ESSENTIALS PRESENTATIONS.pptx
IT3401-WEB ESSENTIALS PRESENTATIONS.pptx
 
GENERAL CONDITIONS FOR CONTRACTS OF CIVIL ENGINEERING WORKS
GENERAL CONDITIONS  FOR  CONTRACTS OF CIVIL ENGINEERING WORKS GENERAL CONDITIONS  FOR  CONTRACTS OF CIVIL ENGINEERING WORKS
GENERAL CONDITIONS FOR CONTRACTS OF CIVIL ENGINEERING WORKS
 
SATELITE COMMUNICATION UNIT 1 CEC352 REGULATION 2021 PPT BASICS OF SATELITE ....
SATELITE COMMUNICATION UNIT 1 CEC352 REGULATION 2021 PPT BASICS OF SATELITE ....SATELITE COMMUNICATION UNIT 1 CEC352 REGULATION 2021 PPT BASICS OF SATELITE ....
SATELITE COMMUNICATION UNIT 1 CEC352 REGULATION 2021 PPT BASICS OF SATELITE ....
 
Phase noise transfer functions.pptx
Phase noise transfer      functions.pptxPhase noise transfer      functions.pptx
Phase noise transfer functions.pptx
 
Strategies of Urban Morphologyfor Improving Outdoor Thermal Comfort and Susta...
Strategies of Urban Morphologyfor Improving Outdoor Thermal Comfort and Susta...Strategies of Urban Morphologyfor Improving Outdoor Thermal Comfort and Susta...
Strategies of Urban Morphologyfor Improving Outdoor Thermal Comfort and Susta...
 
Basic Principle of Electrochemical Sensor
Basic Principle of  Electrochemical SensorBasic Principle of  Electrochemical Sensor
Basic Principle of Electrochemical Sensor
 
دليل تجارب الاسفلت المختبرية - Asphalt Experiments Guide Laboratory
دليل تجارب الاسفلت المختبرية - Asphalt Experiments Guide Laboratoryدليل تجارب الاسفلت المختبرية - Asphalt Experiments Guide Laboratory
دليل تجارب الاسفلت المختبرية - Asphalt Experiments Guide Laboratory
 
UNIT4_ESD_wfffffggggggggggggith_ARM.pptx
UNIT4_ESD_wfffffggggggggggggith_ARM.pptxUNIT4_ESD_wfffffggggggggggggith_ARM.pptx
UNIT4_ESD_wfffffggggggggggggith_ARM.pptx
 
Guardians and Glitches: Navigating the Duality of Gen AI in AppSec
Guardians and Glitches: Navigating the Duality of Gen AI in AppSecGuardians and Glitches: Navigating the Duality of Gen AI in AppSec
Guardians and Glitches: Navigating the Duality of Gen AI in AppSec
 

비동기 회고 발표자료

  • 2. • Dooray! Mail-API 개발, 운영 • www.pikicast.com 개발, 운영 • Cloud orchestration system 개발, 운영 • Caller-Ring 개발, 운영
  • 3. “Non-Blocking” or “Blocking” vs “Asynchronous” or “Synchronous” Return 의 여부에 따라… Thread 상태에 따라서…
  • 4. “The C10K Problem” http://www.kegel.com/c10k.html It’s time for web servers to handle ten thousand clients simultaneously. The web is a big place now
  • 5. The C10K Problem • Scale-out • Load-balancer • Haproxy, Nginx • AWS, Azure, Google Cloud and OpenStack • NoSQL • Redis, HazelCast (IMDG) • Zero-copy, limits on file descriptors
  • 7. The C10K Problem • Event Loop – Nginx, node.js, Play, Netty, Vert.X • Event Driven Programming • Reactor • Java Executor Service Event Loop Thread PoolEvent Queue
  • 8. The C10K Problem • Callback Functions • Functional Programming vs imperative programming • Lambda in Java – Functional programming – Modern Java Event Loop Thread PoolEvent Queue Register Callback Complete CallbackTrigger Callback
  • 9. The C10K Problem • 객체 지향형 프로그래밍 • 함수형 프로그래밍 Event Loop Thread PoolEvent Queue Th #1 Th #2 Th #3 Th #1 Th #2 VS
  • 13. Asynchronous Programming with Spring 4.3 • DeferredResult • @Async • AsyncRestTemplate & ListenableFuture • CompletableFuture
  • 14. @Async & DeferredResult example Sample : AsyncController.java 단점은?
  • 16. “그런데 JDBC 는 blocking IO 라며?” “왜 비동기 프로그래밍을 할까?”“하지만 많은 기능을 구현할 수 있습니다.”
  • 18. “휴지통에 있는 메일을 삭제하 는 API” - 데이터 베이스에 record 삭제 - 메일 파일(Mime) 삭제 - 타 시스템에 메일이 삭제 되었 음을 알려줌. 검색서버, History 서버
  • 19. 그런데… 휴지통에 1만건이 존재한다면? - DB 작업 : 300ms- 파일 삭제 작업 : 10ms * 10,000 = 100sec - 타시스템 연동작업 : 30ms * 10,000 * 3 = 900 sec Total : 약 1000 sec, 16분
  • 20. @Async • @Async 를 쓸때는 별도의 ThreadPool 을 사용하자. • @Async 와 @Transactional 을 쓸때는 주의하자. – @Transactional 의 원리를 이해하고 사용해야 함. – 부하
  • 21. @Transactional • AOP-Proxy 에서는 public method 로 선언 – @EnableTransactionManagement(proxyTargetClass = true) – <tx:annotation-driven/> – AspectJ 에서는 상관없음. • Bean 내부의 다른 method에서 다른 Transaction을 실행할때 – 선언적 Transaction 을 실행한다. – Self-autowiring in Spring 4.3+ – @Async 에서 많이 놓치는 실수.
  • 22. “DB Connection 객체는 비싸 다!” 비동기 프로그래밍에서도 마찬 가지 입니다.
  • 23. @Transactional(readOnly = false) public boolean removeMailsByMailFolderId(Long mailFolderId){ List<Long> mailIds = mailRepository.findIds(mailFolderId); mailRepository.deleteByMailFolderId(mailFolderId); mailIds.stream.forEach(mailId -> { storageComponent.deleteByMailId(mailId); searchComponent.deleteByMailId(mailId); historyComponent.deleteByMailId(mailId); ); return true; } Th#1 Storage Search History DB
  • 24. @Transactional(readOnly = false) public boolean removeMailsByMailFolderId(Long mailFolderId){ List<Long> mailIds = mailRepository.findIds(mailFolderId); mailRepository.deleteByMailFolderId(mailFolderId); mailIds.stream.forEach(mailId -> { storageComponent.deleteByMailId(mailId); searchComponent.deleteByMailId(mailId); historyComponent.deleteByMailId(mailId); ); return true; } 실행시간 = TX 시간 Long Transaction End Transaction Begin Transaction Th#1 Storage Search History DB 장애유발 가능성
  • 25. DB Connection을 아끼는 방법들 • @Async • LazyConnectionDataSourceProxy • Facade Pattern • TransactionSynchronizationManager, TransactionSynchronizationAdaptor
  • 27. @Transactional(readOnly = false) public void removeMailsByMailFolderId(Long mailFolderId){ List<Long> mailIds = mailRepository.findIds(mailFolderId); Lists.partition(mailIds, 10).stream .forEach(submailIds -> selfService.removeByAsync(submailIds); } @Async @Transactional(readOnly = false) public void removeByAsync(List<Long> submailIds){ mailRepository.deleteByMailIdIn(subMailIds) subMailIds.stream.forEach(mailId -> { storageComponent.deleteByMailId(mailId); searchComponent.deleteByMailId(mailId); historyComponent.deleteByMailId(mailId); ); } Th#1 DB Th#4 Th#3 Th#2 Storage Search History DB DB DB
  • 28. @Transactional(readOnly = false) public void removeMailsByMailFolderId(Long mailFolderId){ List<Long> mailIds = mailRepository.findIds(mailFolderId); mailRepository.deleteByMailIdIn(mailIds) selfService.removeByAsync(mailIds) } @Async public void removeByAsync(List<Long> mailIds){ mailIds.stream.forEach(mailId -> { storageComponent.deleteByMailId(mailId); searchComponent.deleteByMailId(mailId); historyComponent.deleteByMailId(mailId); ); } Th #1 Th#1 DB Th#4 Storage Search History
  • 30. @Bean(value = “dataSource”, destoryMethod=“close”) public DataSource dataSource(){ BasicDataSource ds = new BasicDataSource(); //… more configuration return ds; } @Bean(value = “lazyDataSource”) public LazyConnectionDataSourceProxy lazyDataSource(){ return new LazyConnectionDataSourceProxy(datasource()); } @Transactional(readOnly = false) public boolean removeMailsByMailFolderId(Long mailFolderId){ List<Long> mailIds = mailRepository.findIds(mailFolderId); mailRepository.deleteByMailFolderId(mailFolderId); mailIds.stream.forEach(mailId -> { storageComponent.deleteByMailId(mailId); searchComponent.deleteByMailId(mailId); historyComponent.deleteByMailId(mailId); ); return true; } After What is Difference? Before
  • 31. Façade Pattern 을 이용한 예제
  • 32. public class MailFacade { public void removeMailsByMailFolderId(Long mailFolderId){ List<Long> removedMailIds = mailService.findIds(mailFolderId); removedMailIds.stream.forEach(mailId -> { storageComponent.deleteByMailId(mailId); searchComponent.deleteByMailId(mailId); historyComponent.deleteByMailId(mailId); ); } } public class MailServiceImpl implements MailService { @Transactional(readOnly = false) public List<Long> removeMailEntities(Long mailFolderId){ List<Long> mailIds = mailRepository.findIds(mailFolderId); mailRepository.deleteByMailIdIn(mailIds) return mailIds } } Façade Layer Service Layer
  • 34. @Async @Transactional(readOnly = false) public void removeByAsync(List<Long> submailIds){ mailRepository.deleteByMailIdIn(subMailIds) subMailIds.stream.forEach(mailId -> { storageComponent.deleteByMailId(mailId); searchComponent.deleteByMailId(mailId); historyComponent.deleteByMailId(mailId); ); } DB Transaction 이 묶일 필요가 있나요?
  • 35. @Transactional(readOnly = false) public void removeByAsync(List<Long> submailIds){ mailRepository.deleteByMailIdIn(subMailIds) TransactionSynchronizationManager.registerSynchronization(new TransactionSynchronizationAdapter() { @Override public void afterCommit() { subMailIds.stream.forEach(mailId -> { storageComponent.deleteByMailId(mailId); searchComponent.deleteByMailId(mailId); historyComponent.deleteByMailId(mailId); }); } } } afterCommit() – invoke after transaction commit afterCompletion() - invoke after transaction commit / rollback beforeCommit() – invoke before trasnaction commit beforeCompletion() - invoke before transaction commit / rollback
  • 36. “WAS 스레드, DBPool 갯수 설 정이야기 해볼까요?” Peek Time 에 비동기로 프로그래 밍된 api 를 호출하면? DBPool 은 괜찬은가요? “DataSource 를 분리합니다”
  • 37. DataSource Configuration • 비동기용와 API 처리용 DataSource를 분리한다. – Peek time 때, 비동기 프로세스는 늦게 처리되도 된다. – 장애는 전파하지 말자. • AbstractRoutingDataSource class – 일종의 Proxy DataSource – setTargetDataSources method - key 에 따른 DataSource 등록 – determineCurrentLookupKey method – key 에 따른 DataSource 획득
  • 38. @Slf4j public class ExecutionRoutingDataSource extends AbstractRoutingDataSource { @Override protected Object determineCurrentLookupKey() { return ExecutionContextHolder.getExecutionType(); } }
  • 39. public class ExecutionRoutingDataSource extends AbstractRoutingDataSource { @Override protected Object determineCurrentLookupKey() { return ExecutionContextHolder.getExecutionType(); } } @Bean("routingDataSource") public ExecutionRoutingDataSource routingDataSource() { Map<Object, Object> targetDataSources = new HashMap<>(); targetDataSources.put(ExecutionType.API, apiDataSource()); targetDataSources.put(ExecutionType.BATCH, batchDataSource()); ExecutionRoutingDataSource routingDataSource = new ExecutionRoutingDataSource(); routingDataSource.setTargetDataSources(targetDataSources); routingDataSource.setDefaultTargetDataSource(apiDataSource()); return routingDataSource; } LookUpKey 에 따라 동적으로 DataSource 를 사용 미리 등록된 enum ExecutionType.API or BATCH
  • 40. @TransactionalExecutionType( executionType = ExecutionType.BATCH ) @Transactional( readOnly = true, isolation = Isolation.READ_COMMITTED ) public List<CompletableFuture<List<Long>>> partition(Long taskQueueId) { // Business logic and query to DB } @Target({ElementType.METHOD}) @Retention(RetentionPolicy.RUNTIME) public @interface TransactionalExecutionType { ExecutionType executionType() default ExecutionType.API; }
  • 41. AsyncRestTemplate • ListenableFuture – Callback Hell • Factory 종류 – Netty4ClientHttpRequestFactory – SimpleClientHttpRequestFactory – OkHttp3ClientHttpRequestFactory – Pros and Cons • Backpressure, Throttling • Circuit Break Pattern
  • 42. AsyncRestTemplate 은 RestTemplate 과 같다. Spring 의 PSA (Portable Service Abstraction) Sample : AsyncRestTemplateTest.java getWords(), asyncGetWords(), asyncCombinedProcess()
  • 43. AsyncRestTemplate • Callback Hell 을 피하기 위해서는? – 람다식 – Default Method – CompletableFuture 객체로 랩핑하는 Util 클래스를 만든다. – Continue..
  • 44. public class CompletableFutureBuilder { public static <T> CompletableFuture<T> build(ListenableFuture<T> listenableFuture) { CompletableFuture<T> completableFuture = new CompletableFuture<T>() { @Override public boolean cancel(boolean mayInterruptIfRunning) { boolean result = listenableFuture.cancel(mayInterruptIfRunning); super.cancel(mayInterruptIfRunning); return result; } }; listenableFuture.addCallback(new ListenableFutureCallback<T>() { @Override public void onFailure(Throwable ex) { completableFuture.completeExceptionally(ex); } @Override public void onSuccess(T result) { completableFuture.complete(result); } }); return completableFuture; }
  • 45. public class CompletableFutureBuilder { public static <T> CompletableFuture<T> build(ListenableFuture<T> listenableFuture) { CompletableFuture<T> completableFuture = new CompletableFuture<T>() { @Override public boolean cancel(boolean mayInterruptIfRunning) { boolean result = listenableFuture.cancel(mayInterruptIfRunning); super.cancel(mayInterruptIfRunning); return result; } }; listenableFuture.addCallback(new ListenableFutureCallback<T>() { @Override public void onFailure(Throwable ex) { completableFuture.completeExceptionally(ex); } @Override public void onSuccess(T result) { completableFuture.complete(result); } }); return completableFuture; }
  • 46. public class CompletableFutureBuilder { public static <T> CompletableFuture<T> build(ListenableFuture<T> listenableFuture) { CompletableFuture<T> completableFuture = new CompletableFuture<T>() { @Override public boolean cancel(boolean mayInterruptIfRunning) { boolean result = listenableFuture.cancel(mayInterruptIfRunning); super.cancel(mayInterruptIfRunning); return result; } }; listenableFuture.addCallback(new ListenableFutureCallback<T>() { @Override public void onFailure(Throwable ex) { completableFuture.completeExceptionally(ex); } @Override public void onSuccess(T result) { completableFuture.complete(result); } }); return completableFuture; }
  • 47. CompletableFuture • 비동기 프로그래밍을 위한 클래스 – NonBlocking 으로 처리 가능하다. (vs Future class) – Blocking 으로도 처리 가능하다. – 여러개의 작업들을 엮을 수 있다. – 여러개의 작업들을 합칠 수 있다. – 예외 사항을 처리할 수 있다. • Implements CompletionStage<T> – 테스크를 포함하는 모델을 의미한다. – 테스크는 체인패턴의 기본 요소이다.
  • 48. CompletableFuture • 접미사를 확인 – thenApply() vs thenApplyAsync() • 파라미터를 확인 – thenApplyAsync(Function<? super T, ? extends U> fn) – thenApplyAsync(Function<? super T, ? extends U> fn, Executor executor) • 비동기 작업 시작 method – runAsync(), supplyAsync() • 작업들을 연결하는 method – thenApplyAsync(), thenComposeAsync(), anyOf(), allOf() • 예외처리를 하는 method – exceptionally()
  • 49. CompletableFuture Method AsyncMethod Arguments Returns thenAccept thenAcceptAsync 이전 Stage 의 결과 Noting thenRun thenRunAsync Noting thenApply thenApplyAsync 이전 Stage 의 결과 Result of current stage thenCompose thenComposeAsync 이전 Stage 의 결과 Future result of current stage thenCombine thenCombineAsync 이전 두 Stage 의 결과 Result of current stage whenComplete whenCompleteAsync 이전 두 Stage 의 결과 or Exception Noting
  • 50. CompletableFuture • Building a CompletableFuture Chain – Example. CompletableFutureTest.java chainMethods(); – Example. CompletableFutureTest.java allOfMethods(); – Example. CompletableFutureTest.java allOfAcceptMethods(); • Exception handling – Example. CompletableFutureTest.java handleMethods();
  • 51. CompletableFuture • Thread pool 공유시 – 당연하지만 순서보장이 안될 수 있다. – Method chaining 시 공정한 처리가 어려울수 있다. – Starvation 가능성 – executor.setRejectedExecutionHandler(); – AbortPolicy – DiscardPolicy – DiscardOldestPolicy – CallerRunsPolicy Event Loop Thread PoolEvent Queue Register Callback Complete CallbackTrigger Callback
  • 52. Transaction Saving Programming - Avoid MySQL Lock wait - Avoid MySQL Dead Lock - Query Tuning on JPA…
  • 53. - 디버깅이 어렵다. - TC를 만들기 어렵다. - 항상 RDB 와의 싸움이다.
  • 55. Q & A

Notes de l'éditeur

  1. https://www.javacodegeeks.com/2015/07/understanding-callable-and-spring-deferredresult.html
  2. JoinForkPool 같이 Managed 되지 않는 것 보다는 Thread Pool 을 이용하여 관리하라. JoinForkPool 이 나쁘다는것은 아니다.
  3. TX 전체 시간은 DB Query time + 각각의 컴포넌트들을 삭제하는 시간. 이중에 필요한 시간은?? TX 가 길어지면 뭐가 문제가 될까? -> DB Pool 고갈 -> Lock wait -> Timeout 이상황에서 API 는 멀티 스레드이므로 계속해서 부가가 들어오고 장애가 타 시스템으로 전파될 가능성이 높다
  4. @Async 를 이용해서 TX를 짧게 쪼개는 방법. 장점 - Long TX 를 방지 할수 있음 단점 – exception 이 발생하면 일부는 삭제되고 일부는 남아있음
  5. @Async 를 이용해서 TX를 짧게 쪼개는 방법. 단점 – 댕글링 파일이나 정보가 남을 수 있음 장점 – TX 실패시 Async 로 안넘어갈수 있음.
  6. 스토리지와 검색과 히스토리는 어디로? 그런데 Exception 이 발생하면?
  7. TransactionSynchronizationAdaptor 인터페이스
  8. DataSource 의 한 일종이다. determineCurrentLookupKey 에 의해서 runtime 에 어떤 데이터소스를 사용할지 결정된다. Bean 으로 등록해서 사용한다
  9. https://www.callicoder.com/java-8-completablefuture-tutorial/