Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

of

Networking in Java with NIO and Netty Slide 1 Networking in Java with NIO and Netty Slide 2 Networking in Java with NIO and Netty Slide 3 Networking in Java with NIO and Netty Slide 4 Networking in Java with NIO and Netty Slide 5 Networking in Java with NIO and Netty Slide 6 Networking in Java with NIO and Netty Slide 7 Networking in Java with NIO and Netty Slide 8 Networking in Java with NIO and Netty Slide 9 Networking in Java with NIO and Netty Slide 10 Networking in Java with NIO and Netty Slide 11 Networking in Java with NIO and Netty Slide 12 Networking in Java with NIO and Netty Slide 13 Networking in Java with NIO and Netty Slide 14 Networking in Java with NIO and Netty Slide 15 Networking in Java with NIO and Netty Slide 16 Networking in Java with NIO and Netty Slide 17 Networking in Java with NIO and Netty Slide 18 Networking in Java with NIO and Netty Slide 19 Networking in Java with NIO and Netty Slide 20 Networking in Java with NIO and Netty Slide 21 Networking in Java with NIO and Netty Slide 22 Networking in Java with NIO and Netty Slide 23 Networking in Java with NIO and Netty Slide 24 Networking in Java with NIO and Netty Slide 25 Networking in Java with NIO and Netty Slide 26 Networking in Java with NIO and Netty Slide 27 Networking in Java with NIO and Netty Slide 28 Networking in Java with NIO and Netty Slide 29 Networking in Java with NIO and Netty Slide 30 Networking in Java with NIO and Netty Slide 31 Networking in Java with NIO and Netty Slide 32 Networking in Java with NIO and Netty Slide 33 Networking in Java with NIO and Netty Slide 34 Networking in Java with NIO and Netty Slide 35 Networking in Java with NIO and Netty Slide 36 Networking in Java with NIO and Netty Slide 37 Networking in Java with NIO and Netty Slide 38 Networking in Java with NIO and Netty Slide 39 Networking in Java with NIO and Netty Slide 40 Networking in Java with NIO and Netty Slide 41 Networking in Java with NIO and Netty Slide 42 Networking in Java with NIO and Netty Slide 43 Networking in Java with NIO and Netty Slide 44 Networking in Java with NIO and Netty Slide 45 Networking in Java with NIO and Netty Slide 46 Networking in Java with NIO and Netty Slide 47 Networking in Java with NIO and Netty Slide 48 Networking in Java with NIO and Netty Slide 49 Networking in Java with NIO and Netty Slide 50 Networking in Java with NIO and Netty Slide 51 Networking in Java with NIO and Netty Slide 52 Networking in Java with NIO and Netty Slide 53 Networking in Java with NIO and Netty Slide 54 Networking in Java with NIO and Netty Slide 55 Networking in Java with NIO and Netty Slide 56 Networking in Java with NIO and Netty Slide 57 Networking in Java with NIO and Netty Slide 58 Networking in Java with NIO and Netty Slide 59 Networking in Java with NIO and Netty Slide 60

YouTube videos are no longer supported on SlideShare

View original on YouTube

Upcoming SlideShare
What to Upload to SlideShare
Next

Share

Networking in Java with NIO and Netty

Let's dive under the hood of Java network applications. We plan to have a deep look to classic sockets and NIO having live coding examples. Then we discuss performance problems of sockets and find out how NIO can help us to handle 10000+ connections in a single thread. And finally we learn how to build high load application server using Netty.

https://github.com/kslisenko/java-networking

Related Books

Free with a 30 day trial from Scribd

See all

Related Audiobooks

Free with a 30 day trial from Scribd

See all

Networking in Java with NIO and Netty

  1. 1. 1CONFIDENTIAL NETWORKING IN JAVA WITH NIO AND NETTY KANSTANTSIN SLISENKA JUN 1, 2017
  2. 2. 2CONFIDENTIAL ABOUT ME Java Backend engineer Speaker at Java Tech Talks, SEC Online, CMCC Tech Talks, IT Week I’m interested in Complex Java backend, SOA, databases High load, fault-tolerant, distributed systems KANSTANTSIN SLISENKA EPAM Systems, Lead Software Engineer
  3. 3. 3CONFIDENTIAL WHY IS IMPORTANT TO DIG INTO JAVA NETWORKING Classic Java project Tomcat + Spring + Hibernate + … High load, latency-sensitive Custom TCP/UDP solution
  4. 4. 4CONFIDENTIAL AGENDA Blocking vs non-blocking resources1 Blocking I/O2 Non-blocking I/O3 Netty: high performance networking framework4
  5. 5. 5CONFIDENTIAL BLOCKING VS NON-BLOCKING RESOURCES client thread read/write response Blocking resources Database transactions, RPC/WS calls blocked
  6. 6. 6CONFIDENTIAL BLOCKING VS NON-BLOCKING RESOURCES client thread read/write response client thread read/write nope read/write response client thread read/write callback with response read/write nope register callback Blocking resources Non-blocking resources Database transactions, RPC/WS calls Queues, WebSockets, Non-blocking data storage blocked
  7. 7. 7CONFIDENTIAL NETWORK SOCKETS Socket is blocking API over OS and Network hardware not a physical object PERMANENT KNOWN PORT ANY UNUSED PORT SOCKET SOCKET Max ports = 65535
  8. 8. 8CONFIDENTIAL JAVA SOCKETS = BLOCKING IO Client Server
  9. 9. 9CONFIDENTIAL SOCKET SERVER LIVE CODING EXAMPLE
  10. 10. 10CONFIDENTIAL Server 3 2 Client 2 Client 3 Client 1 SOCKET SERVER ARCHITECTURE no data no data data transfer in progress 1 Socket Socket SocketThread (blocked*) Thread (running) Thread (blocked*) *until keep alive timeout Fixed thread pool Requests queue
  11. 11. 11CONFIDENTIAL Server 3 2 Client 2 Client 3 Client 1 SOCKET SERVER ARCHITECTURE no data no data data transfer in progress 1 Socket Socket SocketThread (blocked*) Thread (running) Thread (blocked*) Client 4 *until keep alive timeout Fixed thread pool Requests queue new connection
  12. 12. 12CONFIDENTIAL Server 3 2 Client 2 Client 3 Client 1 SOCKET SERVER ARCHITECTURE no data no data data transfer in progress 1 Socket Socket SocketThread (blocked*) Thread (running) Thread (blocked*) Client 4 *until keep alive timeout Fixed thread pool Requests queue new connection drop by timeout
  13. 13. 13CONFIDENTIAL SOCKET SERVER CONCLUSION 1. Ineffective thread utilization – many threads in waiting/blocked state – extra memory for stack and thread local resources 2. Poor performance – context switching overhead 3. Limited number of connections – when thread pool is fully consumed Benefits Drawbacks 1. Simple development well known Input/OutputStream API 2. No check for partially received messages thread is blocked until fully reads message
  14. 14. 14CONFIDENTIAL IO VS NIO IO (blocking) Thread Thread Thread socket socket socket BLOCKING READ/WRITE WAIT WAIT WAITWAIT
  15. 15. 15CONFIDENTIAL IO VS NIO IO (blocking) NIO (non-blocking) NON BLOCKING READ/WRITE Thread buffer buffer buffer socket channel socket channel socket channel Thread Thread Thread socket socket socket BLOCKING READ/WRITE WAIT WAIT WAITWAIT ALWAYS RUNNING NOTIFICATIONS selector
  16. 16. 16CONFIDENTIAL NIO DATA SENDING Thread buffer buffer buffer socket channel socket channel selector socket channel
  17. 17. 17CONFIDENTIAL NIO DATA SENDING Thread buffer buffer buffer socket channel socket channel selector socket channel 1 WRITE READY
  18. 18. 18CONFIDENTIAL NIO DATA SENDING Thread buffer buffer buffer socket channel socket channel selector PUT DATA2 socket channel 1 WRITE READY
  19. 19. 19CONFIDENTIAL NIO DATA SENDING Thread buffer buffer buffer socket channel socket channel selector PUT DATA2 SEND DATA3 socket channel 1 WRITE READY
  20. 20. 20CONFIDENTIAL NIO DATA RECEIVING Thread buffer buffer buffer socket channel socket channel selector socket channel
  21. 21. 21CONFIDENTIAL NIO DATA RECEIVING Thread buffer buffer buffer socket channel socket channel selector RECEIVE DATA1 socket channel
  22. 22. 22CONFIDENTIAL NIO DATA RECEIVING Thread buffer buffer buffer socket channel socket channel selector RECEIVE DATA1 socket channel 2 READ READY
  23. 23. 23CONFIDENTIAL NIO DATA RECEIVING Thread buffer buffer buffer socket channel socket channel selector GET DATA3 RECEIVE DATA1 socket channel 2 READ READY
  24. 24. 24CONFIDENTIAL NIO BYTE BUFFER POSITION LIMIT CAPACITY 0 1 2 3 4 5 6 7 8 9 Empty buffer BYTE ARRAY
  25. 25. 25CONFIDENTIAL NIO BYTE BUFFER POSITION LIMIT CAPACITY 0 1 2 3 4 5 6 7 8 9 1 0 1 1 0 LIMIT CAPACITY Empty buffer Writing mode BYTE ARRAY BYTE ARRAY POSITION 0 1 2 3 4 5 6 7 8 9
  26. 26. 26CONFIDENTIAL NIO BYTE BUFFER POSITION LIMIT CAPACITY 0 1 2 3 4 5 6 7 8 9 1 0 1 1 0 LIMIT CAPACITY 1 0 1 1 0 POSITION LIMIT CAPACITY Empty buffer Writing mode Reading mode BYTE ARRAY BYTE ARRAY BYTE ARRAY buffer.flip()buffer.clear() POSITION 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9
  27. 27. 27CONFIDENTIAL DIRECT BUFFERS 0 1 2 3 4 5 6 7 8 JVM Network Interface Card 0 1 2 3 4 5 0 1 2 3 4 5 Physical memory 9 Shared memory address space Shared memory address space Memory should be continuous because NICs are block- oriented devices
  28. 28. 28CONFIDENTIAL NIO SERVER LIVE CODING EXAMPLE
  29. 29. 29CONFIDENTIAL Server BASIC NIO SERVER ARCHITECTURE buffer Client #2 Client #1 no data data transfer in progress socket channel #3 Client #3 data transfer in progress buffer socket channel #2 socket channel #1 WORKER THREAD SELECTOR SELECTOR WORKER THREAD buffer Business logic
  30. 30. 30CONFIDENTIAL Server BASIC NIO SERVER ARCHITECTURE buffer Client #2 Client #1 no data data transfer in progress socket channel #3 Client #3 data transfer in progress buffer socket channel #2 socket channel #1 WORKER THREAD SELECTOR SELECTOR WORKER THREAD buffer Business logic
  31. 31. 31CONFIDENTIAL Server BASIC NIO SERVER ARCHITECTURE buffer Client #2 Client #1 no data data transfer in progress socket channel #3 Client #3 data transfer in progress buffer get ready channels socket channel #2 socket channel #1 WORKER THREAD SELECTOR SELECTOR WORKER THREAD buffer Business logic
  32. 32. 32CONFIDENTIAL Server BASIC NIO SERVER ARCHITECTURE buffer Client #2 Client #1 no data data transfer in progress socket channel #3 Client #3 data transfer in progress get buffer put read write get ready channels socket channel #2 socket channel #1 Business logic WORKER THREAD SELECTOR SELECTOR WORKER THREAD buffer
  33. 33. 33CONFIDENTIAL Server BASIC NIO SERVER ARCHITECTURE new connection buffer Client #2 Client #4 Client #1 no data data transfer in progress new connection socket channel #3 Client #3 data transfer in progress get buffer put register in selector read write get ready channels socket channel #2 socket channel #1 WORKER THREAD SELECTOR SELECTOR WORKER THREAD Business logic buffer
  34. 34. 34CONFIDENTIAL CAN WE GO TO PRODUCTION WITH A SINGLE THREAD? https://www.pinterest.com/banjolele/one-man-band/
  35. 35. 35CONFIDENTIAL PRODUCTION-READY NIO SERVER new connection
  36. 36. 36CONFIDENTIAL SELECTOR PRODUCTION-READY NIO SERVER ServerSocket Channel acceptor thread new connection Acceptor threads Separate thread to not lose incoming connections when selector threads are overloaded
  37. 37. 37CONFIDENTIAL SELECTOR SELECTOR PRODUCTION-READY NIO SERVER ServerSocket Channel selector thread … acceptor thread Socket Channel SELECTOR selector threadSocket Channel SELECTOR selector threadSocket Channel new connection Selector threads Multiple processing threads to get benefit from multi core CPU Acceptor threads Separate thread to not lose incoming connections when selector threads are overloaded
  38. 38. 38CONFIDENTIAL SELECTOR SELECTOR Application thread pools (long running tasks) PRODUCTION-READY NIO SERVER ServerSocket Channel selector thread … worker thread worker thread worker thread application thread acceptor thread Socket Channel SELECTOR selector threadSocket Channel SELECTOR selector threadSocket Channel new connection Selector threads Multiple processing threads to get benefit from multi core CPU Acceptor threads Separate thread to not lose incoming connections when selector threads are overloaded Application threads For long tasks (back-end, DB, computation), to avoid blocking of selector threads
  39. 39. 39CONFIDENTIAL PARTIAL MESSAGE RECEIVING PROBLEM Message reader splits/joins data blocks into messages Data block Message Message Message Message Message Message message reader Data block Data block MessageMessage Data block
  40. 40. 40CONFIDENTIAL NIO CONCLUSION Complexity • complex code • learning curve for developers • handling partially received messages • combining async with sync code (need different thread groups) Benefits Drawbacks Single shared thread for many connections • less memory • less context switching overhead
  41. 41. 41CONFIDENTIAL NETTY ONE FRAMEWORK TO RULE THEM ALL
  42. 42. 42CONFIDENTIAL NETTY: JAVA NETWORK FRAMEWORK Some projects using Netty
  43. 43. 43CONFIDENTIAL NETTY SERVER ARCHITECTURE Network
  44. 44. 44CONFIDENTIAL transport Boss EventLoopGroup NIO NETTY SERVER ARCHITECTURE Network Socket Native JNI Thread pool for new connections
  45. 45. 45CONFIDENTIAL transport Boss EventLoopGroup channel channel pipeline Worker EventLoopGroup EventExecutorGroup NIO NETTY SERVER ARCHITECTURE Inbound Handler Inbound Handler Inbound Handler Outbound Handler Outbound Handler Outbound Handler Network Socket Native JNI Separate thread pool for blocking logic I/O thread pool For non-blocking logic Thread pool for new connections
  46. 46. 46CONFIDENTIAL channel channel pipeline NETTY HANDLERS SSL Handler encrypted encrypted
  47. 47. 47CONFIDENTIAL channel channel pipeline NETTY HANDLERS HTTP Request Decoder SSL Handler encrypted encrypted bytes
  48. 48. 48CONFIDENTIAL channel channel pipeline NETTY HANDLERS HTTP Request Decoder HTTP Object Aggregator SSL Handler encrypted encrypted bytes http messages buffer
  49. 49. 49CONFIDENTIAL channel channel pipeline NETTY HANDLERS HTTP Request Decoder HTTP Object Aggregator SSL Handler encrypted encrypted bytes http messages Your Business Logic Handler full http messages buffer
  50. 50. 50CONFIDENTIAL channel channel pipeline NETTY HANDLERS HTTP Request Decoder HTTP Object Aggregator HTTP Request Encoder SSL Handler encrypted encrypted bytes http messages bytes Your Business Logic Handler full http messages buffer full http messages
  51. 51. 51CONFIDENTIAL NETTY SERVER LIVE CODING EXAMPLE
  52. 52. 52CONFIDENTIAL WHAT IS FASTER: IO OR NIO?
  53. 53. 53CONFIDENTIAL WHAT IS FASTER: IO OR NIO? May be yes, may be not * ? ? ? ? NIO is faster Context switching is very slow Threads take up too much memory Thread synchronization will kill you
  54. 54. 54CONFIDENTIAL WHAT IS FASTER: IO OR NIO? May be yes, may be not * * Always point to holy-wars, see www.mailinator.com/tymaPaulMultithreaded.pdf ? ? ? Context switching is very slow Threads take up too much memory Thread synchronization will kill you IO is 25-30% faster (see benchmark*) NIO is faster
  55. 55. 55CONFIDENTIAL WHAT IS FASTER: IO OR NIO? May be yes, may be not * * Always point to holy-wars, see www.mailinator.com/tymaPaulMultithreaded.pdf ? ? Context switching is very slow Threads take up too much memory Thread synchronization will kill you IO is 25-30% faster (see benchmark*) NIO is faster New POSIX library in Linux > 2.6 IDLE thread costs almost zero Context switching is much much faster
  56. 56. 56CONFIDENTIAL WHAT IS FASTER: IO OR NIO? May be yes, may be not * * Always point to holy-wars, see www.mailinator.com/tymaPaulMultithreaded.pdf ? Context switching is very slow Threads take up too much memory Thread synchronization will kill you IO is 25-30% faster (see benchmark*) NIO is faster New POSIX library in Linux > 2.6 IDLE thread costs almost zero Context switching is much much faster Don’t overuse thread-local objects!
  57. 57. 57CONFIDENTIAL WHAT IS FASTER: IO OR NIO? May be yes, may be not * * Always point to holy-wars, see www.mailinator.com/tymaPaulMultithreaded.pdf Context switching is very slow Threads take up too much memory Thread synchronization will kill you IO is 25-30% faster (see benchmark*) NIO is faster New POSIX library in Linux > 2.6 IDLE thread costs almost zero Context switching is much much faster Don’t overuse thread-local objects! Use non-blocking data structures
  58. 58. 58CONFIDENTIAL CONCLUSION • Huge number of connections • Medium or slow data rate per connection • Asynchronous world (event-based) • Non blocking database • Queues • Web-sockets • Callbacks, listeners • Data streaming • Don’t use standard NIO, use frameworks like Netty • Highly skilled team because of complex code Use sockets* Use NIO* *Not a strict rule, always point to holy-wars • Small number of connections • Huge data rate per connection • Synchronous world (procedure-based) • Classic enterprise applications • Request-response protocols • Blocking database • Blocking RPC or web-service calls • Bottleneck is not server, but blocking resource • Make servers stateless and scale horizontally
  59. 59. 59CONFIDENTIAL Java Network Programming, 4’th edition O’REILLY Elliotte Rusty Harold BUY A BOOK, MAKE THEM RICH Netty in Action Manning Norman Maurer Java NIO O’REILLY Ron Hitchens • Code examples: github.com/kslisenko/java-networking • NIO performance: www.mailinator.com/tymaPaulMultithreaded.pdf • NIO tutorial: tutorials.jenkov.com/java-nio/index.html • ITT 2015 - Heinz Kabutz - The Multi-threading, Non Blocking IO: youtube.com/watch?v=uKc0Gx_lPsg • Netty - One Framework to rule them all by Norman Maurer: youtube.com/watch?v=DKJ0w30M0vg • Scalable IO in Java: http://gee.cs.oswego.edu/dl/cpjslides/nio.pdf
  60. 60. 60CONFIDENTIAL THANK YOU! QUESTIONS? kslisenko@gmail.com kslisenko linkedin.com/in/kslisenko/ Konstantin Slisenko kanstantsin_slisenka@epam.com
  • SalapuraMarina

    Feb. 8, 2021
  • x-maverick

    Jan. 27, 2021
  • YiLv4

    Nov. 19, 2020
  • SunghwanCho8

    Sep. 3, 2020
  • raghavendrabadiger7

    Aug. 27, 2020
  • RadhaKrishnaProddatu

    Aug. 11, 2020
  • ssuser17451a

    Aug. 3, 2020
  • ssuser5f054f

    Jul. 24, 2020
  • ssuserf5a6c1

    Jul. 20, 2020
  • JongMinKim42

    Jul. 8, 2020
  • guangmingdong

    Jun. 17, 2020
  • NaveenHegde3

    Jun. 2, 2020
  • IndikaGunawardana1

    May. 3, 2020
  • ssuser3d5a4f

    Mar. 29, 2020
  • AndreyShpitko

    Mar. 21, 2020
  • nahoya

    Mar. 19, 2020
  • jinohoang1994

    Mar. 14, 2020
  • s4553711

    Mar. 13, 2020
  • liyanjiu

    Feb. 11, 2020
  • guptavaibhav05

    Feb. 5, 2020

Let's dive under the hood of Java network applications. We plan to have a deep look to classic sockets and NIO having live coding examples. Then we discuss performance problems of sockets and find out how NIO can help us to handle 10000+ connections in a single thread. And finally we learn how to build high load application server using Netty. https://github.com/kslisenko/java-networking

Views

Total views

10,622

On Slideshare

0

From embeds

0

Number of embeds

569

Actions

Downloads

0

Shares

0

Comments

0

Likes

64

×