What is going on under the hood in JVM?

1. Serkan ÖZAL
JVM UNDER THE HOOD

2. AGENDA
I. JVM Concepts
II. Memory Management
III. GC
IV. Class Loading
V. Execution Engine
VI. Multi-Threading
VII. JNI

3. I. JVM CONCEPTS

4. TERMS
• JVM
• Runs the bytecodes
• Doesn't understand Java source code
• JRE
• Execute java programs
• = JVM + Packages Classes (lang, util, math, ...) + Runtime libraries (native .dll/.so libs)
• JDK
• Compiles and builds java programs
• = JRE + Development/Debugging Tools (javac, javap, javadoc, jdb, …)

5. HOTSPOT
• Implementation of the JVM
• Originally developed by Sun and now owned by Oracle
• Initially available as an add-on for Java 1.2 (1999)
• Became the default Sun JVM in Java 1.3 (2000)
• = Interpreter + JIT + GC + Class loading + ...

6. II. MEMORY MANAGEMENT

7. HEAP MEMORY

8. NON-HEAP MEMORY

9. STACK

10. OBJECT LAYOUT
• Objects are 8 bytes aligned by default
• On JDK 8+, Can be configured by «-XX:ObjectAlignmentInBytes=?»
• Can only be power of 2 between 8-256
• All fields are type aligned
• Fields are packed in the order of their size (except references which are last)
• Sub and parent class fields are never mixed
• Use JOL (Java Object Layout) to analyze object layout

11. OBJECT SCHEMA
• Objects
• Mark word [4/8 bytes]
• Class pointer [4/8 bytes]
• Fields ...
• Arrays
• Mark word [4/8 bytes]
• Class pointer [4/8 bytes]
• Array length [4 bytes]
• Elements …

12. COMPRESSED OOPS
• Based on 8x object alignment rule
• Encode/decode by bit shifting (3 bit shift for default (8 bytes) object alignment)
• Can be configured by «-XX:+UseCompressedOops»
• Enabled by default on JDK 7+ (when heap <= 32GB)
• «-XX:+UseCompressedClassPointers» for classes on JDK 8+

13. OBJECT REFERENCES
• Types of references:
• Strong Reference
• Soft Reference
• Weak Reference
• Phantom Reference
• ReferenceQueue

14. III. GC

15. COLLECTION TECHNIQUES
• Copy Collection
• Mark & Sweep (+ Compact) Collection

16. MINOR GC
• Collects garbage from the Young (Eden + Survivors) space
• Triggered when there is not enough space in Eden for new allocation
• References from Old space to Young space are found via card marking table

17. MAJOR GC (≅ FULL GC)
• Collects garbage from the Old (Tenured) space
• Mostly involves compaction
• Many Major GCs are triggered by Minor GCs because of promotion failure
• Also targets PermGen (JDK 7-) and Metaspace (JDK 8+)

18. SERIAL GC
• Uses STW mark-copy collector for Young space
• Uses STW mark-sweep-compact collector for Old space
• Both of these collectors are single-threaded collectors
• Enabled by «-XX:+UseSerialGC»

19. PARALLEL GC
• Uses STW mark-copy collector for Young space
• Uses STW mark-sweep-compact collector for Old space
• Young space is collected by multi-threaded collector
• Old space is collected by single-threaded collector
• Enabled by «-XX:+UseParallelGC» (default for JDK 8-)
• «-XX:+UseParallelOldGC» for parallel Old space collector
• «-XX:ParallelGCThreads=?» (number of cores by defaults)

20. CMS
• Uses STW mark-copy collector for Young space
• Uses concurrent mark-sweep collector for Old space
• No compaction normally, but uses free-lists
• FullGC with compaction when “promotion failure”
• Enabled by «-XX:+UseConcMarkSweepGC»

21. G1
• Heap is split into a number (typically about 2048) smaller heap regions
• The regions that contain the most garbage are collected first
• Evacuation Fully Young
• Remembered Sets
• Humongous regions and objects
• Concurrent marking
• Evacuation Mixed
• Enabled by «-XX:+UseG1GC» (default at JDK 9)
• «–XX:MaxGCPauseMillis=?», default value = 200ms

22. SHENANDOAH
• JEP 189: Shenandoah: An Ultra-Low-Pause-Time Garbage Collector
• Drived by Red Hat
• First released on Fedore 24 on June 2016
• Not a generational garbage collector
• Concurrent, parallel and region-based garbage collector
• Supports concurrent compaction by Brooks forwarding pointers

23. IV. CLASS LOADING

24. CLASS LOADING
• Classes are loaded by classloaders
• Some core classes (java.lang, …) eagerly
• Others lazily when they are first requested
• JVM finds, parses, verifies bytecodes and generates metadata
• Class metadata is saved into Permgen (JDK 7-) / Metaspace (JDK 8+)
• Classes are searched hierarchically through classloaders
• If a class has already been loaded, it returns it
• Otherwise, it delegates the search to the parent class loader
• If the parent class loader does not find the class, tries to find and load the class itself

25. CLASS LOADER DELEGATION MODEL

26. V. EXECUTION ENGINE

27. INTERPRETER
• Converts bytecode into assembly code using a template table
• Template table has assembly code for each bytecode instruction
• Runtime flag «-Xint» can be used to force interpreted mode
• No compiler optimisation is performed
• Start up behaviour for most JVMs

28. JIT
• JVM continually monitors the code for the following critical metrics:
• Method entry counts
• Loop back branch counts
• Escape Analysis: No Escape, Arg Escape, Global Escape
• CHA: Monomorphic, Bimorphic, Megamorphic calls
• OSR (On-Stack Replacement)
• Inlining
• Intrinsic
• De-optimization

29. COMPILERS
• C1
• Level 1: no profiling
• Level 2: basic profiling (with invocation and back-edge counters)
• Level 3: full profiling (Inlining, CHA, ...)
• C2
• Level 4: advanced profiling (Loop unrolling, Dead Code Elimination, Escape analysis, ...)
• Tiered (C1 + C2)
• Introduced at JDK 7
• Default since JDK 8

30. VI. MULTI-THREADING

31. THREAD MANAGEMENT
• 1:1 mapping between Java threads and native OS threads
• Thread states:
• Internal VM Threads:
• VM Thread
• Periodic Task Thread
• GC Threads
• Compiler Threads
• Signal Dispatcher Thread
• Finalizer Thread
• Reference Handler Thread
• Attach Listener Thread
• ...
• _thread_new
• _thread_in_Java
• _thread_in_vm
• _thread_in_native
• _thread_blocked

32. SYNCHRONIZATION
• Biased Locking
• Can be disabled by «-XX:-UseBiasedLocking»
• Delay can be configured by «-XX:BiasedLockingStartupDelay»
• Can be rebiased to another thread
• Can be revoked to lightweight lock
• Lightweight Locking
• In case of contention, inflated to heavyweight lock
• Heavyweight Locking
• Uses OS mutex/condition variables

33. SAFEPOINT
• On safepoint
• All threads running java code (interpreted or compiled) are suspended
• Threads running native code may continue to run as long as they do not
• attempt to contact with JVM via JNI API
• return from native to java
• A thread is at safepoint
• while running JNI code
• if it is blocked, waiting or parked
• A thread is not at safepoint while running java code (interpreted or compiled)

34. WHEN SAFEPOINTS ARE USED?
• Garbage collection pauses
• Code deoptimization
• Class redefinition (e.g. hot swap or instrumentation)
• Biased lock revocation & Monitor deflation
• Various debug operation (e.g. deadlock check or stacktrace dump)
• Heap dumping

35. HOW SAFEPOINTS WORK?
• Safepoint check (polling) is implemented as memory reads a barrier
• When safepoint is required, JVM unmaps page with that address provoking page fault
• Safepoint polling exist
• between any 2 bytecodes while running in the interpreter (effectively)
• on non-counted loop back edge in C1/C2 compiled code
• on method return in C1/C2 compiled code.

36. VII. JNI

37. HOW TO USE JNI?
• Native libraries are looked up at «java.library.path»
• On Windows, it maps to «PATH»
• On Linux, it maps to «LD_LIBRARY_PATH»
• On OS X, it maps to «DYLD_LIBRARY_PATH»
• (New|Delete)GlobalRef
• (New|Delete)LocalRef
• (Get|Release)???ArrayElements
• (Get|Set)???ArrayRegion
• (Get|Release)(PrimitiveArray|String)Critical
• Exception(Occurred|Check|Clear)
• Use «-Xcheck:jni» for enabling additional checks

38. CRITICAL NATIVES
• Critical natives can be inlined
• Supported only in HotSpot JVM starting from JDK 7
• Must satisfy the following conditions:
• must be static and not synchronized
• argument types must be primitive or primitive arrays
• implementation must not call JNI functions
• Starts with «JavaCritical_» instead of «Java_»
• You need both critical and standard implementation

39. THANKS