Annoyed by some bug in your Java application? Need to profile that performance bottleneck in your JVM? And all of this inside a container? Don't worry: we got you covered. Join this session to learn more about using JDK Flight Recorder (JFR) and JDK Mission Control (JMC) to troubleshoot, monitor, and profile Java applications. Also learn how to do all of this for Java applications inside containers, with ContainerJFR.

1. V0000000
JDK Flight Recorder + OpenShift
Introduction to
ContainerJFR
Andrew Azores
Senior Software Engineer
1

2. V0000000
What we’ll
discuss today
Agenda
2
Java Profiling
JDK Flight
Recorder
JDK Mission
Control
JFR in Containers
ContainerJFR
DEMO
Security
How to Set Up
Your Applications

3. V0000000
3
A brief high-level overview
of profiling and monitoring
with Java and JVMs
Java Profiling

4. V0000000
Overview
4
JVMs, being virtual machines, can provide a lot of information about what they,
and your application, are doing at runtime, through various mechanisms:
▸ JMX and MXBeans
▸ JVM Agents
▸ Flight Recorder (open-sourced in 2018, previously commercial/proprietary)

5. V0000000
Examples of Metrics
5
Some examples of useful metrics that can be captured through any or all of these
mechanisms:
▸ CPU load
▸ Memory usage
▸ Heap allocation
▸ Garbage Collection occurrences
▸ Garbage Collection pause times
▸ Thread states
▸ Disk or Network I/O
▸ And many more...

6. V0000000
6
Built-in framework in
modern OpenJDKs to
enable minimal-overhead
diagnostic and profiling
data about your JVM
applications
JDK Flight
Recorder

7. V0000000
Flight Recorder vs MXBeans
7
What does Flight Recorder offer that MXBeans don’t?
▸ Inherent ability to persist to disk
▸ Circular buffer to limit data collected by age or buffer size
▸ JVM can perform a dump of buffer to disk on exit/crash
▸ Flight Recorder gathers data within the JVM itself, no JMX client connection required

8. V0000000
Flight Recorder vs Agents
8
What does Flight Recorder offer that Monitoring Agents don’t?
▸ Inherent ability to persist to disk, no need to roll your own
▸ Circular buffer to limit data collected by age or buffer size, no need to roll your own
▸ Built-in JFR events are implemented by the JVM itself, even in native code. May be
difficult or impossible to get performance overhead so low with an Agent capturing
equivalent data
▸ Flight Recorder is very stable as a core component of the JVM, removing risk of a bad
Agent implementation causing instability
▸ Standardized and extremely efficient/compact binary representation of data

9. V0000000
Additional Features
9
JFR also offers many other great features:
▸ Present in all OpenJDKs 11+ (and more recent 8us)
▸ Simple API for applications to define custom event types at compile time and hook into
the JFR infrastructure
･ Maintains the same great performance characteristics and minimal overhead of JFR
･ Allows events to be tailored to your specific application, ex. an event when a service request is
received and an event when the response is written, or an event when a database connection is
opened/closed, etc.
▸ Recordings can be configured to capture various different subsets of events, and
enabling different recordings with (potentially overlapping) sets of events does not
start a new thread or incur any other intrinsic overhead
･ Recordings can also be configured to only capture events with a duration above a certain
threshold, or to only capture samples of an event at a specific rate, etc.
▸ Recordings can be configured with JVM flags to ensure that applications always have
JFR enabled from startup to exit
▸ Recordings can be started/stopped at runtime over a JMX connection or using jcmd

10. V0000000
Desktop application for
opening and analyzing
Flight Recorder files,
retrieving them from
(local?) JVMs, and
MXBean data
10
JDK Mission
Control

11. V0000000
JDK Mission Control
11
JMC has tons of tools and features and is too large for the scope of this talk. In
brief, it’s the go-to tool for analyzing Flight Recording files, and can also be used
for starting and retrieving Flight Recordings.

12. V0000000
Mission Control to OpenShift
12
▸ How do you connect JMC, a desktop application, to your JVM?
･ JMC can discover local JVMs, but what if your JVM is in a container?
･ What if that container is running inside a Pod in OpenShift?
▸ Route?
･ How do you configure a route for a JMX service URL (not HTTP)?
▸ NodePort? LoadBalancer? ExternalIP?
･ Is this convenient? Does it make sense to expose the application this
way?
▸ IngressController!
･ This might work, but you need JMX over TLS+SNI and some more
Ingress setup work

13. V0000000
How does an end user
interact with JFR and
consume Flight
Recordings? How does this
fit with containerization?
13
JFR in Containers

14. V0000000
JFR in Containers
14
Can we skip JMC for now and just use JFR directly with our containerized
applications, then get that data into JMC directly later?

15. V0000000
JFR in Containers
15
▸ JFR can dump to a file
･ But that’s to the container’s local filesystem - how do we get the file
out?
▸ JFR can be configured with flags at startup time
･ What if you want to change configuration later after noticing a
performance degradation? Do a rolling replacement of all replicas just
to change recording settings? Can you still capture the performance
degradation details after this?
▸ JMX can start and retrieve recordings!
･ We’re back to Routes/NodePorts/ExternalIPs/IngressControllers to
get a JMX connection to the application

16. V0000000
ContainerJFR is a bridge
between yourself and your
JVM applications in the
cloud
16
ContainerJFR

17. V0000000
Container-Native
Runs as a “sidecar” pod alongside your application
Uses JMX from within the Namespace
Allows start/stop/retrieval of recordings at runtime
Routable HTTPS API
Exposes an API with cluster auth to export recordings
Online Analysis
Provides web-based tools for basic analysis in-cluster
ContainerJFR
How does ContainerJFR address these challenges?
17

18. V0000000
18
Demo Time

19. V0000000
How ContainerJFR keeps
your recordings and
application data safe
19
Safety & Security

20. V0000000
Safety & Security
How does ContainerJFR keep
application data safe and
secure?
20
No agent, no library to bake-in to your application. Only
standard JMX must be enabled on your application via
JVM flags - no recompile or attachment.
Non-Interference
SSL/TLS over JMX is supported and HTTPS is enabled by
default.
Encrypted
Users must authenticate to both the cluster and the target
before accessing data or performing actions.
Multiple Factor Authentication

21. V0000000
The few small steps
needed to allow your
applications to talk to
ContainerJFR
21
Configuring Your
Applications

22. V0000000
Configuring Your Applications
How to Set Up Your Application for ContainerJFR
22
Just add some JVM flags to your application’s entrypoint script or command arguments. An example:
▸ -Dcom.sun.management.jmxremote.port=9091 # listen on port 9091, or any number if the port is named “jfr-jmx”
▸ -Dcom.sun.management.jmxremote.rmi.port=9091
▸ -Dcom.sun.management.jmxremote.authenticate=true # enable JMX authentication
▸ -Dcom.sun.management.jmxremote.password.file=/app/resources/jmxremote.password # define users for JMX auth
▸ -Dcom.sun.management.jmxremote.access.file=/app/resources/jmxremote.access # set permissions for JMX users
▸ -Dcom.sun.management.jmxremote.ssl=true # enable JMX SSL
▸ -Dcom.sun.management.jmxremote.registry.ssl=true
▸ -Djavax.net.ssl.keyStore=/app/resources/keystore # set your SSL keystore
▸ -Djavax.net.ssl.keyStorePassword=somePassword # set your SSL keystore password
JMX authentication and SSL are technically optional, but in any production deployment these should be enabled. After enabling SSL for your
application’s JMX server, copy or POST your certificate to ContainerJFR’s truststore so that ContainerJFR trusts the connection.

23. V0000000
linkedin.com/company/red-hat
youtube.com/user/RedHatVideos
facebook.com/redhatinc
twitter.com/RedHat
Red Hat is the world’s leading provider of enterprise
open source software solutions. Award-winning
support, training, and consulting services make
Red Hat a trusted adviser to the Fortune 500.
Thank you
23