monitoring and troubleshooting tools...termination of hotspot jvm based java applications...
TRANSCRIPT
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Monitoring and Troubleshooting Tools Available in Your Java 9 “bin” Folder
Poonam Parhar Consulting Member of Technical Staff JVM Sustaining Engineer, Oracle
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Safe Harbor Statement
The following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracle’s products remains at the sole discretion of Oracle.
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
What is Troubleshooting ?
• Systematic approach to solving problems
• Three simple steps:
1. Understand the problem/error
2. Collect the required diagnostic data
3. Analyze the collected data
• Good Troubleshooting tools are our friends!
3
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
JDK Troubleshooting Tools
• Tools that allow us to
– Monitor Java applications
– Profile Java applications
– Extract useful diagnostic information from Java applications
– Analyze diagnostic data
– Even perform after-the-fact analysis
• Can work locally or remotely
4
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Agenda
5
Tools available in JDK 9 ‘bin’ folder
1. Monitoring Tools
2. Troubleshooting Tools
3. Newly Added Tools
4. Decommissioned Tools
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Monitoring Tools
• jconsole - Starts a graphic console to monitor and manage Java applications.
• jps - Experimental: Lists the instrumented Java Virtual Machines (JVMs) on the target system.
• jstat - Experimental: Monitors JVM statistics
• jstatd - Experimental: Monitors JVMs and enables remote monitoring tools to attach to JVMs.
• jmc – Java Mission Control: JMX console, and JFR recording and analysis tool
6
Automated Analysis
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
JConsole
• JConsole is a GUI based monitoring tool for Java applications
• JMX client
• Helps in monitoring and managing
– Java Applications and the JVM
– Live and remote applications
• jconsole executable available in JDK/bin
• Using JConsole locally is not recommended for production environments
• For remote monitoring: com.sun.management.jmxremote.port=portNum
7
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Java
Ap
plic
atio
n
8
JMX Client Application
Inst
rum
en
ted
JV
M
JMX
Age
nt
MBean Server
MBean MBean
MBean
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | 9
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | 10
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | 11
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | 12
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | 13
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | 14
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
jps (experimental) – Java Process Status tool
• Lists the JVMs running on the system
• Obtains processes information from /tmp/hsperfdata_user/<pid> files
• List of the JVMs reported can be limited by the OS level permissions granted to the user
• Can list the JVMs on local or remote hosts
• For remote monitoring ‘jstatd’ should be running on the remote system
• Accepts optional argument <protocol:><//hostname>:[<port>]
• On local hosts, the VM identifier is typically the OS process id of the application
15
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | 16
$ jps -h illegal argument: -h usage: jps [-help] jps [-q] [-mlvV] [<hostid>] Definitions: <hostid>: <hostname>[:<port>] $ jps -m 9848 GCBasher.jar -time:100000000 7084 Jps –m $ jps -l 3056 jdk.jcmd/sun.tools.jps.Jps 9848 GCBasher.jar $ jps -v 6324 Jps -Dapplication.home=d:\Java\jdk-9 -Xms8m -Djdk.module.main=jdk.jcmd 9848 GCBasher.jar C:\Users\pobajaj.ORADEV>jps -q 9876 9848
jps examples
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
jstat (experimental) – JVM Statistics
• Tool to monitor JVM statistics
• Performance statistics about HotSpot JVM
• Reads information from /tmp/hsperfdata_user/<pid> file
• Accepts various options to print statistics about different components of the JVM
– class, compiler, gc, gccapacity, gccause, gcnew, gcnewcapacity, gcold, gcoldcapacity, gcmetacapacity, gcutil, printcompilation
• For remote monitoring ‘jstatd’ must be running on the remote system
• Accepts optional argument <protocol:><//hostname>:[<port>]
17
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
jstatd (experimental) - jstat Daemon
• Launches an RMI server application that monitors the creation and termination of HotSpot JVM based java applications
• Provides an interface to allow remote monitoring tools to attach to Java virtual machines running on the system.
• Note: There is no encryption or authentication with jstatd
18
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
JMC - Java Mission Control
• Non-intrusive, low-overhead tool to monitor, manage and profile Java applications
• Suitable for production environments
• JMC Tools Chain
– JMX Console • Monitoring and managing live java applications
• JMX MBeans Browser
– Java Flight Recorder • Engine built in the Java Runtime
• Record interesting events at the application and the JVM level
• Compact binary proprietary format
• Tracking events leading up to a problem provides great help in troubleshooting
19
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | 20
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | 21
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | 22
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | 23
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | 24
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
JFR in Java 9
• Supported JFR APIs
– We can create our own custom JFR events and record them in the produced recordings
• Performance enhancements in data collection and recording
• Ability to dump recordings even at VM crashes or OOMs
• Improved and new Events (safepoint, codecache, compiler, G1, module)
25
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
JMC in Java 9
• Improved user interface
• Automated analysis of flight recordings
– Provides analysis what could be going wrong with suggestions for solutions and links to documentation
– Based on a set of defined rules
– Custom rules can be added
– Individual rules can be modified or disabled
• Headless automated analysis
• http://hirt.se/blog/
26
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | 27
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | 28
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | 29
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | 30
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Monitoring Tools
• jconsole - Starts a graphic console to monitor and manage Java applications.
• jps - Experimental: Lists the instrumented Java Virtual Machines (JVMs) on the target system.
• jstat - Experimental: Monitors JVM statistics
• jstatd - Experimental: Monitors JVMs and enables remote monitoring tools to attach to JVMs.
• jmc – Java Mission Control: JMX console, and JFR recording and analysis tool
31
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Troubleshooting Tools
• jcmd – Sends diagnostic commands to a running JVM
• jdb – java platform debugger
• jinfo – Experimental: Obtains and updates configuration information
• jmap – Experimental: Can create heap dumps or heap histograms for a java process
• jstack – Experimental: Prints Java thread stack traces for a Java process
• jhsdb – Attaches to a hanging process or a crash dump file for postmortem analysis
32
New tool in Java 9
New diagnostic commands
in Java 9
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
jcmd – Java Command
• Java Diagnostic Command utility
• Utility to send diagnostic commands to a running JVM
• Uses Attach API to execute d-commands (helper routines) in the JVM
• Can be used only on the same local machine as the target JVM
• New diagnostics commands added in Java 9
33
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
jcmd: popular commands
• jcmd <process id/main class> VM.version
• jcmd <process id/main class> VM.system_properties
• jcmd <process id/main class> VM.flags
• jcmd <process id/main class> GC.class_histogram
• jcmd <process id/main class> GC.class_stats
• jcmd <process id/main class> GC.heap_dump filename=heapdump
• jcmd <process id/main class> Thread.print
34
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
jcmd: Java Flight Recordings commands
• jcmd <process id/main class> JFR.start name=MyRecording settings=profile delay=20s duration=2m filename=/tmp/myrecording.jfr
• jcmd <process id/main class> JFR.check
• jcmd <process id/main class> JFR.stop
• jcmd <process id/main class> JFR.dump name=MyRecording filename=/tmp/myrecording.jfr
35
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | 36
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | 37
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | 38
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | 39
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | 40
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | 41
jcmd Attach API
Diagnostic Routines
Commands
Output
Local machine
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
jdb – Java Debugger
• Command-line debugger for Java Class files
• jdb <options> <class> <arguments>
• Uses Java Debug Interface (JDI)
• JDI is a component of Java Platform Debugger Architecture (JPDA)
• Can perform inspection and debugging of a local or remote Java Virtual Machine
42
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | 43
jinfo jmap jstack
Local live process
Core dump file
Remote Debug Server JDK 8 JDK 9
Serviceability Agent
Attach API
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
jinfo – Java Configuration Information
• Configuration (flags) information of a running Java process
• Can update the manageable flags at runtime
• It can:
– Print the value of a VM flag
– Enable or disable a specified VM flag
– Set the value of a VM flag
– Print VM flags and system properties
44
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
jmap (experimental) – Java Memory Map
• Heap objects information and Heap dumps
• Classloader Statistics
• -clstats <pid>
• Finalization Information
• -finalizerinfo <pid>
• Class Histograms
• -histo[:live] <pid>
• Heap Dumps
• -dump:<dump-options> <pid>
• JDK 9 removed: heap info and shared memory mappings
45
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
jstack (experimental) – Java Stack
• Thread dump
• Deadlock detection
• jstack –l to see the concurrent locks information
• Removed support for –F and –m options
46
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
jcmd – replacement for old tools
jcmd Individual tools Function
jcmd without any argument jps List Java Processes
jcmd with VM.system_properties, VM.flags, VM.set_flag jinfo VM Configuration
jcmd with GC.class_stats, GC.class_histogram, GC.heap_dump, GC.finalizer_info
jmap Generate Heap Dump and Class Histogram, get Class Statistics and Finalization information
jcmd with Thread.print jstack Thread Dump
jcmd PerfCounter.print jstat Print Performance Counters
47
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
jhsdb- Java HotSpot Debugger
• Can be used to launch a postmortem debugger (SA) to analyze core dump files
• Can be used against live java processes too
jhsdb clhsdb [--pid pid | --exe executable --core coredump] jhsdb debugd [options] pid [server-id]|[option] executable core [server-id] jhsdb hsdb [--pid pid | --exe executable --core coredump] jhsdb jstack [--pid pid | --exe executable --core coredump] [options] jhsdb jmap [--pid pid | --exe executable --core coredump] [options] jhsdb jinfo [--pid pid | --exe executable --core coredump] [options] jhsdb jsnap [options] [--pid pid | --exe executable --core coredump]
48
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | 49
jinfo jmap jstack
Local live process
Core dump file
Remote Debug Server JDK 9
Serviceability Agent
Attach API
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | 50
jhsdb
Local live process
Core dump file
Remote Debug Server JDK 9
Serviceability Agent
Non-cooperative manner
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | 51
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | 52
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | 53
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | 54
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | 55
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | 56
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | 57
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | 58
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | 59
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | 60
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Troubleshooting Tools
• jcmd – Sends diagnostic commands to a running JVM
• jdb – java platform debugger
• jinfo – Experimental: Generates configuration information
• jmap – Experimental: Can create heap dumps or heap histograms for a java process
• jstack – Experimental: Prints Java thread stack traces for a Java process
• jhsdb – Attaches to a hanging process or a crash dump file for postmortem analysis
61
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Tools Removed in JDK 9
• Java VisualVM
• jhat
• jsadebugd
62
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Java VisualVM
• Other tools (profiling and live monitoring) available: Java Mission Control, JConsole
• Several Open Source heap dumps analysis tools available
– Eclipse MAT
• Not included with JDK 9
• Open Source Project: https://visualvm.github.io/download.html
63
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
jhat
• Web application that can be launched with the ‘jhat’ command available in the JDK/bin folder
• Enables heap dump analysis by browsing objects in the heap dump using any web browser
– By default the web server is started at port 7000.
• jhat supports a wide range of pre-designed queries and Object Query Language(OQL) to explore the objects in heap dumps
• Several other heap dump analysis tools available
• Removed in JDK 9
64
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
jsadebugd
• Attaches to a Java process or core file and acts as a debug server
• Up until JDK 8, tools such as jstack, jmap, and jinfo could remotely attach to a debug server using Java RMI
• Removed in Java 9
• The debug server can now be launched with ‘jhsdb debugd’
65
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Summary
• Enhanced Tools
– jcmd – we encourage to use this tool instead of other individual tools
– Java Mission Control
• New Tools – jhsdb for postmortem analysis
• Removed Tools
– Java VisualVM
– jhat
– jsadebugd
66
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Oracle MOOC: Java Virtual Machine Troubleshooting
• Troubleshooting Memory Problems in Java Applications
• Enrollments still open until Oct 11
• After enrollment ends, 1 week of grace period to finish the quizzes
http://www.oracle.com/goto/jvm
67
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Stay connected
• Join us: DevOps Corner (Developer Lounge – Moscone West)
• Learn more: openjdk.java.net | wercker.com/java
• Follow: @OpenJDK, @wercker #JavaOne #DevOps
68