BENCHMARKING ONJVM

Introduction to JMHVladimir Tsanev

ABOUT MEDeveloper

WHAT IS JMHJava Microbenchmark Harness

Building, running and analysing benchmarks written inJava

Licensed under andBSD for samples

Developed @

GPL 2, with the Classpath Exception

OpenJDK

WHYWarm up

Dead code elimination

Loop unrolling

Branch prediction

False sharing

etc…

GET STARTEDIt is as simple as generating project with maven archetype

mvn archetype:generate \ ­DinteractiveMode=false \ ­DarchetypeGroupId=org.openjdk.jmh \ ­DarchetypeArtifactId=jmh­java­benchmark­archetype \ ­DgroupId=jpro.introjmh \ ­DartifactId=get­started \ ­Dversion=1.0­SNAPSHOT

Archetypes for kotlin, groovy, scala and java areprovided.

@BENCHMARKJMH benchmarks code is placed in methods annotated with

@Benchmarkpackage jpro.introjmh;

import org.openjdk.jmh.annotations.Benchmark;

public class MyBenchmark

@Benchmark public void testMethod() // This is a demo/sample template for building your JMH benchmarks. Edit as needed. // Put your benchmark code here.

POM.XML <!­­ This is the demo/sample template build script for building Java benchmarks with JMH. Edit as needed. ­­>

<prerequisites> <maven>3.0</maven> </prerequisites>

<dependencies> <dependency> <groupId>org.openjdk.jmh</groupId> <artifactId>jmh­core</artifactId> <version>$jmh.version</version> </dependency> <dependency> <groupId>org.openjdk.jmh</groupId>

DEMO - FIRST BENCHMARKjava ­jar benchmarks.jar ­help

RTFM (annotations are good place to get started)

Look around in the

This presentation :-)

JavaDoc

Samples

Mailing List

ANNOTATIONS

@AuxCounters @Benchmark @BenchmarkMode

@CompilerControl @Fork @Group

@GroupThreads @Measurement @OperationsPerInvocation

@OutputTimeUnit @Param @Setup

@State @TearDown @Threads

@Timeout @Warmup

@BENCHMARK@Benchmark

Annotates the benchmark method

@BENCHMARKMODE@BenchmarkMode(Mode.Throughput)

May be put at @Benchmark method or at enclosing class

Available modes

Throughput - operations per time unit

AverageTime - average time per operation

SampleTime - sample time per operation

SingleShotTime - for single operation

any combination of the above

@STATE@State(Scope.Benchmark)public static class BenchmarkState volatile double x = Math.PI;

Encapsulate benchmark state

May be injected in @Benchmark methods and fixtures

May be used on the benchmark instance - default state.

Available scopes

Benchmark

Group

Thread

@SETUP AND @TEARDOWN@State(Scope.Thread)public static class BenchmarkState @Setup(Level.Trial) public void up()

@TearDown(Level.Trial) public void down()

State fixtures

Available levels

Trial - executed before/after each run

Iteration - executed before/after each iteration

Invocation - HERE BE DRAGONS!

@PARAM@Param("1", "1", "2", "3", "5", "7")public int arg;

Mark benchmark parameter

Supports primitive/wrapper, strings and enums

Should be non-final fields

Value is given as string and coerced if needed

When multiple @Param JMH will compute the outerproduct

@GROUP AND @GROUPTHREADS@Benchmark@Group("g")@GroupThreads(3)public int produce()

@Benchmark@Group("g")@GroupThreads(1)public int consume()

Very useful for concurrent (producer/consumer)benchmarks

DEFAULTS@Warmup

@Measurement

@OutputTimeUnit

@Fork

@Threads

@Timeout

MISC@OperationsPerInvocation - number of operations perinvocation

@CompilerControl - affect JIT compilation

@AuxCounters - state object int/long fields and methods

INFRASTRUCTURE

BenchmarkParams Blackhole Control

IterationParams ThreadParams

DCEDead-Code Elimination

return

Blackhole

WAITING VS CPUBlackholes can consume not only the values, but also the

timeBlackhole.consumeCPU(1000);

INFRASTRUCTURE PARAMETERSBenchmarkParams - benchmark configuration

IterationParams - type (warmup/measurement),count, time, batch size

ThreadParams - group configuration

DEMO - DEAD CODE & FALSESHARING

PROFILERS

PROFILERSjava ­jar benchmarks.jar ­lprof

Internal profiler - run in the benchmark JVM, and mayquery the internal facilities

External profiler - run outside of JVM

STANDARD MBEANSProfiling via standard MBeans

cl - Classloader profiling

comp - JIT compiler profiling

gc - GC profiling

HOTSPOT (TM)Profiling via implementation-specific MBeans

hs_cl - HotSpot (tm) classloader profiling

hs_comp - HotSpot (tm) JIT compiler profiling

hs_gc - HotSpot (tm) memory manager (GC) profiling

hs_rt - HotSpot (tm) runtime profiling

hs_thr - HotSpot (tm) threading subsystem

SAMPLINGstack - Simple and naive Java stack profiler

PERF/XPERFperf - Linux perf Statistics

perfnorm - Linux perf statistics, normalized by operationcount

perfasm - Linux perf + PrintAssembly Profiler

xperf - Windows xperf + PrintAssembly Profiler

For PrintAssemby profiler you will need hsdis pluginhttps://wiki.openjdk.java.net/display/HotSpot/PrintAssembly

DEMO - BRANCH PREDICTOR

GENERATORSAnnotation processor

JMH Bytecode Generator

Reflection

ASM

RESULTS FORMATjava ­jar benchmarks.jar ­lrf

Text

CSV - (but also SCSV)

JSON

LaTeX

rfc4180

DEMO - URL VALIDATORjava ­jar benchmarks.jar ­rf csv ­rff url.csv

RESOURCEShttp://shipilev.net/talks/jvmls-July2014-benchmarking.pdf

http://shipilev.net/blog/2014/nanotrusting-nanotime

http://shipilev.net/talks/devoxx-Nov2013-benchmarking.pdf

http://psy-lob-saw.blogspot.bg/p/jmh-related-posts.html

Q & A