The following is a comparison of running the sample (basically an empty method) of jmh benchmark. Hardware platform is armv7l with 1GB of RAM. Software is Arch Linux with kernel 4.1.5-1-ARCH, running Oracle Java 1.8.0_51 with HotSpot Server VM.
Using default jmh settings, i.e. 10 forks of 20 iterations, each with 20 warmup iterations.
When run without -Xcomp:
Result "testMethod": 46954362.329 ?(99.9%) 121531.693 ops/s [Average] (min, avg, max) = (44999194.538, 46954362.329, 47305769.412), stdev = 514572.799 CI (99.9%): [46832830.635, 47075894.022] (assumes normal distribution) # Run complete. Total time: 00:06:55 Benchmark Mode Cnt Score Error Units MyBenchmark.testMethod thrpt 200 46954362.329 ? 121531.693 ops/s
When run with -Xcomp:
Result "testMethod": 47114913.482 ?(99.9%) 89902.070 ops/s [Average] (min, avg, max) = (46113318.172, 47114913.482, 47304787.674), stdev = 380650.996 CI (99.9%): [47025011.412, 47204815.553] (assumes normal distribution) # Run complete. Total time: 00:07:41 Benchmark Mode Cnt Score Error Units MyBenchmark.testMethod thrpt 200 47114913.482 ? 89902.070 ops/s
1. When -Xcomp is specified, the overall runtime is 11% longer. That is because the VM needs to wait the code to be compiled. i.e. the efficiency is lower
2. The error (i.e. variation) is 26% lower when -Xcomp is specified. That is probably because all code has been compiled and so no need to spend time on compilation during execution.
3. The average score (i.e. throughput) is almost the same. That is because the jmh does warmup iteration before each round of fork.