Coder Social home page Coder Social logo

java-ubench-agent's Introduction

Microbenchmarking Agent for Java

A JVMTI agent to be used with microbenchmarks. Basic features:

  • Reports major JVM events such as GC and JIT runs.
  • Can collect performance counters through JNI.
  • Can collect accurate time through JNI.

Usage

private static final int LOOPS = 10;
private static final String[] EVENTS = {
	/* The most precise clock available. */
	"SYS:wallclock-time",
	/* Number of JIT compilation events. */
	"JVM:compilations",
	/* L1 cache misses (only Linux with PAPI). */
	"PAPI:PAPI_L1_DCM"
};

public static void myBenchmark() {
	/*
	 * We would have LOOPS measurements and we want to record these EVENTS.
	 *
	 * But we limit ourselves to supported events (i.e. in this case it
	 * would remove PAPI:PAPI_L1_DCM event if libpapi would not be available
	 * on the system.
	 */
	Benchmark.init(LOOPS, Measurement.filterSupportedEvents(EVENTS));

	for (int i = 0; i < LOOPS; i++) {
		/* Start the measurement. */
		Benchmark.start();

		/* Here goes your code that ought to be measured. */

		/* Stop the measurement. */
		Benchmark.stop();
	}

	/* Get the results (available as Iterable<long[]>). */
	BenchmarkResults results = Benchmark.getResults();

	/* Either print them in CSV (to be later processed)... */
	BenchmarkResultsPrinter.toCsv(results, System.out);

	/* ... or as a space-padded table. */
	BenchmarkResultsPrinter.table(results, System.out);
}

To run your program with our agent, add the ubench-agent.jar to your classpath and start JVM with the C-agent: -agentpath:libubench-agent.so (GNU/Linux) or -agentpath:ubench-agent.dll (Windows).

A more generic interface Measurement is available if you wish to bind the measurement to a specific thread or if you need to measure more things at once (though internal limitations of Linux perf subsystem may apply).

Compilation

You will need recent version of Ant and GCC. Then simple

ant && ant test test-junit

shall compile the agent and run the built-in tests (you might need to set JAVA_HOME if you have JDK in non-standard location).

You might also wish to execute

ant lib

to create a JAR with the agent class files (it also copies the C-agent to the same folder for easier use).

Demo

Package cz.cuni.mff.d3s.perf.demo contains a small demo: the MeasureHashing class measures performance of a file hashing with Java built-in MD5. Run it with

ant demo

to produce results that could look like this:

demo:
     [java]   SYS_WALLCLOCK JVM_COMPILATIO   PAPI_TOT_INS    PAPI_L1_DCM
     [java]         1516055              9        5539366         107004
     [java]         1082882              0        3994763          14362
     [java]          983808              1        3690908          13472
     [java]          258992              0        1551475           7049
     [java]           69000              1         468639           1405
     [java]           67542              0         470675           1296
     ...
     <truncated>
     ...
     [java]           67091              0         470675           1286
     [java]           68760              1         469812           1295
     [java] Hash is d41d8cd98f00b204e9800998ecf8427e.

Supported events

To list supported events, run the ListEvents demo program or execute ant list-events.

  • SYS:wallclock-time
    • The most precise clock available. clock_gettime(CLOCK_MONOTONIC) on Linux, QueryPerformanceCounter() on Windows.
  • SYS:thread-time
    • CPU thread time (i.e. not counting when thread is waiting).
  • SYS:thread-time-rusage
    • Same as SYS:thread-time but uses data from getrusage() call on Linux. Seems to be much less precise but it may save you one extra call if you also query SYS:forced-context-switches. Linux only.
  • SYS:forced-context-switches
    • Number of forced context switches (i.e. quantum was exhausted). Linux only.
  • JVM:compilations
    • Number of JIT compilation events.
  • PAPI:*
    • When built on Linux with libpapi available, the agent can collect any event supported by PAPI (note that you can use all the events reported by papi_avail and papi_native_avail).
    • Note that to use different components (e.g. perf and rapl), different event sets has to be created.

java-ubench-agent's People

Contributors

ceresek avatar jysunhy avatar lbulej avatar vhotspur avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar

java-ubench-agent's Issues

Fix builds with GCC on Windows

Using the default Windows configuration for GitHub Actions ends with the following bug:

compile-agent-gcc:
    [apply] D:\a\java-ubench-agent\java-ubench-agent\src\c\events.c: In function 'getter_thread_time':
    [apply] D:\a\java-ubench-agent\java-ubench-agent\src\c\events.c:96:60: warning: unused parameter 'bench' [-Wunused-parameter]
    [apply]    96 | static long long getter_thread_time(const benchmark_run_t *bench, const ubench_event_info_t *UNUSED_PARAMETER(info)) {
    [apply]       |                                     ~~~~~~~~~~~~~~~~~~~~~~~^~~~~
    [apply] In file included from D:\a\java-ubench-agent\java-ubench-agent\src\c\jnifunc.c:22:
    [apply] D:\a\java-ubench-agent\java-ubench-agent\src\c\jnifunc.c: In function 'Java_cz_cuni_mff_d3s_perf_NativeThreads_getNativeId':
    [apply] D:\a\java-ubench-agent\java-ubench-agent\out\agent/cz_cuni_mff_d3s_perf_NativeThreads.h:11:63: error: invalid suffix "i64" on integer constant
    [apply]    11 | #define cz_cuni_mff_d3s_perf_NativeThreads_INVALID_THREAD_ID -1i64
    [apply]       |                                                               ^~~~
    [apply] D:\a\java-ubench-agent\java-ubench-agent\src\c\jnifunc.c:39:32: note: in expansion of macro 'cz_cuni_mff_d3s_perf_NativeThreads_INVALID_THREAD_ID'
    [apply]    39 |                 return (jlong) cz_cuni_mff_d3s_perf_NativeThreads_INVALID_THREAD_ID;
    [apply]       |                                ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Probably caused by javah generating MSVC-specific header as i64 suffix is a non-standard MS extension.

Support for measuring JVMCI threads does not work with JDK 9

Trying to measure JVMCI threads with JDK 9 ends with an exception:

Exception in thread "main" cz.cuni.mff.d3s.perf.MeasurementException: PAPI_attach failed: Invalid argument.
at cz.cuni.mff.d3s.perf.Measurement.createAttachedEventSetNative(Native Method)
at cz.cuni.mff.d3s.perf.Measurement.createAttachedEventSet(Measurement.java:27)
at cz.cuni.mff.d3s.perf.SamplingHelper.lambda$__initJvmciExtraEvents$0(SamplingHelper.java:283)
at java.base/java.util.stream.ReferencePipeline$4$1.accept(ReferencePipeline.java:212)
at java.base/java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1494)
at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:484)
at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:474)
at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:550)
at java.base/java.util.stream.AbstractPipeline.evaluateToArrayNode(AbstractPipeline.java:260)
at java.base/java.util.stream.IntPipeline.toArray(IntPipeline.java:516)
at cz.cuni.mff.d3s.perf.SamplingHelper.__initJvmciExtraEvents(SamplingHelper.java:286)
at cz.cuni.mff.d3s.perf.SamplingHelper.(SamplingHelper.java:201)
at cz.cuni.mff.d3s.perf.SamplingHelper.(SamplingHelper.java:153)
at spec.harness.LaunchPlain.main(LaunchPlain.java:38)

API to retrieve list of supported events

Following shall be probably enough for a list:

String[] Benchmark.getSupportedEvents();

and maybe extended with

String Benchmark.getEventDescription(String event);

for more detailed event description (when possible).

Check for permissions when filtering supported events

Currently, the filtering of supported events is based solely on the ability to resolve the event name.

But when running in some kind of sandbox, the event may exist but may not be available due to other restrictions.

For PAPI events, we should create a dummy event set, add the event there and only if this is OK we can mark the event as supported.

This is easily demonstrated on CI builds on GitHub actions where PAPI is available but no events are permitted.

Allow access to collected data from Java

Now, data are stored to a standalone file, identified by name. Extend the API so the caller can collect the data in a better form: either print to a provided stream or return plain array and leave the processing completely on the Java (not JNI) side.

Individual thread sampling with JVMCI can return negative numbers

In some of the ScalaBench measurements with JVMCI merge threads turned OFF, the values in the instruction count column are occassionally negative. This could be a problem in the ScalaBench sampling helper class, but since it does very little with the data, it is more likely an agent issue.

Get PAPI component from event name

Now, PAPI component has to be specified manually (via PAPI/comp_name:event_name). However, if the event name is valid, it should be possible to retrieve its component id via PAPI_get_event_info. This should simplify event specification a little bit.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.