Coder Social home page Coder Social logo

jrt's Introduction

jRT


Version: 0.0.1


jRT is a instrumentation tool that logs and records networking I/O operations "response times" (applicaion response time if be correct).

jRT can be executed in one of three main ways:

  1. It can be run as a Java agent (using: java -javaagent:jRT.jar)

  2. It can be injected into a running application (using: jRT -pid )


Using jRT as a Java agent:

jRT is most often used as a java agent. This is useful for platforms and environments where a java agent is simpler to integrate into launch scripts, or in environments where using the bash jRT wrapper script is not practical (e.g. Windows, and environments where java is not directly launched from the command line).

jRT.jar can be used as a java agent using the following launch syntax:

% java -javaagent:jRT.jar MyProgram

or

% java -javaagent:jRT.jar="" MyProgram.jar -a -b -c

You can find the available options for the Java agent mode by running:

% java -javaagent:jRT.jar="-h"

Here is a Java agent usage example with explicit parameters:

% java -javaagent:jRT.jar="-start=10000,-si=1000,-l=jRTs.%p-%h.%d.%i,-mode=i2o" MyProgram.jar -a -b -c

This example will record response times experienced during the running of MyProgram.jar in log file jRTs.< PID >-< HOST NAME >.< DATE >.< INSTANCE# >.< i2o | o2i >.hlog. Measurement will start in 10 second delay, and interval data will be records every 1 second.

Useful java agent related notes:

Note 1: When used as a java agent, jRT will treat spaces, commas, and semicolons as delimiting characters ([ ,;]+). For example, the option string "-start=0 -si=1000" is equivalent to the option string "-start=0,-si=1000". This is useful for environments where placing space delimiters into quoted strings is difficult or confusing.

Note 2: I find that a common way people add jRT as a java agent is by using the _JAVA_OPTIONS environment variable. This often allows one to add the jRT java agent without significant launch script surgery. For example:

export _JAVA_OPTIONS='-javaagent:/path/to/jRT/target/jRT.jar="-start=20000 -si=1000"'


Reading and processing the jRT log with jRTLogProcessor:

jRT logs response time information in a histogram log (see HdrHistogram.org). This histogram log contains a full, high fidelity histogram of all collected result sin each interval, in a highly compressed form (typically using only ~200-400 bytes per interval). However, other than the timestamp and maximum response time magnitude found in the given interval, the rest of the log line for each interval is not human readable (it is a base64 encoding of a compressed HdrHistogram).

To translate the jRT log file to a more human-readable form, the jRTLogProcessor utility is provided. In it's simplest form, this utility can be used as such

% jRTLogProcessor -i mylog.hlog -o mylog

Which will produce log file mylog and mylog.hgrm containing a human readable interval log (with selcted percentiles in each interval), as well as a human readable histogram percentile distribution log.

jRTLogProcessor can also be used to produce log files for an arbitrary section of the jRT log, by using the optional -start and -end parameters.

See jRTLogProcessor -h for more details.


Launching jRT by attaching it to existing, running application:

The jRT agent can be injected into a live, running Java application if the environment supports the java attach API (which is typically available in java environments running Java SE 6 or later).

$ java -Xbootclasspath/a:jRT.jar -jar jRT.jar -pid

NOTE: In order to attach to a running java application, the running application needs to have ${JAVA_HOME}/lib/tools.jar in it's classpath. While this is commonly the case already for many IDE and desktop environments, and for environments that involve or enable other attachable agents (such as profilers), you may find that it is not included in your application's classpath, and that it needs to be added if attaching jRT at runtime is needed (launching jRT as a Java agent per the below may be a good alternative).


Response time Charts: Plotting jRT results

A jRTPlotter.xls Excel spreadsheet is included to conveniently plot jRT log files produced by jRTLogProcessor in "Response time Chart"form. To use the spreadsheet, load it into Excel, (make sure to enable macros), and follow the 2-step instructions in the main menu worksheet to automatically import the log files and produce the Response time Chart.

Note that jRTPlotter.xls reads the log files produced by jRTLogProcessor, (the interval log and the .hgrm histogram percentile distribution log), and not the .hlog log format that jRT outputs directly.


Supported/Tested platforms:

The jRT command is expected to work and has been tested on the following frameworks:

  • tomcat
  • VolanoMark
  • cassandra
  • netty
  • jetty

If you use jRT on other applications, please report back on your experience so that we can expand the list.


Example: adding jRT to Tomcat runs:

In Tomcat's catalina.sh script, replace the following line: exec "$_RUNJAVA" "$LOGGING_CONFIG" $JAVA_OPTS $CATALINA_OPTS

with: exec "$_RUNJAVA" -javaagent:$JRT_HOME/jRT.jar "$LOGGING_CONFIG" $JAVA_OPTS $CATALINA_OPTS


Note: Use of HdrHistogram.

jRT depends on and makes systemic use of HdrHistogram to collected and report on the statistical distribution of response times. This package includes an HdrHistogram.jar jar file to support this functionality. HdrHistogram sources, documentation, and a ready to use jar file can all be found on GitHub, at http://giltene.github.com/HdrHistogram


Building jRT:

jRT can be (re)built from source files using Maven:

% mvn clean package

jrt's People

Contributors

fijiol avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

jrt's Issues

Change to use getIntervalHistogramInto()

We should change the log writer to use the non-allocating variant getIntervalHistogramInto() (into an already allocated histogram) instead of the current getIntervalHistogram() (which creates a new histogram each time it is called). This should significantly reduce the allocation pressure ioHiccup produces in the instrumented process.

missing jRT metrics when "Attached" midway thru load test

Hello,

Thanks for sharing jRT, this is a unique and great measurement tool.

Not sure what can be done about this, but I've found a problem that I can reproduce readily.
If I start a load test that looks like this:
JMeter --> Jetty 1 --> Jetty 2
...where each "-->" is an http connection,
....and once the load is running at steady state, then I attach jRT,
....then all my jRT response time metrics show up as 0's.

But if I attach jRT before I start applying load, then it all works fine.....
I'm using the exact same jRT startup arguments for the good and bad run.

If I'm reproducing the problem and then I restart my JMeter test, all the 0.00's in the jRT data change to valid HTTP response times.

If I were a betting man, I'd say that jRT relies on capturing the socket creation to capture good data.
But when a connection pool is already warmed up, as in my steady state example, there are no socket creations (by definition of an HTTP connection pool), thus impairing jRT's ability to trace.
Here is how I'm starting jRT, but I'm pretty sure these arguments don't impact the issue much:
java -Xbootclasspath/a:$MY_CP -jar jRT.jar -pid=$MY_PID -agentargs='-mode=i2o,remote-port=8676,local-port=8676,sample-interval=5000'

Below is the problem data. Note that all the "Interval_Max" times are 0.000.

`"StartTimestamp","Interval_Length","Interval_Max","Interval_Compressed_Histogram"

[StartTime: 1469990446.956 (seconds since epoch), Sun Jul 31 13:40:46 CDT 2016]

0.008,0.461,0.000,HISTggAAACN42pNpmdzIwMDAwQABTBCK+QUQuxnsWMBg/4EBBQAAm+sFqg==
0.469,5.010,0.000,HISTggAAACN42pNpmdzIwMDAwQABTBCK+QUQuxnsWMBg/4EBBQAAm+sFqg==
5.479,5.005,0.000,HISTggAAACN42pNpmdzIwMDAwQABTBCK+QUQuxnsWMBg/4EBBQAAm+sFqg==
10.484,5.003,0.000,HISTggAAACN42pNpmdzIwMDAwQABTBCK+QUQuxnsWMBg/4EBBQAAm+sFqg==
15.487,5.005,0.000,HISTggAAACN42pNpmdzIwMDAwQABTBCK+QUQuxnsWMBg/4EBBQAAm+sFqg==
20.492,5.006,0.000,HISTggAAACN42pNpmdzIwMDAwQABTBCK+QUQuxnsWMBg/4EBBQAAm+sFqg==`

...and here is the good data:

55.299,5.003,3019.899,HISTggAAAFZ42u3YQQmAQBQE0FmtYQgbiBd7CTawwMYwkkcjKCwm8CbvwfDvcxr+sO1rMtc0XTv9+WQZj5rpCgAAAAAAAAAAfFVUAADYMwAAAAAA/Nj7Ry43LkUGhw== 60.302,5.005,3019.899,HISTggAAAGV42u3UUQmAQAwA0J2KKQxhA/HHXoINLGAMI/lpBIUzg6D3Hox9jo1t3bLOEeMWWZVTfdwx9fsWwxkAAAAAAACUIxkB9hwo6M7TS3XTy3195Z+mn80DAAAAAAAAgLK1T24uRAQGlw== 65.307,5.002,3019.899,HISTggAAAG142u3RsQmAMBAF0FPBwhkcwg3Exr0EN3ABx3AkS0dQ0MpGDZbvQfgEAvmX1OM0RHRznPIzivVYfbPM0W4BAAAAAAAAAJAiu+XTubf7r/c/9coSe6f2+3uev/8L7wUAAAAAAADAd9WV5Q7/4Aaj 70.309,5.002,3019.899,HISTggAAAHB42u3QwQmDQBAF0L8GcrGFFGEHIZf0FUgHacAyLMljSkhAT4Lgqsf3YBj4AzMwt/fnlTz6TJqpXcZ/Pbuhz/0bAAAAAAAAAKBOqcyX87JznoP71zSVeXbeKSf/Mwf/BQAAAAAAAACwVTv36w8yoQal 75.311,5.002,3019.899,HISTggAAAGp42u3SsQmEQBAF0NkVTGzhirADMbEv4Tq4BizDkgwtQUEjQfRC5T0Yhg/zs/l8f31EO8Qmb6uY1unqcYhmDgAAAAAAAAB4q3zI6eI+neR8s/dv/6r3NMnLAQAAAAAAAAAvVe27XAAbeAak 80.313,5.006,3019.899,HISTggAAAG142u3PsQmAQAwF0KiFuINDuIHYuJfgBi7gGI5k6QgK2gpeIYi8B+E3SS5Xj9MQ0c1xys8o1qP6Zpmj3QIAAAAAAAAA/ia7ybu+/OF8lvh+JM6l7n36/3jpnrfuBwAAAAAAAAD4mvLKagc84Aak

I used my PR #12 to make sure my filter criteria is ok, see port 8676 info below.

remote-addr:remote-port <-> 127.0.0.1:local-port # /127.0.0.1:59061 <-> 127.0.0.1:8676 === true remote-addr:remote-port <-> 127.0.0.1:local-port # localhost/127.0.0.1:8676 <-> 127.0.0.1:59061 === true 2016-07-31 10:44:43.062 INFO 12028 --- [qtp134999601-73] /__admin : RequestHandlerClass from context returned com.github.tomakehurst.wiremock.http.AdminRequestHandler. Normlized mapped under returned 'null' remote-addr:remote-port <-> 127.0.0.1:local-port # /127.0.0.1:59062 <-> 127.0.0.1:8675 === false remote-addr:remote-port <-> 127.0.0.1:local-port # /127.0.0.1:59063 <-> 127.0.0.1:8676 === true remote-addr:remote-port <-> 127.0.0.1:local-port # localhost/127.0.0.1:8676 <-> 127.0.0.1:59063 === true 2016-07-31 10:44:48.330 INFO 12028 --- [qtp134999601-74] / : RequestHandlerClass from context returned com.github.tomakehurst.wiremock.http.StubRequestHandler. Normlized mapped under returned 'null' remote-addr:remote-port <-> 127.0.0.1:local-port # /127.0.0.1:59064 <-> 127.0.0.1:8675 === false remote-addr:remote-port <-> 127.0.0.1:local-port # localhost/127.0.0.1:8676 <-> 127.0.0.1:59065 === true remote-addr:remote-port <-> 127.0.0.1:local-port # /127.0.0.1:59065 <-> 127.0.0.1:8676 === true remote-addr:remote-port <-> 127.0.0.1:local-port # /127.0.0.1:59066 <-> 127.0.0.1:8675 === false

If you can reproduce this, at a minimum it seems like we need a comment in the doc for this.

Thanks again for a great monitoring tool, I plan on using it a lot.
--Erik

fix main // errors while running >> java -jar jRT.jar <<

Of course, it is written that jRT is an agent, and there is a readme.. but wouldn’t it be better to have more user-friendly output than we have in case when we try to just run it?

$ java -jar jRT.jar
Error: A JNI error has occurred, please check your installation and try again
Exception in thread "main" java.lang.NoClassDefFoundError: com/sun/tools/attach/AgentInitializationException
at java.lang.Class.getDeclaredMethods0(Native Method)
at java.lang.Class.privateGetDeclaredMethods(Class.java:2701)
at java.lang.Class.privateGetMethodRecursive(Class.java:3048)
at java.lang.Class.getMethod0(Class.java:3018)
at java.lang.Class.getMethod(Class.java:1784)
at sun.launcher.LauncherHelper.validateMainClass(LauncherHelper.java:544)
at sun.launcher.LauncherHelper.checkAndLoadMain(LauncherHelper.java:526)
Caused by: java.lang.ClassNotFoundException: com.sun.tools.attach.AgentInitializationException
at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
... 7 more

port discovery enhancement request

Creating a jRT filter for port number or host name is difficult if you're not completely familiar with a JVM's network traffic.

Would be nice for jRT to somehow inform the end user which local and remote port numbers and possibly host names were seeing network traffic.

--Erik

write README file

Readme file should contain information:

  • functionality description
  • how to build/install
  • how to run
  • description of parameters
  • where to file new issues
  • how to plot graphs

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.