fbacchella / jrds

Another monitoring application, intentend to be simple to use and extensible.

What is it?

Jrds is performance collector, much like cacti or munins. But it intends to be more easy to use and able to collect a high number of machines in a very short time. It's fully written in java and avoid call external process to increase performances. It uses RRD4J, a clone of rrdtool written in java.

The site is jrds.fr

How it works?

JRDS is a java web application, that can run in any servlet server like tomcat or resin. It can also run in a standalone mode, using jetty.

It uses threads to parallelize work. Each host is collected within the same thread and the number of simultaneous threads can be configured. It use only one thread for each host to avoid overload of a server.

It use mainly snmp to collect data, but can be easily extended. There is also some jdbc probes, a agent using RMI for the communication, and it can also parse XML data collected with HTTP. The currently available probes can be found here. Additional collectors can be used using external jars.

jrds's Issues

configurable archives data period

It would be great to be able to customize the period of data storage

The next/previous day buttons should set the date to a full day 00:00 -> 23:59

No warnings if step/timeout does not follow the "step - 2 x timeout" rule

If you configure a step and a timeout where "step - 2 x timeout" is less or equal to 0, the timer won't collect anything but it won't be obvious why.

There is no warning and no error in logs.

Sum is broken

<sum> <element /><elements/>... </sum> in graphs is broken

UML diagram

Provide a pseudo UML diagram showing configuration elements relationship

Possible NPE in HcHttpProbe

Offending piece of code:

    public Map<String, Number> getNewSampleValues() {
        log(Level.DEBUG, "Getting %s", getUrl());
        HttpClientStarter httpstarter = find(HttpClientStarter.class);
        HttpClient cnx = httpstarter.getHttpClient();

The method find() may return NULL if the class is not found.

I added a new probe to support multi columns and multi rows from jdbc datasource

the probe class is jrds.probe.jdbc.PowerJdbcProbe
and the probe desc is

DaasNodePerf
DaasNodePerf
jrds.probe.jdbc.PowerJdbcProbe
SHOW @@DataNode
NAME;EXECUTE;ACTIVE


EXECUTE
counter




ACTIVE
gauge




DataNodeFluxGraph
DataNodeConPoolGraph

for the given table as follows

NAME EXECUTE ACTIVE

datanode1 2 10
datanode2 4 20

the probe will dynamic create RRD DSs like follows:

key value

---

datanode1.EXECUTE 2
datanode1.ACTIVE 10
datanode2.EXECUTE 4
datanode2.ACTIVE 20

and the probe also dynamic create RRD Graphs of the follow names:

DataNodeFluxGraph.datanode1.EXECUTE
DataNodeFluxGraph.datanode2.EXECUTE
DataNodeConPoolGraph.datanode1.ACTIVE
DataNodeConPoolGraph.datanode2.ACTIVE

the url of graph is like this :
http://localhost:8080/jrds/graph/localdaas/DataNodeFluxGraph_dn_1.EXECUTE?probe=daasnodeperf

Switching tab in navigator using shortcut also change tab in jrds

Apparently Dojo had the exquisite idea to use the same shortcut as the browser to navigate within a TabContainer:

http://dojotoolkit.org/reference-guide/1.10/dijit/layout/TabContainer.html#keyboard

The problem is that when I switch tab on my browser using shortcut (ctrl+pageup/down) it will do the same on jrds and refresh the content (which will loose everything, the filter, the opened folder, etc..)

Buttons unclickable on IE

Clicking on buttons rights to the charts (Popup, details, history, save) has no effect (Tested with IE 8). It works with FF and Chromium.

Probe arguments must be named

At the moment, probe's arguments are ordonner, and not named.

Currently:

        <connection type="jrds.probe.JMXConnection">
                <arg type="Integer" value="8999" />
                <arg type="String" value="user" />
                <arg type="String" value="password" />
        </connection>

So it's not possible to have something like a default user, and a specific (by host) password.

Expected:

        <connection type="jrds.probe.JMXConnection">
                <arg name="port" value="8999" />
                <arg name="user" value="user" />
                <arg name="password" value="password" />
        </connection>

Collect stop if a step is changed

If the step changed for a given probe, no collect will be done anymore.

REST API interval parameter should be a long

Interval given in second is multiplied by 1000 to then work with millisecond.
So, for example, one month (2 628 000 seconds * 1 000) overflow MAX_INT (2 147 483 647)

No timeout for graph requested with a period

If you retrieve the graph with an url of type http://hostname/graph/host_name/graph_name?period_definition jrds put it on cache without any timeout. So if you ask the same graph a second time with the same period definition you will get the same graph with no new values.

graph url with / gives 404 error

http://jrds.myworld.com/graph/hostname/diskwait-cciss/c0d0
while
http://jrds.myworld.com:8080/graph/hostname/cpuraw
will work perfectly

Invalid JaCoCo plugin configuration in pom.xml

In pom.xml, the JaCoCo plugin is configured like this:

 <configuration>
   <destfile>${basedir}/target/coverage-reports/jacoco-unit.exec</destfile>
   <datafile>${basedir}/target/coverage-reports/jacoco-unit.exec</datafile>
 </configuration>

Whereas it should be:

 <configuration>
   <destFile>${basedir}/target/coverage-reports/jacoco-unit.exec</destFile>
   <dataFile>${basedir}/target/coverage-reports/jacoco-unit.exec</dataFile>
 </configuration>

According to the plugin configuration:

http://www.eclemma.org/jacoco/trunk/doc/dump-mojo.html#destFile

Removal of method getCollectStrings() from ProbeDesc prevents the use of older versions of jrds-agent

Stack trace:

[2019-01-22 14:25:09.383] [WARN ] [binjr-async-thread-1] [132] Error during probe creation of type DiskIOPdh with args []: 
java.lang.NoSuchMethodError: jrds.ProbeDesc.getCollectStrings()Ljava/util/Map;
	at jrds.probe.PdhProbeUniqueInstance.buildArgs(PdhProbeUniqueInstance.java:29)
	at jrds.probe.PdhProbe.configure(PdhProbe.java:17)
	at jrds.probe.PdhProbeUniqueInstance.configure(PdhProbeUniqueInstance.java:21)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
	at jrds.factories.ProbeFactory.configure(ProbeFactory.java:101)
	at jrds.configuration.HostBuilder.makeProbe(HostBuilder.java:328)
	at jrds.configuration.HostBuilder.parseFragment(HostBuilder.java:185)
	at jrds.configuration.HostBuilder.makeHost(HostBuilder.java:89)
	at jrds.configuration.HostBuilder.build(HostBuilder.java:62)
	at jrds.configuration.HostBuilder.build(HostBuilder.java:42)
	at jrds.configuration.ConfigObjectFactory.getObjectMap(ConfigObjectFactory.java:85)
	at jrds.configuration.ConfigObjectFactory.setHostMap(ConfigObjectFactory.java:154)
	at jrds.HostsList.configure(HostsList.java:142)
	at jrds.HostsList.<init>(HostsList.java:76)
	at com.exalead.tools.binjr.sources.cvdiag.adapters.CvdiagDataAdapter.getBindingTree(CvdiagDataAdapter.java:93)
	at eu.binjr.core.controllers.MainViewController.buildTreeViewForTarget(MainViewController.java:947)
	at eu.binjr.core.controllers.MainViewController.lambda$showAdapterDialog$34(MainViewController.java:759)
	at eu.binjr.core.data.async.AsyncTaskManager$2.call(AsyncTaskManager.java:86)
	at javafx.concurrent.Task$TaskCallable.call(Task.java:1425)
	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
	at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
	at java.base/java.lang.Thread.run(Thread.java:834)

ApacheStatusDetails cannot listen to port 443

I tried to listen to the port 443 with the ApacheStatusDetails:

<probe type="ApacheStatusDetails">
    <attr name="port">443</attr>
</probe>

But it won't work because it will try to retrieve apache's status using the HTTP procotol when HTTPS is required in this case.

Specifying a wrong -DpropertiesFile should warn the user / abort jrds

I tried to install a standalone jrds by following http://jrds.fr/installunix , but specified a -DpropertiesFile that was not here (typo).

The interface started happily:

ERROR - [jrds.HostsList] Probes directory not configured, can't configure
INFO - Application jrds started
^C
INFO - Application jrds will stop
INFO - Average wait time: NaN ms
INFO - Application jrds stopped

With only a "minor looking" error at the start, but a completely empty (but usable) web interface.

It might be a good idea to throw an error if at the start some necessary properties are not set properly, or if the declared properties (either coming from the file I gave or from the defaults) point to non existing directories?

Thanks!

Hosts list and probes list in REST API

Hi

I have some requests to the REST API :

• have the hosts list when you do a GET query on /graph/
• have the probes list when you do a GET query on /graph//

Thanks

Data stores produced by RPN operations do not have a persistent name

The names of data storse generated through RPN functions are not stable across JRDSruntime sessions (i.e. they change each time JRDS is restarted).

Since the name of the data store is the only element that allow to identify which column in the csv returned by the download service corresponds to which store, this makes it quite tricky for applications that uses this service to persist their state or configurations across runtime sessions.

Splunk's span like option

For some graphs, it could interesting to have a span option (like the splunk's one) to add values in a given time range. The goal is have a better look graph when you have only few values by minutes.

To achieve that, jrds must compute the sum (or any other aggregate function) in the given span timeperiod for the given field, and then use these values to graph.

IllegalAccessError to jrds.JuliToLog4jHandler.<init>()V when loading old probes

Following 438229b, jrds.JuliToLog4jHandler.<init>()V is private. This breaks loading probes that were compiled before this commit (eg. probes from CloudView 15x.SP3 in my case, but most likely every version before that as well) with JRDS standalone:

ERROR - [jrds.HostsList] Top level starter class jrds.probe.JRDSSocketFactory failed to register: tried to access method jrds.JuliToLog4jHandler.<init>()V from class jrds.probe.JRDSSocketFactory
ERROR - Error stack: 
ERROR - java.lang.IllegalAccessError: tried to access method jrds.JuliToLog4jHandler.<init>()V from class jrds.probe.JRDSSocketFactory
        at jrds.probe.JRDSSocketFactory.<clinit>(JRDSSocketFactory.java:26)
        at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
        at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
        at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
        at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
        at java.lang.Class.newInstance(Class.java:442)
        at jrds.HostsList.configure(HostsList.java:179)
        at jrds.HostsList.<init>(HostsList.java:76)
        at jrds.Configuration.<init>(Configuration.java:50)
        at jrds.Configuration.configure(Configuration.java:20)
        at jrds.webapp.StartListener.contextInitialized(StartListener.java:67)
        at org.eclipse.jetty.server.handler.ContextHandler.callContextInitialized(ContextHandler.java:800)
        at org.eclipse.jetty.servlet.ServletContextHandler.callContextInitialized(ServletContextHandler.java:444)
        at org.eclipse.jetty.server.handler.ContextHandler.startContext(ContextHandler.java:791)
        at org.eclipse.jetty.servlet.ServletContextHandler.startContext(ServletContextHandler.java:294)
        at org.eclipse.jetty.webapp.WebAppContext.startWebapp(WebAppContext.java:1349)
        at org.eclipse.jetty.webapp.WebAppContext.startContext(WebAppContext.java:1342)
        at org.eclipse.jetty.server.handler.ContextHandler.doStart(ContextHandler.java:741)
        at org.eclipse.jetty.webapp.WebAppContext.doStart(WebAppContext.java:505)
        at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
        at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:132)
        at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:114)
        at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:61)
        at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
        at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:132)
        at org.eclipse.jetty.server.Server.start(Server.java:387)
        at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:114)
        at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:61)
        at org.eclipse.jetty.server.Server.doStart(Server.java:354)
        at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
        at jrds.standalone.Jetty.start(Jetty.java:148)
        at jrds.bootstrap.BootStrap.main(BootStrap.java:81)

I'm not sure whether using old probes with JRDS standalone is supported or not, but we do it a lot. Would it break anything to make the constructor public again?

Thanks!

Add a magnifier to the graphs

It would be great to add a magnifier to zoom inside graphs.

This might be done using a library such as:
http://marqueetool.net/

An example of use:
http://marqueetool.net/examples/send-cropped-image-coordinates-to-the-server/

Does the project still maintain ？

I downloaded the project, but there is no way to run it. Is the project still under maintenance?

check time scale order

When filling the time scale, if End is anterior to Begin, the graph crash without any explication.

jrds.fr is down

Looks like the website http://jrds.fr is currently down.
Could you please have a look?

Thanks.

refresh button

Add a refresh button in the button list in a graph

Thumbnail graphs

I'd like to have a way to get thumbnail pictures of the graphs.
This is to build custom dashboards in order to compare at a glance various servers of a cluster.
Another idea is to show the graphs of all the ports of the full switch stacks on a single view, in order to see very quickly (still at a glance) which ports the trafic is going through.
BTW, thanks for providing us with jrds. Easy to use and powerful software!

Create a PID file when start in standalone mode

When a probe failed, default values are used anyway

There is no clean way to notify of a failed probe collect, an custom exception, or a plain runtime exception, should be thrown, not silly catch {return emptyMap}.

Project Homepage http://jrds.fr/ is down

http://jrds.fr/ has been down for a few days.

Maybe consider adding some screenshots to the repo README.

Here is a link for an archived copy: https://web.archive.org/web/20190211013511/http://jrds.fr/

Global timer step for a probe type

It should be good to have the capability to set a step timer for a probe type.

For exemple, "timer.IfXSnmp.step=10" will set a timer of 10s between collect.
But, If we want to set a timer for a particular probe (and override the global timer step), the actual set method () should be keeped.

Debug target that embbed JavaScript source maps files

It would be very helpful when trying to work with the UI to have the JS source maps files (*.map) of the Dojo framework.

Reformat logging to me more compact

A good idea take from http://lightbody.net/blog/2008/12/tip-compact-logging-in-java.html.

Unit type is not returned by the graphdesc service

The graphdesc service (i.e.: http://localhost:8080/graphdesc?id=1755186310) that returns the description of a graph does not seem to return the unit type (typically "SI" or "binary") as part of its XML response.

The graphdesc DTD (https://github.com/fbacchella/jrds/blob/master/dtd/graphdesc.dtd) suggests that this piece of information could be part of the XML response, in the form of a element called "unit", populated with either "SI", "binary and the unit's arithmatical's base as an int, but I have never seen it present in my tests.

Scroll only graph list, not option panel above

When reading a long list of graphs, sometimes I want to change some option like scale or time range. This forces to go back to the top. I'd rather have the option panel stay at the top of the screen independently of the graphs scrolling.

Also, maybe making the option panel more compact would avoid wasting too much space if it does not scroll.

probeName and graphName should be called probeId and graphId

The distinction should be easier.

Compilation and installation problems

Is it possible to provide an installation document to guide the compilation and installation?

JDK1.8
An error is reported when running mvn clean install -DskipTests

sort algo

Sort algo is wrong :
GigabitEthernet9/0/19 is before GigabitEthernet9/0/1 (ng83)

Resizing the graph tree will make the graphs go down

The size of the "dateForm" keep increasing when resizing the width of the graph tree.

Current date in filter change when clicking on it

How to reproduce:

• Select "Last hour"
  • Click in the "from time" input

The scale will change to "Manual" (expected) but the from date will change to the T-24h, it should not !

Avaliability of jrdswmi.jar?

The link on http://jrds.fr doesn't work anymore and i would like to explore using it to monitor windows machines..

/dana

Apache probe must have an (optional) URL argument

The apache probe expect the mod_status URL is avaiable at /server-status, while it's a configurable parameter in apache. Must be configurable in jrds too

front end for smokeping

smokeping is a good tool, but the web front end is broken by design. A virtual probe using smokeping's rrd should be implemented.

No process cpu / memory jrds charts on solaris

Those informations are not provided by kstat, it must be parsed in /proc.

jrds as a local agent

jrds with jrdsagent should be able to run without configuration.
It will just probe the localhost and autodiscover it's configuration.
The only parameters should then be the run directory and the listening port.

Of course both jrds and jrdsagent would then be packaged together.

Automatically unfold the tree when accessing to the host with ?host

Hello

It should be good to have a new var to automatically unfold the tree when access to the host with ?host var.
For example : ?host=myhostname&tree