psyinfra / prometheus-raritan-pdu-exporter

Python-based Raritan PDU exporter for prometheus.io.

Prometheus Raritan PDU Exporter

Python-based Raritan PDU exporter for prometheus.io.

This exporter uses the Raritan JSON-RPC API to find inlet, outlet, pole, and device sensors to expose their readings to the Prometheus monitoring system.

We have purposely opted not to use the Raritan PDU Python API since direct calls to the JSON-RPC API are very straight-forward. Furthermore, only a handful of methods are used (getInlets, getOutlets, getMetaData, getDeviceSlots, getDevice, getReading and getState) on the /bulk endpoint, ignoring most of the methods included in the Python API. As a result we do not have to bundle the Raritan PDU Python API with this project.

Installation

git clone [email protected]:psyinfra/prometheus-raritan-pdu-exporter.git
cd prometheus_raritan_pdu_exporter
pip install .

Usage for PDU collection

raritanpdu [-h] -c config [-w LISTEN_ADDRESS] [-l LOG_LEVEL [LOG_LEVEL ...]]

optional arguments:
  -h, --help            show this help message and exit
  -c config, --config config
                        configuration json file containing PDU addresses 
                        and login info
  -w LISTEN_ADDRESS, --web.listen-address LISTEN_ADDRESS
                        Address and port to listen on (default = :9950)
  -l LOG_LEVEL [LOG_LEVEL ...], --log LOG_LEVEL [LOG_LEVEL ...]
                        Specify logging level for internal and external 
                        logging, respectively (Default is WARNING,CRITICAL)

Example

raritanpdu --web.listen-address :9950

The entry points raritanpdu and prometheus_raritan_pdu_exporter are identical and can be used interchangeably.

Health checks

For every HTTP endpoint other than /healthcheck, a collection of metrics from all of the PDUs in the configuration file will be performed and Prometheus style metrics will be returned.

The /healthcheck endpoint will skip a collection of metrics from the PDUs and instead immediately return a 200 response with the text "Server is running". This /healthcheck endpoint can be useful when running in Kubernetes or other platforms to provide a healthcheck that the HTTP server is still successfully running and isn't hanging.

Debugging

To enable debugging, set -l debug to log debug messages. Note that this will provide a lot of additional information and is therefore not a recommended setting for long-term use in production.

Docker Image

A Docker image can be built with:

docker build -t raritanpdu .

To run the image, expose port 9950 and mount the configuration file:

docker run -p 9950:9950 -v <path/to/config.json>:/prometheus-raritan-pdu-exporter/config.json raritanpdu

Testing

Run tests using tox

pip3 install tox
tox

Testing without VCR cassettes

Our tests use vcrpy to record responses from the Raritan PDU JSON-RPC API, because access to this data is not guaranteed. If you want to test the Raritan PDU Exporter on your PDU setup, copy your config.json file to tests/fixtures/config.json and remove the tests/fixtures/vcr_cassettes/ folder and its contents. Tests can then be run as usual, but will take longer as PDUs are requested for data to re-establish the VCR cassettes with data from the PDUs in the config file.

prometheus-raritan-pdu-exporter's Issues

Expose a healthcheck endpoint

Thanks for this, we had a bunch of pain trying to get SNMP exporting work with the SNMP exporter for our Raritan PDUs while this just worked out of the box without my brain having to think about OIDs and MIB files 😹

We are deploying this into a Kubernetes cluster but liveness probes cause a scrape of the configured Raritan PDUs because the prometheus_client.start_http_server returns the metrics for any requests. This is slower and more expensive than we'd like for a liveness probe and it initially caused the pod to be restarted aggressively. While we've backed off the probes and allowed a longer timeout on them it would be nice to instead have a basic /healthcheck or /healthz etc endpoint that just shows that the HTTP server is up and running.

This example shows how you can expose a /metrics endpoint as part of a larger Flask app and I was thinking it might be nice to do that but just provide a single other healthcheck endpoint in Flask and then push /metrics to the prometheus_client.make_wsgi_app method.

Error Handling when pdu is being updated.

I was happily testing away on this exporter when I updated the software on one. When the pdu is down this is the error from stderr.

[2023-06-06 09:57:44,709] ERROR: (pdu-idfel-1-red1#-) Uncaught Exception: 0, message='Attempt to decode JSON with unexpected mimetype: text/html', url=URL('https://pdu-idfel-1-red1.admin.evergreen.laika.com/bulk')

The exporter itself just acts like the pdu doesn't exist.

If you are monitoring you probably want to know when something is unreachable and you definitely want something more useful than "I couldn't parse the json".

No Response returned when running the raritan-pdu-exporter

Hi,

I am trying to get the raritan-pdu-exporter to work. I configured a config file like the config.json-example file. When running the raritanpdu I get an Error: No response returned. I would like to know if someone has an idea what i could try next or what I am doing wrong.

The command I use is: raritanpdu -c config --web.listen-address :9950

And here is the log of the debug message.

[2022-10-27 11:48:52,072] INFO: Internal log level: DEBUG
[2022-10-27 11:48:52,072] INFO: External log level: CRITICAL
[2022-10-27 11:48:52,072] INFO: Loading configuration file 'config'
[2022-10-27 11:48:53,957] DEBUG: (pdu1_name#-) No _sensors_from_connectors response for pDevSlot.0, pDevSlot.1, pDevSlot.2, pDevSlot.3, pDevSlot.4, pDevSlot.5, pDevSlot.6, pDevSlot.7, pDevSlot.8, pDevSlot.9, pDevSlot.10, pDevSlot.11, pDevSlot.12, pDevSlot.13, pDevSlot.14, pDevSlot.15, pDevSlot.16, pDevSlot.17, pDevSlot.18, pDevSlot.19, pDevSlot.20, pDevSlot.21, pDevSlot.22, pDevSlot.23, pDevSlot.24, pDevSlot.25, pDevSlot.26, pDevSlot.27, pDevSlot.28, pDevSlot.29, pDevSlot.30, pDevSlot.31
[2022-10-27 11:48:54,255] ERROR: No responses returned
[2022-10-27 11:48:54,255] CRITICAL: Exporter shut down due to an error during the setup procedure. Please contact your administrator.

Thank you in advance.

Startup issue when authentication fails.

Startup fails when authentication fails.

When the program attempts to contact the pdu with incorrect authentication startup fails with an error indicating the incorrect mime type.

[2023-10-16 16:42:52,911] ERROR: 0, message='Attempt to decode JSON with unexpected mimetype: text/html', url=URL('https://pdu-rdc-17-red1.evergreen.laika.com/bulk')
[2023-10-16 16:42:52,912] CRITICAL: Exporter shut down due to an error during the setup procedure. Please contact your administrator.

The pdu does correctly throw a 401 and if you are browsing to the above it presents a dialog for logging in which may be the source of mimetype error.

[root@core-vm-tsdb-staging prometheus_raritan_pdu_exporter]# wget -v --user=baduser --password=bogus --no-check-certificate https://pdu-rdc-39-blue1.admin.evergreen.laika.com/bulk
--2023-10-16 11:40:12--  https://pdu-rdc-39-blue1.admin.evergreen.laika.com/bulk
Resolving pdu-rdc-39-blue1.admin.evergreen.laika.com (pdu-rdc-39-blue1.admin.evergreen.laika.com)... 10.4.98.153
Connecting to pdu-rdc-39-blue1.admin.evergreen.laika.com (pdu-rdc-39-blue1.admin.evergreen.laika.com)|10.4.98.153|:443... connected.
WARNING: cannot verify pdu-rdc-39-blue1.admin.evergreen.laika.com's certificate, issued by ‘/CN=Generic Demo CA’:
  Unable to locally verify the issuer's authority.
    WARNING: certificate common name ‘Demo Certificate’ doesn't match requested host name ‘pdu-rdc-39-blue1.admin.evergreen.laika.com’.
HTTP request sent, awaiting response... 401 Unauthorized
Connecting to pdu-rdc-39-blue1.admin.evergreen.laika.com (pdu-rdc-39-blue1.admin.evergreen.laika.com)|10.4.98.153|:443... connected.
WARNING: cannot verify pdu-rdc-39-blue1.admin.evergreen.laika.com's certificate, issued by ‘/CN=Generic Demo CA’:
  Unable to locally verify the issuer's authority.
    WARNING: certificate common name ‘Demo Certificate’ doesn't match requested host name ‘pdu-rdc-39-blue1.admin.evergreen.laika.com’.
HTTP request sent, awaiting response... 401 Unauthorized
Authorization failed.

If its possible to catch and warn on this as the unreachable hosts that would be awesome.
Not sure why the 401 doesn't throw an exception.