athennamind / opnsense-exporter

OPNsense Exporter for Prometheus

OPNsense Prometheus Exporter

The missing OPNsense exporter for Prometheus

Still under heavy development. The full metrics list is not yet implemented. May include breaking changes of the configuration and metrics list.

Table of Contents

• About
• Grafana Dashboard
• Metrics List
• Contributing
• OPNsense User Permissions
• Usage
  • Docker
  • Docker Compose
  • Systemd
  • K8s
• Configuration
  • OPNsense API
  • SSL/TLS
  • Exporters
  • All Options

About

Focusing specifically on OPNsense, this exporter provides metrics about OPNsense, the plugin ecosystem and the services running on the firewall. However, it's recommended to use it with node_exporter. You can combine the metrics from both exporters in Grafana and in your Alert System to create a dashboard that displays the full picture of your system.

While the node_exporter must be installed on the firewall itself, this exporter can be installed on any machine that has network access to the OPNsense API.

Grafana Dashboard

OPNsense Exporter Dashboard

Finaly we have a Grafana dashboard to visualize the data from this exporter. The dashboard can be imported into Grafana by using the id 21113 or by importing the deploy/grafana/dashboard-v1.json file. Please give a review to the dashboard if you like our work. Thank you!

OPNsense user permissions

TODO

Usage

Docker

The following command will start the exporter and expose the metrics on port 8080. Replace ops.example.com, your-api-key, your-api-secret and instance1 with your own values.

docker run -p 8080:8080 ghcr.io/athennamind/opnsense-exporter:latest \
      /opnsense-exporter \
      --log.level=debug \
      --log.format=json \
      --opnsense.protocol=https \
      --opnsense.address=ops.example.com \
      --opnsense.api-key=your-api-key \
      --opnsense.api-secret=your-api-secret \
      --exporter.instance-label=instance1 \
      --web.listen-address=:8080

TODO: Add example how to add custom CA certificates to the container.

Docker Compose

• With environment variables

version: '3'
services:
  opnsense-exporter:
    image: ghcr.io/athennamind/opnsense-exporter:latest
    container_name: opensense-exporter
    restart: always
    command:
      - --opnsense.protocol=https
      - --opnsense.address=ops.example.com
      - --exporter.instance-label=instance1
      - --web.listen-address=:8080
      #- --exporter.disable-arp-table
      #- --exporter.disable-cron-table
      #- ....
    environment:
      OPNSENSE_EXPORTER_OPS_API_KEY: "<your-key>"
      OPNSENSE_EXPORTER_OPS_API_SECRET: "<your-secret>"
    ports:
      - "8080:8080"

• With docker secrets

Create the secrets

echo "<your-key>" | docker secret create opnsense-api-key -
echo "<your-secret>" | docker secret create opnsense-api-secret -

Run the compose

version: '3'
services:
  opnsense-exporter:
    image: ghcr.io/athennamind/opnsense-exporter:latest
    container_name: opensense-exporter
    restart: always
    command:
      - --opnsense.protocol=https
      - --opnsense.address=ops.example.com
      - --exporter.instance-label=instance1
      - --web.listen-address=:8080
      #- --exporter.disable-arp-table
      #- --exporter.disable-cron-table
      #- ....
    environment:
      OPS_API_KEY_FILE: /run/secrets/opnsense-api-key
      OPS_API_SECRET_FILE: /run/secrets/opnsense-api-secret
    secrets:
      - opnsense-api-key
      - opnsense-api-secret
    ports:
      - "8080:8080"

Systemd

TODO

Configuration

The configuration of this tool is following the standard alongside the Prometheus ecosystem. This exporter can be configured using command-line flags or environment variables.

OPNsense API

To configure where the connection to OPNsense is, use the following flags:

• --opnsense.protocol - The protocol to use to connect to the OPNsense API. Can be either http or https.
• --opnsense.address - The hostname or IP address of the OPNsense API.
• --opnsense.api-key - The API key to use to connect to the OPNsense API.
• --opnsense.api-secret - The API secret to use to connect to the OPNsense API
• --exporter.instance-label - Label to use to identify the instance in every metric. If you have multiple instances of the exporter, you can differentiate them by using different value in this flag, that represents the instance of the target OPNsense. You must not start more then 1 instance of the exporter with the same value in this flag.

SSL/TLS

For self-signed certificates, the CA certificate must be added to the system trust store.

If you want to disable TLS certificate verification, you can use the following flag:

• --opnsense.insecure - Disable TLS certificate verification. Defaults to false.

Exporters

Gathering metrics for specific subsystems can be disabled with the following flags:

• --exporter.disable-arp-table - Disable the scraping of ARP table. Defaults to false.
• --exporter.disable-cron-table - Disable the scraping of Cron tasks. Defaults to false.
• --exporter.disable-wireguard - Disable the scraping of Wireguard service. Defaults to false.
• --exporter.disable-unbound - Disable the scraping of Unbound service. Defaults to false.
• --exporter.disable-openvpn - Disable the scraping of OpenVPN service. Defaults to false.
• --exporter.disable-firewall - Disable the scraping of Firewall (pf) metrics. Defaults to false.

To disable the exporter metrics itself use the following flag:

• --web.disable-exporter-metrics - Exclude metrics about the exporter itself (promhttp_, process_, go_*). Defaults to false.

All Options

Flags:
  -h, --[no-]help                Show context-sensitive help (also try --help-long
                                 and --help-man).
      --[no-]exporter.disable-arp-table
                                 Disable the scraping of the ARP table
                                 ($OPNSENSE_EXPORTER_DISABLE_ARP_TABLE)
      --[no-]exporter.disable-cron-table
                                 Disable the scraping of the cron table
                                 ($OPNSENSE_EXPORTER_DISABLE_CRON_TABLE)
      --[no-]exporter.disable-wireguard
                                 Disable the scraping of Wireguard service
                                 ($OPNSENSE_EXPORTER_DISABLE_WIREGUARD)
      --[no-]exporter.disable-unbound
                                 Disable the scraping of Unbound service
                                 ($OPNSENSE_EXPORTER_DISABLE_UNBOUND)
      --[no-]exporter.disable-openvpn
                                  Disable the scraping of OpenVPN service
                                  ($OPNSENSE_EXPORTER_DISABLE_OPENVPN)
      --[no-]exporter.disable-firewall
                                  Disable the scraping of Firewall (pf) metrics
                                  ($OPNSENSE_EXPORTER_DISABLE_FIREWALL)
      --web.telemetry-path="/metrics"
                                 Path under which to expose metrics.
      --[no-]web.disable-exporter-metrics
                                 Exclude metrics about the exporter
                                 itself (promhttp_*, process_*, go_*).
                                 ($OPNSENSE_EXPORTER_DISABLE_EXPORTER_METRICS)
      --runtime.gomaxprocs=2     The target number of CPUs that the Go runtime will
                                 run on (GOMAXPROCS) ($GOMAXPROCS)
      --exporter.instance-label=EXPORTER.INSTANCE-LABEL
                                 Label to use to identify the instance in
                                 every metric. If you have multiple instances
                                 of the exporter, you can differentiate them
                                 by using different value in this flag, that
                                 represents the instance of the target OPNsense.
                                 ($OPNSENSE_EXPORTER_INSTANCE_LABEL)
      --[no-]web.systemd-socket  Use systemd socket activation listeners instead of
                                 port listeners (Linux only).
      --web.listen-address=:8080 ...
                                 Addresses on which to expose metrics and web
                                 interface. Repeatable for multiple addresses.
      --web.config.file=""       Path to configuration file that can
                                 enable TLS or authentication. See:
                                 https://github.com/prometheus/exporter-toolkit/blob/master/docs/web-configuration.md
      --log.level="info"         Log level. One of: [debug, info, warn, error]
      --log.format="logfmt"      Log format. One of: [logfmt, json]
      --opnsense.protocol=OPNSENSE.PROTOCOL
                                 Protocol to use to connect to
                                 OPNsense API. One of: [http, https]
                                 ($OPNSENSE_EXPORTER_OPS_PROTOCOL)
      --opnsense.address=OPNSENSE.ADDRESS
                                 Hostname or IP address of OPNsense API
                                 ($OPNSENSE_EXPORTER_OPS_API)
      --opnsense.api-key=""      API key to use to connect to OPNsense API.
                                 This flag/ENV or the OPS_API_KEY_FILE my be set.
                                 ($OPNSENSE_EXPORTER_OPS_API_KEY)
      --opnsense.api-secret=""   API secret to use to connect to OPNsense API. This
                                 flag/ENV or the OPS_API_SECRET_FILE my be set.
                                 ($OPNSENSE_EXPORTER_OPS_API_SECRET)
      --[no-]opnsense.insecure   Disable TLS certificate verification
                                 ($OPNSENSE_EXPORTER_OPS_INSECURE)

opnsense-exporter's Issues

[BUG] K8s OPNSENSE_EXPORTER_DISABLE_WIREGUARD=true is not working

{"caller":"collector.go:192","collector_name":"wireguard","component":"collector","err":"opnsense-client api call error: endpoint: api/wireguard/service/show; failed status code: 400; msg: {\"message\":\"controller OPNsense\\\\Core\\\\Api\\\\IndexController not found\",\"status\":400}","level":"error","msg":"failed to update","ts":"2024-04-04T14:51:40.675Z"}

Not sure if this is wireguard related, tried to turn it off with K8s OPNSENSE_EXPORTER_DISABLE_WIREGUARD=true but it still fails here and pod restarts.

Version

• OPNSense router: 23.7.12
• OPNsense exporter version: 0.0.4 (ghcr.io/athennamind/opnsense-exporter:latest)

[FEATURE]: CARP monitoring

What would you like to be added?

First: thanks for the project! I use it with prometheus and alert manager to monitor my servers.

I have two servers, where one is primary and the other secondary -- it'd be nice to have some way to detect which one is "MASTER" vs "BACKUP" for a given interface. I don't know if this is possible or not, but thought it's worth posing the question.

I'm happy to do some tests and/or help a bit with coding, but I don't know a lot about the opnsense APIs

Why is this needed?

To allow monitoring and alerting in case you lose a network interface and your servers fail over (which happened to me recently and I didn't notice for almost 24 hours)

Any implementation details that are not related to source code may be included here.

No response

[QUESTION]: thanks for the project

hey just wanted to say thanks, i recently started a project for the OPNsense which might be interesting for one or the other.

https://github.com/badsmoke/LogSense2Loki

The service serves as a syslog server -> it receives all logs from the OPNsense, parses them into JSON and then sends them to the LOKI log server, so you can then create nice Grafana dashboards

certificate issue

When use https, the docker log show a error

caller=client.go:151 level=error msg="failed to send request; retrying" component=opnsense-client err="Get \"https://192.168.1.2/api/core/service/search\": tls: failed to verify certificate: x509: cannot validate certificate for 192.168.1.2 because it doesn't contain any IP SANs"

Use --no-opnsense.insecure env is no help
And use --opnsense.insecure=true is show another error

opnsense-exporter: error: unexpected true, try --help

[QUESTION]: Newbie: How do I get this to work when connecting to Grafana?

I'm new to Prometheus, but I thought there should be an /api/v1 URL exposed for Grafana to pull, correct?
Via browser, I see data correctly coming across under /metrics so I know the exporter is successfully talking to OPNsense.
But if I visit /api/v1/... I just get an HTML file and the grafana log shows:

logger=tsdb.prometheus t=2024-06-15T21:30:56.53085898Z level=warn msg="failed to get prometheus heuristics" err="failed to get buildinfo: failed to unmarshal JSON: invalid character '<' looking for beginning of value"

My data source configuration is very benign, the only input is the url: http:myloggingserver.mydomain:8080
and clicking 'Save and Test' gives a green message about validating the URL.

Thanks for this, can't wait for it to work for me!

[FEATURE]: Secure API keys and secrets

What would you like to be added?

Hello, thanks for this project!

I would like an option to load API key and secret from a docker secret (so from inside the container : load from a file).

Something similar to what mariadb does with *_FILES variables for example.

Why is this needed?

Avoid clear secrets in configuration files, docker-compose or swarm stacks definitions.

Any implementation details that are not related to source code may be included here.

No response

[FEATURE]: Add cpu and memory usage

What would you like to be added?

cpu and memory usage

Why is this needed?

cpu and memory usage

Any implementation details that are not related to source code may be included here.

No response

[BUG] - Parse Error on `api/unbound/diagnostics/stats`

The exporter seems to be unhappy with something coming back from the API when inquiring about Unbound DNS.

ts=2024-03-30T04:11:09.033Z caller=collector.go:138 level=error msg="failed to update" component=collector collector_name=unbound_dns err="opnsense-client api call error: endpoint: api/unbound/diagnostics/stats; failed status code: 0; msg: error parsing  to int: strconv.Atoi: parsing \"\": invalid syntax" 

Versions

OPNsense 24.1.4-amd64
FreeBSD 13.2-RELEASE-p10
OpenSSL 3.0.13

Using exporter v0.0.3

[BUG] Failed to unmarshal response body on api/core/system/status

Describe the bug

I am setting up a docker container to gather metrics from OpnSense, and it appears to connect ok, but fails to unmarshal the data.

Host operating system: output of uname -a

Linux truenas 6.1.74-production+truenas #2 SMP PREEMPT_DYNAMIC Wed Feb 21 20:30:38 UTC 2024 x86_64 GNU/Linux

Versions

- OPNsense version: 24.1.5_1
- OPNsense exporter version: 0.0.4
- Prometheus version: 2.51.1

Configuration of the exporter

Provide the configuration of the exporter in flags or in ENV variables.
Hide the sensitive information.

--opnsense.protocol=http
--opnsense.address=192.168.10.1
--exporter.instance-label=test
--web.listen-address=:17012
--opnsense.insecure

ENV's for API_KEY and API_SECRET

Expected behavior

Data should unmarshal correctly.

Screenshots

2024-04-04 20:03:47.507886-04:00ts=2024-04-05T00:03:47.507Z caller=main.go:34 level=info msg="starting opnsense-exporter" version=v0.0.4
2024-04-04 20:03:47.514329-04:00ts=2024-04-05T00:03:47.514Z caller=tls_config.go:313 level=info msg="Listening on" address=[::]:17012
2024-04-04 20:03:47.514382-04:00ts=2024-04-05T00:03:47.514Z caller=tls_config.go:316 level=info msg="TLS is disabled." http2=false address=[::]:17012
2024-04-04 20:04:12.045354-04:00http://192.168.10.1/api/core/system/status
2024-04-04 20:04:12.045475-04:00{"CrashReporter":{"statusCode":-1,"message":"An issue was detected and can be reviewed using the firmware crash reporter.","logLocation":"\/crash_reporter.php","timestamp":1712275450,"status":"Error","age":"2 seconds ago"},"Firewall":{"statusCode":2,"message":"No problems were detected.","logLocation":"\/ui\/diagnostics\/log\/core\/firewall","timestamp":"0","status":"OK"},"System":{"status":"Error"}}
2024-04-04 20:04:12.045385-04:00ts=2024-04-05T00:04:12.045Z caller=collector.go:169 level=error msg="failed to fetch system status" err="opnsense-client api call error: endpoint: api/core/system/status; failed status code: 200; msg: failed to unmarshal response body: json: cannot unmarshal number into Go struct field .CrashReporter.timestamp of type string"
2024-04-04 20:04:20.424316-04:00http://192.168.10.1/api/core/system/status
2024-04-04 20:04:20.424429-04:00{"CrashReporter":{"statusCode":-1,"message":"An issue was detected and can be reviewed using the firmware crash reporter.","logLocation":"\/crash_reporter.php","timestamp":1712275460,"status":"Error","age":"0 seconds ago"},"Firewall":{"statusCode":2,"message":"No problems were detected.","logLocation":"\/ui\/diagnostics\/log\/core\/firewall","timestamp":"0","status":"OK"},"System":{"status":"Error"}}
2024-04-04 20:04:20.424347-04:00ts=2024-04-05T00:04:20.424Z caller=collector.go:169 level=error msg="failed to fetch system status" err="opnsense-client api call error: endpoint: api/core/system/status; failed status code: 200; msg: failed to unmarshal response body: json: cannot unmarshal number into Go struct field .CrashReporter.timestamp of type string"