gowee / traceroute-map-panel

📍🗺️ Visualize traceroute paths on a map in a Grafana panel

Traceroute Map Panel

Traceroute Map Panel is a Grafana panel that visualize the traceroute hops in a map, just like Besttrace.

Online Demo: https://trmp-demo.bamboo.workers.dev/

Data

The panel expects traceroute data in the following schema, where fields order does not matter:

Field	Type	Description
host	string	source host where the data is collected
dest	string	host where the trace is destinated
hop	number	nth hop
ip	string	IP address of the hop
rtt	number	round-trip time
loss	number	packet loss
time	number	timestamp, not used yet

Typically, the traceroute data is collected from MTR via Telegraf and stored in InfluxDB. It is also possible to use other datasource as long as the schema matches.

The wiki of Telegraf has a sample config utilizing the built-in [[inputs.exec]] for this.

Setup

1. Install & Configure Telegraf and InfluxDB (with InfluxQL) properly. (Or other datasources that matches the expected schema.)
2. Refer to the wiki of Telegraf to configure MTR data collection as an input.
3. Optionally, explore database via the influx CLi tool, so that to make sure data is collected as expected. See the query section.
4. Install the Traceroute Map Panel plugin to Grafana. Two methods are available.
   • Install manually via tarball:
     1. Download the latest tarball, which is meant for Grafana 7.0+/8.0+. For Grafana 6.7.x, there is v0.1.0.
     2. Decompress & put tarball content into the Grafana plugin directory (usually /var/lib/grafana/plugins).
     3. Restart Grafana.
   • Install via the grafana-cli tool.
5. Create a new panel in Grafana:
   1. Choose visualization "Traceroute Map Panel"
   2. In the query editor of InfluxDB (in InfluxQL), toggle the text edit mode by clicking the pen icon and enter the query as suggested in the query section.
   3. At the bottom of the editor, choose FORMAT AS Table instead of Time Series.
6. Setup a GeoIP API key and tweak other options, optionally. Finally, Save the panel.

Query

Query in Grafana

For Grafana 7.1.2 and later.

select mean(avg) as rtt, mean(loss) as loss from mtr WHERE ${__to}ms - 5m <= time AND time <= ${__to}ms group by hop, ip, host, dest

& Format as Table.

Note: The query is in InfluxQL instead of Flux. In InfluxDB v1.x, only InfluxQL is available. So there is nothing to note especially. In InfluxDB v2.0+, the default query lanugage is Flux. To successfully apply the query above, create DBRP mappings to buckets and be sure to configure the InfluxDB datasource in Grafana with InfluxQL.

Alternative queries

For Grafana before 7.1.2, the following one is available.

select mean(avg) as rtt, mean(loss) as loss from mtr WHERE now() - 5m < time group by hop, ip, host, dest

Or (less recommended, see Notes):

select mean(avg) as rtt, mean(loss) as loss from mtr WHERE $timeFilter group by hop, ip, host, dest

& Format as Table.

Preview via the CLi tool of InfluxDB

Note: The section is assuming InfluxDB v1.x / InfluxQL is used. Skip it when using InfluxDB v2.0+.

When previewing / exploring mtr data in InfluxDB via its CLi client. This following query that groups data by host->dest pair gives clearer result for reading.

select hop, ip, rtt, loss from (select mean(avg) as rtt, mean(loss) as loss from mtr WHERE now() - 6h < time AND time < now() group by hop, ip, host, dest) group by host, dest

Geo IP

The panel relies on external API services for Geo IP resolving in the browser (not the backend). It ships with several built-in Geo IP APIs for convenience, which are not affiliated in any way with the panel.

An alternative way is to specify custom API or custom function (proper CORS header is a must). A sample Cloudflare Worker script that proxies requests to some third-party service is located in ipip-cfworker.js.^{_{(No longer working due to the deprecation of that 3-rd party API. To be updated.)}}

Be careful when sharing the panel or its snapshot publicly, as it implies publicizing API keys ever saved in the options editor even for those which are not selected to be active. For now, the only safe way to share a snapshot is to create a new panel from scratch (instead of duplicating) and choose an API without the requirement of a key.

Notes & Known Issues

Time filter

The panel is not really time-series aware in the sense that it expects only a static route path for one src-dest pair. If routing is dynamic (e.g. multiple route paths for one src-dest pair), paths displayed on the map may be fairly chaotic.

So it is generally a good idea to replace the default $timeFilter with a small range filter, such as ${__to}ms - 5m <= time AND time <= ${__to}ms (last 5 mins in the time range).

Telegraf config

A typical execution of mtr might take roughly a quarter of seconds to finish its hop probing. To avoid unnecessary work, especially when multiple mtr commands are configured, a bigger collection interval should be specified under [[inputs.exec]] to overwrite the global short default. Depending on the actual workload, interval = 5m or 1m / 3m / 10m / ... would be reasonable. It is recommended to match this with the one specifed in query. Besides, sometimes timeout needs to be increased (e.g. to 60s) when there are multiple mtr commands to counter jam even though they are parallelized.

The sample config given in the wiki of Telegraf presumes mtr v0.94. If a different mtr is installed, the config may need adapting to match its actual output format.

Invalid Schema

For "Invalid Schema" error, the problem might be just mismatched data schema. Or, the query data might not be Formatted as Table) for the InfluxDB datasource.

Spline and Animation

Since v0.3.0, the panel introduces two new options that allow converting the route path to spline (smooth curve) with optional animation. Both spline and animation are off by default due to the performance degration it brings. Obvious lags at a high zoom level is a known issue when animated spline is activated. Unfortunately, there is no plan to improve it for now.

---

^{_{This project is still too young to be stable enough. 🚀 Feel free to open a new issue if when encountering problems or having suggestions (even pointing out a typo would help a lot).}}

traceroute-map-panel's Issues

Where do i need to add the Geop Api key ?

Can you please tel me where i have to add the API key and which is the preferred one ?

I get following error

The error has been logged in the Debugging Console: TypeError: n.geoIPProviders is undefined

Can't store data into influxdb

The telegraf config:
[[inputs.exec]]
commands=["mtr -C -n 192.168.1.2"]
timeout = "40s"
data_format = "csv"
csv_skip_rows = 1
csv_column_names=[ "", "", "status","dest","hop","ip","loss","snt","", "","avg","best","worst","stdev"]
name_override = "mtr"
csv_tag_columns = ["dest", "hop", "ip"]

[[outputs.influxdb]]
urls = ["http://127.0.0.1:8086"]
database = "my_telegraf3"
timeout = "5s"

telegraf config file testing pass, but no mtr information display:
telegraf --config /etc/telegraf/telegraf.conf --test

influxdb database creased normally, but no mtr measurement:
Using database my_telegraf3

show measurements
name: measurements
name

---

cpu
disk
diskio
kernel
mem
processes
swap
system

Please help to resovle this problem , what should I do?
Many Thanks.

Error -Invalid Schema

Hi, I am using Grafana 7.3.6 with InfluxDB.

Objective is to track gps device, where gps device is storing latitude and longitude data in to the influxdb in float format.

Query :SELECT last("Lat") as latitude, last("Long") as longitude FROM "DishaLite" WHERE ${__to}ms - 5m <= time AND time <= ${__to}ms group by IMEI

I am getting Invalid Schema: Error
Please help

Unable to see GeoIP options in Grafana 7.4

The panel tab in Grafana does not show the GeoIP provider/token options in Grafana 7.4. Is this plugin compatible with this version of Grafana? I was sure to download the latest zip and install from that.

Refactor to query GeoIP in backend

Querying GeoIP in browsers is a compromise due to the limitation of the previous Grafana plugins system (or its doc?). It complicates caching resolution results, slows down the load time, and exposes security risks when sharing panels.

Now it is possible to query GeoIP in a separated backend plugin.

The resources capability allows a backend plugin to handle custom HTTP requests sent to the Grafana HTTP API and respond with custom HTTP responses.

Ref:
https://grafana.com/docs/grafana/latest/developers/plugins/backend/#resources
https://github.com/grafana/grafana-plugin-sdk-go/blob/main/proto/backend.proto

Plugin at risk of deprecation

Hi there from Grafana Labs 👋

Firstly, thank you for contributing to the Grafana plugin ecosystem.

I'm reaching out as this plugin is at risk of being deprecated from the Grafana plugin catalog as it has not been updated in a number of years and it is unclear whether it is still maintained.

We recognise that priorities change and that it may no longer be possible to keep plugins up to date. For your awareness, deprecation has no impact on existing users of the plugin, but discourages further adoption to prevent users from starting to rely on a component which won't receive updates.

Please advise if you believe the plugin is stable and wish for it to remain in the catalog, or alternatively that you agree the plugin should be deprecated. In the case of the latter, if there is any specific guidance you would like to offer users of the plugin - please let me know and we can include this in the deprecation notice.

Feel free to email me at [email protected] if preferred; apologies, we don't have an email address associated with this plugin so I had to raise an issue.

If we don't hear from you within 30 days, I'll go ahead and deprecate the plugin.

Thank you again
David

Error proccessing data - Expected integer number, got NaN

Hi,

Using Grafana 9.2.4. And got that error.

It does not tell me what field exactly but.

{
"request": {
"method": "GET",
"url": "api/datasources/proxy/1569/query",
"params": {
"db": "so_tools",
"q": "select mean(loss) as loss,mean(rtt) as rtt from tracebot_results group by hop, ip, host, dest",
"epoch": "ms"
},
"data": null,
"precision": "ms",
"hideFromInspector": false
},
"response": {
"results": [
{
"statement_id": 0,
"series": [
{
"name": "tracebot_results",
"tags": {
"dest": "",
"hop": "",
"host": "",
"ip": ""
},
"columns": [
"time",
"loss",
"rtt"
],
"values": [
[
0,
0,
55.583333333333336
]
]
}
]
}
],
"executedQueryString": "select mean(loss) as loss,mean(rtt) as rtt from tracebot_results group by hop, ip, host, dest"
}
}

---

Did not understand why in the example query has "select mean(avg) as rtt, mean(loss) as loss from" <-- avg there , what is that?

That query wont work either, returns null for the first field as don't have filed "avg"

BR,
Keeks

Show rtt and loss on the map

Hello.
Didn't find how to display rtt and loss on the map.
is it possible?

Migrating to Grafana 7.x PanelOptions

Starting Grafana 7.0, PanelPlugin.setEditor and PanelPlugin.setDefaults are deprecated in favor of PanelPlugin.setOptions. The former two are the imperative way to write an option configurator. On the contrary, the latter is the newly-introduced declarative way.

The imperative way allows writing a fairly flexible UI, just like writing a normal HTML form, of the option configurator while bringing much hassle in maintaining the UI styles/components consistent with those provided by @grafana/ui.

The declarative way solves the problem by allowing specifying each field structurally with its name, description, type, and default value. But at same the time, it prevents the inherent flexibility.

---

Upon initiating, the project is at the stage where Grafana was migrating from 6.x to 7.x. At first, the panel was started with the boilerplate project depending on Grafana 7.0 which was not released yet at that time (grafana/grafana-starter-panel#18 (comment)). To avoid incompatibility, the project was reverted to an older commit of the boilerplate by acd2df0, resulting in the fact that the old PanelPlugin.{setEditor,setDefaults} were kept.

To handle the complex tagged-union-style GeoIPProvider option

traceroute-map-panel/src/types.ts

Line 21 in 8aa886c

geoIPProviders: {

, the current implementation uses a select with dynamic text inputs 85e79f9 which appears to be infeasible with the new declarative PanelPlugin.setOptions.

---

As is reported in issue #4, PanelPlugin.setEditor and PanelPlugin.setDefaults turn to be unavailable even though it is just marked as deprecated (instead of removed). So it is inevitable to migrate to PanelPlugin.setOptions.

After the migration, the custom-api and custom-function might be removed due to the limitation of fields. Nevertheless, how to allow specifying a token (e.g. for ipinfo)?

GeoIP Token?

How/where do we input the GeoIP token for IPinfo?

Add a sample query for InfluxDB / Flux

Since InfluxDB 2.0+, the default query language has been changed to Flux which is significantly different than the previous InfluxDB. It takes some extra steps to configure Grafana / InfluxDB to work with InfluxDB, which is inconvenient.

A sample query in Flux should be added to README.

Disscussions moved from issue #8 about problems on CentOS 8

cddisk2000 commented 5 days ago:

Use Centos 7 Always Fail
The Main Points Are As Follows

Centos 8 Steam or Centos 8
The MTR was above 0.94 Offline Install , cannot be used yum
I hope Can Help Everyone

Gowee commented 3 days ago:

@ cddisk2000

Hi.

I cannot reproduce the problem on a newly installed CentOS Linux release 8.4.2105. What is the version of telegraf and influxdb? Also, I am not sure what do you mean by "Always Fail". Is the telegraf failing to start? Or no data is written to influxdb?

Would you mind opening an issue where more information can be attached?

cddisk2000 commented 3 hours ago:

@ Gowee
This is my system environment for your reference

    offline install mtr-0.94-2.hs.el8.x86_64.rpm
    Telegraf version 1.19.0
    InfluxDB version: 1.8.6
    CentOS Stream Release 8
    Please confirm that the format must be output using the following command
    mtr -C -n www.google.com
    =======================================================
    Mtr_Version,Start_Time,Status,Host,Hop,Ip,Loss%,Snt, ,Last,Avg,Best,Wrst,StDev,
    MTR.0.94,1624880440,OK,www.google.com,1,192.168.8.254,0.00,10,0,0.24,0.29,0.19,0.41,0.08
    MTR.0.94,1624880440,OK,www.google.com,2,???,100.00,10,10,0.00,0.00,0.00,0.00,0.00
    =======================================================

If it doesn't work, you can browse my blog,I hope you can succeed
https://my-fish-it.blogspot.com/2021/06/ss-grafana-traceroute-map-panel.html