czerwonk / atlas_exporter Goto Github PK
View Code? Open in Web Editor NEWMetric exporter for RIPE Atlas measurement results
License: GNU Lesser General Public License v3.0
Metric exporter for RIPE Atlas measurement results
License: GNU Lesser General Public License v3.0
Hey Daniel,
great work. Your atlas_exporter is very useful and works great. Thanks for that.
I played around with grafana worldmap panel and was able to draw the metrics served by the exporter on the map. I took your code and added another endpoint to get the geometry data of the probes which are present in the cache.
I want to know what you think of this in general. Adding another endpoint makes the exporter serve more than just the metrics, but I think the advantage one can get from this feature would be very useful, as it automatically serves the probe data from a reliable source.
Thanks
Sebi
Thanks for your awesome project!
I have encountered strange issue caused by streaming API socket timeout that causes exporter to return stale statistics. Once I restart it manually, problem is solved.
For indexation purposes I will attach a few lines myself:
time="2024-05-07T18:51:41Z" level=info msg="Got result for xxx from probe y"
time="2024-05-07T18:53:41Z" level=error msg="Timeout reached for measurement #xxx. Trying to reconnect."
time="2024-05-07T18:56:02Z" level=info msg="Cleaning up cache..."
time="2024-05-07T18:56:02Z" level=info msg="Items removed: 0"
In prod environment with machines without direct access to Internet, it would be very handy to have atlas_exporter able to perform requests towards atlas.ripe.net via a HTTP/HTTPS proxy.
It implies to add first http proxy support to the following lib: https://github.com/DNS-OARC/ripeatlas or migrate atlas_exporter to this alternative lib already implementing http proxy: https://github.com/keltia/ripe-atlas
What do you think ?
How to configure this for traceroute and other options?
While testing the exporter against some of my ongoing DNS test I noticed that the rtt value isn't recognised when the response contains multiple results in a resultset.
According to the ripe documentation;
resultset - [optional] an array of objects containing all the fields of a DNS result object, except for the fields: fw, from, msm_id, prb_id, and type. Available for queries sent to each local resolver.
This was added in version 4750.
Test ID as a reference: 16878571
Here's a lazy workaround that's super ugly, but i can't get the nicer version working (See later).
diff --git a/dns/exporter.go b/dns/exporter.go
index d7819ed..55e7bf5 100644
--- a/dns/exporter.go
+++ b/dns/exporter.go
@@ -41,8 +41,18 @@ func (m *dnsExporter) Export(res *measurement.Result, probe *probe.Probe, ch cha
var rtt float64
if res.DnsResult() != nil {
rtt = res.DnsResult().Rt()
+ recordRtt(rtt, labelValues, ch)
+ } else if res.DnsResultsets() != nil {
+ for _, s := range res.DnsResultsets() {
+ if s.Result() != nil {
+ rtt = s.Result().Rt()
+ recordRtt(rtt, labelValues, ch)
+ }
+ }
}
+}
+func recordRtt(rtt float64, labelValues []string, ch chan<- prometheus.Metric) {
if rtt > 0 {
ch <- prometheus.MustNewConstMetric(successDesc, prometheus.GaugeValue, 1, labelValues...)
ch <- prometheus.MustNewConstMetric(rttDesc, prometheus.GaugeValue, rtt, labelValues...)
I'd much prefer to do something like this, but I can't quite get my head around golangs slices/arrays pointers/references etc to make the popping of the first element off the DNSResultSets() array response not cause a segfault.
diff --git a/dns/exporter.go b/dns/exporter.go
index d7819ed..b6706d4 100644
--- a/dns/exporter.go
+++ b/dns/exporter.go
@@ -41,6 +41,8 @@ func (m *dnsExporter) Export(res *measurement.Result, probe *probe.Probe, ch cha
var rtt float64
if res.DnsResult() != nil {
rtt = res.DnsResult().Rt()
+ } else if res.DnsResultsets() != nil {
+ rtt = res.DnsResultsets()[0].Result().Rt()
}
if rtt > 0 {
There was part of a pull request (24cfba9) that added NSID as part of the result. That would be really, really helpful for determining where these results came from for the DNS query sets. Can this be re-added?
Thanks for a fantastic app.
I read your RIPE Blog post that talked about AS-lookup and caching but I can't find any documentation around what is the intended data to be looked up? Is the purpose purely to include the ASN in the resulting prometheus data, or is the system supposed to pull other useful information such as the name of the network too?
Hello,
Thanks for the tool :)
Since the beginning of my use of the exporter, I could see the exporter using lots of memory until it gets OOM killed.
See screen attached:
It uses the RIPE streaming API for < 20 measurements.
I've been installing the exporter via go get
or building it from master, same result.
Is there anything I can do to help tshoot this ?
Does anyone else is experiencing the same behavior ?
@++
Lodpp
ps: as a band-aid / workaround, I've added a daily cronjob to restart the exporter
When querying the end point with prometheus or through the URL it opens a white screen and no metrics are displayed. I've verified the exporter is running as a service and has permissions, also that the prometheus yml file is correct and pointing at the IP and port number. Is this just a Docker exporter? Or can it run outside of the Docker Environment?
I think i installed the requisite dependency packages, but this error is in a vendor/ package, so it came along for the ride :)
morrowc@build:~/atlas_exporter (master)$ make build
[atlas_exporter] Build
../.go/src/github.com/czerwonk/atlas_exporter/vendor/github.com/miekg/dns/msg_helpers.go:144: base32.HexEncoding.WithPadding undefined (type *base32.Encoding has no field or method WithPadding)
../.go/src/github.com/czerwonk/atlas_exporter/vendor/github.com/miekg/dns/msg_helpers.go:144: undefined: base32.NoPadding
Makefile:55: recipe for target 'build' failed
make: *** [build] Error 2
note that at least current golang reports there is a 'WithPadding' method on an Encoding struct:
https://golang.org/src/encoding/base32/base32.go?s=569:650#L72
so I suspect something else is going on here, and perhaps just upgrading away to the newer version of miekg's dns package is the right path?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.