matrix-org / matrix-federation-tester

Tester for matrix federation written in golang.

Matrix Federation Tester

Checks that federation is correctly configured on a matrix server.

Building

Requires Go 1.15.

git clone https://github.com/matrix-org/matrix-federation-tester
cd matrix-federation-tester
go build

Running

BIND_ADDRESS=:8080 ./matrix-federation-tester

Using

The federation tester may be accessed using the following templated URLs. Please replace <server_name> with your server name (eg: matrix.org).

Full JSON report

https://matrix.org/federationtester/api/report?server_name=<server_name>

Plain text response

Returns GOOD if the federation is ok and BAD if it's not ok.

https://matrix.org/federationtester/api/federation-ok?server_name=<server_name>

matrix-federation-tester's Issues

Certificate tests should account for parallel RSA and ECDSA certificates

Some servers run both certificate types in parallel. If the server reports both RSA and ECDSA ciphers available, the tester should connect twice, once with only RSA ciphers enabled and once with only ECDSA ciphers enabled to retrieve and test both certificates.

Currently it only fetches a single certificate based on the first matching cipher.

fed tester follows SRV records which point to CNAMEs

RFC2782 says it's invalid for a SRV record to point to a CNAME.

Consider reworking how server discovery is represented

@ace:kittenface.studio and @grin:grin.hu pointed out that the way the federation tester displays how a server was discovered can be confusing.

@ace:kittenface.studio says that instead of having individual lines saying that well-known wasn't found or an SRV record wasn't found it would be better to say which method was used to discover the server. For example something like

Server was discovered via .well-known and SRV
or
Server was discovered directly on port 8448

That way an admin who thought they had setup a correct well-known file, but saw the server was only discovered using the SRV record for example would know that something was wrong with their well-known file. Using something like this, server admins who don't need a well-known (or don't need an SRV) wouldn't be confused that they were missing some required thing which a lot of people seem to think when they see well-known not found in the federation tester currently.

https://matrix.to/#/!QtykxKocfZaZOUrTwp:matrix.org/$155300756116915ZJdON:kittenface.studio?via=matrix.org&via=half-shot.uk&via=linuxgaming.life

Moved from a comment in #70

AllChecksOK does not mean that all checks are ok

Apparently some checks aren't included in AllChecksOK, which is super misleading

Federation tester doesn't work with multi domain reversed proxy server

Because the federation tester is not actually connecting to the server using the domain/_matrix but instead resolving the IP for the domain and connecting to that IP, the tester is receiving the wrong certificate if the IP connection returns another certificate (the main certificate).

It should directly connect to the correct domain instead of trying to use the IP for the connection.

Certs are marked as invalid if port specified

Adding the (correct) port shouldn't make ValidCertificates fail.

Should still do the SRV check even if using .well-known

So people can have confidence in their fallback

`cannot unmarshal number` when /key request gives a 404

This guy had a misconfigured reverse-proxy, so that the /_matrix/key/v2/server request returned a 404:

{
  "DNSResult": {
    "SRVCName": "_matrix._tcp.bruder.space.",
    "SRVRecords": [
      {
        "Target": "synapse.bruder.space.",
        "Port": 443,
        "Priority": 0,
        "Weight": 10
      }
    ],
    "SRVError": null,
    "Hosts": {
      "synapse.bruder.space.": {
        "CName": "synapse.bruder.space.",
        "Addrs": [
          "185.147.237.114"
        ],
        "Error": null
      }
    },
    "Addrs": [
      "185.147.237.114:443"
    ]
  },
  "ConnectionReports": {},
  "ConnectionErrors": {
    "185.147.237.114:443": {
      "Message": "json: cannot unmarshal number into Go value of type gomatrixserverlib.ServerKeyFields"
    }
  }
}

Reads the certificate from the domain and not the host

On a configuration with both SRV and .well-known set up, the tester should return the correct information for both delegation mechanisms. It looks like it basically reads the cert from the wrong server.

What happened:

1. Running https://federationtester.matrix.org/api/report?server_name=hoeg.com, I am given errors about using the wrong certificate "x509: certificate is valid for hoeg.com, not matrix.hoeg.com"

2. Using testssl.sh, I can see the correct certificate:

testssl.sh https://matrix.hoeg.com:8448
<snip>
 Common Name (CN)             matrix.hoeg.com
 subjectAltName (SAN)         matrix.hoeg.com

Further to #61

Don't parse tls fingerprints

The check has been removed (#18) however the field is still parsed, which can cause errors when testing a domain. The tester should not parse the field.

fed tester seems to be testing the cert in ip:port instead of hostname:port if .well-known is used

Example here: https://matrix.org/federationtester/api/report?server_name=bubu1.eu

It should be getting the cert from matrix.bubu1.eu:443, instead it connects to 176.9.145.28:443 which has a different cert.

Api endpoint down

after the infrastructure problem at matrix homeserver, the federation tester hasnt come back yet. just a reminder in case you forgot about it! :)

Failing to supply a server_name causes a 502 which causes a Cloud Flare screen

This isn't a big deal because it should be obvious what your mistake is, but it's rather unfriendly to show the user a screen saying matrix.org is down rather than asking them to specify a server_name.

give the federation tester a UI

It would be nice if people could self-service on the federation tester, rather than having to be handheld. It was designed to have a UI added later; we should do so

Federation Tester complains about invalid certs when using SRV to an IP address

If a server is using an SRV record that leads to an IP address and that IP address is not included in the target server's federation TLS certificate, then the certificate gets marked as valid even if example.com is included (it's a case of trying and failing to validate xx.xx.xx.xx:8448 vs. example.com:8448).

I'm of the mind that we should still fail, but use the new errors output to say that this is the case and that people should move to .well-known.

Fedtester sometimes takes forever to complete a test

Testing a server such as amorgan.xyz returns immediately, whereas testing sw1v.org takes a very long time.

We seem to be waiting on some request and then timing out after some amount of time, causing the tests to still display correctly.

Need to figure out what request is timing out.

Support ipv6

It looks like the current installation of federation tester doesn't support ipv6. It'd be a nice thing to have.

"error": "No .well-known found" when well-known unused

I kinda know why it says this, but it's really unintuitive. Can we call this something like result instead of error?

SRV record causes certificate check against domain name, not host name

When using a SRV entry for domain, the certificate check fails, because it expects the domain name to match instead of the host name, it connected to.

What happens

For:

_matrix._tcp.example.com. 3600 IN	SRV	10 0 8448 matrix.example.com.

The tester connects to matrix.example.com but checks for certificate for example.com.

What I expect

The tester connects to matrix.example.com and checks for certificate for matrix.example.com.

It is not only impracticable to use the main domain (causes a lots of problems with automatically issued certificates), it also differs from the behavior that occurs with the .well-known entry on web server hosting on example.com:

{"m.server": "matrix.exaple.com:8448"}

remove check on tls_fingerprints

tls_fingerprints is no longer relevant.

we should time out requests to non-responsive servers rather than letting the upstream proxy return a 504

Federation tester doesn't appear to be matching server names correctly with .well-known enabled

In a setup where example.org is delegated to matrix.example.org:443 via .well-known, the federation tester says that the server names don't match. It appears to be checking that example.org === matrix.example.org, which is wrong because the certificate should be for matrix.example.org and therefore it should be checking that.

"x509: certificate signed by unknown authority" with Let's Encrypt certificates on (some) servers with .well-known

When running the federation tester against my homeserver (abolivier.bzh), which is serving a valid Let's Encrypt certificates, and is delegating traffic via .well-known, the federation tester errors at cert verification with x509: certificate signed by unknown authority.

.well-known on server_name appears to be insufficent?

I just set up my own matrix server at cyphar.com, with both of the following delegations set up:

1. https://cyphar.com/.well-known/matrix/server gives you { "m.server": "matrix.cyphar.com:8448" }.
2. _matrix._tcp.cyphar.com has a SRV record containing 10 0 8448 matrix.cyphar.com..

However it appears this is not sufficient -- the federation tester gives a notice that well-known isn't in use.

In order to get well-known recognised I have to add https://matrix.cyphar.com/.well-known/matrix/server -- which doesn't make much sense since the whole point of the .well-known setup is that you need it to redirect from the Matrix server name.

Is this caused by me having both methods set up?

Unhelpful output when .well-known lacks an m.server field

We have a Matrix server running on matrix.fachschaften.org:8449 (for the server port) with SRV and .well-known entries on fachschaften.org (the domain used for Matrix). This doesn't seem to be properly recognised, checking fachschaften.org or matrix.fachschaften.org yields:

{
  "WellKnownResult": {
    "m.server": ""
  },
  "DNSResult": {
    "SRVCName": "",
    "SRVRecords": null,
    "SRVError": {
      "Message": "lookup _matrix._tcp. on 1.1.1.1:53: no such host"
    },
    "Hosts": {
      "": {
        "CName": "",
        "Addrs": null,
        "Error": {
          "Message": "lookup : no such host"
        }
      }
    },
    "Addrs": null
  },
  "ConnectionReports": {},
  "ConnectionErrors": {}
}

Using nslookup agains 1.1.1.1 though seems to resolve our SRV entry just fine:

Non-authoritative answer:
_matrix._tcp.fachschaften.org   service = 10 0 8449 matrix.fachschaften.org.

Checking against the s2s Port of our installation yields a cert error:

"MatchingServerName": false,

I'm guessing this is because the query is done with a port and the cert has no port? Can this be ignored?

Thanks.

(Oh, and so far our Homeserver has worked well with another one we are administering, as well as other servers, matrix.org being one of which).

WellKnownInUse is in the wrong place

WellKnownInUse is reported for each Connection (ie, each IP address), which makes no sense.

We should either put it at the top level, or make the UI infer it from the WellKnownResult property. Either way, let's get rid of it at the Connection level.

move `gomatrixserverlib.LookupServer` into federation tester

LookupServer isn't useful functionality for matrix servers in general (they should be using ResolveServer as of matrix-org/gomatrixserverlib#118). We should move it in here.

fedtester writes all its output on one line

Not helpful:

tail -n 1 ~/.forever/fedtester.log | wc
      1    2086   28045

federation tester doesn't work for russian homeservers

https://isitblockedinrussia.com/?host=94.237.61.73

Should federation tester check /_matrix/federation/v1/version?

Maybe the federation tester should display your Synapse version (and warn if you are on an old version?)

May help with matrix-org/synapse#3220 since

• using the federation/v1/version endpoint (doesn't work for those who don't have federation endpoints exposed on the client port)

doesn't really apply to the federation tester since it knows which port to use

Human-readable errors for known configuration issues

Some simple errors for known issues if you end up in a certain situation.

Something like:

if MatchingServerName and !FutureValidUntilTS {
    error = "your server is doing this dumb thing, try ..."
}

Consider adding support for Synapse 0.34 backwards-compatibility SRV records

Some people are using SRV records on their matrix domain as well as having a .well-known which delegates to a different domain, for compatibility with Synapse-pre-0.99, and have reported (#34) finding it confusing that the federation tester ignores that SRV record.

Maybe we could re-run the checks in a Synapse-0.34-compatibility mode?

missing time-out because of firewall drop

I was trying https://matrix.org/federationtester/api/report?server_name=matrix.example.com but I was receiving after a minute (without any other response):

504 Gateway Time-out

The server didn't respond in time.

When I ran this in my laptop, I didn't have IPv6 setup, so http://localhost:8080/api/report?server_name=matrix.example.com returns:

(...)
  "ConnectionErrors": {
    "[2a00:1508:100:ffff::d]:8448": {
      "Message": "dial tcp [2a00:1508:100:ffff::d]:8448: connect: network is unreachable"
    }
  }

After opening the firewall https://matrix.org/federationtester/api/report?server_name=matrix.example.com returns me:

  "ConnectionErrors": {
    "[2a00:1508:100:ffff::d]:8448": {
      "Message": "dial tcp [2a00:1508:100:ffff::d]:8448: getsockopt: connection refused"
    }
  }

(so yes I have a problem with matrix listening IPv6)

but I proved that in IPv6 this program requires a TCP timeout because of a firewall dropping packets. Probably the same can be reproduced with IPv4.

To easily reproduce it, use for example: https://matrix.org/federationtester/api/report?server_name=google.com

distinguish between dead server and dead tester

`run.sh` assumes installation in $HOME

tester does not give useful information about certificate fails

The federation tester just seems to give a binary "ValidCertificates": false, which isn't terribly helpful (and the UI turns into a warning about self-signed certs, whether or not the problem is that the cert is self-signed).

We should give better feedback on what's wrong with the cert - possibly in the Errors or Info fields of the connection report.

Require actual X509 certs on Synapse release v0.99+

Add the self-signed cert check to AllChecksOK once Synapse v0.99 is out.

Build on go 1.12 fails

$ go version
go version go1.12 linux/amd64
$ go get -u github.com/constabulary/gb/...
$ gb build
golang.org/x/sys/unix.ioctl: relocation target golang.org/x/sys/unix.Syscall not defined for ABIInternal (but is defined for ABI0)
golang.org/x/sys/unix.munmap: relocation target golang.org/x/sys/unix.Syscall not defined for ABIInternal (but is defined for ABI0)
golang.org/x/sys/unix.mmap: relocation target golang.org/x/sys/unix.Syscall6 not defined for ABIInternal (but is defined for ABI0)
# github.com/matrix-org/matrix-federation-tester
github.com/matrix-org/matrix-federation-tester
FATAL: command "build" failed: exit status 2

Cloudflare 524 error

Hello,

I'm getting the following error while testing the following domain: mental.af. Everything is fine with other domains (.org, .fr...). Is it really related to the TLD?

My server is working fine, I can access it and use all functionalities through the official Riot.im server (https://riot.im/app).

Thanks in advance,

502 on port closed / fallback on ipv4?

I have experienced a kind of special issue: the hostname I gave as input was valid, with a real server behind. However, the port was closed on ipv6 for matrix, but open on ipv4, by my firewall (my mistake). The API returned as Bad gateway error instead of falling back to ipv4, or even returning any helpful error message.

Test both ipv6 and ipv4 for .well-known request

I've had several problems due to a misconfigured nginx server that didn't respond to ipv6 but did for ipv4. For example the .well-known wasn't accessible for that reason on ipv6, but I spent a long time searching for the issue, because wget apparently preferred ipv4 (or automatically tried both).

It would be great if the federation tester would test both ipv4 and ipv6 and report issues for them separately.

ConnectionErrors "invalid character '<' looking for beginning of value"

I get this error message when testing my server: https://matrix.org/federationtester/api/report?server_name=bubu1.eu

Replace gometalinter in CI

According to alecthomas/gometalinter#590, gometalinter is being deprecated and needs to be replaced with golangci-lint.

Error Generating Report: "strconv.ParseUint: parsing \"//grml.de\": invalid syntax"

When I try to use the fed-tester with my own server_name grml.de it exits with Error 500 on matrix.org and after I managed building it locally this is the error output:

Error Generating Report: "strconv.ParseUint: parsing "//grml.de": invalid syntax"

Federation tester sends the wrong SNI

It appears to attempt to read the SNI to send from a query param. This is completely bogus.

SRV records should be looked up for root domain, not server hostname

On a configuration with both SRV and .well-known set up, the tester should return the correct information for both delegation mechanisms.

What happened

Given the recommended setup for a delegated matrix host on a domain:

• Domain: draak.fr
• Server hostname: matrix.draak.fr
• SRV record: _matrix._tcp.draak.fr
• .well-known: https://draak.fr/.well-known/matrix/server

DNS results ( from https://matrix.org/federationtester/api/report?server_name=draak.fr) came back with the following:

"DNSResult": {
  "SRVCName": "",
  "SRVRecords": null,
  "SRVError": null,
  "Hosts": {
    "matrix.draak.fr": {
      "CName": "matrix.draak.fr.",
      "Addrs": [
        "5.196.94.91"
      ],
      "Error": null
    }
  },
  "Addrs": [
    "5.196.94.91:8448"
  ]
}

Notice the "null" SRVCName and SRVRecords.

What should happen

The SRV record information should be populated with the correct information, with both SRVCName and SRVRecords filled in.

Some investigation

It looks like there is a logic error in the tester's handling of root domain vs. server hostname when looking up .well-known and SRV records.

// Host address of the server (can be different from the serverName through SRV/well-known)
serverHost := serverName

// Check for .well-known
var err error
var wellKnown *gomatrixserverlib.WellKnownResult
if wellKnown, err = gomatrixserverlib.LookupWellKnown(serverName); err == nil {
	// Use well-known as new host
	serverHost = wellKnown.NewAddress
}
var report ServerReport
dnsResult, err := gomatrixserverlib.LookupServer(serverHost)
if err != nil {
	return nil, err
}

When a .well-known is successfully detected and read, the serverHost changes to that of the actual server's host, and then the tester goes on to try to lookup _matrix._tcp.<serverHost> which may (most probably) not exist. DNS results should only ever be tested against the root domain (i.e. serverName) for the purpose of reading SRV records.

In case .well-known and SRV record both exist but differ in values, .well-known should probably be the source of truth and the discrepancy should be conveyed that the SRV should match .well-known.

If MatchingServerName is false add a warning saying what they are probably doing wrong

It is such a common question to see "why is MatchingServerName showing an error". 99% of the time it is because they did not enter the server_name. I've only helped one person where it wasn't caused by that. If that is the only error, the federation tester should add a sentence to the top saying something like

The MatchingServerName error is most likely caused by you using the wrong URL in the federation tester. You should enter in the server_name, not the location where your server is.

Related: #16

Certs marked as valid on matrix.example.org, but invalid on example.org

Testing matrix.snopyta.org, we get ValidCertificates == true, but testing snopyta.org gives false.

Well-known seems to be set up correctly, so this doesn't make much sense.

#26 may be the cause however.

AllChecksOk ignores ipv6 problems

we should have a higher-level FederationOk or something field right at the beginning or the end to make it easy to check overall health

Run GetVersion in a goroutine

When fetching the server version, if the endpoint times out, it means that we have to wait an extra 30s before getting our results, meaning that if everything times out we have to wait for 40s. Doing so is not necessary for tests, thus the request can be run in parallel with the connection reports, and save 10s.

Bonus points if we standardise the timeout by creating the gomatrixserverlib client using NewClientWithTimeout instead of NewClient so that it takes up to as much time as other requests.