tgivslife / open-rmbt

RTR-NetTest Open Source repository

Open-RMBT

Open-RMBT is an open source, multi-threaded bandwidth measurement system.

It consists of the following components:

• Web site
• JavaScript client
• Android client +
• iOS client
• Measurement server
• QoS measurement server (in this repository)
• Control server +
• Statistics server +
• Map server (in this repository)

+) These components are available in separate repositories. This repository still contains outdated and incompatible versions of these components. They shall not be used for production purposes. Plese contact us if you are uncertain about the compatiblity of repositories.

Open-RMBT is released under the Apache License, Version 2.0. It was developed by the Austrian Regulatory Authority for Broadcasting and Telecommunications (RTR-GmbH).

Related material

• RMBT specification
• RTR-NetTest/rmbt-server - Test Server for conducting measurements based on the RMBT protocol
• RTR-NetTest/rmbtws - JavaScript client for conducting RMBT-based speed measurements
• RTR-NetTest/open-rmbt-control - Control server
• RTR-NetTest/open-rmbt-statistics - Statistics server
• RTR-NetTest/open-rmbt-ios - iOS app
• RTR-NetTest/open-rmbt-android - Android app
• RTR-NetTest/rtr-nettest/open-rmbt-website - Web site

System requirements

• 1-3 servers

• Everything can be installed on a single server

• The test server (RMBT-Websocket) should run on a physical machine

• Base system Debian 11 or newer (or similar)

• At least a single static public IPv4 address (IPv6 support recommended, more addresses allow to run more services on port 443)

  NOTE: other Linux distributions can also be used, but commands and package names may be different

Installation

For each server:

1. Setup IP/DNS/hostname
2. firewall (e.g. iptables)
3. Install git
4. Install and configure sshd
5. Install and configure ntp
6. dpkg-reconfigure locales (database requires en_US.UTF-8)
7. dpkg-reconfigure tzdata
8. Install and configure letsencrypt

Database Server

1. Install:

   • postgresql-13
   • postgresql-contrib
   • postgresql-13-cron
   • postgis
   • postgresql-13-postgis-3
   • for quantile extension; Install:
     • devscripts
     • sudo
     • postgresql-server-dev-13
     • pgxnclient
     • Run: pgxn install quantile

2. Configure pg_cron, add in /etc/postgresql/13/main/postgresql.conf

   # -- extension pg_cron
   # add to postgresql.conf
   
   # required to load pg_cron background worker on start-up
   shared_preload_libraries = 'pg_cron'
   # optionally, specify the database in which the pg_cron background worker should
   cron.database_name = 'rmbt'
   
   # Schedule jobs via background workers instead of localhost connections
   cron.use_background_workers = on
   

3. Run:

   # Restart database to enable pg_cron
   systemctl restart postgresql
   su - postgres
   # Create database users
   createuser -lSRDP rmbt     # (set password)
   createuser -lSRDP rmbt_control     # (set password)
   createuser -LSRD rmbt_group_control
   createuser -LSRD rmbt_group_read_only
   createuser -LSRD rmbt_group_read_only
   # The following two users are within schema, but not required
   # for basic functionality
   createuser -LSRD kibana # (for export)
   createuser -LSRD nagios # (for monitoring) 
   # Additional users might be required for replication
   echo 'GRANT rmbt_group_read_only TO rmbt_group_control;' | psql
   echo 'GRANT rmbt_group_control TO rmbt_control;' | psql
   
   # Create database
   createdb -O rmbt rmbt 
   
   # import database scheme    
   cat rmbt.sql | psql rmbt -1
   # import basic configuration (modifiy according to your needs)
   cat rmbt_init.sql | psql rmbt -1

   (optional: add additional 3rd party open data, eg. Corine)

4. Edit table "test_server"

   You need to add the test server key to the test_server table.

5. Edit /etc/postgresql/13/main/postgresql.conf to allow remote access for the ControlServer and other servers (change listen_addresses) and configure /etc/postgresql/13/main/pg_hba.conf accordingly

6. Optimise postgres settings

   Check the values of

   • shared_buffers (approx. 25% of available memory)
   • work_mem (approx25% of mem/max_connections)
   • effective_cache_size (approx. 50% of mem)
   • maintenance_work_mem (approx. 5% of mem)
   • max_worker_processes (as number of CPUs)
   • max_parallel_workers_per_gather (as number of CPUs)
   • max_parallel_workers (as number of CPUs)

MapServer

1. Install:

• Apache Tomcat 9 or higher
• nginx (optional, highly recommended)
• openjdk-11-jre (do not use a higher version)
• libservlet3.1-java

2. Edit /etc/tomcat9/context.xml (substitute parts with []), add to <Context>:

   <Context>
   <!-- [...] -->
   <Resource 
      name="jdbc/rmbtro" 
      auth="Container"
      type="javax.sql.DataSource"
      maxActive="200" maxIdle="10" maxWait="10000"
      url="jdbc:postgresql://[db host]/rmbt"
      driverClassName="org.postgresql.Driver"
      username="rmbt" password="[read only pass]"
      description="DB RO Connection" />
   <!-- [...] -->
   </Context>
    

3. Build the server:

   The map server can be built with gradle:

   ./gradlew :RMBTMapServer:war 

   The war file is located in RMBTMapServer/build/lib.

4. Copy RMBTMapServer.war to /var/lib/tomcat9/webapps/

5. Add the package libpostgresql-jdbc-java from Postgresql JDBC and restart tomcat9.

6. Optimize tomcat settings

   Check the values in /etc/default/tomcat9

   • JAVA_OPTS -Xmms MEM -Xmx MEM

Get in Touch

• RTR-Netztest on the web