jimmytsai16 / mktxp-stack

MKTXP Exporter monitoring stack

Description

MKTXP-Stack is a dockerized monitoring stack for MKTXP Exporter.

As an out-of-the-box solution, it lets you quickly get up & running with MKTXP, Prometheus, and Grafana and have multiple Mikrotik RouterOS devices monitored with least amount of configuration.

While complementary to MKTXP, this project also adds some extra capabilities such an centralized Mikrotik log processing based on a preconfigured syslog-ng / promtail / Loki stack.

The project offers multiple docker-compose configurations, for loading only relevant parts of the stack as well as for multiple log management options.

Requirements:

Docker Compose

Install & Getting Started:

• Clone this repository (or download zip with wget)

git clone https://github.com/akpw/mktxp-stack.git
cd mktxp-stack

MKTXP Exporter configuration

• Following the steps described in MKTXP Getting Started, let's:
  a) edit the main mktxp config file, adding your Mikrotik device IP address & authentication info to provided sample entry:

  nano mktxp/mktxp.conf
  

  b) if needed, add a dedicated API user from the mktxp config to your RouterOS device:

  /user group add name=mktxp_group policy=api,read
  /user add name=mktxp_user group=mktxp_group password=mktxp_user_password
  

With that out the of way, things should be ready for running docker-compose:

docker-compose -f ./docker-compose-mktxp-stack.yml up -d

Now give the containers some time to start up, and then point a Web browser to Grafana. You should see the default MKTXP Exporter Dashboard:

 

Mikrotik Centralized Logging configuration

In addition to RouterOS devices monitoring, MKTXP-Stack provides a preconfigured syslog-ng / promtail / Loki stack to receieve & process logs from multiple Mikrotik RouterOS devices in a centralized location:

To make this work, we need to configure our Mikrotik devices to send their logs to a specified log server target. Let's first configure the corresponding remote logging action (replace XX.XX.XX.XX with your docker-compose host IP address):

/system logging action
set remote bsd-syslog=yes name=remote remote=XX.XX.XX.XX remote-port=514 src-address=0.0.0.0 syslog-facility=local0 syslog-severity=auto target=remote

Next, let's modify relevant log topics to use with this remote action:

/system logging
set 0 action=remote prefix=:Info
set 1 action=remote prefix=:Error
set 2 action=remote prefix=:Warning
set 3 action=remote prefix=:Critical

add action=remote disabled=no prefix=:Firewall topics=firewall
add action=remote disabled=no prefix=:Account topics=account
add action=remote disabled=no prefix=:Caps topics=caps
add action=remote disabled=no prefix=:Wireles topics=wireless

You can extend the list above as needed, following Mikrotik's description of the log topics used by various RouterOS facilities

Now all should be ready and, unless you already done so during the previous MKTXP Exporter configuration, it's time for docker-compose:

docker-compose -f ./docker-compose-mktxp-stack.yml up -d

As soon as the containers are up & running, just point your Web browser to included Grafana dashboards and open the one called "Mikrotik Loki Logs".

Alternative docker-compose configurations

The project offers multiple docker-compose files, for loading relevant parts of the stack as well as multiple log management options.

Default stack

To go with default full stack, just run docker-compose as described above:

docker-compose -f ./docker-compose-mktxp-stack.yml up -d

File-system based logs

If you want more control over managing your routers' logs, such a specific file-system location and separate log files:

docker-compose -f ./docker-compose-mktxp-stack-fs.yml up -d

This configuration makes it easy to implement log rotation or any additional management functionality on top. By default, the devices' logs will be send to syslog-ng/logs/ where you can check it out with:

ls -l syslog-ng/logs/

💡 With mktxp-stack up & running, it might take a while for newly added log files to appear in the dashboard. In case this happens, an easy solution is to restart the promtail and syslog containers via docker restart promtail; docker restart syslog-ng

To set a different location on your mktxp-stack host, just edit the device path in the mktxp-stack/docker-compose-mktxp-stack-fs.yml configuration:

volumes:
  mktxp-logs:
    driver_opts:
       o: bind
       type: none
       device: $PWD/syslog-ng/logs

MKTXP Exporter only

Finally, in case you need just MKTXP Exporter functionality and no logs:

docker-compose -f ./docker-compose-mktxp-stack-no-logs.yml up -d

Overview of components used in this project:

• MKTXP Exporter: an open-source Prometheus Exporter for Mikrotik RouterOS devices
• Prometheus: an open-source monitoring & alerting toolkit for cloud / native environments
• Grafana: an open-source analytics & interactive visualization platform
• Loki: an open-source log aggregation system inspired by Prometheus
• promtail: an open-source agent to deliver the logs to Loki ]
• syslog-ng: an open-source log server implementing the syslog protocol

Included Dashboards

• MKTXP Exporter Dashboard: set as default Grafana dashboard
• Mikrotik Loki Logs: logging dashboard designed for this project
• Grafana Internals: Grafana-related stats for system overivew, credited to Grafana community
• Prometheus 2.0 Stats: Prometheus-related stats for system overivew, credited to Grafana community