danmutblix / calert

Prometheus' Alertmanager - Google Chat Integration :bell:

calert

Overview

calert is a lightweight binary to push Alertmanager notifications to Google Chat via webhook integration.

Table of Contents

• Getting Started

  • How it Works
  • Installation
  • Quickstart
  • Testing a sample alert

• Advanced Section

  • Configuration options
  • API Usage
  • Setting up Prometheus

• Ansible Playbook

Getting Started (Quickstart)

How it Works

calert uses Alertmanager webhook receiver to receive alerts payload, and pushes this data to Google Chat webhook endpoint.

Installation

There are multiple ways of installing/running calert.

• Precompiled binaries

  Precompiled binaries for released versions are available in the Releases section.

• Compiling the binary

  You can checkout the source code and build manually:

  $ mkdir -p $GOPATH/src/github.com/mr-karan/
  $ cd $GOPATH/src/github.com/mr-karan
  $ git clone https://github.com/mr-karan/calert.git
  $ cd calert
  $ make build
  $ cp config.toml.sample config.toml
  $ ./calert
  

• Running as docker container

  docker pull mrkaran/calert:latest

You can find all the tags here.

• Deploying on Kubernetes cluster

  kustomize build ./kustomize/overlays/prod | kubectl apply -f -

Quickstart

mkdir calert-example && cd calert-example/ # copy the binary and config.toml.sample in this folder
cp config.toml.sample config.toml # change the settings like hostname, address, google chat webhook url, timeouts etc in this file.
./calert.bin # this command starts a web server (calert API) and is ready to receive events from alertmanager

• Set the webhook URL from Google Chat in [app.chat.<yourRoomName>.notification_url] section of config.toml. You can refer to the official documentation for more details.

You are now ready to send alerts to Google Chat!

Testing a sample alert

To help you quickly get started, you can POST a dummy payload which is similar to Alertmanager payload, present in examples/send_alert.sh.

Advanced Section

Configuration Options

• [server]

  • address: Port which the server listens to.
  • socket: Optional, if you want to connect to the server using unix socket.
  • name: Optional, human identifier for the server.
  • read_timeout: Duration (in milliseconds) for the request body to be fully read) Read this blog for more info.
  • write_timeout: Duration (in milliseconds) for the response body to be written.
  • keepalive_timeout: Duration (in milliseconds) the server waits for the connection to be re-used in case the request is sent with Keep-Alive header.

• [app]

  • template_file: Path to template file used for parsing Alertmanager payload to Google Chat message. You can configure the default template for notification and create your own. Create a template file, similar to message.tmpl and set the path of this file in this setting.
  • max_size: The maximum size of a single push to the webhook URL.
  • http_client
    • max_idle_conns: Optional, Maximum count of keep-alive collections per host.
    • request_timeout: Duration (in milliseconds) to wait for the response.
  • chat.your_room_name
    • notification_url: Webhook URL of Google Chat Room where Incoming Webhooks are configured.
  • http_client
    • max_idle_conns: Optional, Maximum count of keep-alive collections per host.
    • request_timeout: Duration (in milliseconds) to wait for the response.
  • chat.your_room_name
    • notification_url: Webhook URL of Google Chat Room where Incoming Webhooks are configured.

NOTE: You can use --config.file parameter to supply a custom config file path while running calert.

API Usage

• POST /create?room_name=<> (Used to receive new alerts and push to Google Chat.)

# example request. (See examples/send_alert.sh)
➜ curl -XPOST -d"$alerts1" http://localhost:6000/create?room_name=<room> -i
{"status":"success","message":"Alert sent","data":null}

room_name param is required. The same room_name should be present in app.chat section. You can refer to the config.sample for examples.

• GET /ping (Health check endpoint)

➜ curl http://localhost:6000/ping
{"status":"success","data":{"buildVersion":"025a3a3 (2018-12-26 22:04:46 +0530)","buildDate":"2018-12-27 10:41:52","ping":"pong"}}

Setting Up Prometheus

• Alertmanager has the ability of group similar alerts together and fire only one event, clubbing all the alerts data into one event. calert leverages this and sends all alerts in one message by looping over the alerts and passing data in the template. You can configure the rules for grouping the alerts in alertmanager.yml config. You can read more about it here.

• Configure Alertmanager config file (alertmanager.yml) and give the address of calert web-server. You can refer to the official documentation for more details.

Ansible Playbook

• You can refer to the calert role in my Ansible Server Logs-Monitoring playbook.

Contribution

PRs on Feature Requests, Bug fixes are welcome. Feel free to open an issue and have a discussion first. Read CONTRIBUTING.md for more details.

License

MIT