redborder / dswatcher Goto Github PK

View Code? Open in Web Editor NEW

3.0 5.0 0.0 2.33 MB

Service for discover new netflow sensors by watching an option template

Home Page: https://redborder.github.io/dswatcher/

License: GNU Affero General Public License v3.0

Makefile 3.98% Go 96.02%

redborder redborder-ng rpm service

dswatcher's Introduction

dswatcher (Dynamic Sensors Watcher)

Overview
Installing
Usage
Configuration

Overview

Service for dynamically add and remove Teldat sensors on the Netflow collector by updating the information on the Chef node.

When a new sensor starts to send data to the Netflow collector, the data will be discarded to a Kafka topic.
dswatcher will analyze the discarded Netflow data looking for a specific Option Template that carries a Serial Number.
dswatcher will look up on the Chef sensor nodes for a node with the Serial number. If this sensor exists, the IP address for the sensor and the Observation ID will be updated with the IP address and Observation ID of the Netflow sender.
dswatcher will listen for alerts about sensors that reached their limits. The sensor will be marked as blocked on the Chef node. When no UUID is specified, i.e. uuid == "*" then all sensors will be blocked.
dswatcher will listen for alerts about counters resets. When this message is received all the sensors block status will be set to false.
dswatcher can check if the Product Type on the Netflow data matches the Product Type specified on the database (Chef Node).

Installing

To install this application ensure you have the GOPATH environment variable set and glide installed.

curl https://glide.sh/get | sh

And then:

Clone this repo and cd to the project:

git clone https://github.com/redBorder/dswatcher.git && cd dswatcher

Install dependencies and compile:
```
make
```

Install on desired directory:

prefix=/opt/dynamic-sensors-watcher/ make install

Usage

Usage of dswatcher:

--version
    Show version info
--config string
    Config file
--debug
    Print debug info

Configuration

broker:
  address: kafka:9092        # Kafka host
  consumer_group: dswatcher  # Kafka consumer group ID
  netflow_topics:
    - flow_discard_topic     # Topic to look up for the Option Template where the serial number is
  limits_topics:
    - limits_topic           # Topic listen for notification about sensors limits

decoder:
  element_id: 300              # Netflow element id of the serial number
  option_template_id: 258      # ID of the Option Template where the serial number is
  product_type_element_id: 144 # Element ID of the field used to verify the DeviceID

updater:
  chef_server_url: <chef_server_url>            # URL of the Chef server
  node_name: <node_name>                        # Node name on Chef
  client_key: key.pem                           # Path to the key used for Chef authorization
  serial_number_path: org/serial_number         # Path to the serial number of the sensor on Chef
  sensor_uuid_path: org/sensor_uuid             # Path to the UUID of the sensor on Chef
  ipaddress_path: org/ipaddress                 # Path to the IP address of the sensor to update
  observation_id_path: org/observation_id       # Path to the Observation Domain ID to update
  product_type_path: org/product_type           # Path to the Product Type to verify
  fetch_interval_s: 60                          # Time between updates of the internal sensors database
  blocked_status_path: org/blocked              # Path to the block status
  update_interval_s: 30                         # Time between updates of the Chef node
  organization_uuid_path: org/organization_uuid # Organization UUID path of the key used to block sensors
  license_uuid_path: org/license_uuid           # License UUID path of the key used to block sensors
  data_bag_name: rBglobal                       # Name of the data bag where the licenses are stored
  data_bag_item: licenses                       # Item in the data bag where the licenses are stored
  fetch_interval_s: 60                          # Time between updates of the internal sensors database
  update_interval_s: 30                         # Time between updates of the Chef node

dswatcher's People

Contributors

Stargazers

Watchers

dswatcher's Issues

Block sensors by license UUID instead organization UUID

Sprint 2

Teldat sensors send a IPFIX field ElementID 144: (deviceID) which is its product type. This product type is in the Chef node of the sensor.

This deviceID have to be the same on the netflow stream and in the Chef node. If not, dswatcher should ignore the sensor.**

Sprint 5

Decode product type from option template

KafkaConsumer

Create a Kafka consumer to consume the discarded flow.

Create Netflow Decoder

The decoder should decode a Netflow v10 / IPFIX packet and extract a specific field (should be configurable).

Block sensors

Don't block Teldat sensors

When a limit_reached signal is received for an organization, all sensors are blocked, even Teldat sensors.

Teldat sensors should not be blocked on limit_reached signal.

Hint: product_type can be used to identify Teldat sensors.

When a license_expired signal is received, every sensor associated to the license should be blocked, regardless the sensor type.

#redmine-9893

Verify deviceID

Allow unblock sensors that belongs to an specific organization

dswatcher should be able to unblock sensors belonging to a specific organization.

A message like the following should unblock all sensors. Note the "*" on the organization_uuid.

{
  "monitor": "alert",
  "type": "counters_reset",
  "organization_uuid": "*",
  "timestamp": 1496996336
}

To unblock only sensors that belongs to an organization, the message should be:

{
  "monitor": "alert",
  "type": "counters_reset",
  "organization_uuid": "e4fb8711-feb4-49d6-9316-c6915aeface1",
  "timestamp": 1496996336
}

#redmine-9702

Error updating node with serial number 757/00157: Node not found

But it should display the IP address of the sensor.