gradiuscypher / grids

My network monitoring solution and tools that go along with it.

grIDS

My network monitoring solution and tools that go along with it. This setup is designed to be contained in an all-in-one sort of system, but services can be separated into individual hosts if you have a higher load that requires more resources.

The goal for this project is to help people combine multiple open-source tools to have a useful network monitoring solution. Within this project, I'll also include custom scripts that help make this system even more useful.

Feedback, corrections, and suggestions are welcomed and appreciated. Reach out to me on Twitter or here on Github as an Issue.

Manual Configuration Steps

Wiki

To get started with configuring the system, follow the steps found in the Wiki. Each of the Configuration sections walks you through the parts of building a working all-in-one IDS system.

Docker Configuration Steps

To deploy the NSM via Docker containers, follow the instructions found under the Docker Configuration section of the Wiki.

Screenshots

Screenshots of some Kibana Dashboards that have been created and powered using this setup

Future Additions + Modifications + Ideas

This is a list of future tools that could be added to this toolset for even more features. Also includes modifications.

Features

• Sysmon logging
• Bro logging
• Centralized Logging
• Webhook integration for alerts
• FPC and usability tools
• Pi-Hole DNS service
• Include EveBox for alerting
• Include Scirius rule management.

Modifications

• Performance tuning for Elastic Stack
• Performance tuning of Suricata - spread load between CPU threads

Ideas

• IDS Sensor on a Raspberry Pi
• Spreading services out among hosts, multiple sensor configuration and management
• Setting up an iptables sensor
• Setting up a hardware network tap
• Process for upgrading/updating the OS

grids's Issues

Improved documentation for splitting services onto separate hosts

Elastic data expiration

Setup a graceful way to expire old Elasticsearch documents with configurable rates.

Include TheHive

Python tooling for rule updates

Need to put together some tools for updating the Suricata rules in the Docker container.

Also need to consider actually tuning the rules and picking particular sections of ET's rulesets. Could also write a script to help users pick the right rulesets.

Instructions on installing Elastic monitoring tools

The Elastic monitoring X-Pack is super helpful for managing Elasticsearch. Should include a guide for setting it up.

Include Moloch

Rebuild Dashboards

Threat Hunting Documentation

How to leverage the platform for data exploration and threat hunting.

Move Documentation to Wiki rather than MD files

Improved documentation and process for the Hardware section

The hardware section is light on details and is a bit hand-wavy. Need to improve this with actual recommendations and some more math behind hardware choices.

events/sec on IDS Monitoring Dashboard is incorrect

The events/sec metric on the IDS Monitoring Dashboard isn't calculating properly.

Alerting scripts to send notifications for various events

Would like to write up something to publish events from the IDS platform to webhooks for things like Slack/Discord.

Might also consider leveraging Push Notifications with a platform like Firebase.

Automated Setup Scripts

Automated scripts for each section of setup, using something like Chef/Ansible preferred.

Integrate piHole DNS adblocking

Suricata Configuration Deep Dive

Finish the Suricata Configuration deep dive, as mentioned in this Wiki page: https://github.com/gradiuscypher/grIDS/wiki/Suricata-Container

Elasticsearch Docker - Performance changes

Two things need to be done to ensure that the Elasticsearch Docker containers have enough resources:

• Deploy the Elasticsearch containers with more than once node
• Modify the host vm_max_map to a higher value. Reference here.

Machine learning and anomaly detection

Exploration into machine learning to highlight anomalous traffic.

Suricata is configured to a specific interface

Suricata is configured to use a specific interface name. This will more than likely not match other's hardware, so we need a way to change the interface name, either during a script or letting the user know they need to change the interface name.

nginx proxy for kibana

nginx proxy for kibana for more control and configuration.

Integrate Sysmon Logging

Full Packet Capture

Full packet capture along with a useful way of navigating PCAPs.

Logstash not generating index

On the docker configuration when i launch the logstash container it doesnt create an index in kibana.

Ignoring the 'pipelines.yml' file because modules or command line options are specified.
No persistent UUID file found. Generating new UUID.

Integrate Bro