ccdcoe / frankencoding

You're busted!

Frankencoding

You're busted! - and just with Bubble Gum And Baling Wire... ok and some Duct Tape is also used.

About

This repository houses a list of open-source tools, libraries, projects, etc that can be used to build awesome security stacks.

Contents

Packet capture and intrusion detection

• Suricata IDS - Suricata is a network IDS, IPS and NSM engine.
  • Documentation
  • CCDCOE course materials
  • Suricata update
  • Detect newly created TLS certificates
• Moloch - Moloch is an open source, large scale, full packet capturing, indexing, and database system.
  • Wiki deprecated
  • Official documentation
  • API endpoints
  • CCDCOE course materials, example WISE plugin, writing a wise plugin
• Zeek - Zeek is a powerful network analysis framework that is much different from the typical IDS you may know.
  • Documentation
  • Scripts for detecting cobal strike

Libraries

• py-idstools - idstools: Snort and Suricata Rule and Event Utilities in Python (Including a Rule Update Tool)
• Go NIDS - gonids is a library to parse IDS rules, with a focus primarily on Suricata rule compatibility.
• Gopacket - Provides packet processing capabilities for Go

Logging

Sources

• Snoopy - Log every executed command to syslog (a.k.a. Snoopy Logger).
  • Improve logging format, normalize snoopy messages using liblognorm
• Sysmon - Make Windows logging great again
  • SwiftOnSecurity configs - Sysmon configuration file template with default high-quality event tracing
  • Sysmon modular - A repository of sysmon configuration modules

Collecting, shipping, stream processing

• Rsyslog - RSYSLOG is the rocket-fast system for log processing.
  • Documentation
  • liblognorm - A fast samples-based log normalization library.
• Syslog-ng - free and open-source implementation of the syslog protocol for Unix and Unix-like systems.
• Fever - fast, extensible, versatile event router for Suricata's EVE-JSON format

Correlation

• Simple Event Correlator - SEC is an event correlation tool for advanced event processing.

Databases

Search engines

• Visibility Across Space and Time (VAST) - is a scalable foundation for a security operations center (SOC): a rich data model for security data, high-throughput ingestion of telemetry, low-latency search, and flexible export in various formats.

Graph

• Neo4j - Neo4j is the world’s leading Graph Database.

SQL

• SQLite - SQLite is a relational database management system contained in a C library. In contrast to many other database management systems, SQLite is not a client–server database engine. Rather, it is embedded into the end program.
• MariaDB - MariaDB server is a community developed fork of MySQL server.
• CockroachDB - the open source, cloud-native SQL database.

NoSQL

• Cassandra - Apache Cassandra is a highly-scalable partitioned row store.
• Elasticsearch - Open Source, Distributed, RESTful Search Engine.
  • Install elastic with docker
• InfluxDB - Scalable datastore for metrics, events, and real-time analytics.
• Prometheus - The Prometheus monitoring system and time series database.

Web interfaces and API-s

Alerts

• Scirius - Scirius is a web application for Suricata ruleset management.
  • scirius-docker
• Evebox - Web Based Event Viewer (GUI) for Suricata EVE Events in Elastic Search
• Alerta - Alerta monitoring system
  • Alerta web UI - Alerta Web UI 7.0

Generic Viz

• Kibana - Your window into the Elastic Stack
• Grafana - The tool for beautiful monitoring and metric analytics & dashboards for Graphite, InfluxDB & Prometheus & More

Programming languages and dev tools

• Rust - Rust’s rich type system and ownership model guarantee memory-safety and thread-safety — and enable you to eliminate many classes of bugs at compile-time.
  • Learn
  • Dark arts
• Golang - Go is an open source programming language that makes it easy to build simple, reliable, and efficient software.
  • Learn
• Julia - Julia is a high-level, high-performance dynamic language for technical computing.
• R - R is a free software environment for statistical computing and graphics.
  • nvim-r - Vim plugin to work with R

Editors

• neovim - Vim-fork focused on extensibility and usability
  • Conquer of Completion - Intellisense engine for vim8 & neovim, full language server protocol support as VSCode
    • coc-rls - Rust language server support for coc.nvim
    • coc-python - Python extension for coc.nvim
  • plug - Minimalist Vim Plugin Manager
  • vim-go - Go development plugin for Vim
• vscode - Visual Studio Code is a streamlined code editor with support for development operations like debugging, task running, and version control.
• juno - Juno is a powerful, free environment for the Julia language.
• Jupyter lab - An extensible environment for interactive and reproducible computing, based on the Jupyter Notebook and Architecture.
  • IJulia - IJulia is a Julia-language backend combined with the Jupyter interactive environment (also used by IPython).
  • gophernotes - The Go kernel for Jupyter notebooks and nteract.

Libraries

• Pandas - powerful Python data analysis toolkit
• matplotlob - plotting with Python

Data science

Tools

• LogCluster - experimental Perl-based tool for log file clustering and mining line patterns from log files

Libraries

• scikit-learn - machine learning in Python
• TensorFlow - An Open Source Machine Learning Framework for Everyone
  • TensorFlow-Examples - TensorFlow Tutorial and Examples for Beginners (support TF v1 & v2)
• Ngraph - a set of graph related algorithms.
  • ngraph.pixel - fast graph renderer based on low level ShaderMaterial from three.js
• Cayley - an open-source graph to be a part of the developer's toolbox

Message queue and data pipelining

• Heka - Heka is a tool for collecting and collating data from a number of different sources, performing "in-flight" processing of collected data, and delivering the results to any number of destinations for further analysis.
• Hindsight - Hindsight is lighter weight and faster data pipeline with delivery guarantees to replace Heka.
• nanomsg - The nanomsg library is a simple high-performance implementation of several "scalability protocols".
• mangos - Package mangos is an implementation in pure Go of the SP ("Scalable Protocols") protocols.
• Kapacitor - Framework for processing, monitoring, and alerting on time series data.
• Kafka - A distributed streaming platform.
• Apache Pulsar - Distributed pub-sub messaging system.

Hunting

Intelligence platforms

• Malware Information Sharing Platform and Threat Sharing (MISP) - Open Source Threat Intelligence and Sharing Platform
• Semi-Automated Cyber Threat Intelligence - ACT - The main objective of the ACT project is to develop a platform for cyber threat intelligence to uncover cyber attacks, cyber espionage and sabotage.
• MITRE ATT&CK - Globally-accessible knowledge base of adversary tactics and techniques based on real-world observations.

Playbooks

• Threat hunters playbook - A Threat hunter's playbook to aid the development of techniques and hypothesis for hunting campaigns.

Metrics and alerting

• Telegraf - Telegraf is an plugin-driven agent for collecting & reporting metrics.

Reading materials

---

see also

• Cyber Defence Monitoring Course Suite
• Technical Courses
• Cyber Defence Exercises