tlugger / oopsgenie

Functions to run analysis on an exported OpsGenie alert CSV

OpsGenie Alert Classifier

Helpful functions for analyzing an export of OpsGenie alerts and collecting useful data on them.

Functions

• Clean (--clean cloumn1 column2 ...)
  • Specify rows from a *raw.csv file into a *clean.csv file
  • Remove (--remove keyword1 keyword2 ...)
    • A keyword to filter each cleaned row against. Matches on any value in the 'Message' column.
• Count (--count column)
  • Count the number of alerts matching a specified column name. Passing a column name of "all" will return the total count of alerts
  • Limit (--limit limit)
    • Limit the number of results returned, ordered by count highest to lowest
  • Time Interval Filtering (--interval hour1 hour2)
    • An interval to filter the "CreatedAtDate" hour of each alert. This can be added to count.
  • Keyword matching (--match keyword)
    • A keyword to filter the specified --count column against
  • Minutes between update (--update-minutes minutes)
    • A filter matching alerts that update within x minutes (between "CreatedAt" and "UpdatedAt" timestamps)
  • Output file (--outfile filename)
    • A file to output the results of --count
• Fuzzy Count (--fuzzy-count column)
  • Count the number of alerts for a specified column name using fuzzy matching.
  • Threshold (--threshold threshold)
    • A threshold of tolerance for fuzzy matching on your --fuzzy-count. This is based on Levenshtein Distance; Default to 90.
  • Remove Numbers (--remove-numbers boolean)
    • Remove numbers from the alert alias before performing fuzzy matching in --count. This defaults to False and should be used in conjunction with the fuzzy threshold flag.
  • Alias Strip List File (--alias-strip-list csv)
    • Remove strings from the alert alias before performing matching in --fuzzy-count. Input for this flag is a csv without headers that contains a single column of strings to strip out of the alias (e.g. server names).

Future functionality

• Implement matplotlib (or some other package) graphing for alerts over time
• Wrap oopsgenie into a pip installable CLI tool

Running

This requires Python3

pip install virtualenv
virtualenv -p python3 venv
source venv/bin/activate
pip install -r requirements.txt

Usage Examples:

# Clean alert-data-raw.csv to only include the columns "Alias", "CreatedAtDate", and "Teams" (creates alert-data-clean.csv)
python oopsgenie.py alert-data-raw.csv --clean Alias CreatedAtDate Teams

# Clean alert-data-raw.csv to only include the column "Alias", "CreatedAtDate", and "Teams" but exclude any message containing "staging"
python oopsgenie.py alert-data-raw.csv --clean Alias CreatedAtDate Teams --remove staging

# Get a count of all alerts
python oopsgenie.py alert-data-raw.csv --count

# Get a count of alerts grouped by the column "Alias"
python oopsgenie.py alert-data-raw.csv --count Alias

# Get a count of alerts grouped by the column "Alias" with server names stripped out
python oopsgenie.py alert-data-raw.csv --count Alias --alias-strip-list server_names.csv

# Get a count of alerts grouped by the column "Alias" and with a fuzzy matching threshold of 80%
python oopsgenie.py alert-data-raw.csv --fuzzy-count Alias --threshold 80

# Get a count of alerts grouped by the column "Alias" and with a fuzzy matching threshold of 90% and numbers removed from the alias before the fuzzy matching
python oopsgenie.py alert-data-raw.csv --fuzzy-count Alias --threshold 90 --remove-numbers True

# Get a count of all alerts grouped by the column "Alias" that are created between the hours of 04 and 13 (UTC)
python oopsgenie.py alert-data-raw.csv --count Alias --interval 4 13

# Get a count of all alerts grouped by the column "Alias" that match the keyword "gdpr"
python oopsgenie.py alert-data-raw.csv --count Alias --match gdpr

# Get a count of all alerts grouped by the column "Alias" that updated within 5 minutes of creation
python oopsgenie.py alert-data-raw.csv --count Alias --update-minutes 5

# Get a count of alerts grouped by the column "Alias" and store in a file named "alias-count.csv
python oopsgenie.py alert-data-raw.csv --count Alias --outfile alias-count.csv

oopsgenie's Issues

Re-write in Go.

Pretty self-explanatory.

Add matplotlib graphing for alerts over time

It would be nice to have a --graph flag to follow alerts over time. This should take a matching column (Alias, Message, etc) and graph by hour/day/week/month using CreatedAtDate