Checkout my profile and other projects at amit.cloud
Script to delete logs older then set number of days (File beat, metric beat, cloudtrail).
- Copy the script to the ELK master server.
- Install the Extra Packages if needed.
pip install elasticsearch
pip install curator
- Update the days (Default set to 14 days) in the script.
- Run the script as
python clean_elk.py
Following three indices are considered for cleanup:
- Filebeat
- Metricbeat
- Cloudtrail You could append your custom indices or remove the once not required by updating line #10 in code:
index = {"fb": "filebeat-", "mb": "metricbeat-", "ct": "cloudtrail-"}
This script would delete old logs from ELK master/data nodes and ensure that old logs does not occupy all the space in nodes.
- Python Version = 3.7.2
- elasticsearch (Python module) = 7, 0, 2
- curator (Python module) = 5.5.4