enterprise-unified-logging's Introduction

Enterprise Unified Logging

This repository contains example configurations for running an Elasticsearch stack to monitor and log ShotGrid Enterprise. The visualizations and dashboards provided are examples of what could be useful for monitoring ShotGrid Enterprise, clients may choose to modify the configuration as needed.

The solution uses Fluentd as the data collector between the ShotGrid application and the Elasticsearch database.

Getting started

First, you will need to build the Fluentd image and the Kibana image using the following command.

docker-compose build

Then start Fluentd and Elasticsearch along with Kibana.

docker-compose up

Finally, you will need to change the ShotGrid application logging driver in its docker-compose.yml file from json-file to fluentd:

Remove

# json-file
logging:
  driver: "json-file"
  options:
    max-size: "2g"
    max-file: "20"

And add

# fluentd
logging:
  driver: "fluentd"
  options:
    fluentd-address: "<fluentd_server_address>:24224"
    tag: "sg.app.{{.ID}}"
    mode: "non-blocking"

How to access logs

Kibana

Logs can be access via Kibana at http://localhost:5601/

From there you can create your indexes (ex: shotgrid_logs-* already created by default) and then query Elasticsearch.

Saved Objects

Customizations (visualizations, dashboards, etc) can be saved. Please refer to Kibana documentation on how to manage saved objects. However, once changes are made in Kibana they persist as long as the database is not removed.

Saved objects can be provisioned by default in a new stack by modifying the appropriate ndjson files in the kibana/saved_objects/ directory.