mveitas / loggly-rsyslog

Chef cookbook to configure rsyslog to send logs to Loggly

Loggly rsyslog Cookbook

Installs and configures rsyslog for use with Loggly. This cookbook was built upon the work from an existing cookbook, https://github.com/kdaniels/loggly-rsyslog.

Requirements

• Chef 11 or higher
• Ruby 1.9.3 or higher

Platform

Tested against Ubuntu 12.04

Data Bags

By default, this cookbook depends on the use of encrypted data bags to store the token to be used. For more information about data bags see the Chef Data Bags documentation. By default, the data bag needs to be named loggly and contains an item token:

{
    "id": "token",
    "token": "<your token goes here>"
}

You may change the name of the data bag and item via the node['loggly']['token']['databag'] and node['loggly']['token']['databag_item'] attributes respectively. Also, if you do not want this cookbook to load the credentials for you, and instead want to set them yourself, set node['loggly']['token']['from_databag'] to false, and then set the credentials via node['loggly']['token']['value'].

Attributes

• node['loggly']['tags'] - A list of event tags to apply to a message (https://www.loggly.com/docs/tags/) (optional)

• node['loggly']['log_files'] - A list of files rsyslog should monitor. (optional). Below is an example of a hash used to describe a file to monitor.

    {
        :filename => "/var/log/filename.log",
        :tag => "tag you want for this logfile",
        :statefile => "unique-name-for-statefile"
    }
  

• node['loggly']['log_dirs'] - A list of directories to monitor (optional). The loggly configuration template will create an imfile block for each file ending in '.log' in that directory. Each logdir in the list is of the format:

  {
      :directory => "/var/log/directory",
      :tag => "tag for all files in this directory"
  }
  

• node['loggly']['tls']['enabled'] - Set to true if communication to the remote service should use TLS (defaults to true)

• node['loggly']['tls']['cert_path'] - Directory where the loggly certificate should be placed

• node['loggly']['tls']['cert_url'] - Url to the loggly.com certificate

• node['loggly']['tls']['cert_checksum'] - Cchecksum of the loggly.com certificate

• node['loggly']['tls']['intermediate_cert_url'] - Url to the intermediate certificate

• node['loggly']['tls']['intermediate_cert_checksum'] - Checksum of the intermediate certificate

• default['loggly']['token']['from_databag'] - Whether to load the Loggly token from a Data Bag (defaults to true)

• default['loggly']['token']['databag'] - The name of the Data Bag to load the credentials from (defaults to "loggly")

• default['loggly']['token']['databag_item'] - The name of the Data Bag Item to load the credentials from (defaults to "token")

• default['loggly']['token']['value'] - The Loggly token. Set from the Data Bag above by default.

• node['loggly']['rsyslog']['conf'] - Name of the loggly rsyslog confiugration file (defaults to /etc/rsyslog.d/10-loggly.conf)

• node['loggly']['rsyslog']['host'] - Name of the remote loggly syslog host (defaults to logs-01.loggly.com)

• node['loggly']['rsyslog']['port'] - Port of the remote loggly syslog host (defaults to 514 and if TLS is enabled to 6514)

• node['loggly']['rsyslog']['input_file_poll_interval'] - Specifies how often files are to be polled for new data (defaults to 10)

Recipes

Include the default recipe in the run list or a cookbook. The cookbook includes the rsyslog cookbook that will install the rsyslog package and start the service if it does not exist. The rsyslog service will restart after changes to the loggly rsyslog configuration file are made.

Running Locally with Vagrant

Since the cookbook relies on using an encrypted data bag, there is some additional steps that are needed in order to run the cookbook locally using Vagrant. The create_data_bag.rb is a helper script that can be used to create the data bag for loggly for use with Vagrant and Chef Solo. The script expects a single arguement, the value of the loggly token.

Add the following two lines into your Vagrantfile in the chef_solo provisioner configuration:

chef.data_bags_path = './data_bags'
chef.encrypted_data_bag_secret_key_path = './encrypted_data_bag_secret'

License & Authors

• Author: Matt Veitas [email protected]

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

    http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.

loggly-rsyslog's Issues

Update rsyslog dependency

Hey,

This recipe depends on rysyslog ~> 1.5.0 and claims to support ubuntu >= 12.04.

Since Ubuntu 13.10 rsyslog no longer uses init.d to manage the service and requires upstart. Chef is unable to restart rsyslog with this recipe on Ubuntu >= 13.10.

A fix was added to the core rsyslog recipe to allow usage of upstart https://tickets.opscode.com/browse/COOK-4126.

Could you update the dependency to use a version of rsyslog that supports upstart?

I tested by changing the dependency to pull the latest version of rsyslog (1.12) and it seems to work okay.

Cheers,
David

Deprecated Statefile with rsyslog 8

Looks like statefiles are deprecated for rsyslog 8. When validating the new config, syslog executes https://github.com/chef-cookbooks/rsyslog/blob/master/recipes/default.rb#L42-L46 and throws a warning, however somehow chef interprets this as a failure.

      * execute[validate_config] action run

           ================================================================================
           Error executing action `run` on resource 'execute[validate_config]'
           ================================================================================

           Mixlib::ShellOut::ShellCommandFailed
           ------------------------------------
           Expected process to exit with [0], but received '1'
           ---- Begin output of rsyslogd -N 1 -f /etc/rsyslog.conf ----
           STDOUT: 
           STDERR: rsyslogd: version 8.16.0, config validation run (level 1), master config /etc/rsyslog.conf
           rsyslogd: error during parsing file /etc/rsyslog.d/99-files.conf, on or before line 9: parameter 'statefile' deprecated but accepted, consider removing or replacing it [v8.16.0 try http://www.rsyslog.com/e/2207 ]
           ---- End output of rsyslogd -N 1 -f /etc/rsyslog.conf ----
           Ran rsyslogd -N 1 -f /etc/rsyslog.conf returned 1

           Cookbook Trace:
           ---------------
           /tmp/kitchen/cache/cookbooks/compat_resource/files/lib/chef_compat/monkeypatches/chef/runner.rb:41:in `run_action'

           Resource Declaration:
           ---------------------
           # In /tmp/kitchen/cache/cookbooks/rsyslog/recipes/default.rb

            42: execute 'validate_config' do
            43:   command "rsyslogd -N 1 -f #{node['rsyslog']['config_prefix']}/rsyslog.conf"
            44:   action  :nothing
            45: end
            46: 

           Compiled Resource:
           ------------------
           # Declared in /tmp/kitchen/cache/cookbooks/rsyslog/recipes/default.rb:42:in `from_file'

           execute("validate_config") do
             action [:nothing]
             retries 0
             retry_delay 2
             default_guard_interpreter :execute
             command "rsyslogd -N 1 -f /etc/rsyslog.conf"
             backup 5
             returns 0
             declared_type :execute
             cookbook_name "rsyslog"
             recipe_name "default"
           end

           Platform:
           ---------
           x86_64-linux

         * service[rsyslog] action restart
           - restart service service[rsyslog]

       Running handlers:
       [2016-08-16T20:27:50+00:00] ERROR: Running exception handlers
       Running handlers complete
       [2016-08-16T20:27:50+00:00] ERROR: Exception handlers complete
       Chef Client failed. 12 resources updated in 11 seconds
       [2016-08-16T20:27:50+00:00] FATAL: Stacktrace dumped to /tmp/kitchen/cache/chef-stacktrace.out
       [2016-08-16T20:27:50+00:00] FATAL: Please provide the contents of the stacktrace.out file if you file a bug report
       [2016-08-16T20:27:50+00:00] ERROR: execute[validate_config] (rsyslog::default line 42) had an error: Mixlib::ShellOut::ShellCommandFailed: Expected process to exit with [0], but received '1'
       ---- Begin output of rsyslogd -N 1 -f /etc/rsyslog.conf ----
       STDOUT: 
       STDERR: rsyslogd: version 8.16.0, config validation run (level 1), master config /etc/rsyslog.conf
       rsyslogd: error during parsing file /etc/rsyslog.d/99-files.conf, on or before line 9: parameter 'statefile' deprecated but accepted, consider removing or replacing it [v8.16.0 try http://www.rsyslog.com/e/2207 ]
       ---- End output of rsyslogd -N 1 -f /etc/rsyslog.conf ----

Copyright mismatch

Your README says Apache 2.0, your recipe files say "All rights reserved - Do Not Redistribute".