davidcollom / puppet-serf

Puppet Module for Serf

#puppet-Serf

##Overview

The Serf module installs, configures, and manages a Serf agent.

##Module Description

The Serf module manages the installation and configuration of Serf

##Setup

include serf

##Usage

class{'serf':
    bind            =>  $::ipaddress_eth1
    event_handler   =>  ['/root/bin/handler.sh']
}

###Parameters

#####version The version you wish to install

#####bind The address that Serf will bind to for communication with other Serf nodes

Note Changing the bind port also requires you specify the rpc_addr otherwise rpc_addr will remain the default $::ipaddress fact

#####config_dir The directory you wish to store the serf config file in, defaults to: '/etc/serf/'

#####config_file The name of the config file you wish to be generated, stored within config_dir eg:${config_dir}/${config_file}

#####encrypt Secret key to use for encryption of Serf network traffic

#####event_handler Array of event handlers, NOTE: No validation is performed if these files exist.

#####join Array of IP's known to the Serf network already, This was added to allow for hiera data to be passed and allow for a "master" node which is aware of all serf clusters but is in a "monitoring" state.

#####log_level Change logging levels, Added for debugging purposes, defaults to: info #####node Node name, defaults to $fqdn

#####protocol Protocol Version to use- this is for backwards compatability and defaults to 1

#####role Optional Role for specific node

#####rpc_addr The address that Serf will bind to for the agent's internal RPC server, defaults to: "${::ipaddress}:7373"

#####advertise The address that Serf will provide to other nodes to reach it; defaults to ```${::ipaddress}`.

Eg:

$::ipaddress='10.5.2.xxx'
class{'serf':
    bind => '10.6.2.xxxx'
}

rpc_addr would be '10.5.2.xxxx:7373'

#####install_path (Private) Installation path to install serf executable, this should be within your normal users path. defaults to: /usr/local/bin

#####install_url Where to download serf from, this defaults to: https://dl.bintray.com/mitchellh/serf/${version}_linux_${::architecture}.zip

#####install_method Defaults to url but can be package if you want to install via a system package.

#####package_name Only valid when the install_method == package. Defaults to serf.

#####package_ensure Only valid when the install_method == package. Defaults to present.

#####config_owner and config_group Specify who is to become owner of $config_file

##Limitations

This module has been tested on:

• Ubuntu 12.04
• CentOS 6.4

Testing on other platforms has been light and cannot be guaranteed.

Vagrant example

1. Install Vagrant and VirtualBox
2. Clone this repository: git clone https://github.com/danieldreier/serf-demo.git
3. cd serf-demo then run vagrant up node0 and wait for the first node to start
4. vagrant ssh -c 'serf monitor' node0 to monitor the communication the first node sees
5. in another terminal window, run vagrant up and watch members join the cluster
6. vagrant ssh node1 then run serf event foo bar will trigger an event titled "foo" with a payload "bar"
7. On node1, run serf members to get a list of serf cluster members
8. exit out to your regular command line, run vagrant halt node2, then repeat step 7 to see node2's status change to 'failed'.

When you're done, run vagrant halt to shut them down, or vagrant destroy to remove the VMs entirely.

You can modify the number of nodes created by editing INSTANCES=5 in the Vagrantfile to some other value, then running vagrant up again to bring up the new nodes. If you add other services to the nodes, you'll probably need to increase MEMORY=128 to MEMORY=256 or greater.

Authors

This module is based on the example provided by Mitchell Hashimoto at https://github.com/hashicorp/serf/

puppet-serf's Issues

need to specify rpc_addr when overiding bind

If you specify a custom bind value, for instance $::ipaddress_eth1 then the value for rpc_addr will remain with the default value of $::ipaddress. The README indicates otherwise.

Protocol versioning is broken

There are both a protocol and protocol_version member in serf::params. Only the protocol member is coupled with a corresponding parameter in serf and the template responds to protocol_version. This makes it impossible to set the protocol higher than the default value of 1.

invalid keys bind_addr

I get an error when running serf has invalid keys: bind_addr. I have tried specifying the bind address and leaving it out, still the same thing. I changed the key to bind and it started working.

role deprecated

in 0.4.0 now "role" is deprecated in favor of "tag". I have some updates for 0.4.0 I've been working on so i'll try to package them up as a PR

Interest in more sophisticated vagrantfile?

I recently put together a puppet-serf demo. Any interest in me merging that into the Vagrantfile on puppet-serf? I don't want to do a bunch of work if you prefer a very simple vagrant setup to begin with.

install.pp failure on AWS

$osfamily evaluates as "Linux" on Amazon Linux boxes so I updated this case statement:

     13     case $::osfamily {
     14       'Debian': {
     15         $install_url = "https://dl.bintray.com/mitchellh/serf/${::serf::version}_linux_${::architecture}.zip"
     16       }
     17       'Linux': {
     18         $install_url = "https://dl.bintray.com/mitchellh/serf/${::serf::version}_linux_${::architecture}.zip"
     19       }
     20       default: {
     21         fail "Operating system ${::osfamily} ${::operatingsystem} is not supported yet."
     22       }
     23     }

but then when i try to use it like this:

      2 include serf
      3 
      4 class  { 'serf':
      5     bind            =>  $::ipaddress_eth1
      6     event_handler   =>  ['/root/bin/handler.sh']
      7 }

i get this error on the client

root@ip-172-31-8-10 ~]# puppet agent -t apply
err: Could not retrieve catalog from remote server: Error 400 on SERVER: no such file to load -- json
warning: Not using cache on failed catalog
err: Could not retrieve catalog; skipping run

any ideas?

Puppet functions to query serf members?

What do you think about puppet functions that could return serf members for use in manifests? Like:

$webserver_array = serf_members("webserver")
notify { $webserver_array: }

I'm thinking it would take a parameter of the role and maybe an optional param for alternate RPC addresses.

Do you think this would be useful to others? It isn't very "serf"-like, but it would be eventually consistent. It would have to execute the serf cli on the puppet master, which could be confusing to some. It would also have to scrape the output, which would probably break as serf changes. (until they have a stable api)

If you think this would be useful, would you accept a PR if it did this + tests?

Encryption is broken

The puppet class has a parameter called encrypt but the template responds to an ivar called 'encrypt_key'. As a result, the config file never contains an encryption key and encryption is never enabled.

Option to use system packages instead of wget?

In larger installations, wgetting is not "web scale"

Would you accept a PR for parameters like:
install_source => (url / package )
package_name =
package_ensure =

I can leave the default behavior to be url so the behavior doesn't change, but enable to option to use a package if desired?

Config template does not work with current serf release (0.6.3)

The current serf release does not recognize the bind_addr JSON property in the config file. It expects binding information to be supplied via a bind property.

Version change does not restart service

When changing the version parameter, installation of the new version does not trigger a restart of the service.