sous-chefs / rsync

Development repository for the rsync cookbook

rsync Cookbook

Rsync cookbook with rsyncd LWRP. More info on ryncd options can be found in the Docs.

Requirements

Platforms

• Debian/Ubuntu
• RHEL/CentOS/Scientific/Amazon/Oracle

Chef

• Chef >= 15.3

Cookbooks

• none

Attributes

node['rsyncd']['config'] (Hash) default: "/etc/rsyncd.conf"

Path to the rsyncd config file. This is the default, but the serve resource can write config files to arbitrary paths independant of this.

node['rsyncd']['globals'] (Hash) default: {}

This is where you can store key-value pairs that coincide with rsyncd globals.

node['rsyncd']['options'] (String) default: ''

Additional daemon arguments.

• motd file: This parameter allows you to specify a "message of the day" to display to clients on each connect. This usually contains site information and any legal notices. The default is no motd file.
• pid file: This parameter tells the rsync daemon to write its process ID to that file. If the file already exists, the rsync daemon will abort rather than overwrite the file.
• port: You can override the default port the daemon will listen on by specifying this value (defaults to 873). This is ignored if the daemon is being run by inetd, and is superseded by the --port command-line option.
• address: You can override the default IP address the daemon will listen on by specifying this value. This is ignored if the daemon is being run by inetd, and is superseded by the --address command-line option.
• socket options: This parameter can provide endless fun for people who like to tune their systems to the utmost degree. You can set all sorts of socket options which may make transfers faster (or slower!). Read the man page for the setsockopt() system call for details on some of the options you may be able to set. By default no special socket options are set. These settings can also be specified via the --sockopts command-line option.

Refer to the documentation for rsyncd for more info.

Recipes

default

This recipe simply installs the rsync package, nothing more.

Resources/Providers

serve

This LWRP implements a rsync server module. The folowing params are chef-only, the rest implement the feature as described in the [rsyncd docs][1]

Parameters

Required

• path - Path which this module should server

Optional

Unless specified these paramaters use the rsyncd default values as refed in the [Rsyncd docs][1]. Params are Strings unless specified otherwise.

• name - The name of this module that will be refrenced by rsync://foo/NAME. Defaults to the resource name.
• config_path - Path to write the rsyncd config Defaults to `node['rsyncd']['config']
• comment - Comment when rsync gets the list of modules from the server.
• read_only - Boolean - Serve this as a read-only module.
• write_only- Boolean - Serve this as a write-only module.
• list - Boolean - Add this module the the rsync modules list
• uid - String - This parameter specifies the user name or user ID that file transfers to and from that module should take place as when the daemon was run as root.
• gid - String - This parameter specifies the group name or group ID that file transfers to and from that module should take place as when the daemon was run as root.
• auth_users - This parameter specifies a comma and space-separated list of usernames that will be allowed to connect to this module. [more info][1]
• secrets_file - This parameter specifies the name of a file that contains the username:password pairs used for authenticating this module. [more info][1]
• hosts_allow - This parameter allows you to specify a list of patterns that are matched against a connecting clients hostname and IP address. If none of the patterns match then the connection is rejected. [more info][1]
• hosts_deny - This parameter allows you to specify a list of patterns that are matched against a connecting clients hostname and IP address. If the pattern matches then the connection is rejected. [more info][1]
• max_connections - Fixnum - *Default: 0 - The maximum number of simultaneous connections you will allow.
• munge_symlinks - Boolean - *Default: true - This parameter tells rsync to modify all incoming symlinks in a way that makes them unusable but recoverable. [more info][1]
• use_chroot - Boolean - the rsync daemon will chroot to the "path" before starting the file transfer with the client.
• nemeric_ids - Boolean - *Default: true - Enabling this parameter disables the mapping of users and groups by name for the current daemon module.
• fake_super - Boolean - This allows the full attributes of a file to be stored without having to have the daemon actually running as root.
• exclude_from - This parameter specifies the name of a file on the daemon that contains daemon exclude patterns. [more info][1]
• exclude - This parameter specifies the name of a file on the daemon that contains daemon exclude patterns. [more info][1]
• include_from - Analogue of exclude_from
• include - Analogue of exclude
• strict_modes - Boolean - If true, then the secrets file must not be readable by any user ID other than the one that the rsync daemon is running under.
• log_file - Path where you should store this modules log file.
• log_format - The format is a text string containing embedded single-character escape sequences prefixed with a percent (%) character. An optional numeric field width may also be specified between the percent and the escape letter (e.g. "%-50n %8l %07p"). [more info][1]
• transfer_logging - This parameter enables per-file logging of downloads and uploads in a format somewhat similar to that used by ftp daemons. The daemon always logs the transfer at the end, so if a transfer is aborted, no mention will be made in the log file.
• timeout - Fixnum - Default: 600 - Using this parameter you can ensure that rsync won't wait on a dead client forever. The timeout is specified in seconds. A value of zero means no timeout.
• dont_compress - This parameter allows you to select filenames based on wildcard patterns that should not be compressed when pulling files from the daemon
• lock_file - This parameter specifies the file to use to support the "max connections" parameter. The rsync daemon uses record locking on this file to ensure that the max connections limit is not exceeded for the modules sharing the lock file. The default is /var/run/rsyncd.lock
• refuse_options - This parameter allows you to specify a space-separated list of rsync command line options that will be refused by your rsync daemon.
• prexfer_exec - A command to run before each transfer to or from this module. If this command fails, the transfer will be aborted.
• postxfer_exec - A command to run after each transfer to or from this module.

Usage

After loading the rsync cookbook you have access to the rsync_serve resource for serving up a generic rsyncd module with many options.

You must include the rsync::server recipe before you can use the LWRP as shown in the examples below.

Examples

Serve a directory:

include_recipe 'rsync::server'

rsync_serve 'tmp' do
  path '/tmp'
end

Serve a directory with read only and specify uids:

include_recipe 'rsync::server'

rsync_serve 'tmp' do
  path      '/tmp'
  uid       'nobody'
  gid       'nobody'
  read_only true
end

A more complex example with networking:

include_recipe 'rsync::server'

rsync_serve 'centos-prod' do
  path             '/data/repos/prod/centos'
  comment          'CentOS prod mirror'
  read_only        true
  use_chroot       true
  list             true
  uid              'nobody'
  gid              'nobody'
  hosts_allow      '127.0.0.1, 10.4.1.0/24, 192.168.4.0/24'
  hosts_deny       '0.0.0.0/0'
  max_connections  10
  transfer_logging true
  log_file         '/tmp/centos-sync'
  postxfer_exec    '/usr/local/bin/collect_things.sh'
end

Contributors

This project exists thanks to all the people who contribute.

Backers

Thank you to all our backers!

Sponsors

Support this project by becoming a sponsor. Your logo will show up here with a link to your website.

rsync's Issues

action:remove missing in server resource

Cookbook version

2.0

Chef-client version

13.8

Scenario:

Removal of a serve resource

Steps to Reproduce:

Upgrade to version 2.0,

try to remove some rsync_serve resource

Expected Result:

resource will be removed

Actual Result:

Chef::Exceptions::ValidationFailed

Option action must be equal to one of: nothing, add! You passed :remove.

The rsync_serve LWRP is not writing to rsync.conf. You end up with a blank rsync.conf.

None of the data makes it from the LWRP to the rsync.conf. The tests for this cookbook are also failing for the same reason.

bundle exec kitchen test simple-centos-66
-----> Starting Kitchen (v1.4.1)
-----> Cleaning up any prior instances of <simple-centos-66>
-----> Destroying <simple-centos-66>...
       Finished destroying <simple-centos-66> (0m0.00s).
-----> Testing <simple-centos-66>
-----> Creating <simple-centos-66>...
       Bringing machine 'default' up with 'virtualbox' provider...
==> default: Matching MAC address for NAT networking...
       ==> default: Setting the name of the VM: kitchen-rsync-simple-centos-66_default_1438641755854_61171
       ==> default: Fixed port collision for 22 => 2222. Now on port 2203.
       ==> default: Clearing any previously set network interfaces...
       ==> default: Preparing network interfaces based on configuration...
           default: Adapter 1: nat
       ==> default: Forwarding ports...
           default: 22 => 2203 (adapter 1)
       ==> default: Booting VM...
       ==> default: Waiting for machine to boot. This may take a few minutes...
           default: SSH address: 127.0.0.1:2203
           default: SSH username: vagrant
           default: SSH auth method: private key
           default: Warning: Connection timeout. Retrying...
           default: 
           default: Vagrant insecure key detected. Vagrant will automatically replace
           default: this with a newly generated keypair for better security.
           default: 
           default: Inserting generated public key within guest...
           default: Removing insecure key from the guest if it's present...
           default: Key inserted! Disconnecting and reconnecting using new SSH key...
       ==> default: Machine booted and ready!
       ==> default: Checking for guest additions in VM...
       ==> default: Checking for host entries
       ==> default: Setting hostname...
       ==> default: Machine not provisioned because `--no-provision` is specified.
       [SSH] Established
       Vagrant instance <simple-centos-66> created.
       Finished creating <simple-centos-66> (0m43.56s).
-----> Converging <simple-centos-66>...
       Preparing files for transfer
       Preparing dna.json
       Resolving cookbook dependencies with Berkshelf 3.3.0...
       Removing non-cookbook files before transfer
       Preparing validation.pem
       Preparing client.rb
-----> Installing Chef Omnibus (install only if missing)
       Downloading https://www.chef.io/chef/install.sh to file /tmp/install.sh
       Trying wget...
       Download complete.
       Downloading Chef  for el...
       downloading https://www.chef.io/chef/metadata?v=&prerelease=false&nightlies=false&p=el&pv=6&m=x86_64
         to file /tmp/install.sh.2395/metadata.txt
       trying wget...
       url      https://opscode-omnibus-packages.s3.amazonaws.com/el/6/x86_64/chef-12.4.1-1.el6.x86_64.rpm
       md5      f09a9ae589bb9040232c600b008ef812
       sha256   95150a4b3c3b2313bd206876e09e2fcf742f2fa4611951d52c79225becb32928
       downloaded metadata file looks valid...
       downloading https://opscode-omnibus-packages.s3.amazonaws.com/el/6/x86_64/chef-12.4.1-1.el6.x86_64.rpm
         to file /tmp/install.sh.2395/chef-12.4.1-1.el6.x86_64.rpm
       trying wget...
       Comparing checksum with sha256sum...

       WARNING WARNING WARNING WARNING WARNING WARNING WARNING WARNING WARNING

       You are installing an omnibus package without a version pin.  If you are installing
       on production servers via an automated process this is DANGEROUS and you will
       be upgraded without warning on new releases, even to new major releases.
       Letting the version float is only appropriate in desktop, test, development or
       CI/CD environments.

       WARNING WARNING WARNING WARNING WARNING WARNING WARNING WARNING WARNING

       Installing Chef 
       installing with rpm...
       warning: /tmp/install.sh.2395/chef-12.4.1-1.el6.x86_64.rpm: Header V4 DSA/SHA1 Signature, key ID 83ef826a: NOKEY
       Preparing...                ########################################### [100%]
          1:chef                   ########################################### [100%]
       Thank you for installing Chef!
       Transferring files to <simple-centos-66>
       Starting Chef Client, version 12.4.1
       Creating a new client identity for simple-centos-66 using the validator key.
       [2015-08-03T22:43:51+00:00] WARN: Child with name 'dna.json' found in multiple directories: /tmp/kitchen/dna.json and /tmp/kitchen/dna.json
       [2015-08-03T22:43:51+00:00] WARN: Child with name 'dna.json' found in multiple directories: /tmp/kitchen/dna.json and /tmp/kitchen/dna.json
       resolving cookbooks for run list: ["fake::simple"]
       [2015-08-03T22:43:51+00:00] WARN: Child with name 'dna.json' found in multiple directories: /tmp/kitchen/dna.json and /tmp/kitchen/dna.json
       Synchronizing Cookbooks:
         - fake
         - rsync
       Compiling Cookbooks...
       Converging 4 resources
       Recipe: rsync::default

           - install version 3.0.6-12.el6 of package rsync
       Recipe: rsync::server


           - update content in file /etc/init.d/rsyncd from none to 1bfbf2
           --- /etc/init.d/rsyncd       2015-08-03 22:44:13.181073877 +0000
           +++ /tmp/chef-rendered-template20150803-2496-z4x1s2  2015-08-03 22:44:13.180073377 +0000
           @@ -1 +1,71 @@
           +#!/bin/sh
           +#
           +# Rsyncd init script
           +#
           +# chkconfig: 2345 98 02
           +# description: Remote file copy program daemon
           +#
           +# Source function library.
           +. /etc/init.d/functions
           +. /etc/sysconfig/network
           +
           +#
           +# Set default options.  Override the defaults and define
           +# custom options in /etc/sysconfig/rsyncd
           +#
           +OPTS=""
           +[ -f /etc/sysconfig/rsyncd ] && . /etc/sysconfig/rsyncd
           +
           +# Check that networking is configured.
           +[ ${NETWORKING} = "no" ] && exit 0
           +
           +function start {
           +  echo -n $"Starting rsyncd: "
           +  if [ $UID -ne 0 ]; then
           +    RETVAL=1
           +    failure
           +  else
           +    daemon /usr/bin/rsync --daemon $OPTS
           +    RETVAL=$?
           +    [ $RETVAL -eq 0 ] && touch /var/lock/subsys/rsyncd
           +  fi;
           +  echo
           +  return $RETVAL
           +}
           +
           +function stop {
           +  echo -n $"Shutting down rsyncd:"
           +  if [ $UID -ne 0 ]; then
           +    RETVAL=1
           +    failure
           +  else
           +    killproc /usr/bin/rsync
           +    RETVAL=$?
           +    [ $RETVAL -eq 0 ] && rm -f /var/lock/subsys/rsyncd
           +  fi;
           +  echo
           +  return $RETVAL
           +}
           +

           +start)
           +  start
           +;;
           +stop)
           +  stop
           +;;
           +restart)
           +  stop
           +  start
           +;;
           +status)
           +  status rsync
           +  exit $?
           +;;
           +*)
           +  echo "Usage: rsyncd {start|stop|restart|status}"
           +  exit 1
           +esac
           +

           - change mode from '' to '0755'
           - change owner from '' to 'root'

           - restore selinux security context
         * service[rsyncd] action enable (skipped due to only_if)
         * service[rsyncd] action start (skipped due to only_if)
       Recipe: fake::simple
         * rsync_serve[tmp] action add[2015-08-03T22:44:13+00:00] WARN: Cloning resource attributes for service[rsyncd] from prior resource (CHEF-3694)
       [2015-08-03T22:44:13+00:00] WARN: Previous service[rsyncd]: /tmp/kitchen/cache/cookbooks/rsync/recipes/server.rb:39:in `from_file'
       [2015-08-03T22:44:13+00:00] WARN: Current  service[rsyncd]: /tmp/kitchen/cache/cookbooks/rsync/providers/serve.rb:47:in `write_conf'
        (up to date)


           - update content in file /etc/rsyncd.conf from none to f57720
           --- /etc/rsyncd.conf 2015-08-03 22:44:13.350158375 +0000
           +++ /tmp/chef-rendered-template20150803-2496-zfv0dx  2015-08-03 22:44:13.350158375 +0000
           @@ -1 +1,6 @@
           +# Rsync Config file written by chef
           +#
           +
           +

           - change mode from '' to '0640'
           - change owner from '' to 'root'


         * service[rsyncd] action nothing (skipped due to action :nothing)

           - restart service service[rsyncd]

       Running handlers:
       Running handlers complete
       Chef Client finished, 4/5 resources updated in 24.783508016 seconds
       Finished converging <simple-centos-66> (1m6.30s).
-----> Setting up <simple-centos-66>...
       Finished setting up <simple-centos-66> (0m0.00s).
-----> Verifying <simple-centos-66>...
       Preparing files for transfer
-----> Installing Busser (busser)
Fetching: thor-0.19.0.gem (100%)
       Successfully installed thor-0.19.0
Fetching: busser-0.7.1.gem (100%)
       Successfully installed busser-0.7.1
       2 gems installed
-----> Setting up Busser
       Creating BUSSER_ROOT in /tmp/verifier
       Creating busser binstub
       Installing Busser plugins: busser-bats
       Plugin bats installed (version 0.3.0)
-----> Running postinstall for bats plugin
       Installed Bats to /tmp/verifier/vendor/bats/bin/bats
       Suite path directory /tmp/verifier/suites does not exist, skipping.
       Transferring files to <simple-centos-66>
-----> Running bats test suite
 ✓ rsync is running
 ✗ rsync is serving /tmp
          (in test file /tmp/verifier/suites/bats/verify_running.bats, line 6)
            `rsync rsync://127.0.0.1 | grep "tmp"' failed

       2 tests, 1 failure
       !!!!!! Command [/tmp/verifier/vendor/bats/bin/bats /tmp/verifier/suites/bats] exit code was 1
>>>>>> Verify failed on instance <simple-centos-66>.
>>>>>> Please see .kitchen/logs/simple-centos-66.log for more details
>>>>>> ------Exception-------
>>>>>> Class: Kitchen::ActionFailed
>>>>>> Message: SSH exited (1) for command: [sh -c '

BUSSER_ROOT="/tmp/verifier"; export BUSSER_ROOT
GEM_HOME="/tmp/verifier/gems"; export GEM_HOME
GEM_PATH="/tmp/verifier/gems"; export GEM_PATH
GEM_CACHE="/tmp/verifier/gems/cache"; export GEM_CACHE

sudo -E /tmp/verifier/bin/busser test
']
>>>>>> ----------------------

LWRP should use use_inline_resources

LWRP should use use_inline_resources

LWRP needs to support why-run mode

LWRP needs to support why-run mode

rsyncd init script not needed on RHEL 7

You folks provide an rsyncd init script for RHEL, however, I think (though I am not 100% sure) that that is only needed for RHEL < 7, RHEL7 rsync has systemd unit files for rsync:

rpm -ql rsync | grep systemd /usr/lib/systemd/system/rsyncd.service /usr/lib/systemd/system/rsyncd.socket /usr/lib/systemd/system/[email protected]

So if we can set a check in there for RHEL < 7, then we don't end up with multiple init scripts/systemd units.

failure with Chef 12.0.0

Running kitchen test fails with:

[2014-11-25T19:51:11+00:00] ERROR: rsync_serve[tmp] (fake::simple line 3) had an error: NoMethodError: undefined method `global_modules' for Chef::Resource::Template
       [2014-11-25T19:51:11+00:00] FATAL: Chef::Exceptions::ChildConvergeError: Chef run process exited unsuccessfully (exit code 1)
>>>>>> Converge failed on instance <simple-ubuntu-1204>

(The failure is not specific to Ubuntu. It also fails on CentOS.)

After trying out different Chef versions by changing require_chef_omnibus in .kitchen.yml, it appears that this was broken between Chef 11.16.4 and 12.0.0.

Making global_modules and rsync_modules public instead of protected appears to fix it, but I don't know whether that is a proper fix.

Dependency Dashboard

This issue lists Renovate updates and detected dependencies. Read the Dependency Dashboard docs to learn more.

This repository currently has no open or pending branches.

Detected dependencies

github-actions

.github/workflows/ci.yml

• sous-chefs/.github 3.1.1
• actions/checkout v4
• actionshub/chef-install 3.0.0
• actionshub/test-kitchen 3.0.0

.github/workflows/stale.yml

• actions/stale v9

---

• Check this box to trigger a request for Renovate to run again on this repository