rodjek / rspec-puppet

RSpec tests for your Puppet manifests

RSpec tests for your Puppet manifests & modules

---

Note that as of release 2.10.0, this project is being maintained in the main puppetlabs namespace.

---

Table of Contents

• Installation
• Starting out with a new module
• Configure manifests for Puppet 4
• Configuration
• Naming conventions
• Example groups
• Defined Types, Classes & Applications
• Functions
• Hiera integration
• Producing coverage reports
• Related projects

Installation

gem install rspec-puppet

Note for ruby 1.8 users: while rspec-puppet itself supports ruby 1.8, you'll need to pin rspec itself to ~> 3.1.0, as later rspec versions do not work on old rubies anymore.

Starting out with a new module

When you start out on a new module, create a metadata.json file for your module and then run rspec-puppet-init to create the necessary files to configure rspec-puppet for your module's tests.

Configure manifests for Puppet 4

With Puppet 3, the manifest is set to $manifestdir/site.pp. However Puppet 4 defaults to an empty value. In order to test manifests you will need to set appropriate settings.

Puppet configuration reference for manifest can be found online:

• Puppet 3: https://docs.puppet.com/puppet/3.8/reference/configuration.html#manifest
• Puppet 4: https://docs.puppet.com/puppet/4.8/reference/configuration.html#manifest

Configuration is typically done in a spec/spec_helper.rb file which each of your spec will require. Example code:

# /spec
base_dir = File.dirname(File.expand_path(__FILE__))

RSpec.configure do |c|
  c.module_path     = File.join(base_dir, 'fixtures', 'modules')
  c.manifest_dir    = File.join(base_dir, 'fixtures', 'manifests')
  c.manifest        = File.join(base_dir, 'fixtures', 'manifests', 'site.pp')
  c.environmentpath = File.join(Dir.pwd, 'spec')

  # Coverage generation
  c.after(:suite) do
    RSpec::Puppet::Coverage.report!
  end
end

Configuration

rspec-puppet can be configured by modifying the RSpec.configure block in your spec/spec_helper.rb file.

RSpec.configure do |c|
  c.<config option> = <value>
end

manifest_dir

Type	Default	Puppet Version(s)
String	Required	2.x, 3.x

The path to the directory containing your basic manifests like site.pp.

module_path

Type	Default	Puppet Version(s)
String	Required	2.x, 3.x, 4.x, 5.x

The path to the directory containing your Puppet modules.

default_facts

Type	Default	Puppet Version(s)
Hash	{}	2.x, 3.x, 4.x, 5.x

A hash of default facts that should be used for all the tests.

hiera_config

Type	Default	Puppet Version(s)
String	"/dev/null"	3.x, 4.x, 5.x

The path to your hiera.yaml file (if used).

default_node_params

Type	Default	Puppet Version(s)
Hash	{}	4.x, 5.x

A hash of default node parameters that should be used for all the tests.

default_trusted_facts

Type	Default	Puppet Version(s)
Hash	{}	4.x, 5.x

A hash of default trusted facts that should be used for all the tests (available in the manifests as the $trusted hash). In order to use this, the trusted_node_data setting must be set to true.

trusted_node_data

Type	Default	Puppet Version(s)
Boolean	false	>=3.4, 4.x, 5.x

Configures rspec-puppet to use the $trusted hash when compiling the catalogues.

trusted_server_facts

Type	Default	Puppet Version(s)
Boolean	false	>=4.3, 5.x

Configures rspec-puppet to use the $server_facts hash when compiling the catalogues.

confdir

Type	Default	Puppet Version(s)
String	"/etc/puppet"	2.x, 3.x, 4.x, 5.x

The path to the main Puppet configuration directory.

config

Type	Default	Puppet Version(s)
String	Puppet's default value	2.x, 3.x, 4.x, 5.x

The path to puppet.conf.

manifest

Type	Default	Puppet Version(s)
String	Puppet's default value	2.x, 3.x

The entry-point manifest for Puppet, usually $manifest_dir/site.pp.

template_dir

Type	Default	Puppet Version(s)
String	nil	2.x, 3.x

The path to the directory that Puppet should search for templates that are stored outside of modules.

environmentpath

Type	Default	Puppet Version(s)
String	"/etc/puppetlabs/code/environments"	4.x, 5.x

The search path for environment directories.

parser

Type	Default	Puppet Version(s)
String	"current"	>= 3.2

This switches between the 3.x (current) and 4.x (future) parsers.

ordering

Type	Default	Puppet Version(s)
String	"title-hash"	>= 3.3, 4.x, 5.x

How unrelated resources should be ordered when applying a catalogue.

• manifest - Use the order in which the resources are declared in the manifest.
• title-hash - Order the resources randomly, but in a consistent manner across runs (the order will only change if the manifest changes).
• random - Order the resources randomly.

strict_variables

Type	Default	Puppet Version(s)
Boolean	false	>= 3.5, 4.x, 5.x

Makes Puppet raise an error when it tries to reference a variable that hasn't been defined (not including variables that have been explicitly set to undef).

stringify_facts

Type	Default	Puppet Version(s)
Boolean	true	>= 3.3, 4.x, 5.x

Makes rspec-puppet coerce all the fact values into strings (matching the behaviour of older versions of Puppet).

enable_pathname_stubbing

Type	Default	Puppet Version(s)
Boolean	false	2.x, 3.x, 4.x, 5.x

Configures rspec-puppet to stub out Pathname#absolute? with it's own implementation. This should only be enabled if you're running into an issue running cross-platform tests where you have Ruby code (types, providers, functions, etc) that use Pathname#absolute?.

setup_fixtures

Type	Default	Puppet Version(s)
Boolean	true	2.x, 3.x, 4.x, 5.x

Configures rspec-puppet to automatically create a link from the root of your module to spec/fixtures/<module name> at the beginning of the test run.

derive_node_facts_from_nodename

Type	Default	Puppet Version(s)
Boolean	true	2.x, 3.x, 4.x, 5.x

If true, rspec-puppet will override the fdqn, hostname, and domain facts with values that it derives from the node name (specified with let(:node).

In some circumstances (e.g. where your nodename/certname is not the same as your FQDN), this behaviour is undesirable and can be disabled by changing this setting to false.

Naming conventions

For clarity and consistency, I recommend that you use the following directory structure and naming convention.

module/
  ├── manifests/
  ├── lib/
  └── spec/
       ├── spec_helper.rb
       │
       ├── classes/
       │     └── <class_name>_spec.rb
       │
       ├── defines/
       │     └── <define_name>_spec.rb
       │
       ├── applications/
       │     └── <application_name>_spec.rb
       │
       ├── functions/
       │     └── <function_name>_spec.rb
       │
       ├── types/
       │     └── <type_name>_spec.rb
       │
       ├── type_aliases/
       │     └── <type_alias_name>_spec.rb
       │
       └── hosts/
             └── <host_name>_spec.rb

Example groups

If you use the above directory structure, your examples will automatically be placed in the correct groups and have access to the custom matchers. If you choose not to, you can force the examples into the required groups as follows.

describe 'myclass', :type => :class do
  ...
end

describe 'mydefine', :type => :define do
  ...
end

describe 'myapplication', :type => :application do
  ...
end

describe 'myfunction', :type => :puppet_function do
  ...
end

describe 'mytype', :type => :type do
  ...
end

describe 'My::TypeAlias', :type => :type_alias do
  ...
end

describe 'myhost.example.com', :type => :host do
  ...
end

Defined Types, Classes & Applications

Matchers

Checking if the catalog compiles

You can test whether the subject catalog compiles cleanly with compile.

it { is_expected.to compile }

To check the error messages of your class, you can check for raised error messages.

it { is_expected.to compile.and_raise_error(/error message match/) }

Checking if a resource exists

You can test if a resource exists in the catalogue with the generic contain_<resource type> matcher.

it { is_expected.to contain_augeas('bleh') }

You can also test if a class has been included in the catalogue with the same matcher.

it { is_expected.to contain_class('foo') }

Note that rspec-puppet does none of the class name parsing and lookup that the puppet parser would do for you. The matcher only accepts fully qualified classnames without any leading colons. That is a class foo::bar will only be matched by foo::bar, but not by ::foo::bar, or bar alone.

If your resource type includes :: (e.g. foo::bar simply replace the :: with __ (two underscores).

it { is_expected.to contain_foo__bar('baz') }

You can further test the parameters that have been passed to the resources with the generic with_<parameter> chains.

it { is_expected.to contain_package('mysql-server').with_ensure('present') }

If you want to specify that the given parameters should be the only ones passed to the resource, use the only_with_<parameter> chains.

it { is_expected.to contain_package('httpd').only_with_ensure('latest') }

You can use the with method to verify the value of multiple parameters.

it do
  is_expected.to contain_service('keystone').with(
    'ensure'     => 'running',
    'enable'     => 'true',
    'hasstatus'  => 'true',
    'hasrestart' => 'true'
  )
end

The same holds for the only_with method, which in addition verifies the exact set of parameters and values for the resource in the catalogue.

it do
  is_expected.to contain_user('luke').only_with(
    'ensure' => 'present',
    'uid'    => '501'
  )
end

You can also test that specific parameters have been left undefined with the generic without_<parameter> chains.

it { is_expected.to contain_file('/foo/bar').without_mode }

You can use the without method to verify that a list of parameters have not been defined

it { is_expected.to contain_service('keystone').without(
  ['restart', 'status']
)}

Checking the number of resources

You can test the number of resources in the catalogue with the have_resource_count matcher.

it { is_expected.to have_resource_count(2) }

The number of classes in the catalogue can be checked with the have_class_count matcher.

it { is_expected.to have_class_count(2) }

You can also test the number of a specific resource type, by using the generic have_<resource type>_resource_count matcher.

it { is_expected.to have_exec_resource_count(1) }

This last matcher also works for defined types. If the resource type contains ::, you can replace it with __ (two underscores).

it { is_expected.to have_logrotate__rule_resource_count(3) }

NOTE: when testing a class, the catalogue generated will always contain at least one class, the class under test. The same holds for defined types, the catalogue generated when testing a defined type will have at least one resource (the defined type itself).

Relationship matchers

The following methods will allow you to test the relationships between the resources in your catalogue, regardless of how the relationship is defined. This means that it doesn’t matter if you prefer to define your relationships with the metaparameters (require, before, notify and subscribe) or the chaining arrows (->, ~>, <- and <~), they’re all tested the same.

it { is_expected.to contain_file('foo').that_requires('File[bar]') }
it { is_expected.to contain_file('foo').that_comes_before('File[bar]') }
it { is_expected.to contain_file('foo').that_notifies('File[bar]') }
it { is_expected.to contain_file('foo').that_subscribes_to('File[bar]') }

An array can be used to test a resource for multiple relationships

it { is_expected.to contain_file('foo').that_requires(['File[bar]', 'File[baz]']) }
it { is_expected.to contain_file('foo').that_comes_before(['File[bar]','File[baz]']) }
it { is_expected.to contain_file('foo').that_notifies(['File[bar]', 'File[baz]']) }
it { is_expected.to contain_file('foo').that_subscribes_to(['File[bar]', 'File[baz]']) }

You can also test the reverse direction of the relationship, so if you have the following bit of Puppet code

notify { 'foo': }
notify { 'bar':
  before => Notify['foo'],
}

You can test that Notify[bar] comes before Notify[foo]

it { is_expected.to contain_notify('bar').that_comes_before('Notify[foo]') }

Or, you can test that Notify[foo] requires Notify[bar]

it { is_expected.to contain_notify('foo').that_requires('Notify[bar]') }

Match target syntax

Note that this notation does not support any of the features you're used from the puppet language. Only a single resource with a single, unquoted title can be referenced here. Class names need to be always fully qualified and not have the leading ::. It currently does not support inline arrays or quoting.

These work

• Notify[foo]
• Class[profile::apache]

These will not work

• Notify['foo']
• Notify[foo, bar]
• Class[::profile::apache]

Recursive dependencies

The relationship matchers are recursive in two directions:

• vertical recursion, which checks for dependencies with parents of the resource (i.e. the resource is contained, directly or not, in the class involved in the relationship). E.g. where Package['foo'] comes before File['/foo']:

class { 'foo::install': } ->
class { 'foo::config': }

class foo::install {
  package { 'foo': }
}

class foo::config {
  file { '/foo': }
}

• horizontal recursion, which follows indirect dependencies (dependencies of dependencies). E.g. where Yumrepo['foo'] comes before File['/foo']:

class { 'foo::repo': } ->
class { 'foo::install': } ->
class { 'foo::config': }

class foo::repo {
  yumrepo { 'foo': }
}

class foo::install {
  package { 'foo': }
}

class foo::config {
  file { '/foo': }
}

Autorequires

Autorequires are considered in dependency checks.

Type matcher

When testing custom types, the be_valid_type matcher provides a range of expectations:

• with_provider(<provider_name>): check that the right provider was selected
• with_properties(<property_list>): check that the specified properties are available
• with_parameters(<parameter_list>): check that the specified parameters are available
• with_features(<feature_list>): check that the specified features are available
• with_set_attributes(<param_value_hash>): check that the specified attributes are set

Type alias matchers

When testing type aliases, the allow_value and allow_values matchers are used to check if the alias accepts particular values or not:

describe 'MyModule::Shape' do
  it { is_expected.to allow_value('square') }
  it { is_expected.to allow_values('circle', 'triangle') }
  it { is_expected.not_to allow_value('blue') }
end

Writing tests

Basic test structure

To test that

sysctl { 'baz'
  value => 'foo',
}

Will cause the following resource to be in included in catalogue for a host

exec { 'sysctl/reload':
  command => '/sbin/sysctl -p /etc/sysctl.conf',
}

We can write the following testcase (in spec/defines/sysctl_spec.rb)

describe 'sysctl' do
  let(:title) { 'baz' }
  let(:params) { { 'value' => 'foo' } }

  it { is_expected.to contain_exec('sysctl/reload').with_command("/sbin/sysctl -p /etc/sysctl.conf") }
end

Specifying the title of a resource

let(:title) { 'foo' }

Specifying the parameters to pass to a resources or parameterised class

Parameters of a defined type, class or application can be passed defining :params in a let, and passing it a hash as seen below.

let(:params) { {'ensure' => 'present', ...} }

For passing Puppet's undef as a paremeter value, you can simply use :undef and it will be translated to undef when compiling. For example:

let(:params) { {'user' => :undef, ...} }

For references to nodes or resources as seen when using require or before properties, or an application resource you can pass the string as an argument to the ref helper:

let(:params) { 'require' => ref('Package', 'sudoku') }

Which translates to:

mydefine { 'mytitle': require => Package['sudoku'] }

Another example, for an application setup (when using app_management):

let(:params) { { 'nodes' => { ref('Node', 'dbnode') => ref('Myapp::Mycomponent', 'myapp') } } }

Will translate to:

site { myapp { 'myimpl': nodes => { Node['dbnode'] => Myapp::Mycomponent['myimpl'] } } }

Specifying the FQDN of the test node

If the manifest you're testing expects to run on host with a particular name, you can specify this as follows

let(:node) { 'testhost.example.com' }

Specifying the environment name

If the manifest you're testing expects to evaluate the environment name, you can specify this as follows

let(:environment) { 'production' }

Specifying the facts that should be available to your manifest

By default, the test environment contains no facts for your manifest to use. You can set them with a hash

let(:facts) { {'operatingsystem' => 'Debian', 'kernel' => 'Linux', ...} }

Facts may be expressed as a value (shown in the previous example) or a structure. Fact keys may be expressed as either symbols or strings. A key will be converted to a lower case string to align with the Facter standard

let(:facts) { {'os' => { 'family' => 'RedHat', 'release' => { 'major' => '7', 'minor' => '1', 'full' => '7.1.1503' } } } }

You can also create a set of default facts provided to all specs in your spec_helper:

RSpec.configure do |c|
  c.default_facts = {
    'operatingsystem' => 'Ubuntu'
  }
end

Any facts you provide with let(:facts) in a spec will automatically be merged on top of the default facts.

Specifying top-scope variables that should be available to your manifest

You can create top-scope variables much in the same way as an ENC.

let(:node_params) { { 'hostgroup' => 'webservers', 'rack' => 'KK04', 'status' => 'maintenance' } }

You can also create a set of default top-scope variables provided to all specs in your spec_helper:

RSpec.configure do |c|
  c.default_node_params = {
    'owner'  => 'itprod',
    'site'   => 'ams4',
    'status' => 'live'
  }
end

NOTE Setting top-scope variables is not supported in Puppet < 3.0.

Specifying extra code to load (pre-conditions)

If the manifest being tested relies on another class or variables to be set, these can be added via a pre-condition. This code will be evaluated before the tested class.

let(:pre_condition) { 'include other_class' }

This may be useful when testing classes that are modular, e.g. testing apache::mod::foo which relies on a top-level apache class being included first.

The value may be a raw string to be inserted into the Puppet manifest, or an array of strings (manifest fragments) that will be concatenated.

Specifying extra code to load (post-conditions)

In some cases, you may need to ensure that the code that you are testing comes before another set of code. Similar to the :pre_condition hook, you can add a :post_condition hook that will ensure that the added code is evaluated after the tested class.

let(:post_condition) { 'include other_class' }

This may be useful when testing classes that are modular, e.g. testing class do_strange_things::to_the_catalog which must come before class foo.

The value may be a raw string to be inserted into the Puppet manifest, or an array of strings (manifest fragments) that will be concatenated.

Specifying the path to find your modules

I recommend setting a default module path by adding the following code to your spec_helper.rb

RSpec.configure do |c|
  c.module_path = '/path/to/your/module/dir'
end

However, if you want to specify it in each example, you can do so

let(:module_path) { '/path/to/your/module/dir' }

Specifying trusted facts

When testing with Puppet >= 4.3 the trusted facts hash will have the standard trusted fact keys (certname, domain, and hostname) populated based on the node name (as set with :node).

By default, the test environment contains no custom trusted facts (as usually obtained from certificate extensions) and found in the extensions key. If you need to test against specific custom certificate extensions you can set those with a hash. The hash will then be available in $trusted['extensions']

let(:trusted_facts) { {'pp_uuid' => 'ED803750-E3C7-44F5-BB08-41A04433FE2E', '1.3.6.1.4.1.34380.1.2.1' => 'ssl-termination'} }

You can also create a set of default certificate extensions provided to all specs in your spec_helper:

RSpec.configure do |c|
  c.default_trusted_facts = {
    'pp_uuid'                 => 'ED803750-E3C7-44F5-BB08-41A04433FE2E',
    '1.3.6.1.4.1.34380.1.2.1' => 'ssl-termination'
  }
end

Specifying trusted external data

When testing with Puppet >= 6.14, the trusted facts hash will have an additional external key for trusted external data.

By default, the test environment contains no trusted external data (as usually obtained from trusted external commands and found in the external key). If you need to test against specific trusted external data you can set those with a hash. The hash will then be available in $trusted['external']

let(:trusted_external_data) { {'foo' => 'bar'} }

You can also create a set of default trusted external data provided to all specs in your spec_helper:

RSpec.configure do |c|
  c.default_trusted_external_data = {
    'foo' => 'bar'
  }
end

Testing Exported Resources

You can test if a resource was exported from the catalogue by using the exported_resources accessor in combination with any of the standard matchers.

You can use exported_resources as the subject of a child context:

context 'exported resources' do
  subject { exported_resources }

  it { is_expected.to contain_file('foo') }
end

You can also use exported_resources directly in a test:

it { expect(exported_resources).to contain_file('foo') }

Testing applications

Applications in some ways behave as defined resources, but are more complex so require a number of elements already documented above to be combined for testing.

A full example of the simplest rspec test for a single component application:

require 'spec_helper'

describe 'orch_app' do
  let(:node) { 'my_node' }
  let(:title) { 'my_awesome_app' }
  let(:params) do
    {
      'nodes' => {
        ref('Node', node) => ref('Orch_app::Db', title),
      }
    }
  end

  it { should compile }
  it { should contain_orch_app(title) }
end

Each piece is required:

• You must turn on app_management during testing for the handling to work
• The :node definition is required to be set so later on you can reference it in the :nodes argument within :params
• Applications act like defined resources, and each require a :title to be defined
• The :nodes key in :params requires the use of node reference mappings to resource mappings. The ref keyword allows you to provide these (a normal string will not work).

Beyond these requirements, the very basic should compile test and other matchers as you would expect will work the same as classes and defined resources.

Note: for the moment, cross-node support is not available and will return an error. Ensure you model your tests to be single-node for the time being.

Functions

Matchers

All of the standard RSpec matchers are available for you to use when testing Puppet functions.

it 'should be able to do something' do
  subject.execute('foo') == 'bar'
end

For your convenience though, a run matcher exists to provide easier to understand test cases.

it { is_expected.to run.with_params('foo').and_return('bar') }

Writing tests

Basic test structure

require 'spec_helper'

describe '<function name>' do
  ...
end

Specifying the name of the function to test

The name of the function must be provided in the top level description, e.g.

describe 'split' do

Specifying the arguments to pass to the function

You can specify the arguments to pass to your function during the test(s) using either the with_params chain method in the run matcher

it { is_expected.to run.with_params('foo', 'bar', ['baz']) }

Or by using the execute method on the subject directly

it 'something' do
  subject.execute('foo', 'bar', ['baz'])
end

Passing lambdas to the function

A lambda (block) can be passed to functions that support either a required or optional lambda by passing a block to the with_lambda chain method in the run matcher.

it { is_expected.to run.with_lambda { |x| x * 2 }

Testing the results of the function

You can test the result of a function (if it produces one) using either the and_returns chain method in the run matcher

it { is_expected.to run.with_params('foo').and_return('bar') }

Or by using any of the existing RSpec matchers on the subject directly

it 'something' do
  subject.execute('foo') == 'bar'
  subject.execute('baz').should be_an Array
end

Testing the errors thrown by the function

You can test whether the function throws an exception using either the and_raises_error chain method in the run matcher

it { is_expected.to run.with_params('a', 'b').and_raise_error(Puppet::ParseError) }
it { is_expected.not_to run.with_params('a').and_raise_error(Puppet::ParseError) }

Or by using the existing raises_error RSpec matcher

it 'something' do
  expect { subject.execute('a', 'b') }.should raise_error(Puppet::ParseError)
  expect { subject.execute('a') }.should_not raise_error(Puppet::ParseError)
end

Accessing the parser scope where the function is running

Some complex functions require access to the current parser's scope, e.g. for stubbing other parts of the system.

before(:each) { scope.expects(:lookupvar).with('some_variable').returns('some_value') }
it { is_expected.to run.with_params('...').and_return('...') }

Note that this does not work when testing manifests which use custom functions. Instead, you'll need to create a replacement function directly.

before(:each) do
    Puppet::Parser::Functions.newfunction(:custom_function, :type => :rvalue) { |args|
        raise ArgumentError, 'expected foobar' unless args[0] == 'foobar'
        'expected value'
    }
end

Hiera integration

Configuration

Set the hiera config symbol properly in your spec files:

let(:hiera_config) { 'spec/fixtures/hiera/hiera.yaml' }
hiera = Hiera.new(:config => 'spec/fixtures/hiera/hiera.yaml')

Create your spec hiera files

spec/fixtures/hiera/hiera.yaml

---
:backends:
  - yaml
:hierarchy:
  - test
:yaml:
  :datadir: 'spec/fixtures/hiera'

spec/fixtures/hiera/test.yaml

---
ntpserver: ['ntp1.domain.com','ntpXX.domain.com']
user:
  oneuser:
    shell: '/bin/bash'
  twouser:
    shell: '/sbin/nologin'

Use hiera in your tests

  ntpserver = hiera.lookup('ntpserver', nil, nil)
  let(:params) { 'ntpserver' => ntpserver }

Enabling hiera lookups

If you just want to fetch values from hiera (e.g. because you're testing code that uses explicit hiera lookups) just specify the path to the hiera config in your spec_helper.rb

RSpec.configure do |c|
  c.hiera_config = 'spec/fixtures/hiera/hiera.yaml'
end

spec/fixtures/hiera/hiera.yaml

---
:backends:
  - yaml
:yaml:
  :datadir: spec/fixtures/hieradata
:hierarchy:
  - common

Please note: In-module hiera data depends on having a correct metadata.json file. It is strongly recommended that you use metadata-json-lint to automatically check your metadata.json file before running rspec.

Producing coverage reports

You can output a basic resource coverage report with the following in your spec_helper.rb

RSpec.configure do |c|
  c.after(:suite) do
    RSpec::Puppet::Coverage.report!
  end
end

This checks which Puppet resources have been explicitly checked as part of the current test run and outputs both a coverage percentage and a list of untouched resources.

A desired code coverage level can be provided. If this level is not achieved, a test failure will be raised. This can be used with a CI service, such as Jenkins or Bamboo, to enforce code coverage. The following example requires the code coverage to be at least 95%.

RSpec.configure do |c|
  c.after(:suite) do
    RSpec::Puppet::Coverage.report!(95)
  end
end

Resources declared outside of the module being tested (i.e. forge dependencies) are automatically removed from the coverage report. There is one exception for this though: prior to Puppet 4.6.0, resources created by functions (create_resources(), ensure_package(), etc) did not have the required information in them to determine which manifest they came from and so can not be excluded from the coverage report.

Related projects

• puppetlabs_spec_helper: shared spec helpers to setup puppet
• rspec-puppet-augeas: RSpec tests for Augeas resources inside Puppet manifests
• jimdo-rspec-puppet-helpers: Tests the contents of a file with a source
• Ease development of specs
  • puppet-catalog_rspec: Dump the Puppet Catalog as RSpec code at compile time
  • create_specs: A different implementation that takes a compiled catalog and writes out RSpec code with various options
• Fact providers
  • rspec-puppet-facts: Simplify your unit tests by looping on every supported Operating System and populating facts.
  • rspec-puppet-osmash: Provides Operation System hashes and validations for rspec-puppet
  • puppet_spec_facts: Gem to provide puppet fact hashes for rspec-puppet testing

For a list of other module development tools see https://puppet.community/plugins/

rspec-puppet's Issues

Compare to strings containing newline is not possible

Hi,

check for strings containing a tailing newline is not posible.

spec-content:

it { should contain_concat__fragment( "iptables_rule_iptable1" ).with(
      'target'  => '/etc/iptables/rules.v4',
      'content' => '-A INPUT -p tcp --dport 1234 -s 0/0 -d 0/0 -j ACCEPT\n',
    ) }

testresult:

     Failure/Error: ) }
       expected that the catalogue would contain Concat::Fragment[iptables_rule_iptable1] with content set to `"-A INPUT -p tcp --dport 1234 -s 0/0 -d 0/0 -j ACCEPT\\n"` but it is set to `"-A INPUT -p tcp --dport 1234 -s 0/0 -d 0/0 -j ACCEPT\n"` in the catalogue

You will find the whole example at https://github.com/jerger/puppet-iptables/blob/master/spec/defines/iptables_rule_spec.rb.

Best regards,
jerger

You can find the whole example at https://github.com/jerger/puppet-iptables.

Problems with testing exec commands which have whitespaces in their title

Hi,

I'm super happy with this tool but found an error for testing the exec title. Here is the example that produces the errors

The class:

class git::install {
  package { 'git-core':
  ensure => installed
}

exec { 'git-clone a':}

}

The spec for the class:

require 'spec_helper'

describe 'git::install' do

context 'clone the brokenlifts-vm' do
  it { should contain_exec('git-clone a')}
end

end

Running rake spec produces an error

Failure/Error: it { should contain_exec('git-clone ')}          
  expected that the catalogue would contain Exec[git-clone ]

If I leave the whitespaces in the title description of the resource away:
The class:

class git::install {
  package { 'git-core':
  ensure => installed
}

exec { 'git-clone-a':}

}

The spec for the class:

require 'spec_helper'

describe 'git::install' do

context 'clone the brokenlifts-vm' do
  it { should contain_exec('git-clone-a')}
end

end

It is working. If I can help your with this issue in the code base, I'm glad to help.

using other modules fires Invalid resource type

Trying to use the VCSRepo module in a module im building, but backfires!

  1) git::deploy
     Failure/Error: it { should create_class('git::install') }
     Puppet::Error:
       Puppet::Parser::AST::Resource failed with error ArgumentError: Invalid resource type vcsrepo at /etc/puppet/modules/git/spec/fixtures/modules/git/manifests/deploy.pp:28 on node testagent.fritz.box
     # ./spec/classes/deploy_spec.rb:4

Using symlinks doesn't work on Windows and breaks puppet doc and geppetto

Windows doesn't support symlinks so any user trying to use rspec-puppet on Windows is simply out of luck. Also since Java doesn't support symlinks either (mainly due to Windows lack of support for them) once you allow rspec-puppet-init to create the symlinks from the fixtures directory to the top of the modules directory any user trying to work with Geppetto as their editor will be broken as well. Especially if they keep their modules under source control which everyone should be doing. Its no longer possible to use Geppeto for any git operations on my modules and its sees the symlinked directories as containing a separate copy of all of my manifests.

Also after using rspec-puppet-init to generate the layout in my module I'm no longer able to use puppet doc to generate rdoc documentation on any of my modules because it sees the classes defined multiple times.

$ puppet doc --outputdir /tmp/puppet/rdoc --mode rdoc --manifestdir ~/git/puppet/manifests --modulepath ~/git/puppet/modules

/usr/lib/ruby/1.8/rdoc/dot/dot.rb:28: warning: already initialized constant NODE_OPTS
/usr/lib/ruby/1.8/rdoc/dot/dot.rb:46: warning: already initialized constant EDGE_OPTS
/usr/lib/ruby/1.8/rdoc/dot/dot.rb:76: warning: already initialized constant GRAPH_OPTS
Could not generate documentation: Definition 'haproxy::proxy_farm' is already defined at /home/example/git/puppet/modules/haproxy/manifests/proxy_farm.pp:32; cannot be redefined at /home/example/git/puppet/modules/haproxy/spec/fixtures/modules/haproxy/manifests/proxy_farm.pp:32

I need to know what the best way to work around this bug is and another solution to the modulepath issue clearly needs to be found. Why isn't it acceptable to include the parrent directory of the module in the module path?

Implementation-agnostic checking of dependencies

Currently, to check dependencies, you have to tie yourself to a particular implementation. For example, in my current project, I have:

it { should contain_file('/etc/init/puppetmaster.conf').with_require('Package[puppet]') }

It would be nice to have a way to specify that Service[foo] should depend on Package[foo], and have it work for any of the following implementations:

service {'foo':
 # etc...
 require => Package[foo]
}

or

package {'foo':
 before => Service[foo]
}

or

Package[foo] -> Service[foo]

Using rspec-puppet to test template files

How would I use rspec-puppet to test that a template file is rendered properly?

tests fail with rspec=2.11.0

ilya@frohlocken:~/src/rspec-puppet (master)$ bundle exec rspec -b
........F..F...F.......

Failures:

  1) split something
     Failure/Error: expect { subject.call('foo') }.should raise_error(Puppet::ParseError)
       expected Puppet::ParseError, got #<NoMethodError: undefined method `call' for #<RSpec::Expectations::ExpectationTarget:0x7fe6b9007f10>>
     # ./vendor/gems/rspec-expectations-2.11.2/lib/rspec/expectations/fail_with.rb:33:in `fail_with'
     # ./vendor/gems/rspec-expectations-2.11.2/lib/rspec/expectations/handler.rb:19:in `handle_matcher'
     # ./vendor/gems/rspec-expectations-2.11.2/lib/rspec/expectations/syntax.rb:53:in `should'
     # ./spec/functions/split_spec.rb:10
     # ./vendor/gems/rspec-core-2.11.1/lib/rspec/core/example.rb:113:in `instance_eval'
     # ./vendor/gems/rspec-core-2.11.1/lib/rspec/core/example.rb:113:in `run'
     # ./vendor/gems/rspec-core-2.11.1/lib/rspec/core/example.rb:253:in `with_around_each_hooks'
     # ./vendor/gems/rspec-core-2.11.1/lib/rspec/core/example.rb:110:in `run'
     # ./vendor/gems/rspec-core-2.11.1/lib/rspec/core/example_group.rb:378:in `run_examples'
     # ./vendor/gems/rspec-core-2.11.1/lib/rspec/core/example_group.rb:374:in `map'
     # ./vendor/gems/rspec-core-2.11.1/lib/rspec/core/example_group.rb:374:in `run_examples'
     # ./vendor/gems/rspec-core-2.11.1/lib/rspec/core/example_group.rb:360:in `run'
     # ./vendor/gems/rspec-core-2.11.1/lib/rspec/core/command_line.rb:28:in `run'
     # ./vendor/gems/rspec-core-2.11.1/lib/rspec/core/command_line.rb:28:in `map'
     # ./vendor/gems/rspec-core-2.11.1/lib/rspec/core/command_line.rb:28:in `run'
     # ./vendor/gems/rspec-core-2.11.1/lib/rspec/core/reporter.rb:34:in `report'
     # ./vendor/gems/rspec-core-2.11.1/lib/rspec/core/command_line.rb:25:in `run'
     # ./vendor/gems/rspec-core-2.11.1/lib/rspec/core/runner.rb:69:in `run'
     # ./vendor/gems/rspec-core-2.11.1/lib/rspec/core/runner.rb:8:in `autorun'
     # ./vendor/bin/rspec:19
[...]

How to specify two paths for c_module_path

Most of the modules that we have created live in /opt/puppet/modules while the class I want to test using rspec-puppet is in /opt/puppet/services/mymodule

How do I indicate to c.module_path that it should look in one place for the "centralized" modules and in another place for the class I am testing?

Improved default Rakefile

Maybe the following could serve as a improved / advanced default Rakefile

require 'rake'
require 'rspec/core/rake_task'

desc "Run all RSpec code examples"
RSpec::Core::RakeTask.new(:rspec) do |t| 
  t.rspec_opts = File.read("spec/spec.opts").chomp || ""
end

SPEC_SUITES = (Dir.entries('spec') - ['.', '..','fixtures']).select {|e| File.directory? "spec/#{e}" }
namespace :rspec do
  SPEC_SUITES.each do |suite|
    desc "Run #{suite} RSpec code examples"
    RSpec::Core::RakeTask.new(suite) do |t| 
      t.pattern = "spec/#{suite}/**/*_spec.rb"
      t.rspec_opts = File.read("spec/spec.opts").chomp || ""
    end 
  end 
end
task :default => :rspec

## So you have puppet-lint? Allow me to use it for you.
begin
  if Gem::Specification::find_by_name('puppet-lint')
    require 'puppet-lint/tasks/puppet-lint'
    PuppetLint.configuration.ignore_paths = ["spec/**/*.pp", "vendor/**/*.pp"]
    PuppetLint.configuration.log_format = '%{path}:%{linenumber}:%{KIND}: %{message}'
    task :default => [:rspec, :lint]
  end 
rescue Gem::LoadError
end

Obviously the puppet-lint stuff is optional ;)

Note: This depends on rspec >2 (for the t.pattern use)

Support for Puppet 3.0

When running tests with rspec-puppet against Puppet 3.0 the run fails with the message:

Error converting value for param 'statedir': Could not find value for $vardir on node example.local

For example, I ran the test powerdns__package_spec.rb and got the following output:

1) powerdns::package 
   Failure/Error: it { should contain_package('pdns-server').with_ensure('present') }
   Puppet::Error:
     Error converting value for param 'statedir': Could not find value for $vardir on node example.local
   # ./spec/classes/powerdns__package_spec.rb:5:in `block (2 levels) in <top (required)>'

This may be related fixes: puppet issue 13429, mcollective-plugin PR 38.

Thanks!

Puppet 3.1.0 support

let(:node) not working

Hi There,

In the documentation, it says that I can specify a node name with:

let(:node) { 'host.example.com' }

However, the tests are failing unless the hostname of the machine I'm testing on, is defined in the puppet manifests/nodes.pp (which is included in site.pp), with the following error:

"Could not find default node or by name with 'test-host.example.com,test-host.example, test-host on node test-host.example.com".

As soon as I add "test-host.example.com" as a node in my puppet's nodes.pp config, the tests pass, which makes me think rspec-puppet is ignoring the let(:node) definition in my spec. Ideas?

Accessing variables and how rspec-puppet compiles

When building Puppet modules I've come to split off parts into separate classes and chaining the whole thing together in init.pp. Similarly I've split up tests per 'subclass' into their own subclass_spec.rb test.

But, when you do that and in subclass.pp you use a variable declared in init.pp by accessing it like so $module::varname or $::mdoule::varname that variable does not exist for subclass_spec.rb.

It looks like when tests are run through separate files it only compiles that one pp-file to a catalog and therefor the variable we're trying to access at the mdoule-level is not accessible.

I've put up a gist which should show what I mean: https://gist.github.com/daenney/ff2de112c0190d9103bf

This could be solved by inheriting from params everywhere but seems like a terrible solution, I also don't really like the idea of having one spec-file for a complete module.

So, how do we go about doing this?

Add parameters validation to matcher: include_class

Like:

it { should include_class('some::class').with('p1' => 'v1') }

need a mechanism for testing that a class is called with no params

This catches the case where the class should be included with no parameters. I couldn't see an existing way to do this today.

See:

https://github.com/puppetlabs/puppetlabs-opennebula/pull/2/files#L9R46

Support Hiera

Since Hiera is a part of Puppet 3.0, it would be nice to include support for this natively. There is this gem: https://github.com/amfranz/rspec-hiera-puppet but it appears to not work properly with the latest release of Puppet.

This is a feature that I think should be first class in rspec-puppet.

Add support for testing if resource parameters are undefined

it { should contain_file('foo').without_source }

Something like that maybe?

Handling of Relationships inconsistent

Hi.

While trying to write tests for a module I stumbled across an inconsistency with the handling of relationships (->, <-, before, require, >, <, notify, subscribe).

Suppose the only manifest in the module is:

class relationships {
  file { '/tmp/a': 
#   require => File['/tmp'],
  }
  File['/tmp'] -> File['/tmp/a'],
}

Where a usage requirement for the module is said to have some definition for File['/tmp'] somewhere else. (A more likely thing would be to have an exec or package defined elsewhere which you need to run after.)
When running rspec on the above class, puppet barks out because it doesn't see a definition for File['/tmp']. If I comment out the File->File relation, but instead un-comment the require, puppet doesn't report any error while testing the module.

I originally only saw that require/before/subscribe/notify didn't cause errors and created a matcher (see pull request "Add contain_all_deps matcher") do be able to explicitly test for those. I later wanted to make sure I also can test the arrow-notation relationships, and noticed that they actually cause puppet to immediately report an error.

Personally, I think the behavior seen for "require" and related is desirable, since it allows to ignore the dependency issues, but test for them explicitly where wanted. However, the difference in behavior needs some handling, I think. Especially with the arrow-notation relationships only reporting an error location in the rspec files, not the puppet manifests. In other words, you need to find the offending relationship arrow by hand.

Make it simpler to specify complex pre conditions

e.g. by allowing to put them into an array and joining them. Pull request forthcoming.

Need to be able to specify confdir with Puppet Enterprise

I am using the pe_accounts class from Puppet Enterprise which unfortunately hardcodes the location of user data files to the confdir/pe_accounts_users_hash.yaml. When running 'rake spec' this results in the following problem:

Puppet::Error: No such file or directory - /Users/bappelt/.puppet/data/pe_accounts_users_hash.yaml at ...

If I could specify the confdir in the spec_helper, I could easily work around this.

tags in classes in pre_condition don't work

I'm including a class in a pre_condition that tags the node with a tag("sometag"). The class I'm testing is checking for the tag using if tagged("sometag"), but it doesn't succeed:

let(:pre_condition) { 'include myclassthattags' }

if I just call tag myself directly in a pre_condition, it works fine:

let(:pre_condition) { 'tag("sometag") }

Allow multiple entries in modulepath

Hi,

while developing puppet modules with dependencie to other modules it would be nice to use a modulepath with mor than one directory-entry. For defining modulepath I use:

c.module_path = File.join(File.dirname(FILE), '../../')

Best regards,
jerger

should not require a site.pp

When I try to run my rspec tests on my laptop, I get the following error:

Puppet::Error:
Could not parse for environment production: No file(s) found for import of '/Users/danbode/.puppet/manifests/site.pp' at line 3 on node danbodesmac.hsd1.or.comcast.net.
# /Users/danbode/dev/puppet/lib/puppet/parser/compiler.rb:27:in compile' # /Users/danbode/dev/puppet/lib/puppet/indirector/catalog/compiler.rb:77:incompile'
# /Users/danbode/dev/puppet/lib/puppet/util.rb:185:in benchmark' # /Users/danbode/dev/puppet/lib/puppet/indirector/catalog/compiler.rb:75:incompile'
# /Users/danbode/dev/puppet/lib/puppet/indirector/catalog/compiler.rb:35:in find' # /Users/danbode/dev/puppet/lib/puppet/indirector/indirection.rb:189:infind'
# /Users/danbode/dev/rspec-puppet/lib/rspec-puppet/support.rb:12:in build_catalog' # /Users/danbode/dev/rspec-puppet/lib/rspec-puppet/example/class_example_group.rb:51:incatalogue'
# /Users/danbode/dev/rspec-puppet/lib/rspec-puppet/example/class_example_group.rb:7:in `subject'
# ./ntp_spec.rb:8\

I can easily resolve this error by just creating an empty file

Facts are not working

Hi Tim,
documentation states that facts can be set for example:

let(:facts) { {:operatingsystem => 'Debian'} }

However, this is apparently not working and tests that rely on certain fact values are failing. Please take a look at https://github.com/vurbia/puppet-augeas. The tests in spec/classes/augeas_spec.rb are failing.

Cheers,
Atha

:pre_condition seems to fail with anything in spec/defines

Per the hacking on IRC, first we tried this:

describe 'haproxy::global' do
  let(:pre_condition) { '$haproxy::config_file = "some crap"' }

  # other examples down here
end

And that raised the following

  2) haproxy::global 
     Failure/Error: it { should include_class('concat::setup') }
     Puppet::Error:
       Could not parse for environment production: Cannot assign to variables in other namespaces at line 2 on node kiwi.coupleofllamas.com
     # ./spec/defines/global_spec.rb:11:in `block (2 levels) in <top (required)>'

Finished in 0.01426 seconds
2 examples, 2 failures

Failed examples:

rspec ./spec/defines/global_spec.rb:11 # haproxy::global 
rspec ./spec/defines/global_spec.rb:11 # haproxy::global 
/home/tyler/.rvm/gems/ruby-1.9.2-p290@puppet-haproxy/gems/rspec-core-2.9.0/lib/rspec/core/formatters/documentation_formatter.rb:41:in `failure_output': undefined method `strip' for nil:NilClass (NoMethodError)
        from /home/tyler/.rvm/gems/ruby-1.9.2-p290@puppet-haproxy/gems/rspec-core-2.9.0/lib/rspec/core/formatters/documentation_formatter.rb:37:in `example_failed'
        from /home/tyler/.rvm/gems/ruby-1.9.2-p290@puppet-haproxy/gems/rspec-core-2.9.0/lib/rspec/core/reporter.rb:98:in `block in notify'
        from /home/tyler/.rvm/gems/ruby-1.9.2-p290@puppet-haproxy/gems/rspec-core-2.9.0/lib/rspec/core/reporter.rb:97:in `each'
        from /home/tyler/.rvm/gems/ruby-1.9.2-p290@puppet-haproxy/gems/rspec-core-2.9.0/lib/rspec/core/reporter.rb:97:in `notify'
        from /home/tyler/.rvm/gems/ruby-1.9.2-p290@puppet-haproxy/gems/rspec-core-2.9.0/lib/rspec/core/reporter.rb:68:in `example_failed'
        from /home/tyler/.rvm/gems/ruby-1.9.2-p290@puppet-haproxy/gems/rspec-core-2.9.0/lib/rspec/core/example.rb:193:in `finish'
        from /home/tyler/.rvm/gems/ruby-1.9.2-p290@puppet-haproxy/gems/rspec-core-2.9.0/lib/rspec/core/example.rb:166:in `fail_with_exception'
        from /home/tyler/.rvm/gems/ruby-1.9.2-p290@puppet-haproxy/gems/rspec-core-2.9.0/lib/rspec/core/example_group.rb:363:in `block in fail_filtered_examples'
        from /home/tyler/.rvm/gems/ruby-1.9.2-p290@puppet-haproxy/gems/rspec-core-2.9.0/lib/rspec/core/example_group.rb:363:in `each'
        from /home/tyler/.rvm/gems/ruby-1.9.2-p290@puppet-haproxy/gems/rspec-core-2.9.0/lib/rspec/core/example_group.rb:363:in `fail_filtered_examples'
        from /home/tyler/.rvm/gems/ruby-1.9.2-p290@puppet-haproxy/gems/rspec-core-2.9.0/lib/rspec/core/example_group.rb:341:in `rescue in run'
        from /home/tyler/.rvm/gems/ruby-1.9.2-p290@puppet-haproxy/gems/rspec-core-2.9.0/lib/rspec/core/example_group.rb:346:in `run'
        from /home/tyler/.rvm/gems/ruby-1.9.2-p290@puppet-haproxy/gems/rspec-core-2.9.0/lib/rspec/core/command_line.rb:28:in `block (2 levels) in run'
        from /home/tyler/.rvm/gems/ruby-1.9.2-p290@puppet-haproxy/gems/rspec-core-2.9.0/lib/rspec/core/command_line.rb:28:in `map'
        from /home/tyler/.rvm/gems/ruby-1.9.2-p290@puppet-haproxy/gems/rspec-core-2.9.0/lib/rspec/core/command_line.rb:28:in `block in run'
        from /home/tyler/.rvm/gems/ruby-1.9.2-p290@puppet-haproxy/gems/rspec-core-2.9.0/lib/rspec/core/reporter.rb:34:in `report'
        from /home/tyler/.rvm/gems/ruby-1.9.2-p290@puppet-haproxy/gems/rspec-core-2.9.0/lib/rspec/core/command_line.rb:25:in `run'
        from /home/tyler/.rvm/gems/ruby-1.9.2-p290@puppet-haproxy/gems/rspec-core-2.9.0/lib/rspec/core/runner.rb:69:in `run'
        from /home/tyler/.rvm/gems/ruby-1.9.2-p290@puppet-haproxy/gems/rspec-core-2.9.0/lib/rspec/core/runner.rb:10:in `block in autorun'

Even when commenting out the actual setting of the variable, but leaving the :pre_condition block in, I receive the following error:

Failures:

  1) haproxy::global 
     Failure/Error: it { should include_class('concat::setup') }
     NoMethodError:
       undefined method `+' for nil:NilClass
     # ./spec/defines/global_spec.rb:12:in `block (2 levels) in <top (required)>'

  2) haproxy::global 
     Failure/Error: it { should include_class('concat::setup') }
     NoMethodError:
       undefined method `+' for nil:NilClass
     # ./spec/defines/global_spec.rb:12:in `block (2 levels) in <top (required)>'

Finished in 0.00408 seconds
2 examples, 2 failures

Failed examples:

rspec ./spec/defines/global_spec.rb:12 # haproxy::global 
rspec ./spec/defines/global_spec.rb:12 # haproxy::global 
/home/tyler/.rvm/gems/ruby-1.9.2-p290@puppet-haproxy/gems/rspec-core-2.9.0/lib/rspec
/core/formatters/documentation_formatter.rb:41:in `failure_output': undefined method
 `strip' for nil:NilClass (NoMethodError)

add ability to assert number of classes and resources

It is difficult to test that only a given set of classes and a given set of resources are included/contained as it is only possible to test if something is there or not.

A very simple way to implement this would be to make it possible to assert the number of included classes and contained resources. I can then test for inclusion of a, b, c, d etc. and then test that number of classes is exactly 4. For resources, a simple solution would be a count of all resources. Getting slightly more fancy, a count per type as well as the sum would be nice.

Have not written that many rspecs yet to suggest best way of implementing, but something like

it {
  should have_class_count == 4
  should have_resource_count == 11
  should have_file_resource_count == 3
}

would be nice.

This issue is related to issue #42 regarding "with_only" for testing resource attributes.

Requires puppetlabs_spec_helper but doesn't add it as a dependency in the gemspec

From this file

-> % rspec-puppet-init                                                                                                                                                                                                                                                          1 ↵
/home/tyler/.rvm/rubies/ruby-1.9.3-p0/lib64/ruby/site_ruby/1.9.1/rubygems/custom_require.rb:36:in `require': iconv will be deprecated in the future, use String#encode instead.
/home/tyler/.rvm/rubies/ruby-1.9.3-p0/lib64/ruby/site_ruby/1.9.1/rubygems/custom_require.rb:36:in `require': cannot load such file -- puppetlabs_spec_helper/puppetlabs_spec/puppet_internals (LoadError)
    from /home/tyler/.rvm/rubies/ruby-1.9.3-p0/lib64/ruby/site_ruby/1.9.1/rubygems/custom_require.rb:36:in `require'
    from /home/tyler/.rvm/gems/ruby-1.9.3-p0@puppet-module-dev/gems/rspec-puppet-0.1.5/lib/rspec-puppet/example/function_example_group.rb:1:in `<top (required)>'
    from /home/tyler/.rvm/rubies/ruby-1.9.3-p0/lib64/ruby/site_ruby/1.9.1/rubygems/custom_require.rb:36:in `require'
    from /home/tyler/.rvm/rubies/ruby-1.9.3-p0/lib64/ruby/site_ruby/1.9.1/rubygems/custom_require.rb:36:in `require'
    from /home/tyler/.rvm/gems/ruby-1.9.3-p0@puppet-module-dev/gems/rspec-puppet-0.1.5/lib/rspec-puppet/example.rb:4:in `<top (required)>'
    from /home/tyler/.rvm/rubies/ruby-1.9.3-p0/lib64/ruby/site_ruby/1.9.1/rubygems/custom_require.rb:36:in `require'
    from /home/tyler/.rvm/rubies/ruby-1.9.3-p0/lib64/ruby/site_ruby/1.9.1/rubygems/custom_require.rb:36:in `require'
    from /home/tyler/.rvm/gems/ruby-1.9.3-p0@puppet-module-dev/gems/rspec-puppet-0.1.5/lib/rspec-puppet.rb:6:in `<top (required)>'
    from /home/tyler/.rvm/rubies/ruby-1.9.3-p0/lib64/ruby/site_ruby/1.9.1/rubygems/custom_require.rb:36:in `require'
    from /home/tyler/.rvm/rubies/ruby-1.9.3-p0/lib64/ruby/site_ruby/1.9.1/rubygems/custom_require.rb:36:in `require'
    from /home/tyler/.rvm/gems/ruby-1.9.3-p0@puppet-module-dev/gems/rspec-puppet-0.1.5/bin/rspec-puppet-init:5:in `<top (required)>'
    from /home/tyler/.rvm/gems/ruby-1.9.3-p0@puppet-module-dev/bin/rspec-puppet-init:19:in `load'
    from /home/tyler/.rvm/gems/ruby-1.9.3-p0@puppet-module-dev/bin/rspec-puppet-init:19:in `<main>'

rspec version problem with rspec-puppet

I've been putting rspec puppet into fairly simple puppet modules project. I kept seeing these errors:

undefined method `run_all' for []:Array

Some googling found this discussion - puppetlabs/puppet#575 - which said that the same problem had been solved by moving forwards from version 2.3.n to 2.4.n of rspec. I tried downgrading rspec from (I think) 2.11.0 to 2.6.0 and the problem went away.

Is this part of a known problem?

Add support for contain_matching_<resourcetype>

Hi.

Use case:
Testing node manifests to not include any package matching a certain pattern.
We have modified internal kernels for our workstations, which are not to be installed on laptops. As such, we want to make sure no package matching a pattern (/linux-.*gg[0-9]/) is ever installed on a laptop by puppet.

Regards,
Sven

Not sure how to validate variable as file

Looking for a way to test the presence of a file, but the file is created from the variable e.g.

$rpm_version = inline_template('<%= wmq_version.split("_")[1] %>')

file { "/tmp/mq_license_$rpm_version":
  ensure => link,
  target => '/tmp/mq_license',
}

Not sure on the correct way to test this, i have added

let(:facts) { { :wmq_version => 'server_7.5', :rpm_version => '7.5' } }

Yet, not sure how to validate, i have just had to declare it as

it { should contain_file("/tmp/mq_license_7.5").with_ensure('link').with_target('/tmp/mq_license') }

as oppose using the variable

contain_file("/tmp/mq_license_#{rpm_version}")

I guess it makes no difference, the reality is the same, but wondering if there is a better way / accurate way of handling these situations.

rspec-puppet.com links to 0.1.6 as latest release, but matchers document changes only available in the master branch

In the Matchers documentation at http://rspec-puppet.com/matchers/, the have_*_resource_count matcher is documented, but isn't available in the latest release (0.1.6), only from the github master branch.

It looks like those features were added in pull request #75, commit 8dc9ecb after the Jan 2012 release of 0.1.6.

Multiple class tests break in puppet 3 and custom resources

If you have multiple spec files in the classes directory something seems to break the resolution of custom types. For an example see:

https://github.com/nhuff/spec-failure

This will pass with puppet 2.7 but with puppet 3 it gives
Invalid resource type anchor at /tmp/test/spec/fixtures/modules/test/manifests/b.pp:2

It seems like the custom type is only getting resolved in the first class and not in subsequent ones.

should support tags

I have some classes and definitions that have different behaviour when the host is tagged with different tags. I think something like: the facts support?

let(:tags) { ["tag1", "tag2"] }

Add support for exported resource mocks

Like what you have done with the fact mocks, but with exported resources.

This feature would kick arse.

Matching values of generic parameters is not done in a very "intelligent" way

Matching things in https://github.com/rodjek/rspec-puppet/blob/master/lib/rspec-puppet/matchers/create_generic.rb#L38 is likely to fail from time to time on certain circumstances.

Parameters are more or less compared as strings, due to following code parts:

https://github.com/rodjek/rspec-puppet/blob/master/lib/rspec-puppet/matchers/create_generic.rb#L59-L63

and

https://github.com/rodjek/rspec-puppet/blob/master/lib/rspec-puppet/matchers/create_generic.rb#L53-L57

The first - default-branch - solution makes it for example not very safe to compare a parameter that is a hash. As this hash is just converted to a string, tests might fail on some occasions and work on other occasions. As you can not be sure, that a hash is always ordered the same way, the string of the hash from the catalogue might look differently than the string of the hash from your specs. -> While the to_s has probably been taken to avoid the issue that puppet doesn't know about Integers and internally stores them as strings, it introduces problems on more complex data types, such as hashes.

-> We should compare complex datatypes in more intelligent, way.

The second mentioned branch (comparing arrays) is similar to this issue, probably it tried to be intelligent, but fails to do so. It introduces the possibility that something matches although it should not match:

Assume that within the catalog a parameter is ['ab','b' ] but according to our spec it should be ['a','bb'], this will not fail and pass! (at least on ruby 1.8.7)
As we are simply joining the array, both values and up to be 'abb' and hence match, although they shouldn't.
-> We should compare the array differently

In general using to_s is probably a bad idea and we should compare more complex datatypes with casting it to a different type, while for numbers, strings, and so on to_s is probably still safe.

Uncaught dependency cycle

This commit has a depedency cycle which rspec-puppet isn't catching: https://github.com/benburkert/puppet-homebrew/tree/cycle-test

Easier default facts specification

It would be super rad to do something like so:

RSpec::Puppet.configure do |config|
  config.default_facts = {
    :operatingsystem => 'Debian',
    :datacenter      => 'ec2'
  }
end

This would then not only default facts in specs to the default_facts value, but would expose RSpec::Puppet.configuration.default_facts as default_facts, so specs which need to override behavior via merging can do:

describe "some::class" do
  let(:facts) { default_facts.merge(:operating_system => 'Ubuntu') }
end

Support for Ruby 1.9

I believe something like puppetlabs/puppet@0f2bb27 should be applied to lib/rspec-puppet/matchers/create_generic.rb, as I'm getting a lot of failures like the following:

  1) apt when using default class parameters 
     Failure/Error: })
       expected that the catalogue would contain Exec[apt_update] with subscribe set to `"File[sources.list]File[sources.list.d]"` but it is set to `[File[sources.list]{:path=>"sources.list"}, File[sources.list.d]{:path=>"sources.list.d"}]` in the catalogue
     # ./spec/classes/apt_spec.rb:94:in `block (4 levels) in <top (required)>'

That one comes from puppetlabs/puppet-apt, from this test:

  it {
    should contain_exec("apt_update").with({
      'command'     => "/usr/bin/apt-get update",
      'subscribe'   => ["File[sources.list]", "File[sources.list.d]"],
      'refreshonly' => refresh_only_apt_update
    })
  }

undefined method `preferred_run_mode=' for #

trying to run

rspec-puppet-init

gives me

/var/lib/gems/1.8/gems/puppet-3.1.0/lib/puppet/test/test_helper.rb:154:in initialize_settings_before_each': undefined method preferred_run_mode=' for #<Puppet::Util::Settings:0x7fd9b1d3e978> (NoMethodError)

any clue why? mixing up puppet and puppet gem?

Autoloader for f-n's doesn't work well with other tests

Create two specs. One testing a class, one testing a function. Use an already defined puppet in the class.

If you test both, e.g. rspec spec/, then the one executed 2nd will raise this error:

     Failure/Error: output = subject.call({
     Puppet::Error:
       Could not autoload /home/xyz/labs/puppet-riak/spec/fixtures/modules/hiera-puppet/lib/puppet/parser/functions/hiera.rb: Function hiera already defined

From line 10 in function_example.rb, Puppet::Parser::Functions.autoloader.loadall.

Gem 0.1.4.

Note, that the current master doesn't load functions in the same way; so if you're not very much against it, a release would be welcome to try out the new code.

Error in doc and spec re. subject.call

When using subject.call in a spec, it seems like you have to pass in a single argument of type array; i.e. subject.call(['foo']) rather than subject.call('foo').

Both the doc in the README and the spec/functions/split_spec.rb use subject.call('foo'), and the spec doesn't actually test that it passes, so yes subject.call('foo') raises an exception, but not for the same reason should run.with_params('foo') does: in the first case, args is the string foo, so args.length returns 3, whereas in the latter args is an array whose args.length is 1. Both lead to an error being raised as the split function checks for args.length == 2.

Document how to do negative tests for parameter content

Hi-

I assume this is already possible, but it is not documented how this should work.

Today a coworker wrote a class that uses a template to set the content of a file. The template has basically this near the bottom of the file:

<% if is_laptop do %>whatever<% end %>

Now, in his rspec tests, he would like to test that with is_laptop set to false, "whatever" does not occur in the content parameter of the file resource.

I would have expected
should contain_file('/etc/a').without_content(/whatever/)
to check that the file is set, with content not containing anything matching 'whatever'. However, the without_content matcher accepts the argument, but basically discards it, testing for content to be undefined. Which it isn't.

should_not contain_file('/etc/a').with_content(/whatever/)

seems to work, but it seems counter-intuitive, especially since it also reports no problem when the file isn't there at all. So the full test today reads like this:

should_not contain_file('/etc/a').with_content(/whatever/)
should contain_file('/etc/a')

Which really looks weird and not very comprehensible.

Proposal:
Change without matcher to check for content to be undefined if no parameter passed (current behaviour), but make it check that the parameter is either undefined or not matching the parameter.

So that in the above example
should contain_file('/etc/a').without_content(/whatever/)
would make sure that
a) the file resource '/etc/a' is defined
b) the resource has a parameter content that doesn't match /whatever/ (by being undefined or having a content that is not matching the regexp.

Regards,
Sven

Getting an error when using '.with()'

Here's my test:

it do should contain_service('autofs').with(
'ensure' => 'running',
'enable' => 'true',
'hasstatus' => 'true'
) end

When I run the tests, I get:

Failures:

1. autofs::base
   Failure/Error: it do should contain_service('autofs').with(
   NoMethodError:
   undefined method `with' for #RSpec::Puppet::ManifestMatchers::CreateGeneric:0x7f112fba2ad8

   ./modules/autofs/spec/classes/base_spec.rb:13

I was following the example exactly, so I'm not sure what's wrong.

-Andy

Host specs with modules not working on Puppet 3

I have created a simple project that works with puppet 2.7.19 and doesn't with 3.0.1. Using rspec-puppet 0.1.5

https://github.com/carlossg/rspec-puppet-test

For host specs, the first host spec passes, but all the following ones fail to find the modules.

Stub Facter for functions

let(:facts) {} doesn't currently work for functions. @rodjek tells me it's coming soon :)

Rspec fails if file's resource title differs from 'path' parameter value

rspec-puppet checks for:

timezone
 [Debian] timezone class
   should contain Class[timezone]
   should contain Package[tzdata] with ensure => "latest"
   should contain File[/etc/timezone](FAILED - 1)
   should contain Exec[set-timezone] with command => "/usr/sbin/dpkg-reconfigure -f noninteractive tzdata"
 [CentOS] timezone class
   should contain Class[timezone]
   should contain Package[tzdata] with ensure => "latest"
   should contain File[/etc/sysconfig/clock](FAILED - 2)
   should contain Exec[set-timezone] with command => "/usr/sbin/tzdata-update"

Failures:

 1) timezone [Debian] timezone class
    Failure/Error: it { should contain_file('/etc/timezone') }
      expected that the catalogue would contain File[/etc/timezone]
    # ./spec/classes/init_spec.rb:8:in `block (3 levels) in <top (required)>'

 2) timezone [CentOS] timezone class
    Failure/Error: it { should contain_file('/etc/sysconfig/clock') }
      expected that the catalogue would contain File[/etc/sysconfig/clock]
    # ./spec/classes/init_spec.rb:16:in `block (3 levels) in <top (required)>'

Finished in 0.52597 seconds
8 examples, 2 failures

Failed examples:

rspec ./spec/classes/init_spec.rb:8 # timezone [Debian] timezone class
rspec ./spec/classes/init_spec.rb:16 # timezone [CentOS] timezone class

• Puppet code snip:

  file {
  'timezone':
  ensure => present,
  path => $::operatingsystem ?{
  /(RedHat|CentOS)/ => '/etc/sysconfig/clock',
  /(Debian|Ubuntu)/ => '/etc/timezone',
  },
  content => $::operatingsystem ?{
  default => template("timezone/timezone-${operatingsystem}"),
  },
  }

According to http://docs.puppetlabs.com/references/latest/type.html#file :

• path
  (Namevar: If omitted, this parameter’s value defaults to the resource’s title.)
  The path to the file to manage. Must be fully qualified.

Puppet code is executed correctly on both Puppet 2.6.x, 2.7.x, 3.0.x, 3.1.x. But rspec fails, even if the file is actually created correctly from that template.

Is there a workaround at rspec-level? Is it a known issue, or this way 'by design'? Thanks in advance! 👍

"present" not treated as the same as "installed" for package ensure

According to the puppet docs (http://docs.puppetlabs.com/references/stable/type.html#package ),

Valid values are present (also called installed), absent, purged, held, latest.

However, when I use ensure => present in my test, I get the following:

Failures:

1. basenode On an Ubuntu 10.04 OS with no params specified
   Failure/Error: should contain_package('puppet').with( { 'ensure' => 'present' } )
   expected that the catalogue would contain Package[puppet] with ensure set to "present" but it is set to "installed" in the catalogue

   ./spec/classes/basenode_spec.rb:29

The gems I have installed are:
diff-lcs (1.1.3)
hiera (0.3.0)
hiera-puppet (0.3.0)
metaclass (0.0.1)
mocha (0.12.1)
puppet-lint (0.1.13)
puppetlabs_spec_helper (0.2.0)
rake (0.9.2.2)
rspec (2.11.0)
rspec-core (2.11.1)
rspec-expectations (2.11.2)
rspec-mocks (2.11.1)
rspec-puppet (0.1.3)

This is in Ubuntu 10.04, 64bit w/ puppet 2.7.14

Not detecting dependency cycles under 2.7.x

Need to test for dependency cycles in the compiled catalogue

difficult to state 'withoutAnyOther'

It is difficult to test if a resource has exactly a set of attributes/values - and no others as it is then required to list all known attributes for the resource type using without.

I am thinking something like:

x.with(...).withoutAnyOther()

create_* should be renamed contain_*

since the subject for these tests is the catalog, it would make sense to switch create with contains

the catalog should contain_file .... is a more natural way of expressing the tests