Coder Social home page Coder Social logo

puppetlabs / control-repo Goto Github PK

View Code? Open in Web Editor NEW
200.0 50.0 508.0 186 KB

A control repository template

License: Apache License 2.0

Ruby 40.58% Shell 20.05% Puppet 38.87% PowerShell 0.49%
control-repo r10k code-manager config-version puppet puppet-enterprise

control-repo's Introduction

A Puppet Control Repository

What You Get From This control-repo

This is a template control repository that has the minimum amount of scaffolding to make it easy to get started with r10k or Puppet Enterprise's Code Manager.

The important files and items in this template are as follows:

  • Basic example of roles and profiles.
  • An example Puppetfile with various module references.
  • An example Hiera configuration file and data directory with pre-created common.yaml and nodes directory.
    • These match the default hierarchy that ships with PE.
  • An environment.conf that correctly implements:
    • A site-modules directory for roles, profiles, and any custom modules for your organization.
    • A config_version script.
  • An example config_version script that outputs the git commit ID of the code that was used during a Puppet run.

Here's a visual representation of the structure of this repository:

control-repo/
├── data/                                 # Hiera data directory.
│   ├── nodes/                            # Node-specific data goes here.
│   └── common.yaml                       # Common data goes here.
├── manifests/
│   └── site.pp                           # The "main" manifest that contains a default node definition.
├── scripts/
│   ├── code_manager_config_version.rb    # A config_version script for Code Manager.
│   ├── config_version.rb                 # A config_version script for r10k.
│   └── config_version.sh                 # A wrapper that chooses the appropriate config_version script.
├── site-modules/                         # This directory contains site-specific modules and is added to $modulepath.
│   ├── profile/                          # The profile module.
│   └── role/                             # The role module.
├── LICENSE
├── Puppetfile                            # A list of external Puppet modules to deploy with an environment.
├── README.md
├── environment.conf                      # Environment-specific settings. Configures the modulepath and config_version.
└── hiera.yaml                            # Hiera's configuration file. The Hiera hierarchy is defined here.

Copy This Repo Into Your Own Git Server

To get started with using the control-repo template in your own environment and git server, we've provided steps for the three most common servers we see: GitLab, BitBucket, and GitHub.

GitLab

  1. Install GitLab.
  2. After GitLab is installed you may sign in with the root user. If you didn't specify a custom password during installation, a temporary password is located in /etc/gitlab/initial_root_password.
  3. Make a user for yourself.
  4. Make an SSH key to link with your user. You’ll want to do this on the machine you intend to edit code from (most likely not your Puppet master, but your local workstation or laptop).
  5. Create a group called puppet (this is case sensitive).
  6. Add your user to the puppet group as well.
  7. Create a project called control-repo, and set the Namespace to be the puppet group.
  8. Clone this control repository to your laptop/workstation:
    • git clone <repository url>
    • cd control-repo
  9. Remove this repository as the origin remote:
    • git remote remove origin
  10. Add your internal repository as the origin remote:
    • git remote add origin <url of your gitlab repository>
  11. Push the production branch of the repository from your machine up to your git server
    • git push origin production

Bitbucket/Stash

  1. Install Bitbucket
  2. Make a Project called puppet (with a short name of PUP)
  3. Create a repository called control-repo
  4. Create a user called r10k with a password of puppet.
    • Make the r10k user an admin of the PUP project.
  5. Either use the admin user to test pushing code, or create a user for yourself and add your SSH key to that user.
    • If making a user for yourself, give your user account read/write or admin privilege to the PUP project.
  6. Clone this control repository to your laptop/workstation
    • git clone <repository url>
    • cd control-repo
  7. Remove this repository as the origin remote
    • git remote remove origin
  8. Add your internal repository as the origin remote
    • git remote add origin <url of your bitbucket repository>
  9. Push the production branch of the repository from your machine up to your git server
    • git push origin production

GitHub

Follow GitHub's documentation to create your control repository starting from this template.

template button

GitHub Enterprise

  1. Prepare your local git client to authenticate with a local GitHub Enterprise instance.
  2. Create a repository called control-repo in your user account or organization. Ensure that "Initialize this repository with a README" is not selected.
  3. Make a note of your repository URL (HTTPS or SSH, depending on your security configuration).
  4. Clone this control repository to your laptop/workstation:
    • git clone <repository url>
    • cd control-repo
  5. Remove this repository as the origin remote:
    • git remote remove origin
  6. Add your internal repository as the origin remote:
    • git remote add origin <url of your github repository>
  7. Push the production branch of the repository from your machine up to your git server
    • git push origin production

Code Manager Setup

If you use Puppet Enterprise see the official documentation for enabling Code Manager.

control-repo's People

Contributors

16c7x avatar abuxton avatar aharden avatar binford2k avatar cvquesty avatar davidsandilands avatar ekohl avatar fvoges avatar igalic avatar iristyle avatar jdesroch avatar jessereynolds avatar mcka1n avatar milesstoetzner avatar mtsprout avatar mwaggett avatar natemccurdy avatar npwalker avatar petems avatar pjmorr avatar reidmv avatar smortex avatar sshastri avatar stjeanp avatar taikaa avatar vulongtran avatar whatsaranjit avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

control-repo's Issues

Delete additional branches in this repository

Describe the Bug

In the repository there's now 3 branches:

  • production
  • binford2k-patch-1
  • main

Expected Behavior

I would expect only a single branch. That means if users clone the repository they won't end up with redundant environments.

Add validation and test configuration

As proposed in puppetlabs/pdk#333 this repository should serve as a default setup for control repositories generated by PDK.

Hence, we should add a Gemfile, .gitignore, .rubocop.yml (and optionally proposed CI configuration and other files?) that allow running validation and testing in a simple fashion via PDK (see below). Sample files are included in the Puppet Labs pdk-module-template repository.

Example Workflow (PDK)

# create control repo
$ pdk new control
# run static analysis tools
$ pdk validate
# run tests
$ pdk test unit

add shebang to 2 rb scripts in /scripts

the 2 ruby scripts do not have a #!/usr/bin/env ruby

this causes an issue if you clone repo on a windows machine using Git Bash for windows.

this caused an issue where the executable bits for these 2 files were changed on windows due to missing shebang and this caused Puppet agent run failure due to inability of puppet to execute the scripts. Updating the scripts from a linux-based git location made no difference, the exec bits do not get set, unless you wipe out the control repo entirely and start new from a linux host.

Once I added the shebang to the ruby scripts, and pushed from windows git location, the bits were back to normal.

SVN + r10k

A significant number of corporate users are stuck using SVN for the time being. It'd be nice to provide that as an option here. E.g. how to use this as your templated control repo with an existing SVN server.

config_version.sh fails if there is no initalized git repo

Describe the Bug

If git is installed on the system but the control repo is actually not a git repo the scripts/config_version.sh script fails.

Error:

puppet agent -t

Info: Using configured environment 'production'
Info: Retrieving pluginfacts
Info: Retrieving plugin
Info: Retrieving locales
Error: Could not retrieve catalog from remote server: Error 500 on SERVER: Server Error: Execution of config_version command `/etc/puppetlabs/code/environments/production/scripts/config_version.sh /etc/puppetlabs/code/environments production` failed: Execution of '/etc/puppetlabs/code/environments/production/scripts/config_version.sh /etc/puppetlabs/code/environments production' returned 128: fatal: not a git repository: '/etc/puppetlabs/code/environments/production/.git' on node XXXXXXXX
Warning: Not using cache on failed catalog
Error: Could not retrieve catalog; skipping run

Expected Behavior

The script should not pass the "git availability" check if there is no git repo initalized and therefore return a date as config version.

Solution

I currently fixed it by using the following condition:

elif type git >/dev/null && [ -d "$1/$2/.git" ]; then

Steps to Reproduce

Assumptions:

  • there is a production environment that had been once a git repository (see "Additional Context")
  • the previous checks in the script are failing too (no exisiting r10k etc).
  • git is installed

Steps to reproduce the behavior:

  1. rm -rf /etc/puppetlabs/code/environments/production/.git
  2. /etc/puppetlabs/code/environments/production/scripts/config_version.sh /etc/puppetlabs/code/environments production or puppet agent -t

Environment

  • control repo version: latest resp. commitID: 8820892
  • Ubuntu 18.04
  • git version 2.17.1

Additional Context

I deployed the puppetserver using foreman and I currently copy the environment files using scp to the server.

Strange error from config_version when doing puppet-apply

Running puppet apply results in the following error when running under one of my docker containers - this is after doing r10k deploy environment -pv:

Error: Execution of config_version command `/etc/puppetlabs/code/environments/production/scripts/config_version.sh /etc/puppetlabs/code/environments production` failed: Execution of '/etc/puppetlabs/code/environments/production/scripts/config_version.sh /etc/puppetlabs/code/environments production' returned 1: /opt/puppetlabs/puppet/lib/ruby/gems/2.1.0/gems/rugged-0.21.4/lib/rugged/repository.rb:99:in `lookup': Object not found - failed to find pack entry (a782f44552cc90a6b8645e95a17ead5b36e71b40) (Rugged::OdbError)
    from /opt/puppetlabs/puppet/lib/ruby/gems/2.1.0/gems/rugged-0.21.4/lib/rugged/repository.rb:99:in `lookup'
    from /etc/puppetlabs/code/environments/production/scripts/config_version.rb:18:in `<main>' on node dockerbuild.lan.asio

Deployment Key chicken/egg

Quick question about the Deployment key. The README instructions have you create the deploy key manually on the Puppet Server, then add the deployment key to the git Repo.

However, there is also code to do the same, which never runs if you follow the default instructions. Is there another way to solve the bootstrap problem of getting the server connected to the control repo, or is the deployment key code a future enhancement?

side-note: this repo is awesome, thanks a bunch!
Chris

private key location - README does not match the PE docs

The docs for this repo say to place the private key at:

/etc/puppetlabs/puppetserver/code_manager.key

And the PE docs at https://docs.puppetlabs.com/pe/latest/cmgmt_control_repo.html say this:

After PE installation is complete, place this private key in a location that the pe-puppet user can access; we recommend /etc/puppetlabs/puppetserver/ssh/id-control_repo.rsa. The file itself should have the correct permissions for the pe-puppet user, which is created during PE installation, to use that key.

Make repo a GitHub template repository

Use Case

Configure the repository to be a GitHub Template repository.

Describe the Solution You Would Like

Based on the instructions here configure the repository to be a GitHub template repository for easier use for GitHub users and/or new Puppet users.

Describe Alternatives You've Considered

The current method of use would/could apply.

Additional Context

NA

config_version creates chicken-and-egg with Git

The current config version script requires git or rugged to run (depending of weather it is pe or not), meaning that a new master will need git or rugged installed before it can run puppet. Using puppet to install git or rugged will not work as config_version.sh must exit with 0 for the catalog to be compiled. I would recommend having some kind of fall back if neither of the options work. Also if we want to do any acceptance testing we have to install git on the VMs before anything will work using puppet apply

Hunner/Hiera module - Forge or GitHub?

Pretty much a complete newb so I apologize in advance if this is a stupid question:

The control-repo Puppetfile includes Hunner's 'hiera' module from GitHub but it looks like the same module is also available on the Forge (v2.0.0 versus GitHub's 1.4.1). Any reason not to use the module available on the Forge?

Secondary question: Is this even the 'correct' forum for such a question. Kinda feels like it's not.

r10k postrun not running

@npwalker

I am using a slightly modified version of your control-repo, but for some reason my postrun commands are not running, so my Puppet Enterprise console environments are never being updated.

Could you validate if your test rigs are correctly running the postrun commands configured in hiera?

Add example for hiera.yaml

Please add an example for an environment specific hiera.yaml.

Currently the hieradata folder in this example doesen't work with PE because the default hiera.yaml doesen't include a definition for e.g. "node/%{trusted.certname}".

Alternative Git Repository Location?

Is it possible to setup an alternative git repository location and project name? puppet/control-repo isn't really desirable in my environment and it seems like it is hard-coded to that.

Vagrantfile added?

Generally in the control-repos I do for different Puppet setups, I include a Vagrantfile and relevant scripts to be able to test locally the same Puppet and data. Would a PR with such an addition be welcomed? If so I can provide one.

Could not evaluate: no implicit conversion of String into Integer

On the puppetmaster I receive this error during every run.
Granted, this is PE2015.3 and you mentioned it wasn't tested, but I have been able to get everything working using the native Code Management system with the exception of removing these two messages.
Error: /Stage[main]/Profile::Puppetmaster/Git_deploy_key[add_deploy_key_to_puppet_control-]: Could not evaluate: no implicit conversion of String into Integer
Error: /Stage[main]/Profile::Puppetmaster/Git_webhook[web_post_receive_webhook-]: Could not evaluate: no implicit conversion of String into Integer

Also, the webhook and deploy key were put in place manually and are working just fine, I just can't seem to get this error to go away.

Bare control-repo doesn't pass "pdk validate" tests

Describe the Bug

The site-modules/adhoc/plans/myplan.pp file doesn't pass a basic 'pdk validate' run.

Expected Behavior

It's expected that control-repo should be able to pass any PDK tests from the main/master branch.

Steps to Reproduce

Steps to reproduce the behavior:

  1. git clone https://github.com/puppetlabs/control-repo.git
  2. cd control-repo
  3. pdk convert (accept all defaults)
  4. pdk validate

Environment

Red Hat Enterprise Linux 7.9
pdk-2.6.1.0-1.el7.x86_64
puppet-agent-7.16.0-1.el7.x86_64

Additional Context

root@example:~/a/control-repo$  pdk validate
pdk (INFO): Using Ruby 2.7.7
pdk (INFO): Using Puppet 7.22.0
pdk (INFO): Running all available validators...
pdk (INFO): Validator 'puppet-epp' skipped for '/home/a/control-repo'. No files matching '["**/*.epp"]' found to validate.
pdk (INFO): Validator 'task-name' skipped for '/home/a/control-repo'. No files matching '["tasks/**/*"]' found to validate.
pdk (INFO): Validator 'task-metadata-lint' skipped for '/home/a/control-repo'. No files matching '["tasks/*.json"]' found to validate.
┌ [✔] Running metadata validators ...
├── [✔] Checking metadata syntax (metadata.json tasks/*.json).
└── [✔] Checking module metadata style (metadata.json).
┌ [✖] Running puppet validators ...
├── [✖] Checking Puppet manifest syntax (**/*.pp).
└── [✖] Checking Puppet manifest style (**/*.pp).
┌ [✔] Running ruby validators ...
└── [✔] Checking Ruby code style (**/**.rb).
┌ [✔] Running tasks validators ...
├── [✔] Checking task names (tasks/**/*).
└── [✔] Checking task metadata style (tasks/*.json).
┌ [✔] Running yaml validators ...
└── [✔] Checking YAML syntax (**/*.yaml **/*.yml).
pdk (ERROR): puppet-syntax: Could not parse for environment production: Syntax error at 'targets' (site-modules/adhoc/plans/myplan.pp:4:14)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.