Coder Social home page Coder Social logo

ansible-gitlab-runner's Introduction

GitLab Runner Build Status Ansible Role

This role will install the official GitLab Runner (fork from haroldb) with updates. Needed something simple and working, this did the trick for me. Open for changes though.

Requirements

This role requires Ansible 2.7 or higher.

Role Variables

  • gitlab_runner_package_name - Since Gitlab 10.x The package name of gitlab-ci-multi-runner has been renamed to gitlab-runner. In order to install a version < 10.x you will need to define this variable gitlab_runner_package_name: gitlab-ci-multi-runner.
  • gitlab_runner_wanted_version or gitlab_runner_package_version - To install a specific version of the gitlab runner (by default it installs the latest). On Mac OSX and Windows, use e.g. gitlab_runner_wanted_version: 12.4.1. On Linux, use gitlab_runner_package_version instead.
  • gitlab_runner_concurrent - The maximum number of global jobs to run concurrently. Defaults to the number of processor cores.
  • gitlab_runner_registration_token - The GitLab registration token. If this is specified, a runner will be registered to a GitLab server.
  • gitlab_runner_coordinator_url - The GitLab coordinator URL. Defaults to https://gitlab.com.
  • gitlab_runner_sentry_dsn - Enable tracking of all system level errors to Sentry
  • gitlab_runner_listen_address - Enable /metrics endpoint for Prometheus scraping.
  • gitlab_runner_runners - A list of gitlab runners to register & configure. Defaults to a single shell executor.
  • gitlab_runner_skip_package_repo_install- Skip the APT or YUM repository installation (by default, false). You should provide a repository containing the needed packages before running this role.
  • gitlab_runner_config_update_mode- Set to by_config_toml (default) if this role should apply config changes by updating the config.toml itself or set it to by_registering if config changes should be applied by unregistering and regeistering the runner in case the config has changed.
  • gitlab_unregister_runner_executors_which_are_not_longer_configured - Set to true if executors should be unregistered from a runner in case it is are not longer configured in ansible. Default: false

See the defaults/main.yml file listing all possible options which you can be passed to a runner registration command.

Gitlab Runners cache

For each gitlab runner in gitlab_runner_runners you can set cache options. At the moment role support s3, azure and gcs types. Example configurration for s3 can be:

gitlab_runner:
  cache_type: "s3"
  cache_path: "cache"
  cache_shared: true
  cache_s3_server_address: "s3.amazonaws.com"
  cache_s3_access_key: "<access_key>"
  cache_s3_secret_key: "<secret_key>"
  cache_s3_bucket_name: "<bucket_name>"
  cache_s3_bucket_location: "eu-west-1"
  cache_s3_insecure: false

Autoscale Runner Machine vars for AWS (optional)

  • gitlab_runner_machine_options: [] - Foremost you need to pass an array of dedicated vars in the machine_options to configure your scaling runner:

    • amazonec2-access-key and amazonec2-secret-key the keys of the dedicated IAM user with permission for EC2
    • amazonec2-zone
    • amazonec2-region
    • amazonec2-vpc-id
    • amazonec2-subnet-id
    • amazonec2-use-private-address=true
    • amazonec2-security-group
    • amazonec2-instance-type
    • you can also set amazonec2-tags to identify you instance more easily via aws-cli or the console.
  • MachineDriver - which should be set to amzonec2 when working on AWS

  • MachineName - Name of the machine. It must contain %s, which will be replaced with a unique machine identifier.

  • IdleCount - Number of machines, that need to be created and waiting in Idle state.

  • IdleTime - Time (in seconds) for machine to be in Idle state before it is removed.

In addition you could set off peak settings. This lets you select a regular time periods when no work is done. For example most of commercial companies are working from Monday to Friday in a fixed hours, eg. from 10am to 6pm. In the rest of the week - from Monday to Friday at 12am-9am and 6pm-11pm and whole Saturday and Sunday - no one is working. These time periods we’re naming here as Off Peak.

  • gitlab_runner_machine_off_peak_periods
  • gitlab_runner_machine_off_peak_idle_time
  • gitlab_runner_machine_off_peak_idle_count

Read Sources

For details follow these links:

See the config for more options

Example Playbook

- hosts: all
  become: true
  vars_files:
    - vars/main.yml
  roles:
    - { role: riemers.gitlab-runner }

Inside vars/main.yml

gitlab_runner_coordinator_url: https://gitlab.com
gitlab_runner_registration_token: '12341234'
gitlab_runner_runners:
  - name: 'Example Docker GitLab Runner'
    # token is an optional override to the global gitlab_runner_registration_token
    token: 'abcd'
    # url is an optional override to the global gitlab_runner_coordinator_url
    url: 'https://my-own-gitlab.mydomain.com'
    executor: docker
    docker_image: 'alpine'
    tags:
      - node
      - ruby
      - mysql
    docker_volumes:
      - "/var/run/docker.sock:/var/run/docker.sock"
      - "/cache"
    extra_configs:
      runners.docker:
        memory: 512m
        allowed_images: ["ruby:*", "python:*", "php:*"]
      runners.docker.sysctls:
        net.ipv4.ip_forward: "1"

autoscale setup on AWS

how vars/main.yml would look like, if you setup an autoscaling GitLab-Runner on AWS:

gitlab_runner_registration_token: 'HUzTMgnxk17YV8Rj8ucQ'
gitlab_runner_coordinator_url: 'https://gitlab.com'
gitlab_runner_runners:
  - name: 'Example autoscaling GitLab Runner'
    state: present
    # token is an optional override to the global gitlab_runner_registration_token
    token: 'HUzTMgnxk17YV8Rj8ucQ'
    executor: 'docker+machine'
    # Maximum number of jobs to run concurrently on this specific runner.
    # Defaults to 0, simply means don't limit.
    concurrent_specific: '0'
    docker_image: 'alpine'
    # Indicates whether this runner can pick jobs without tags.
    run_untagged: true
    extra_configs:
      runners.machine:
        IdleCount: 1
        IdleTime: 1800
        MaxBuilds: 10
        MachineDriver: 'amazonec2'
        MachineName: 'git-runner-%s'
        MachineOptions: ["amazonec2-access-key={{ lookup('env','AWS_IAM_ACCESS_KEY') }}", "amazonec2-secret-key={{ lookup('env','AWS_IAM_SECRET_KEY') }}", "amazonec2-zone={{ lookup('env','AWS_EC2_ZONE') }}", "amazonec2-region={{ lookup('env','AWS_EC2_REGION') }}", "amazonec2-vpc-id={{ lookup('env','AWS_VPC_ID') }}", "amazonec2-subnet-id={{ lookup('env','AWS_SUBNET_ID') }}", "amazonec2-use-private-address=true", "amazonec2-tags=gitlab-runner", "amazonec2-security-group={{ lookup('env','AWS_EC2_SECURITY_GROUP') }}", "amazonec2-instance-type={{ lookup('env','AWS_EC2_INSTANCE_TYPE') }}"]

NOTE

from https://docs.gitlab.com/runner/executors/docker_machine.html:

The first time you’re using Docker Machine, it’s best to execute manually docker-machine create... with your chosen driver and all options from the MachineOptions section. This will set up the Docker Machine environment properly and will also be a good validation of the specified options. After this, you can destroy the machine with docker-machine rm [machine_name] and start the Runner.

Example:


docker-machine rm test

Run As A Different User

To run the Gitlab Runner as a different user (rather than the default gitlab-runner user), there is a workaround requiring a little extra Ansible to be run. See #277 for details.

Contributors

Feel free to add your name to the readme if you make a PR. A full list of people from the PR's is here

  • Gastrofix for adding Mac Support
  • Matthias Schmieder for adding Windows Support
  • dniwdeus & rosenstrauch for adding AWS autoscale option
  • oscillate123 for fixing Windows config.toml idempotency

ansible-gitlab-runner's People

Contributors

ajdurant avatar britaxx avatar chrisrohr avatar easbar avatar ecno92 avatar fkleon avatar gabor-nagy-stylers avatar gardar avatar guenhter avatar haroldb avatar heytrav avatar hugopuntos avatar icez avatar jarleb avatar jmlx42 avatar joelpet avatar jonnymccullagh avatar kshcherban avatar m-bucher avatar mschmieder avatar paulrbr avatar paulrbr-fl avatar riemers avatar samjetski avatar sezanzeb avatar snoopotic avatar srustem3 avatar sykmschmieder avatar tauffredou avatar volcan01010 avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar

ansible-gitlab-runner's Issues

With gitlab-runner version 10.x package, docker-volumes is wrongly set

Actually, on your 'master' branch, the role does not correctly register the runner with docker executor.

The parameter --docker-volumes is not correctly set since you try to use a mutli-parameter with comma-separeted.

--docker-volumes use its argument as one parameter. If others parameters needed, we have to set several times --docker-volumes like this:

gitlab-runner register --non-interactive -url https://gitlab.com/ --registration-token xxxxxxx --description test-runner --tag test --executor docker --docker-image debian:stable-backports --docker-volumes "/var/run/docker.sock:/var/run/docker.sock" --docker-volumes "/cache"

In order to have:

  [runners.docker]
    tls_verify = false
    image = "debian:stable-backports"
    privileged = false
    disable_cache = false
    volumes = ["/var/run/docker.sock:/var/run/docker.sock", "/cache"]
    shm_size = 0

Instead, actually we've got this:

  [runners.docker]
    tls_verify = false
    image = "debian:stable-backports"
    privileged = false
    disable_cache = false
    volumes = ["[", "/cache"]
    shm_size = 0

Cannot find package in suggested repo - RedHat

vagrant ssh fedora28

 # BEFORE RUNNING ANSIBLE ROLE

[vagrant@fedora28 ~]$ sudo dnf repolist
Fedora 28 - x86_64 - Updates                                                                                                                                                                                  4.0 MB/s |  29 MB     00:07    
Fedora 28 - x86_64                                                                                                                                                                                            4.5 MB/s |  60 MB     00:13    
Last metadata expiration check: 0:00:11 ago on Mon 11 Feb 2019 07:14:51 PM UTC.
repo id                                                                                                   repo name                                                                                                                     status
*fedora                                                                                                   Fedora 28 - x86_64                                                                                                            57,327
*updates                                                                                                  Fedora 28 - x86_64 - Updates                                                                                                  21,147
[vagrant@fedora28 ~]$

# AFTER RUNNING ANSIBLE ROLE
 
[vagrant@fedora28 ~]$ sudo dnf repolist
Failed to synchronize cache for repo 'runner_gitlab-runner', disabling.
Failed to synchronize cache for repo 'runner_gitlab-runner-source', disabling.
Last metadata expiration check: 0:01:48 ago on Mon 11 Feb 2019 07:14:51 PM UTC.
repo id                                                                                                   repo name                                                                                                                     status
*fedora                                                                                                   Fedora 28 - x86_64                                                                                                            57,327
*updates                                                                                                  Fedora 28 - x86_64 - Updates                                                                                                  21,147

 
[vagrant@fedora28 ~]$ cat /etc/yum.repos.d/runner_gitlab-runner.repo 
[runner_gitlab-runner]
name=runner_gitlab-runner
baseurl=https://packages.gitlab.com/runner/gitlab-runner/el/28/$basearch
repo_gpgcheck=1
gpgcheck=0
enabled=1
gpgkey=https://packages.gitlab.com/gpg.key
sslverify=1
sslcacert=/etc/pki/tls/certs/ca-bundle.crt

[runner_gitlab-runner-source]
name=runner_gitlab-runner-source
baseurl=https://packages.gitlab.com/runner/gitlab-runner/el/28/SRPMS
repo_gpgcheck=1
gpgcheck=0
enabled=1
gpgkey=https://packages.gitlab.com/gpg.key
sslverify=1
sslcacert=/etc/pki/tls/certs/ca-bundle.crt




[vagrant@fedora28 ~]$ sudo dnf search gitlab-runner
Failed to synchronize cache for repo 'runner_gitlab-runner', disabling.
Failed to synchronize cache for repo 'runner_gitlab-runner-source', disabling.
Last metadata expiration check: 0:08:51 ago on Mon 11 Feb 2019 07:14:51 PM UTC.
No matches found

restart_gitlab_runner job fail on CentOS 7 Could not find the requested service gitlab-runner: host

Hello,

I'm running riemers.gitlab-runner 1.6.1 playbook with the following parameters

 - role: riemers.gitlab-runner
     gitlab_runner_coordinator_url: --skip--
     gitlab_runner_registration_token: --skip--
     gitlab_runner_package_name: gitlab-runner
     gitlab_runner_concurrent: 1
     gitlab_runner_runners:
       - name: test-run
         concurrent_specific: 1
         tags: [tr-tag]
         run_untagged: false
         locked: true
     become: true

On initial run everythings seems fine but on subsequent runs I'm getting this failure

RUNNING HANDLER [riemers.gitlab-runner : restart_gitlab_runner] *************************************************************************************************************************************************************************************************************** fatal: [test-playbook]: FAILED! => {"changed": false, "msg": "Could not find the requested service gitlab-runner: host"}

Allow multiple runners to be registered

Fow now only une runner can be registered with this playbook.

However gitlab-runner allows for any number of runners to be registered on the same machine.

Error on Assemble new config.toml

Hi.

Using latest 1.5.2 version, my variables:

gitlab_runner_coordinator_url: https://gitlab.***.ru/
gitlab_runner_registration_token: ***
gitlab_runner_runners:
  - name: pbx
    run_untagged: false
    tags:
      - production
      - vpbx

When installing, I get such error:

TASK [riemers.gitlab-runner : Copy gitlab-runner-wrapper.sh] *********************************************************************************************************************************************************************************
ok: [5.9.107.164]

TASK [riemers.gitlab-runner : Assemble new config.toml] **************************************************************************************************************************************************************************************
fatal: [5.9.107.164]: FAILED! => {"changed": false, "msg": "failed to validate: rc:1 error:Runtime platform                                  \u001b[0;m  arch\u001b[0;m=amd64 os\u001b[0;m=linux pid\u001b[0;m=16399 revision\u001b[0;m=577f813d version\u001b[0;m=12.5.0\nRunning in system-mode.                           \u001b[0;m \n                                                  \u001b[0;m \n\u001b[31;1mFATAL: toml: cannot load TOML value of type map[string]interface {} into a Go string\u001b[0;m \n"}

As I see this task just executes this command:

[root@pbx tmp]# /tmp/gitlab-runner-wrapper.sh /etc/gitlab-runner/config.toml
Runtime platform                                    arch=amd64 os=linux pid=3025 revision=577f813d version=12.5.0
Running in system-mode.                            
                                                   
Verifying runner... is alive                        runner=RszkNp8f

So return value seems to be OK and the runner itself is working OK as I can see. My configuration file config.toml:

[root@pbx tmp]# cat /etc/gitlab-runner/config.toml
concurrent = 8
check_interval = 0

[session_server]
  session_timeout = 1800

[[runners]]
  name = "pbx"
  output_limit = 4096
  url = "https://gitlab.***.ru/"
  token = "***"
  executor = "shell"
  [runners.custom_build_dir]
  [runners.cache]
    [runners.cache.s3]
    [runners.cache.gcs]

A can't figure out where is the problem, because everything seems to be OK except the errror... Thanks!

Need to remove .bash_logout for Ubuntu Focal

System Information

TASK [debug] *******************************************************************
ok: [focal] => (item=ansible_os_family: Debian) => {
    "msg": "ansible_os_family: Debian"
}
ok: [focal] => (item=ansible_distribution: Ubuntu) => {
    "msg": "ansible_distribution: Ubuntu"
}
ok: [focal] => (item=ansible_distribution_version: 20.04) => {
    "msg": "ansible_distribution_version: 20.04"
}
ok: [focal] => (item=ansible_distribution_release: focal) => {
    "msg": "ansible_distribution_release: focal"
}

Role Version Tested

v1.6.17

Problem

Shell provisioner on Ubuntu 20.04 (Focal) suffers the same issue as Debian 10 (Buster) - where lines in .bash_logout prematurely kill the job.

I can confirm that Ubuntu 20.04 has the same offending lines in its default .bash_logout script:

root@focal:/home/gitlab-runner# cat .bash_logout 
# ~/.bash_logout: executed by bash(1) when login shell exits.

# when leaving the console clear the screen to increase privacy

if [ "$SHLVL" = 1 ]; then
    [ -x /usr/bin/clear_console ] && /usr/bin/clear_console -q
fi

Looks like this role only looks remedies the issue on Debian 10 (Buster) and not Ubuntu 20.04 (Focal).

You can see that only Debian Buster is accounted for in the following two tasks:

(Debian) Install GitLab Runner
(Debian) Remove ~/gitlab-runner/.bash_logout on debian buster

Proposed Solution

Update the relevant tasks' when statements to check for Ubuntu 20.04 OR Debian 10.

EL8 is not supported

meta/main.yml states that all EL versions are supported, but thats not true.
EL8 cannot be supported as there are no gitlab-runner packages available for EL8.

Changes to some role variables ignored once runner has been registered

Steps to reproduce:

  1. Start from a clean slate (e.g. by running gitlab-runner unregister --all-runners)
  2. Set a variable used in runner registration, e.g. gitlab_runner_docker_volumes: ['/cache']
  3. Run role
  4. Change to gitlab_runner_docker_volumes: ['/tmp']
  5. Run role again

Expected:
/etc/gitlab-runner/config.toml contains a single runner with a [runners.docker] entry volumes = ["/tmp"]

Actual:
/etc/gitlab-runner/config.toml contains a single runner with a [runners.docker] entry volumes = ["/cache"]

Discussion:
Runner registration is skipped as soon as a runner is found among the registered runners that matches the description of the runner for which the gitlab_runner_docker_volumes variable was updated. Ideally, registration should be re-run whenever dependent role variables change.

Default config.toml is not replaced by new one

Hello,

First of all thanks for your role and the work you did.

I've a problem when using it. The default config.toml is not replaced by the new configuration which is generated by Ansible.

When Ansible finish the work, on the server, I've two in the /etc/gitlab-runner directory.

root@buster:/etc/gitlab-runner# tree
.
├── config.toml
└── config.toml.6155.2019-10-06@07:50:13~

0 directories, 2 files

The first one config.toml contains the default content of the runner configuration:

concurrent = 2
check_interval = 0

[[runners]]
  name = "vagrant"
  limit = 1
  output_limit = 4096
  url = "https://gitlab.com/ci"
  token = "..."
  executor = "shell"
  [runners.docker]
    tls_verify = false
    disable_cache = false
    shm_size = 0
  [runners.cache]

  environment = []
[[runners]]
  name = "vagrant"
  limit = 1
  output_limit = 4096
  url = "https://gitlab.com/ci"
  token = "..."
  executor = "shell"
  [runners.cache]
  environment = []

Whereas the second one contains the configuration I want:

concurrent = 2
check_interval = 0

[[runners]]
  name = "vagrant"
  limit = 1
  output_limit = 4096
  url = "https://gitlab.com/ci"
  token = "..."
  executor = "docker"
  [runners.docker]
    tls_verify = false
    image = "alpine:latest"
    privileged = true
    disable_cache = false
    volumes = ["/var/run/docker.sock:/var/run/docker.sock", "/cache"]
    shm_size = 0
  [runners.cache]

[[runners]]
  name = "vagrant"
  limit = 1
  output_limit = 4096
  url = "https://gitlab.com/ci"
  token = "..."
  executor = "shell"
  [runners.cache]

During the process Ansible doesn't display any error.

For information, this my Ansible version information:

ansible 2.8.5
  config file = /etc/ansible/ansible.cfg
  configured module search path = [u'/home/jdecool/.ansible/plugins/modules', u'/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/lib/python2.7/dist-packages/ansible
  executable location = /usr/bin/ansible
  python version = 2.7.16 (default, Jul  9 2019, 16:43:02) [GCC 8.3.0]

Support for docker services in runner config

Currently,

what I try to achieve is also to configure predinefed services for some runner. This could be done through runner/configuration/advanced-configuration.html#the-runnersdockerservices-section

So currently I don't know how this could be achieved by using this role.

What I did try but without success was the following:

gitlab_runner_coordinator_url: "https://gitlab.example.com"
gitlab_runner_listen_address: "0.0.0.0:9097"
gitlab_runner_runners:
  - name: '{{ ansible_hostname }}'
    state: present
    executor: docker
    docker_image: 'docker:19.03-git'
    tags:
      - docker
      - dind
    env_vars: [
      "DOCKER_AUTH_CONFIG={\"auths\":{\"docker.example.com\":{\"auth\":\"<my-secret>\"}},\"HttpHeaders\":{\"User-Agent\":\"Docker-client/18.09.5 (linux)\"}}",
      "DOCKER_DRIVER=\"overlay2\"",
    ]
    run_untagged: false
    # Docker privileged mode
    docker_privileged: true
    docker_volumes:
      - "/cache"
      - "/etc/docker/certs.d:/etc/docker/certs.d"
      - "/var/lib/docker"
    extra_configs:
      runners.docker:
        wait_for_services_timeout: 15
      runners.docker.services: 
        - name: "docker:19.03-dind"

I hope someone can help me. Thanks in advance.

Key 'runners.cache' has already been defined.

My gitlab_runner_runners look as follows (pretty much like the example):

gitlab_runner_runners:
  - name: 'Example Docker GitLab Runner'
    executor: docker
    docker_image: 'alpine'
    tags:
      - nginx
      - python
    docker_volumes:
      - "/var/run/docker.sock:/var/run/docker.sock"
      - "/cache"
    extra_configs:
      runners.docker:
        memory: 512m
        allowed_images: ["nginx:*", "python:*"]
      runners.docker.sysctls:
        net.ipv4.ip_forward: "1"

When I run the playbook, I keep getting this error:
FATAL: Near line 23 (last key parsed 'runners'): Key 'runners.cache' has already been defined.

Any ideas what I am doing wrong? How can I recover from this? Or is it a bug?

Thanks a lot!

ssh_user is not saved correctly in config.toml

See https://github.com/riemers/ansible-gitlab-runner/blob/master/tasks/update-config-runner.yml#L195

It says:

line: '  user = {{ gitlab_runner.ssh_user|default("") | to_json }}'
state: "{{ 'present' if gitlab_runner.cache_s3_insecure is defined else 'absent' }}"

Shouldn't it rather be? Why does cache_s3_insecure have to be defined for the user to be saved?

line: '  user = {{ gitlab_runner.ssh_user|default("") | to_json }}'
state: "{{ 'present' if gitlab_runner.ssh_user is defined else 'absent' }}"

cache_type variable seems to have become a mandatory variable

cache_type variable seems to have become mandatory after this 5c9659b

TASK [riemers.gitlab-runner : Set cache s3 section] ******************************************************************************************************* Wednesday 12 February 2020 14:49:14 -0500 (0:00:00.591) 0:01:40.354 **** fatal: [vps3.cloudalbania.com]: FAILED! => {"msg": "The task includes an option with an undefined variable. The error was: 'dict object' has no attribute 'cache_type'\n\nThe error appears to be in '/home/bzanaj/git/pers/ansible-systems/roles/riemers.gitlab-runner/tasks/update-config-runner.yml': line 128, column 3, but may\nbe elsewhere in the file depending on the exact syntax problem.\n\nThe offending line appears to be:\n\n\n- name: Set cache s3 section\n ^ here\n"}

Multiple registration tokens?

Hi,
Thank you for this great role. I had a question, and maybe am missing something. It looks like the role supports multiple runners per machine, however, they all go off of the same gitlab registration token. Am I seeing that correctly?

The issue that I'm running into is that I created a token in gitlab for a group and registered my runner just fine, however, gitlab does not allow those runners to be available to subgroups only to the projects in the group. I was wondering if I could setup a second runner with a new registration on the same machine.

If this is not something that is supported right now and you think it is a valuable addition I could try to write up a PR for it.

Thank you

Privileged mode requires a second run

Steps to reproduce:

  1. Set gitlab_runner_docker_privileged: true
  2. Run provisioning
  3. Check /etc/gitlab-runner/config.toml on host
  4. Run provisioning again
  5. Check /etc/gitlab-runner/config.toml on host again

Expected:

  • Step 3: privileged is true
  • Step 5: privileged is true

Actual:

  • Step 3: privileged is false
  • Step 5: privileged is true

Travis tests do not generate runners-config

It looks to me as if the travis-tests currently do not generated a config.toml file that includes the configured runners from the test-vars:

TASK [ansible-gitlab-runner : include_tasks] ***********************************

skipping: [localhost] => (item={u'run_untagged': True, u'locked': u'false', u'concurrent_specific': u'0', u'tags': [], u'env_vars': [], u'output_limit': 4096, u'docker_privileged': False, u'state': u'present', u'executor': u'shell', u'docker_image': u'', u'name': u'travis-job-062393db-d09f-40dd-a080-24d8103cafa8'}) 

I think it is due to the follwing section, where [runner]-config is only created for runners that where already present in config.toml.

- include_tasks: update-config-runner.yml
when:
- ('name = "'+gitlab_runner.name|default(ansible_hostname+'-'+gitlab_runner_index|string)+'"') in runner_config
- gitlab_runner.state|default('present') == 'present'
loop: "{{ gitlab_runner_runners }}"
loop_control:
index_var: gitlab_runner_index
loop_var: gitlab_runner

This is never true for travis, because it always makes a fresh install.

Wouldn't it be better to loop over the gitlab-runner-runners variable without a condition?

Ansible galaxy releases

Hello,

Thanks for this role. Please, can you import to galaxy the latest releases because the last on Galaxy is v1.1.5.

Have a nice day

added docker_volumes parameter rendered in wrong section

Greetings!
Let's assume I have such configuration for a runner

  - name: '{{ ansible_hostname }}-docker'
    state: present
    executor: 'docker'
    output_limit: 4096
    concurrent_specific: '0'
    run_untagged: true
    docker_privileged: false
    docker_image: ''
    tags: []
    locked: 'false'
    env_vars: []

and add after tags parameter

    docker_volumes:
      - "/var/run/docker.sock:/var/run/docker.sock"
      - "/cache"

then configuration will be like that

[[runners]]
  name = "localhost-docker"
  limit = 0
  output_limit = 4096
  url = "https://gitlab.infra.local/"
  executor = "docker"
    volumes = ["/var/run/docker.sock:/var/run/docker.sock", "/cache"]
  [runners.custom_build_dir]
  [runners.docker]
    privileged = false
    tls_verify = false
    image = ""
    disable_entrypoint_overwrite = false
    oom_kill_disable = false
    disable_cache = false
    shm_size = 0
  [runners.cache]
    [runners.cache.gcs]

  environment = []

volumes under runners section, not runners.docker

(.venv) [spigell@hammerhead examples]$ ansible-playbook --version
ansible-playbook 2.9.2

riemers.gitlab-runner - v1.5.4

Offpeak option is deprecated and will be removed in version 14.

Hello,

Just wondering how i can configure a subsection under runners.machine?
The current offpeak option will be removed with gitlabrunner version 14.
The new solution is using the runners.machine.autoscaling option: https://docs.gitlab.com/runner/configuration/autoscale.html#autoscaling-periods-configuration

Current config:

extra_configs:
      runners.machine:
        IdleCount: 1
        IdleTime: 1800
        OffPeakPeriods: ["* * 0-7,18-23 * * mon-fri *", "* * * * * sat,sun *"]
        OffPeakIdleCount: 0
        OffPeakIdleTime: 600
        MachineDriver: 'google'

New config:

extra_configs:
      runners.machine:
        IdleCount: 1
        IdleTime: 1800
        MachineDriver: 'google'
        runners.machine.autoscaling:
          Periods: ["* * 0-7,18-23 * * mon-fri *","* * * * * sat,sun *"]
          IdleCount: 0
          IdleTime: 600

output with the new section:

[runners.machine]
    IdleTime = 1800
    IdleCount = 1
    MachineDriver = "google"
    MachineName = "auto-scale-runner-%s"
    runners.machine.autoscaling = {"Periods": ["* * 0-7,18-23 * * mon-fri *", "* * * * * sat,sun *"], "IdleCount": 0, "IdleTime": 600}

should be:

[runners.machine]
    IdleTime = 1800
    IdleCount = 1
    MachineDriver = "google"
    MachineName = "auto-scale-runner-%s"
    [[runners.machine.autoscaling]]
      Periods = ["* * 0-7,18-23 * * mon-fri *", "* * * * * sat,sun *"]
      IdleCount = 0
      IdleTime = 600

Is there an issue with my config in Ansible or is this not supported yet?

gitlab-runner reload fails on reload

Looks like reload is not supported by debian gitlab-runner service.

RUNNING HANDLER [riemers.gitlab-runner : reload_gitlab_runner] ***********************************************************************************************
fatal: [X.X.X.X]: FAILED! => {"changed": false, "msg": "Unable to reload service gitlab-runner: Failed to reload gitlab-runner.service: Job type reload is not applicable for unit gitlab-runner.service.\nSee system logs and 'systemctl status gitlab-runner.service' for details.\n"}

I think restart can be used instead.

gitlab runner repo installation is not idempotent

repo installation script should only run if the repo is missing

this can be achieved in a few ways:
1. Ansible should provide a fact about configured package repositories, run the script conditionally on that fact
2. gather the relevant fact inside this module
3. install repo using Ansible's apt_repository + handle anything else that is in that script

ad 1
such fact doesn't exist at the moment, it's handled by internal logic of apt_repository: https://github.com/ansible/ansible/blob/adc8d607643c3406b88ea9a3b4a9a537f02aa991/lib/ansible/modules/packaging/os/apt_repository.py#L352

ad 2
Probably the quickest and easiest, options:

ad 3
Not really a fan of this approach, the install script/dependencies will evolve in time, so this will introduce constant maintenance overhead. Besides, it feels like redoing somebody else's job.

Add an option not to overwrite existing runner config

I've been using your playbook to install gitlab runner.

My configuration is that the ansible playbook I use is ran against a persistent machine, and not a docker image.
Whenever I make the changes to the playbook, that includes runner config and re-run it, the role ignores existing runner configuration and overwrites it with the new runner config, as a result the old runner stops reporting and I need to re-enter all project-specific settings for that runner in gitlab.

Would it be possible and reasonable to add an option that would make this playbook skip re-registering runner if the runner is already installed and configured on the system? I.e. if config file in /etc/gitlab-runner/config.toml exists, skip registering the runner if skip registration option is enabled.

Got error on `Assemble new config.toml`

Running with config as described in example got an error FATAL: Near line 26 (last key parsed 'runners.docker.sysctls'): bare keys cannot contain '.'

Message:

ASK [riemers.gitlab-runner : Assemble new config.toml] ************************
fatal: [default]: FAILED! => {"changed": false, "msg": "failed to validate: rc:1 error:Runtime platform                                  \u001b[0;m  arch\u001b[0;m=amd64 os\u001b[0;m=linux pid\u001b[0;m=12602 revision\u001b[0;m=0e5417a3 version\u001b[0;m=12.0.1\nRunning in system-mode.                           \u001b[0;m \n                                                  \u001b[0;m \n\u001b[31;1mFATAL: Near line 26 (last key parsed 'runners.docker.sysctls'): bare keys cannot contain '.'\u001b[0;m \n"}

Error "Assemble new config.toml" with executor "docker-machine"

Hi,

I have a runner with "docker+machine" as executor, when I execute my playbook it fails on Assemble new config.toml.

The issue seems to come from the generated config :

  name = "my-gitlab-runner"
  executor = "docker+machine"
  ....
  [runners.docker+machine]
  environment = []
  [runners.docker]
    ...
  [runners.machine]
    ...

My config file done manually:

  name = "my-gitlab-runner"
  executor = "docker+machine"
  ....
  environment = []
  [runners.docker]
    ...
  [runners.machine]
    ...

My ansible config:

gitlab_runner_runners:
  - name: "my-gitlab-runner"
    executor: "docker+machine"
    docker_image: "alpine"
    extra_configs:
      runners.machine:
        MachineDriver: "digitalocean"
        ....

Any advice to fix that ?

Too many loops (Key 'runners.docker' has already been defined)

I'm using ansible-gitlab-runner version 1.2.7 in a playbook. It fails during the assemble with the following error message:

Near line 29 (last key parsed 'runners'): Key 'runners.docker' has already been defined

This may have the same cause as #45. It looks like one of the loops is called too many times, as two temporary files with the runner configuration are created (1 and 2). I think this happens in the isolate runner configuration step.

Playbook code

  tasks:
    - import_role:
        name: riemers.gitlab-runner
      tags:
        - gitlab-runner
      vars:
        # GitLab runner config
        gitlab_runner_package_name: "gitlab-runner"
        gitlab_runner_concurrent: 2
        gitlab_runner_registration_token: "1234567890"
        gitlab_runner_coordinator_url: "https://a.b.c"
        gitlab_runner_runners:
          - name:  "{{ inventory_hostname }}"
            executor: docker
            docker_image: docker
            docker_privileged: "true"  # Require for docker in docker
            tags:
              - mytag
              - anothertag

Temporary files created

ansible.6j9_YPgitlab-runner-config/
├── gitlab-runner.01.YXMYY1
├── gitlab-runner.02.0GG_Ey
├── gitlab-runner.0.qak3J1
├── gitlab-runner.1.dgnKup
└── gitlab-runner.2.FMi2sx

Concatenated files

# cat gitlab-runner.[012].*
concurrent = 2
check_interval = 0

[session_server]
  session_timeout = 1800

  name = "gitlabrunner0"
  url = "https://a.b.c"
  token = "1234567890"
  executor = "docker"
  [runners.docker]
    tls_verify = false
    image = "docker"
    privileged = true
    disable_entrypoint_overwrite = false
    oom_kill_disable = false
    disable_cache = false
    shm_size = 0
  [runners.cache]
    [runners.cache.s3]
    [runners.cache.gcs]

  name = "gitlabrunner0"
  url = "https://a.b.c"
  token = "1234567890"
  executor = "docker"
  [runners.docker]
    tls_verify = false
    image = "docker"
    privileged = true
    disable_entrypoint_overwrite = false
    oom_kill_disable = false
    disable_cache = false
    shm_size = 0
  [runners.cache]
    [runners.cache.s3]
    [runners.cache.gcs]

Full role output

TASK [riemers.gitlab-runner : Get Gitlab repository installation script] *******************************************************************
skipping: [gitlabrunner0]
skipping: [gitlabrunner1]

TASK [riemers.gitlab-runner : Install Gitlab repository] ***********************************************************************************
skipping: [gitlabrunner0]
skipping: [gitlabrunner1]

TASK [riemers.gitlab-runner : set_fact] ****************************************************************************************************
skipping: [gitlabrunner0]
skipping: [gitlabrunner1]

TASK [riemers.gitlab-runner : set_fact] ****************************************************************************************************
skipping: [gitlabrunner0]
skipping: [gitlabrunner1]

TASK [riemers.gitlab-runner : Install GitLab Runner] ***************************************************************************************
skipping: [gitlabrunner0]
skipping: [gitlabrunner1]

TASK [riemers.gitlab-runner : Get Gitlab repository installation script] *******************************************************************
ok: [gitlabrunner1]
ok: [gitlabrunner0]

TASK [riemers.gitlab-runner : Install Gitlab repository] ***********************************************************************************
ok: [gitlabrunner1]
ok: [gitlabrunner0]

TASK [riemers.gitlab-runner : set_fact] ****************************************************************************************************
skipping: [gitlabrunner0]
skipping: [gitlabrunner1]

TASK [riemers.gitlab-runner : set_fact] ****************************************************************************************************
ok: [gitlabrunner0]
ok: [gitlabrunner1]

TASK [riemers.gitlab-runner : Install GitLab Runner] ***************************************************************************************
ok: [gitlabrunner1]
ok: [gitlabrunner0]

TASK [riemers.gitlab-runner : Set concurrent option] ***************************************************************************************
ok: [gitlabrunner1]
ok: [gitlabrunner0]

TASK [riemers.gitlab-runner : Add sentry dsn to config] ************************************************************************************
skipping: [gitlabrunner0]
skipping: [gitlabrunner1]

TASK [riemers.gitlab-runner : List configured runners] *************************************************************************************
ok: [gitlabrunner1]
ok: [gitlabrunner0]

TASK [riemers.gitlab-runner : Register GitLab Runner] **************************************************************************************
included: /home/jostev/devops/gitlab-runner-centos/roles/riemers.gitlab-runner/tasks/register-runner.yml for gitlabrunner1
included: /home/jostev/devops/gitlab-runner-centos/roles/riemers.gitlab-runner/tasks/register-runner.yml for gitlabrunner0

TASK [riemers.gitlab-runner : Register runner to GitLab] ***********************************************************************************
skipping: [gitlabrunner1]

TASK [riemers.gitlab-runner : Register runner to GitLab] ***********************************************************************************
skipping: [gitlabrunner0]

TASK [riemers.gitlab-runner : Get existing config] *****************************************************************************************
ok: [gitlabrunner1]
ok: [gitlabrunner0]

TASK [riemers.gitlab-runner : set_fact] ****************************************************************************************************
ok: [gitlabrunner0]
ok: [gitlabrunner1]

TASK [riemers.gitlab-runner : Create temporary directory] **********************************************************************************
ok: [gitlabrunner1]
ok: [gitlabrunner0]

TASK [riemers.gitlab-runner : include_tasks] ***********************************************************************************************
included: /home/jostev/devops/gitlab-runner-centos/roles/riemers.gitlab-runner/tasks/config-runner.yml for gitlabrunner0, gitlabrunner1
included: /home/jostev/devops/gitlab-runner-centos/roles/riemers.gitlab-runner/tasks/config-runner.yml for gitlabrunner0
included: /home/jostev/devops/gitlab-runner-centos/roles/riemers.gitlab-runner/tasks/config-runner.yml for gitlabrunner0
included: /home/jostev/devops/gitlab-runner-centos/roles/riemers.gitlab-runner/tasks/config-runner.yml for gitlabrunner1
included: /home/jostev/devops/gitlab-runner-centos/roles/riemers.gitlab-runner/tasks/config-runner.yml for gitlabrunner1

TASK [riemers.gitlab-runner : Create temporary file] ***************************************************************************************
ok: [gitlabrunner1]
ok: [gitlabrunner0]

TASK [riemers.gitlab-runner : Isolate runner configuration] ********************************************************************************
ok: [gitlabrunner1]
ok: [gitlabrunner0]

TASK [riemers.gitlab-runner : include_tasks] ***********************************************************************************************
skipping: [gitlabrunner0] => (item={'name': 'gitlabrunner0', 'executor': 'docker', 'docker_image': 'docker', 'docker_privileged': 'true', 'tags': ['docker-builder', 'gitlabrunner']}) 
skipping: [gitlabrunner1] => (item={'name': 'gitlabrunner1', 'executor': 'docker', 'docker_image': 'docker', 'docker_privileged': 'true', 'tags': ['docker-builder', 'gitlabrunner']}) 

TASK [riemers.gitlab-runner : Remove runner config] ****************************************************************************************
skipping: [gitlabrunner0] => (item={'name': 'gitlabrunner0', 'executor': 'docker', 'docker_image': 'docker', 'docker_privileged': 'true', 'tags': ['docker-builder', 'gitlabrunner']}) 
skipping: [gitlabrunner1] => (item={'name': 'gitlabrunner1', 'executor': 'docker', 'docker_image': 'docker', 'docker_privileged': 'true', 'tags': ['docker-builder', 'gitlabrunner']}) 

TASK [riemers.gitlab-runner : Create temporary file] ***************************************************************************************
ok: [gitlabrunner0]

TASK [riemers.gitlab-runner : Isolate runner configuration] ********************************************************************************
ok: [gitlabrunner0]

TASK [riemers.gitlab-runner : include_tasks] ***********************************************************************************************
included: /home/jostev/devops/gitlab-runner-centos/roles/riemers.gitlab-runner/tasks/update-config-runner.yml for gitlabrunner0

TASK [riemers.gitlab-runner : Create temporary file] ***************************************************************************************
ok: [gitlabrunner0]

TASK [riemers.gitlab-runner : Isolate runner configuration] ********************************************************************************
ok: [gitlabrunner0]

TASK [riemers.gitlab-runner : Set concurrent limit option] *********************************************************************************
ok: [gitlabrunner0]

TASK [riemers.gitlab-runner : Set coordinator URL] *****************************************************************************************
ok: [gitlabrunner0]

TASK [riemers.gitlab-runner : Set runner executor option] **********************************************************************************
ok: [gitlabrunner0]

TASK [riemers.gitlab-runner : Set runner docker image option] ******************************************************************************
ok: [gitlabrunner0]

TASK [riemers.gitlab-runner : Set docker privileged option] ********************************************************************************
ok: [gitlabrunner0]

TASK [riemers.gitlab-runner : Set docker volumes option] ***********************************************************************************
changed: [gitlabrunner0]

TASK [riemers.gitlab-runner : Set cache type option] ***************************************************************************************
ok: [gitlabrunner0]

TASK [riemers.gitlab-runner : Set cache path option] ***************************************************************************************
ok: [gitlabrunner0]

TASK [riemers.gitlab-runner : Set cache s3 server addresss] ********************************************************************************
ok: [gitlabrunner0]

TASK [riemers.gitlab-runner : Set cache s3 access key] *************************************************************************************
ok: [gitlabrunner0]

TASK [riemers.gitlab-runner : Set cache s3 secret key] *************************************************************************************
ok: [gitlabrunner0]

TASK [riemers.gitlab-runner : Set cache shared option] *************************************************************************************
ok: [gitlabrunner0]

TASK [riemers.gitlab-runner : Set cache s3 bucket name option] *****************************************************************************
ok: [gitlabrunner0]

TASK [riemers.gitlab-runner : Set cache s3 bucket location option] *************************************************************************
ok: [gitlabrunner0]

TASK [riemers.gitlab-runner : Set cache s3 insecure option] ********************************************************************************
ok: [gitlabrunner0]

TASK [riemers.gitlab-runner : Set ssh user option] *****************************************************************************************
ok: [gitlabrunner0]

TASK [riemers.gitlab-runner : Set ssh host option] *****************************************************************************************
ok: [gitlabrunner0]

TASK [riemers.gitlab-runner : Set ssh port option] *****************************************************************************************
ok: [gitlabrunner0]

TASK [riemers.gitlab-runner : Set ssh password option] *************************************************************************************
ok: [gitlabrunner0]

TASK [riemers.gitlab-runner : Set ssh identity file option] ********************************************************************************
ok: [gitlabrunner0]

TASK [riemers.gitlab-runner : include] *****************************************************************************************************
skipping: [gitlabrunner0]

TASK [riemers.gitlab-runner : Remove runner config] ****************************************************************************************
skipping: [gitlabrunner0] => (item={'name': 'gitlabrunner0', 'executor': 'docker', 'docker_image': 'docker', 'docker_privileged': 'true', 'tags': ['docker-builder', 'gitlabrunner']}) 

TASK [riemers.gitlab-runner : Create temporary file] ***************************************************************************************
ok: [gitlabrunner0]

TASK [riemers.gitlab-runner : Isolate runner configuration] ********************************************************************************
ok: [gitlabrunner0]

TASK [riemers.gitlab-runner : include_tasks] ***********************************************************************************************
included: /home/jostev/devops/gitlab-runner-centos/roles/riemers.gitlab-runner/tasks/update-config-runner.yml for gitlabrunner0

TASK [riemers.gitlab-runner : Create temporary file] ***************************************************************************************
ok: [gitlabrunner0]

TASK [riemers.gitlab-runner : Isolate runner configuration] ********************************************************************************
ok: [gitlabrunner0]

TASK [riemers.gitlab-runner : Set concurrent limit option] *********************************************************************************
ok: [gitlabrunner0]

TASK [riemers.gitlab-runner : Set coordinator URL] *****************************************************************************************
ok: [gitlabrunner0]

TASK [riemers.gitlab-runner : Set runner executor option] **********************************************************************************
ok: [gitlabrunner0]

TASK [riemers.gitlab-runner : Set runner docker image option] ******************************************************************************
ok: [gitlabrunner0]

TASK [riemers.gitlab-runner : Set docker privileged option] ********************************************************************************
ok: [gitlabrunner0]

TASK [riemers.gitlab-runner : Set docker volumes option] ***********************************************************************************
changed: [gitlabrunner0]

TASK [riemers.gitlab-runner : Set cache type option] ***************************************************************************************
ok: [gitlabrunner0]

TASK [riemers.gitlab-runner : Set cache path option] ***************************************************************************************
ok: [gitlabrunner0]

TASK [riemers.gitlab-runner : Set cache s3 server addresss] ********************************************************************************
ok: [gitlabrunner0]

TASK [riemers.gitlab-runner : Set cache s3 access key] *************************************************************************************
ok: [gitlabrunner0]

TASK [riemers.gitlab-runner : Set cache s3 secret key] *************************************************************************************
ok: [gitlabrunner0]

TASK [riemers.gitlab-runner : Set cache shared option] *************************************************************************************
ok: [gitlabrunner0]

TASK [riemers.gitlab-runner : Set cache s3 bucket name option] *****************************************************************************
ok: [gitlabrunner0]

TASK [riemers.gitlab-runner : Set cache s3 bucket location option] *************************************************************************
ok: [gitlabrunner0]

TASK [riemers.gitlab-runner : Set cache s3 insecure option] ********************************************************************************
ok: [gitlabrunner0]

TASK [riemers.gitlab-runner : Set ssh user option] *****************************************************************************************
ok: [gitlabrunner0]

TASK [riemers.gitlab-runner : Set ssh host option] *****************************************************************************************
ok: [gitlabrunner0]

TASK [riemers.gitlab-runner : Set ssh port option] *****************************************************************************************
ok: [gitlabrunner0]

TASK [riemers.gitlab-runner : Set ssh password option] *************************************************************************************
ok: [gitlabrunner0]

TASK [riemers.gitlab-runner : Set ssh identity file option] ********************************************************************************
ok: [gitlabrunner0]

TASK [riemers.gitlab-runner : include] *****************************************************************************************************
skipping: [gitlabrunner0]

TASK [riemers.gitlab-runner : Remove runner config] ****************************************************************************************
skipping: [gitlabrunner0] => (item={'name': 'gitlabrunner0', 'executor': 'docker', 'docker_image': 'docker', 'docker_privileged': 'true', 'tags': ['docker-builder', 'gitlabrunner']}) 

TASK [riemers.gitlab-runner : Create temporary file] ***************************************************************************************
ok: [gitlabrunner1]

TASK [riemers.gitlab-runner : Isolate runner configuration] ********************************************************************************
ok: [gitlabrunner1]

TASK [riemers.gitlab-runner : include_tasks] ***********************************************************************************************
included: /home/jostev/devops/gitlab-runner-centos/roles/riemers.gitlab-runner/tasks/update-config-runner.yml for gitlabrunner1

TASK [riemers.gitlab-runner : Create temporary file] ***************************************************************************************
ok: [gitlabrunner1]

TASK [riemers.gitlab-runner : Isolate runner configuration] ********************************************************************************
ok: [gitlabrunner1]

TASK [riemers.gitlab-runner : Set concurrent limit option] *********************************************************************************
ok: [gitlabrunner1]

TASK [riemers.gitlab-runner : Set coordinator URL] *****************************************************************************************
ok: [gitlabrunner1]

TASK [riemers.gitlab-runner : Set runner executor option] **********************************************************************************
ok: [gitlabrunner1]

TASK [riemers.gitlab-runner : Set runner docker image option] ******************************************************************************
ok: [gitlabrunner1]

TASK [riemers.gitlab-runner : Set docker privileged option] ********************************************************************************
ok: [gitlabrunner1]

TASK [riemers.gitlab-runner : Set docker volumes option] ***********************************************************************************
changed: [gitlabrunner1]

TASK [riemers.gitlab-runner : Set cache type option] ***************************************************************************************
ok: [gitlabrunner1]

TASK [riemers.gitlab-runner : Set cache path option] ***************************************************************************************
ok: [gitlabrunner1]

TASK [riemers.gitlab-runner : Set cache s3 server addresss] ********************************************************************************
ok: [gitlabrunner1]

TASK [riemers.gitlab-runner : Set cache s3 access key] *************************************************************************************
ok: [gitlabrunner1]

TASK [riemers.gitlab-runner : Set cache s3 secret key] *************************************************************************************
ok: [gitlabrunner1]

TASK [riemers.gitlab-runner : Set cache shared option] *************************************************************************************
ok: [gitlabrunner1]

TASK [riemers.gitlab-runner : Set cache s3 bucket name option] *****************************************************************************
ok: [gitlabrunner1]

TASK [riemers.gitlab-runner : Set cache s3 bucket location option] *************************************************************************
ok: [gitlabrunner1]

TASK [riemers.gitlab-runner : Set cache s3 insecure option] ********************************************************************************
ok: [gitlabrunner1]

TASK [riemers.gitlab-runner : Set ssh user option] *****************************************************************************************
ok: [gitlabrunner1]

TASK [riemers.gitlab-runner : Set ssh host option] *****************************************************************************************
ok: [gitlabrunner1]

TASK [riemers.gitlab-runner : Set ssh port option] *****************************************************************************************
ok: [gitlabrunner1]

TASK [riemers.gitlab-runner : Set ssh password option] *************************************************************************************
ok: [gitlabrunner1]

TASK [riemers.gitlab-runner : Set ssh identity file option] ********************************************************************************
ok: [gitlabrunner1]

TASK [riemers.gitlab-runner : include] *****************************************************************************************************
skipping: [gitlabrunner1]

TASK [riemers.gitlab-runner : Remove runner config] ****************************************************************************************
skipping: [gitlabrunner1] => (item={'name': 'gitlabrunner1', 'executor': 'docker', 'docker_image': 'docker', 'docker_privileged': 'true', 'tags': ['docker-builder', 'gitlabrunner']}) 

TASK [riemers.gitlab-runner : Create temporary file] ***************************************************************************************
ok: [gitlabrunner1]

TASK [riemers.gitlab-runner : Isolate runner configuration] ********************************************************************************
ok: [gitlabrunner1]

TASK [riemers.gitlab-runner : include_tasks] ***********************************************************************************************
included: /home/jostev/devops/gitlab-runner-centos/roles/riemers.gitlab-runner/tasks/update-config-runner.yml for gitlabrunner1

TASK [riemers.gitlab-runner : Create temporary file] ***************************************************************************************
ok: [gitlabrunner1]

TASK [riemers.gitlab-runner : Isolate runner configuration] ********************************************************************************
ok: [gitlabrunner1]

TASK [riemers.gitlab-runner : Set concurrent limit option] *********************************************************************************
ok: [gitlabrunner1]

TASK [riemers.gitlab-runner : Set coordinator URL] *****************************************************************************************
ok: [gitlabrunner1]

TASK [riemers.gitlab-runner : Set runner executor option] **********************************************************************************
ok: [gitlabrunner1]

TASK [riemers.gitlab-runner : Set runner docker image option] ******************************************************************************
ok: [gitlabrunner1]

TASK [riemers.gitlab-runner : Set docker privileged option] ********************************************************************************
ok: [gitlabrunner1]

TASK [riemers.gitlab-runner : Set docker volumes option] ***********************************************************************************
changed: [gitlabrunner1]

TASK [riemers.gitlab-runner : Set cache type option] ***************************************************************************************
ok: [gitlabrunner1]

TASK [riemers.gitlab-runner : Set cache path option] ***************************************************************************************
ok: [gitlabrunner1]

TASK [riemers.gitlab-runner : Set cache s3 server addresss] ********************************************************************************
ok: [gitlabrunner1]

TASK [riemers.gitlab-runner : Set cache s3 access key] *************************************************************************************
ok: [gitlabrunner1]

TASK [riemers.gitlab-runner : Set cache s3 secret key] *************************************************************************************
ok: [gitlabrunner1]

TASK [riemers.gitlab-runner : Set cache shared option] *************************************************************************************
ok: [gitlabrunner1]

TASK [riemers.gitlab-runner : Set cache s3 bucket name option] *****************************************************************************
ok: [gitlabrunner1]

TASK [riemers.gitlab-runner : Set cache s3 bucket location option] *************************************************************************
ok: [gitlabrunner1]

TASK [riemers.gitlab-runner : Set cache s3 insecure option] ********************************************************************************
ok: [gitlabrunner1]

TASK [riemers.gitlab-runner : Set ssh user option] *****************************************************************************************
ok: [gitlabrunner1]

TASK [riemers.gitlab-runner : Set ssh host option] *****************************************************************************************
ok: [gitlabrunner1]

TASK [riemers.gitlab-runner : Set ssh port option] *****************************************************************************************
ok: [gitlabrunner1]

TASK [riemers.gitlab-runner : Set ssh password option] *************************************************************************************
ok: [gitlabrunner1]

TASK [riemers.gitlab-runner : Set ssh identity file option] ********************************************************************************
ok: [gitlabrunner1]

TASK [riemers.gitlab-runner : include] *****************************************************************************************************
skipping: [gitlabrunner1]

TASK [riemers.gitlab-runner : Remove runner config] ****************************************************************************************
skipping: [gitlabrunner1] => (item={'name': 'gitlabrunner1', 'executor': 'docker', 'docker_image': 'docker', 'docker_privileged': 'true', 'tags': ['docker-builder', 'gitlabrunner']}) 

TASK [riemers.gitlab-runner : copy] ********************************************************************************************************
ok: [gitlabrunner1]
ok: [gitlabrunner0]

TASK [riemers.gitlab-runner : assemble] ****************************************************************************************************
fatal: [gitlabrunner1]: FAILED! => {"changed": false, "msg": "failed to validate: rc:1 error:Runtime platform                                  \u001b[0;m  arch\u001b[0;m=amd64 os\u001b[0;m=linux pid\u001b[0;m=5489 revision\u001b[0;m=6946bae7 version\u001b[0;m=12.0.0\nRunning in system-mode.                           \u001b[0;m \n                                                  \u001b[0;m \n\u001b[31;1mFATAL: Near line 29 (last key parsed 'runners'): Key 'runners.docker' has already been defined.\u001b[0;m \n"}
fatal: [gitlabrunner0]: FAILED! => {"changed": false, "msg": "failed to validate: rc:1 error:Runtime platform                                  \u001b[0;m  arch\u001b[0;m=amd64 os\u001b[0;m=linux pid\u001b[0;m=18261 revision\u001b[0;m=6946bae7 version\u001b[0;m=12.0.0\nRunning in system-mode.                           \u001b[0;m \n                                                  \u001b[0;m \n\u001b[31;1mFATAL: Near line 29 (last key parsed 'runners'): Key 'runners.docker' has already been defined.\u001b[0;m \n"}

Register re-runs on every execution

Thank you for this great role. I noticed one thing when I was attempting to run my playbook on a Redhat based machine. It looks as if the register runner task was running every time despite the conditionals. This causes extra runners in the config.toml and in gitlab. I think the issue is in the Unix.yml file for both the List configured runners and the Check runner is registered tasks. I believe that a become:yes is needed on both of these.

It seems that we are checking if the runner is already registered but it is looking in the current user's home directory's config.toml which is empty because the role is updating the global one.

Do not work on Ubuntu 16.04

Do not work on Ubuntu 16.04. Failed, when play:

TASK [riemers.gitlab-runner : Get Gitlab repository installation script] *************************************************************************************************
fatal: [gitlab-runners]: FAILED! => {"msg": "The conditional check 'ansible_os_family == 'Debian'' failed. The error was: error while evaluating conditional (ansible_os_family == 'Debian'): 'ansible_os_family' is undefined\n\nThe error appears to have been in '/Users/user/Git/otus-project/search_engine_infra/ansible/roles/riemers.gitlab-runner/tasks/install-debian.yml': line 3, column 3, but may\nbe elsewhere in the file depending on the exact syntax problem.\n\nThe offending line appears to be:\n\n\n- name: Get Gitlab repository installation script\n  ^ here\n"}

How do I unregister a runner?

This is a minor issue, but I was wondering if there was a way to unregister a runner? I set one of my runners to state: absent, but in my Gitlab CI/CD UI, I still my runner as registered even though it's not running on the my gitlab runner. I assumed that it would be unregistered when I set the state to absent.

Only first Docker volume added to runner configuration

The following vars definition

gitlab_runner_docker_volumes:                  
  - "/cache"                                   
  - "/tmp"

results in a registration command like this:

gitlab-runner register ... --docker-volumes "/cache" "/tmp" ...

which does not seem to have the expected result—i.e. only /cache is being added as a volume in /etc/gitlab-runner/config.toml instead of both /cache and /tmp.

With the following code in place at

{% if gitlab_runner_docker_volumes is defined %}
it works as expected (both volumes show up in config.toml):

    {% for volume in gitlab_runner_docker_volumes | default([]) %}
    --docker-volumes "{{ volume }}"
    {% endfor %}

$ gitlab-runner --version
Version:      11.1.0
Git revision: 081978aa
Git branch:   
GO version:   go1.8.7
Built:        2018-07-22T07:24:46+00:00
OS/Arch:      linux/amd64

Example does not work

It seems that the following example playbook from the readme cannot be run:

gitlab_runner_registration_token: 'HUzTMgnxk17YV8Rj8ucQ'
gitlab_runner_runners:
  - name: 'Example Docker GitLab Runner'
    executor: docker
    tags:
      - node
      - ruby
      - mysql
    docker_volumes:
      - "/var/run/docker.sock:/var/run/docker.sock"
      - "/cache"
    extra_configs:
      runners.docker:
        memory: 512m
        allowed_images: ["ruby:*", "python:*", "php:*"]
      runners.docker.sysctls:
        net.ipv4.ip_forward: "1"

As the docker image is not specified and not defined in the default. Extract from defaults:

# The default Docker image to use. Required when executor is `docker`.
    docker_image: ''

I get the following error from ansible:

The docker-image needs to be entered

Possible fix is to update the example to:

gitlab_runner_registration_token: 'HUzTMgnxk17YV8Rj8ucQ'
gitlab_runner_runners:
  - name: 'Example Docker GitLab Runner'
    executor: docker
    docker_image: 'alpine'
    tags:
      - node
      - ruby
      - mysql
    docker_volumes:
      - "/var/run/docker.sock:/var/run/docker.sock"
      - "/cache"
    extra_configs:
      runners.docker:
        memory: 512m
        allowed_images: ["ruby:*", "python:*", "php:*"]
      runners.docker.sysctls:
        net.ipv4.ip_forward: "1"

Role fails when using a vault-encrypted token variable

The check for the token in tasks/main.yml produces the following error if gitlab_runner_registration_token is encrypted with ansible vault:

object of type 'AnsibleVaultEncryptedUnicode' has no len()

According to ansible/ansible#33067, it looks like we can just throw a |string in there to work around it. I'll submit a PR.

'dict object' has no attribute 'executor'

After updating gitlab-runner from version 1.2.3 to 1.6.17 I see the following error:

FAILED! => {"msg": "The task includes an option with an undefined variable. The error was: 'dict object' has no attribute 'executor'\n\nThe error appears to have been in '/home/fxg/.ansible/roles/riemers.gitlab-runner/tasks/update-config-runner.yml': line 82, column 3, but may\nbe elsewhere in the file depending on the exact syntax problem.\n\nThe offending line appears to be:\n\n\n- name: Set runner executor section\n  ^ here\n"}

Explicitly adding the default option shell has resolved the problem.

 gitlab_runner_runners:
   - name: '{{ runner_name }}'
     tags:
       - python3
       - host-iron
+    executor: shell

Has this been changed to a mandatory value? From what I see in the failed task, it seems that it should default to shell anyway.

Dubble runners.machine config with docker+machine

Hello,

I'm using your role (master branch) for installing my gitlab-runner with docker+machine on gcs.
I ran into the issue that runners.machine already exists in my configuration.

After checking the temp config file i do see a double runners.machine entry:

cat gitlab-runner.1.1kG9jL
  name = "gitlab-runner01"
  limit = 0
  output_limit = 4096
  url = "<url>"
  environment = []
  token = "<token>"
  executor = "docker+machine"
  [runners.machine]
  [runners.machine]
    MachineOptions = ["google-project=<id>", "google-machine-type=n1-standard-1", "google-machine-image=https://www.googleapis.com/compute/v1/projects/ubuntu-os-cloud/global/images/family/ubuntu-minimal-1804-lts", "google-tags=gitlab-ci-slave", "google-preemptible=true", "google-zone=europe-west1-d"]
    MachineName = "auto-scale-runner-%s"
    MachineDriver = "google"
    MaxBuilds = 100
    IdleTime = 3600
    IdleCount = 1
  [runners.custom_build_dir]
  [runners.cache]
  [runners.docker]
    image = "ubuntu:latest"
    privileged = true
    disable_entrypoint_overwrite = false
    oom_kill_disable = false
    disable_cache = false
    volumes = ["/var/run/docker.sock:/var/run/docker.sock", "/cache"]
    shm_size = 0

Found following tasks that creates the runners.machine section.

- name: Set runner executor section
  lineinfile:
    dest: "{{ temp_runner_config.path }}"
    regexp: '^\s*\[runners\.{{ gitlab_runner.executor|default("shell") }}\]'
    line: '  [runners.{{ gitlab_runner.executor|replace("docker+machine","machine")|default("shell") }}]'
    state: "{{ 'absent' if (gitlab_runner.executor|default('shell')) == 'shell' else 'present' }}"
    insertafter: '^\s*executor ='
    backrefs: no
  check_mode: no
  notify:
  - restart_gitlab_runner
  - restart_gitlab_runner_macos

This can be skipped if the executor is docker+machine as you loop over the extra_configs in the vars and insert a second runners.machine.

  - name: 'gitlab-runner01'
    state: present
    # token is an optional override to the global gitlab_runner_registration_token
    executor: 'docker+machine'
    # Maximum number of jobs to run concurrently on this specific runner.
    docker_image: 'ubuntu:latest'
    docker_volumes: ["/var/run/docker.sock:/var/run/docker.sock","/cache"]
    docker_privileged: true
    tags:
      - docker
      - dind
    # Indicates whether this runner can pick jobs without tags.
    run_untagged: true
    locked: false
    extra_configs:
      runners.machine:
        IdleCount: 1
        IdleTime: 3600
        MaxBuilds: 100
        MachineDriver: 'google'
        MachineName: 'auto-scale-runner-%s'
        MachineOptions: ["google-project={{ gcloud_project_id }}",
                          "google-machine-type=n1-standard-1",
                          "google-machine-image=https://www.googleapis.com/compute/v1/projects/ubuntu-os-cloud/global/images/family/ubuntu-minimal-1804-lts",
                          "google-tags=gitlab-ci-slave",
                          "google-preemptible=true",
                          "google-zone=europe-west1-d"]

After i added the when statement to the task, my problem has been solved and deploy of the runner was successful.

- name: Set runner executor section
  lineinfile:
    dest: "{{ temp_runner_config.path }}"
    regexp: '^\s*\[runners\.{{ gitlab_runner.executor|default("shell") }}\]'
    line: '  [runners.{{ gitlab_runner.executor|replace("docker+machine","machine")|default("shell") }}]'
    state: "{{ 'absent' if (gitlab_runner.executor|default('shell')) == 'shell' else 'present' }}"
    insertafter: '^\s*executor ='
    backrefs: no
  when: gitlab_runner.executor != "docker+machine"
  check_mode: no
  notify:
  - restart_gitlab_runner
  - restart_gitlab_runner_macos

Does not update to latest version

When re-running a playbook with the gitlab-runner role, the version of the deployed runner is not updated to the lastest version.

I understand from documentation, that this should be the case. The README says (emphasis added):

  • gitlab_runner_wanted_version or gitlab_runner_package_version - To install a specific version of the gitlab runner (by default it installs the latest). On Mac OSX and Windows, use e.g. gitlab_runner_wanted_version: 12.4.1. On Linux, use gitlab_runner_package_version instead.

Are there any caveats? (or is this my user error?)

The runner had been originally deployed with gitlab-runner 1.2.3 and I have tried re-deploying with the same version and the latest 1.6.17.

Missing sudo rights

Thanks for providing this role!
Unfortunately, I am getting this error:

TASK [riemers.gitlab-runner : (Debian) Install GitLab Runner] *************************************************************************************************************************************************************************************************
fatal: [78.46.244.166]: FAILED! => {"cache_update_time": 1571013460, "cache_updated": false, "changed": false, "msg": "'/usr/bin/apt-get -y -o \"Dpkg::Options::=--force-confdef\" -o \"Dpkg::Options::=--force-confold\"      install 'gitlab-runner'' failed: E: Could not open lock file /var/lib/dpkg/lock - open (13: Permission denied)\nE: Unable to lock the administration directory (/var/lib/dpkg/), are you root?\n", "rc": 100, "stderr": "E: Could not open lock file /var/lib/dpkg/lock - open (13: Permission denied)\nE: Unable to lock the administration directory (/var/lib/dpkg/), are you root?\n", "stderr_lines": ["E: Could not open lock file /var/lib/dpkg/lock - open (13: Permission denied)", "E: Unable to lock the administration directory (/var/lib/dpkg/), are you root?"], "stdout": "", "stdout_lines": []}

To me it looks like you are missing become: true for the Install GitLab Runner play in install-debian.yml (and in install-redhat.yml too).
Even when I fix this there seem to be more missing become: true lines in a few places where /etc/gitlab-runner/config.toml is modified:

  • At Get existing config.toml in tasks/config-runners.yml
  • In all of the three plays in tasks/global-setup.yml

Am I supposed to run the whole role with become: true ? And if yes why are there some become: true lines in your role then (e.g. in the Install Gitlab repository play in install-debian.yml) ?

fix typo in tasks/global-setup.yml

file has extra line in the EOF with a single "-" character. this fails ansible-lint as it does not like the empty task

16:55 $ cat roles/riemers.gitlab-runner/tasks/global-setup.yml
---
- name: Set concurrent option
  lineinfile:
    dest: /etc/gitlab-runner/config.toml
   regexp: '^(\s*)concurrent ='
    line: '\1concurrent = {{ gitlab_runner_concurrent }}'
    state: present
    backrefs: yes
  notify: restart_gitlab_runner

- name: Add sentry dsn to config
  lineinfile:
    dest: /etc/gitlab-runner/config.toml
    regexp: '^sentry_dsn ='
    line: 'sentry_dsn = "{{ gitlab_runner_sentry_dsn }}"'
   insertafter: '\s*concurrent.*'
   state: present
  when: gitlab_runner_sentry_dsn != ""
  notify: restart_gitlab_runner
- 



16:54 $ ansible-lint site.yml 
Traceback (most recent call last):
  File "/usr/bin/ansible-lint", line 11, in <module>
    load_entry_point('ansible-lint==3.4.20', 'console_scripts', 'ansible-lint')()
  File "/usr/lib/python2.7/dist-packages/ansiblelint/__main__.py", line 173, in main
matches.extend(runner.run())
  File "/usr/lib/python2.7/dist-packages/ansiblelint/__init__.py", line 229, in run
    for child in ansiblelint.utils.find_children(arg, self.playbook_dir):
  File "/usr/lib/python2.7/dist-packages/ansiblelint/utils.py", line 158, in find_children
    items = _playbook_items(playbook_ds)
  File "/usr/lib/python2.7/dist-packages/ansiblelint/utils.py", line 143, in _playbook_items
    return [item for play in pb_data for item in play.items()]
AttributeError: 'NoneType' object has no attribute 'items'

Unused parameters

Hi. I'm not using the SSH parameters in the gitlab register task. Basically, the command is running on a dedicated instance for a gitlab runner, so no remote connection. Is there a way to avoid using the SSH parameters? Ideally, the task should use only the parameters that have values.

Unspecified runner is added instead of updating existing one

After updating this role today, an additional runner has been installed which I have not defined.

The vars say:

runner_name: hiccup-notebook
gitlab_runner_runners:
  - name: '{{ runner_name }}'
    tags:
      - python3
      - host-iron
    executor: shell

The runner hiccup-notebook had previously been installed with gitlab-runner v1.2.3. Today I am trying to update it to the latest version and have subsequently updated gitlab-runner to v1.6.17.

After running the playbook I see in Gitlab are two runners on the same host:

  • hiccup-notebook on version 12.10.1 with tags python3 and host-iron
  • lxc-gitlab-runner-hiccup-notebook-1 on version 13.2.1 without any tags.

12.10.1 is the old version I am trying to update from.
13.2.1 is the version I want the runner hiccup-notebook to be updated to.

The name of the unexpected runner seems to be automatically picked up from the hostname lxc-gitlab-runner-hiccup-notebook.
Note that this runner is also missing the tags.

Both runners show up in the log when running the playbook:

TASK [riemers.gitlab-runner : (Windows) Write config section for each runner] *********************************************************************************************************************************************************************************************
skipping: [gitlab-runner-hiccup-notebook] => (item=concurrent = 16
check_interval = 0

[session_server]
  session_timeout = 1800

)
skipping: [gitlab-runner-hiccup-notebook] => (item=

  name = "hiccup-notebook"
  limit = 0
  url = "https://gitlab....com/"
  environment = []
  token = "t0kEn"
  executor = "shell"
  output_limit = 4096
  [runners.custom_build_dir]
  [runners.cache]

)
skipping: [gitlab-runner-hiccup-notebook] => (item=

  name = "lxc-gitlab-runner-hiccup-notebook-1"
  output_limit = 4096
  url = "https://gitlab....com/"
  token = "t0kEn"
  executor = "shell"
  [runners.custom_build_dir]
  [runners.cache]
    [runners.cache.s3]
    [runners.cache.gcs]
)

Where does this additional runner come from and how can I avoid it?

Also, how can I make sure the originally installed runner is updated?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.