centos-paas-sig / contra-env-setup Goto Github PK

View Code? Open in Web Editor NEW

12.0 12.0 29.0 505 KB

This repo is for setting up the continuous-infra pipeline environment.

License: GNU General Public License v3.0

Python 32.12% Groovy 8.97% Dockerfile 11.41% Shell 16.30% Jinja 31.20%

contra-env-setup's People

Contributors

Stargazers

Watchers

contra-env-setup's Issues

Upgrade Minishift CentOS image

Minishift v1.13.0+ no longer works with the current custom CentOS image. We need at a fix for minishift/minishift-centos-iso#216. In CVP I've tested with https://github.com/minishift/minishift-centos-iso/releases/download/v1.10.0/minishift-centos7.iso and so far everything works as expected.

Add an example for remote configuration to README

The README currenty only includes examples of using minishift locally, since deploying to a remote OpenShift cluster is a supported use case, we should include an example in the README for how to do that.

We need to have the ansible-executor and linchpin-executor as part of the contra-env-setup

We have certain helper container images we use in minishift or OpenShift setup as part of the contra-env-setup.

ansible-executor allows us to execute ansible playbooks/roles
linchpin-executor allows us to provision resources in clouds and bare metal infrastructures.

@robnester-rh @scoheb make sense?

Use locally installed `oc` binary if its available

When working with a remote OpenShift cluster, the playbook always installs the oc binary in its own. It desirable the use the locally installed one if its available in $PATH to save some time.

Logging in to OpenShift breaks if username or password contain spaces

If the username of password that are user to login to OpenShift contain any spaces, the login would always fail, because the executed command line does not include quoting of the username or password values.

Fail if prerequisites are missing but installation is skipped

The run_prereqs variable allows one to skip the installation of prerequisites on the machine Ansible is running from. This is useful when one is using the playbook for managing a remote OpenShift instance and does not desire to grant the playbook root access to the local machine.

When prerequisites are missing it can result in strange and unexpected outcomes, for example if the jq command is missing, the playbook would finis successfully, but various commands it triggered would have not properly done what they needed to do.

It is desirable to make sure the playbook fails predictably if prerequisites are missing to avoid unexpected results

Add fedora 29 to testing infrastructure

Final freeze for fedora 29 was few days ago [1] and should be released at the enn of this month.

The image in docker hub docker.io/fedora:29 is quite outdated (Upgrade 57 Packages + Install 34 Packages). It is possible to either update all packages or to use image from fedora registry
registry.fedoraproject.org/fedora:29

[1] https://fedoraproject.org/wiki/Releases/29/Schedule

Add the ability to specify adhoc oc commands

With the CVP project, we need to create and apply some Openshift settings that are specific to the CVP project. Rather than have an additional playbook that runs these commands and re-create the Openshift context outside of contra-env-setup (login, url, etc) , it would be great to be able to specify a set of playbooks that would run in the same context as the initial deployment.

Here is an example of a resource we need created:

oc create -f pipeline/config/openshift/quota_management_role.yml

and then we need to apply to some users:

oc adm policy add-cluster-role-to-user quota-management-role ${OCP_USER}

Note that this can extend to any possible deployment command and will be quite powerful.

Therefore, the requirement is:

Given a contra-env-setup deployment
And a list of playbooks defined as a configuration parameter (-e playbook_hooks="config/play1.yml, config/play2.yml")
When the deployment is executed and completes,
Then the list of playbooks are executed with the same context as the initial deployment.

contra-hdsl shared library

The automation should support a DEV environment with contra-hdsl configured as jenkins shared libraries. @ifireball

Jenkins persistent template overriding PV content with image content and boot

The Jenkins persistent template sets environment variables that are responsible for overriding the content of the PV with default content stored in the image. This results in lost jobs and configuration when the Jenkins instance is redeployed in OpenShift which is not exactly what "persistent" means 😃

contra-env-setup/playbooks/roles/os_temps/templates/jenkins-persistent-template.yaml.j2

Lines 161 to 164 in 1be10ff

    
           - name: OVERRIDE_PV_CONFIG_WITH_IMAGE_CONFIG 
        
             value: 'true' 
        
           - name: OVERRIDE_PV_PLUGINS_WITH_IMAGE_PLUGINS 
        
             value: 'true'

Environment variables in question: OVERRIDE_PV_CONFIG_WITH_IMAGE_CONFIG and OVERRIDE_PV_PLUGINS_WITH_IMAGE_PLUGINS. These should be unset or set to false.

Docs: https://github.com/openshift/jenkins#environment-variables

Relevant code:

https://github.com/openshift/jenkins/blob/5a15d42e285de8ff39993b8b2d74c749fa1647f1/2/contrib/s2i/run#L373-L401

Method to clean old deploys

How to reproduce this problem:
1) clean your environment:
$ sed -i".bak" '/minishift/d' ~/.bashrc ; sed -i".bak" '/oc/d' ~/.bashrc ; rm -rf ~/.minishift ; rm -rf ~/minishift ; mkdir ~/minishift ; cp ~/ISOs/minishift.iso ~/minishift/

2) run a new deploy:
$ ansible-playbook -i "localhost," setup.yml -e setup_jenkins=false -e setup_fedmsg=false -e setup_containers=false -e modify_scc=false -e force_repo_clone=false -e remote_user=firemanxbr -k

TASK [minishift : Initialization of minishift cluster with profile minishift] ***************************
fatal: [localhost]: FAILED! => {"changed": true, "cmd": "/home/firemanxbr/minishift/minishift start --profile minishift --disk-size 40gb --memory 6400mb --iso-url file:////home/firemanxbr/minishift/minishift.iso", "delta": "0:00:03.300875", "end": "2018-02-21 14:11:24.919859", "msg": "non-zero return code", "rc": 1, "start": "2018-02-21 14:11:21.618984", "stderr": "E0221 14:11:24.916297 20549 start.go:368] Error starting the VM: Error creating the VM. Error creating machine: Error in driver during machine creation: [Code-9] [Domain-20] operation failed: domain 'minishift' already exists with uuid 9c99a8d4-278b-4bd0-bb3a-6a3d66544971. Retrying.\nError starting the VM: Error creating the VM. Error creating machine: Error in driver during machine creation: [Code-9] [Domain-20] operation failed: domain 'minishift' already exists with uuid 9c99a8d4-278b-4bd0-bb3a-6a3d66544971", "stderr_lines": ["E0221 14:11:24.916297 20549 start.go:368] Error starting the VM: Error creating the VM. Error creating machine: Error in driver during machine creation: [Code-9] [Domain-20] operation failed: domain 'minishift' already exists with uuid 9c99a8d4-278b-4bd0-bb3a-6a3d66544971. Retrying.", "Error starting the VM: Error creating the VM. Error creating machine: Error in driver during machine creation: [Code-9] [Domain-20] operation failed: domain 'minishift' already exists with uuid 9c99a8d4-278b-4bd0-bb3a-6a3d66544971"], "stdout": "-- Starting profile 'minishift'\n-- Checking if requested hypervisor 'kvm' is supported on this platform ... OK\n-- Checking if KVM driver is installed ... \n Driver is available at /usr/local/bin/docker-machine-driver-kvm ... \n Checking driver binary is executable ... OK\n-- Checking if Libvirt is installed ... OK\n-- Checking if Libvirt default network is present ... OK\n-- Checking if Libvirt default network is active ... OK\n-- Checking the ISO URL ... OK\n-- Starting local OpenShift cluster using 'kvm' hypervisor ...\n-- Minishift VM will be configured with ...\n Memory: 6 GB\n vCPUs : 2\n Disk size: 40 GB\n-- Starting Minishift VM ..... FAIL ", "stdout_lines": ["-- Starting profile 'minishift'", "-- Checking if requested hypervisor 'kvm' is supported on this platform ... OK", "-- Checking if KVM driver is installed ... ", " Driver is available at /usr/local/bin/docker-machine-driver-kvm ... ", " Checking driver binary is executable ... OK", "-- Checking if Libvirt is installed ... OK", "-- Checking if Libvirt default network is present ... OK", "-- Checking if Libvirt default network is active ... OK", "-- Checking the ISO URL ... OK", "-- Starting local OpenShift cluster using 'kvm' hypervisor ...", "-- Minishift VM will be configured with ...", " Memory: 6 GB", " vCPUs : 2", " Disk size: 40 GB", "-- Starting Minishift VM ..... FAIL "]}
to retry, use: --limit @/home/firemanxbr/GitHub/contra-env-setup/playbooks/setup.retry

PLAY RECAP **********************************************************************************************
localhost : ok=25 changed=12 unreachable=0 failed=1

Possible solution:

Create a task to remove the 'minishift' profile and clean the local environment:
$ minishift delete --profile minishift --force
Deleting the Minishift VM...
Minishift VM deleted.

$ sed -i".bak" '/minishift/d' ~/.bashrc ; sed -i".bak" '/oc/d' ~/.bashrc ; rm -rf ~/.minishift ; rm -rf ~/minishift ; mkdir ~/minishift ; cp ~/ISOs/minishift.iso ~/minishift/

$ ansible-playbook -i "localhost," setup.yml -e setup_jenkins=false -e setup_fedmsg=false -e setup_containers=false -e modify_scc=false -e force_repo_clone=false -e remote_user=firemanxbr -k

Didn't insert into '~/.bashrc' the path to 'oc' and 'minishift'

I ran the command below:

$ ansible-playbook -i "localhost," contra-env-setup/playbooks/setup.yml --vault-password-file=pipeline/config/password.txt --connection=local -K -e @pipeline/config/contra-env-setup.yml -e BASE64_ANSIBLE_VAULT_PASSWORD="$(cat pipeline/config/password.txt | base64 )" -e @pipeline/config/ansible/vault.yml -e user=$USER --extra-vars='{"hooks": ["{{ project_dir }}/config/post_deployment_tasks.yml"]}'

The output of my ~/.bashrc:
$ cat ~/.bashrc
if [ -f /etc/bashrc ]; then
. /etc/bashrc
fi

Minishift exists in this path: $ ~/.contra-env-setup/minishift/minishift
'oc' exist in this path: $ ~/.minishift/cache/oc/v3.9.0/linux/oc

New releases of contra-env-setup should be associated with a tag

As features are added to contra-env-setup, the CVP project wants to rely on release tags so as to not be affected by non-CVP tested master commits.

Can you create release tags in the project?

Namespace definitions in templates are not set via the variable

The variable openshift_project is used to determine which Kubernetes namespace are we installing things into.

That information is, however, not being passed along when instantiating OpenShift templates to create the various object that are being created.

As a result, the created objects may end up referring to the wrong or even non-existent namespaces. For example the Jenkins deployment can end up being configured to trigger on changed to the jenkins image stream in the contra-sample-project namespace (The default value for the NAMESPACE parameter) as opposed to the image stream that was actually created by by the playbook.

It is theoretically possible to customize the parameters being passed to the template, but merging PR #112 is required to allow that in practice and it may be very cumbersome to do customization in practice as one needs to specify all the relevant template parameters when customizing, as opposed to jest the ones that need to be customized.

Could not set oc CLI context for 'minishift' profile

I'm not sure if this is an issue on my side or not, but I'm seeing an error like this:

Could not set oc CLI context for 'minishift' profile: Error during setting 'minishift' as active profile: The specified path to the kube config '/home/goldmann/.minishift/machines/minishift_kubeconfig' does not exist

Log:

TASK [os_temps : set_fact] ***************************************************************************************************************************************************************************************************************************************************************
task path: /home/goldmann/git/redhat/contra-env-setup/playbooks/roles/os_temps/tasks/main.yml:5
ok: [localhost] => {"ansible_facts": {"run_time": "20190118T115223"}, "changed": false}

TASK [os_temps : Create log directory] ***************************************************************************************************************************************************************************************************************************************************
task path: /home/goldmann/git/redhat/contra-env-setup/playbooks/roles/os_temps/tasks/main.yml:9
changed: [localhost] => {"changed": true, "gid": 1000, "group": "goldmann", "mode": "0755", "owner": "goldmann", "path": "/tmp/contra-env-setup/logs/run-20190118T115223", "secontext": "unconfined_u:object_r:user_tmp_t:s0", "size": 40, "state": "directory", "uid": 1000}

TASK [os_temps : Check if the minishift profile is already up and running] ***************************************************************************************************************************************************************************************************************
task path: /home/goldmann/git/redhat/contra-env-setup/playbooks/roles/os_temps/tasks/start_mcluster.yml:5
changed: [localhost] => {"changed": true, "cmd": "/home/goldmann/.contra-env-setup/minishift/minishift status --profile minishift | head -1 | awk '{print $2}'", "delta": "0:00:00.046462", "end": "2019-01-18 11:52:29.109389", "rc": 0, "start": "2019-01-18 11:52:29.062927", "stderr": "", "stderr_lines": [], "stdout": "Stopped", "stdout_lines": ["Stopped"]}

TASK [os_temps : Start minishift profile minishift] **************************************************************************************************************************************************************************************************************************************
task path: /home/goldmann/git/redhat/contra-env-setup/playbooks/roles/os_temps/tasks/start_mcluster.yml:10
fatal: [localhost]: FAILED! => {"changed": true, "cmd": "/home/goldmann/.contra-env-setup/minishift/minishift start --profile minishift --cpus 4 --disk-size 40gb --memory 8092mb --openshift-version v3.11.0 --iso-url file:////home/goldmann/.contra-env-setup/minishift/minishift.iso", "delta": "0:03:47.311957", "end": "2019-01-18 11:56:16.556755", "msg": "non-zero return code", "rc": 1, "start": "2019-01-18 11:52:29.244798", "stderr": "Could not set oc CLI context for 'minishift' profile: Error during setting 'minishift' as active profile: The specified path to the kube config '/home/goldmann/.minishift/machines/minishift_kubeconfig' does not exist", "stderr_lines": ["Could not set oc CLI context for 'minishift' profile: Error during setting 'minishift' as active profile: The specified path to the kube config '/home/goldmann/.minishift/machines/minishift_kubeconfig' does not exist"], "stdout": "-- Starting profile 'minishift'\n-- Check if deprecated options are used ... OK\n-- Checking if https://github.com is reachable ... OK\n-- Checking if requested OpenShift version 'v3.11.0' is valid ... SKIP\n-- Checking if requested OpenShift version 'v3.11.0' is supported ... SKIP\n-- Checking if requested hypervisor 'kvm' is supported on this platform ... OK\n-- Checking if KVM driver is installed ... \n   Driver is available at /usr/local/bin/docker-machine-driver-kvm ... \n   Checking driver binary is executable ... OK\n-- Checking if Libvirt is installed ... OK\n-- Checking if Libvirt default network is present ... OK\n-- Checking if Libvirt default network is active ... OK\n-- Checking the ISO URL ... OK\n-- Checking if provided oc flags are supported ... OK\n-- Starting the OpenShift cluster using 'kvm' hypervisor ...\n-- Starting Minishift VM ............... OK\n-- Checking for IP address ... OK\n-- Checking for nameservers ... OK\n-- Checking if external host is reachable from the Minishift VM ... \n   Pinging 8.8.8.8 ... OK\n-- Checking HTTP connectivity from the VM ... \n   Retrieving http://minishift.io/index.html ... OK\n-- Checking if persistent storage volume is mounted ... OK\n-- Checking available disk space ... 1% used OK\n-- Writing current configuration for static assignment of IP address ... WARN\n-- OpenShift cluster will be configured with ...\n   Version: v3.11.0\n-- Pulling the Openshift Container Image ......... OK\n-- Copying oc binary from the OpenShift container image to VM ... OK\n-- Starting OpenShift cluster ..............................................\nGetting a Docker client ...\nChecking if image openshift/origin-control-plane:v3.11.0 is available ...\nPulling image openshift/origin-cli:v3.11.0\nE0118 05:53:18.915812    2280 helper.go:173] Reading docker config from /home/docker/.docker/config.json failed: open /home/docker/.docker/config.json: no such file or directory, will attempt to pull image docker.io/openshift/origin-cli:v3.11.0 anonymously\nImage pull complete\nPulling image openshift/origin-node:v3.11.0\nE0118 05:53:21.101159    2280 helper.go:173] Reading docker config from /home/docker/.docker/config.json failed: open /home/docker/.docker/config.json: no such file or directory, will attempt to pull image docker.io/openshift/origin-node:v3.11.0 anonymously\nPulled 5/6 layers, 92% complete\nPulled 6/6 layers, 100% complete\nExtracting\nImage pull complete\nChecking type of volume mount ...\nDetermining server IP ...\nUsing public hostname IP 192.168.42.212 as the host IP\nChecking if OpenShift is already running ...\nChecking for supported Docker version (=>1.22) ...\nChecking if insecured registry is configured properly in Docker ...\nChecking if required ports are available ...\nChecking if OpenShift client is configured properly ...\nChecking if image openshift/origin-control-plane:v3.11.0 is available ...\nStarting OpenShift using openshift/origin-control-plane:v3.11.0 ...\nI0118 05:53:29.401067    2280 config.go:40] Running \"create-master-config\"\nI0118 05:53:30.931278    2280 config.go:46] Running \"create-node-config\"\nI0118 05:53:31.966445    2280 flags.go:30] Running \"create-kubelet-flags\"\nI0118 05:53:32.402127    2280 run_kubelet.go:49] Running \"start-kubelet\"\nI0118 05:53:32.622825    2280 run_self_hosted.go:181] Waiting for the kube-apiserver to be ready ...\nI0118 05:54:00.643630    2280 interface.go:26] Installing \"kube-proxy\" ...\nI0118 05:54:00.644141    2280 interface.go:26] Installing \"kube-dns\" ...\nI0118 05:54:00.644146    2280 interface.go:26] Installing \"openshift-service-cert-signer-operator\" ...\nI0118 05:54:00.644151    2280 interface.go:26] Installing \"openshift-apiserver\" ...\nI0118 05:54:00.644174    2280 apply_template.go:81] Installing \"openshift-apiserver\"\nI0118 05:54:00.644330    2280 apply_template.go:81] Installing \"kube-proxy\"\nI0118 05:54:00.644465    2280 apply_template.go:81] Installing \"kube-dns\"\nI0118 05:54:00.646198    2280 apply_template.go:81] Installing \"openshift-service-cert-signer-operator\"\nI0118 05:54:02.370902    2280 interface.go:41] Finished installing \"kube-proxy\" \"kube-dns\" \"openshift-service-cert-signer-operator\" \"openshift-apiserver\"\nI0118 05:55:40.397833    2280 run_self_hosted.go:242] openshift-apiserver available\nI0118 05:55:40.398523    2280 interface.go:26] Installing \"openshift-controller-manager\" ...\nI0118 05:55:40.398594    2280 apply_template.go:81] Installing \"openshift-controller-manager\"\nI0118 05:55:42.116652    2280 interface.go:41] Finished installing \"openshift-controller-manager\"\nAdding default OAuthClient redirect URIs ...\nAdding registry ...\nAdding sample-templates ...\nAdding web-console ...\nAdding centos-imagestreams ...\nAdding router ...\nAdding persistent-volumes ...\nI0118 05:55:42.134063    2280 interface.go:26] Installing \"openshift-image-registry\" ...\nI0118 05:55:42.134077    2280 interface.go:26] Installing \"sample-templates\" ...\nI0118 05:55:42.134082    2280 interface.go:26] Installing \"openshift-web-console-operator\" ...\nI0118 05:55:42.134086    2280 interface.go:26] Installing \"centos-imagestreams\" ...\nI0118 05:55:42.134094    2280 interface.go:26] Installing \"openshift-router\" ...\nI0118 05:55:42.134099    2280 interface.go:26] Installing \"persistent-volumes\" ...\nI0118 05:55:42.134169    2280 interface.go:26] Installing \"sample-templates/sample pipeline\" ...\nI0118 05:55:42.134174    2280 interface.go:26] Installing \"sample-templates/mongodb\" ...\nI0118 05:55:42.134178    2280 interface.go:26] Installing \"sample-templates/mysql\" ...\nI0118 05:55:42.134182    2280 interface.go:26] Installing \"sample-templates/postgresql\" ...\nI0118 05:55:42.134186    2280 interface.go:26] Installing \"sample-templates/cakephp quickstart\" ...\nI0118 05:55:42.134190    2280 interface.go:26] Installing \"sample-templates/dancer quickstart\" ...\nI0118 05:55:42.134195    2280 interface.go:26] Installing \"sample-templates/django quickstart\" ...\nI0118 05:55:42.134198    2280 interface.go:26] Installing \"sample-templates/mariadb\" ...\nI0118 05:55:42.134203    2280 interface.go:26] Installing \"sample-templates/nodejs quickstart\" ...\nI0118 05:55:42.134207    2280 interface.go:26] Installing \"sample-templates/rails quickstart\" ...\nI0118 05:55:42.134211    2280 interface.go:26] Installing \"sample-templates/jenkins pipeline ephemeral\" ...\nI0118 05:55:42.134256    2280 apply_list.go:67] Installing \"sample-templates/jenkins pipeline ephemeral\"\nI0118 05:55:42.134543    2280 apply_template.go:81] Installing \"openshift-web-console-operator\"\nI0118 05:55:42.134689    2280 apply_list.go:67] Installing \"sample-templates/dancer quickstart\"\nI0118 05:55:42.134698    2280 apply_list.go:67] Installing \"sample-templates/cakephp quickstart\"\nI0118 05:55:42.134794    2280 apply_list.go:67] Installing \"sample-templates/django quickstart\"\nI0118 05:55:42.134821    2280 apply_list.go:67] Installing \"centos-imagestreams\"\nI0118 05:55:42.134881    2280 apply_list.go:67] Installing \"sample-templates/mariadb\"\nI0118 05:55:42.134959    2280 apply_list.go:67] Installing \"sample-templates/nodejs quickstart\"\nI0118 05:55:42.135035    2280 apply_list.go:67] Installing \"sample-templates/rails quickstart\"\nI0118 05:55:42.135121    2280 apply_list.go:67] Installing \"sample-templates/sample pipeline\"\nI0118 05:55:42.135210    2280 apply_list.go:67] Installing \"sample-templates/mongodb\"\nI0118 05:55:42.135263    2280 apply_list.go:67] Installing \"sample-templates/mysql\"\nI0118 05:55:42.135297    2280 apply_list.go:67] Installing \"sample-templates/postgresql\"\nI0118 05:55:47.557836    2280 interface.go:41] Finished installing \"sample-templates/sample pipeline\" \"sample-templates/mongodb\" \"sample-templates/mysql\" \"sample-templates/postgresql\" \"sample-templates/cakephp quickstart\" \"sample-templates/dancer quickstart\" \"sample-templates/django quickstart\" \"sample-templates/mariadb\" \"sample-templates/nodejs quickstart\" \"sample-templates/rails quickstart\" \"sample-templates/jenkins pipeline ephemeral\"\nI0118 05:56:13.757288    2280 interface.go:41] Finished installing \"openshift-image-registry\" \"sample-templates\" \"openshift-web-console-operator\" \"centos-imagestreams\" \"openshift-router\" \"persistent-volumes\"\nLogin to server ...\nCreating initial project \"myproject\" ...\nServer Information ...\nOpenShift server started.\n\nThe server is accessible via web console at:\n    https://192.168.42.212:8443/console\n\nYou are logged in as:\n    User:     developer\n    Password: <any value>\n\nTo login as administrator:\n    oc login -u system:admin", "stdout_lines": ["-- Starting profile 'minishift'", "-- Check if deprecated options are used ... OK", "-- Checking if https://github.com is reachable ... OK", "-- Checking if requested OpenShift version 'v3.11.0' is valid ... SKIP", "-- Checking if requested OpenShift version 'v3.11.0' is supported ... SKIP", "-- Checking if requested hypervisor 'kvm' is supported on this platform ... OK", "-- Checking if KVM driver is installed ... ", "   Driver is available at /usr/local/bin/docker-machine-driver-kvm ... ", "   Checking driver binary is executable ... OK", "-- Checking if Libvirt is installed ... OK", "-- Checking if Libvirt default network is present ... OK", "-- Checking if Libvirt default network is active ... OK", "-- Checking the ISO URL ... OK", "-- Checking if provided oc flags are supported ... OK", "-- Starting the OpenShift cluster using 'kvm' hypervisor ...", "-- Starting Minishift VM ............... OK", "-- Checking for IP address ... OK", "-- Checking for nameservers ... OK", "-- Checking if external host is reachable from the Minishift VM ... ", "   Pinging 8.8.8.8 ... OK", "-- Checking HTTP connectivity from the VM ... ", "   Retrieving http://minishift.io/index.html ... OK", "-- Checking if persistent storage volume is mounted ... OK", "-- Checking available disk space ... 1% used OK", "-- Writing current configuration for static assignment of IP address ... WARN", "-- OpenShift cluster will be configured with ...", "   Version: v3.11.0", "-- Pulling the Openshift Container Image ......... OK", "-- Copying oc binary from the OpenShift container image to VM ... OK", "-- Starting OpenShift cluster ..............................................", "Getting a Docker client ...", "Checking if image openshift/origin-control-plane:v3.11.0 is available ...", "Pulling image openshift/origin-cli:v3.11.0", "E0118 05:53:18.915812    2280 helper.go:173] Reading docker config from /home/docker/.docker/config.json failed: open /home/docker/.docker/config.json: no such file or directory, will attempt to pull image docker.io/openshift/origin-cli:v3.11.0 anonymously", "Image pull complete", "Pulling image openshift/origin-node:v3.11.0", "E0118 05:53:21.101159    2280 helper.go:173] Reading docker config from /home/docker/.docker/config.json failed: open /home/docker/.docker/config.json: no such file or directory, will attempt to pull image docker.io/openshift/origin-node:v3.11.0 anonymously", "Pulled 5/6 layers, 92% complete", "Pulled 6/6 layers, 100% complete", "Extracting", "Image pull complete", "Checking type of volume mount ...", "Determining server IP ...", "Using public hostname IP 192.168.42.212 as the host IP", "Checking if OpenShift is already running ...", "Checking for supported Docker version (=>1.22) ...", "Checking if insecured registry is configured properly in Docker ...", "Checking if required ports are available ...", "Checking if OpenShift client is configured properly ...", "Checking if image openshift/origin-control-plane:v3.11.0 is available ...", "Starting OpenShift using openshift/origin-control-plane:v3.11.0 ...", "I0118 05:53:29.401067    2280 config.go:40] Running \"create-master-config\"", "I0118 05:53:30.931278    2280 config.go:46] Running \"create-node-config\"", "I0118 05:53:31.966445    2280 flags.go:30] Running \"create-kubelet-flags\"", "I0118 05:53:32.402127    2280 run_kubelet.go:49] Running \"start-kubelet\"", "I0118 05:53:32.622825    2280 run_self_hosted.go:181] Waiting for the kube-apiserver to be ready ...", "I0118 05:54:00.643630    2280 interface.go:26] Installing \"kube-proxy\" ...", "I0118 05:54:00.644141    2280 interface.go:26] Installing \"kube-dns\" ...", "I0118 05:54:00.644146    2280 interface.go:26] Installing \"openshift-service-cert-signer-operator\" ...", "I0118 05:54:00.644151    2280 interface.go:26] Installing \"openshift-apiserver\" ...", "I0118 05:54:00.644174    2280 apply_template.go:81] Installing \"openshift-apiserver\"", "I0118 05:54:00.644330    2280 apply_template.go:81] Installing \"kube-proxy\"", "I0118 05:54:00.644465    2280 apply_template.go:81] Installing \"kube-dns\"", "I0118 05:54:00.646198    2280 apply_template.go:81] Installing \"openshift-service-cert-signer-operator\"", "I0118 05:54:02.370902    2280 interface.go:41] Finished installing \"kube-proxy\" \"kube-dns\" \"openshift-service-cert-signer-operator\" \"openshift-apiserver\"", "I0118 05:55:40.397833    2280 run_self_hosted.go:242] openshift-apiserver available", "I0118 05:55:40.398523    2280 interface.go:26] Installing \"openshift-controller-manager\" ...", "I0118 05:55:40.398594    2280 apply_template.go:81] Installing \"openshift-controller-manager\"", "I0118 05:55:42.116652    2280 interface.go:41] Finished installing \"openshift-controller-manager\"", "Adding default OAuthClient redirect URIs ...", "Adding registry ...", "Adding sample-templates ...", "Adding web-console ...", "Adding centos-imagestreams ...", "Adding router ...", "Adding persistent-volumes ...", "I0118 05:55:42.134063    2280 interface.go:26] Installing \"openshift-image-registry\" ...", "I0118 05:55:42.134077    2280 interface.go:26] Installing \"sample-templates\" ...", "I0118 05:55:42.134082    2280 interface.go:26] Installing \"openshift-web-console-operator\" ...", "I0118 05:55:42.134086    2280 interface.go:26] Installing \"centos-imagestreams\" ...", "I0118 05:55:42.134094    2280 interface.go:26] Installing \"openshift-router\" ...", "I0118 05:55:42.134099    2280 interface.go:26] Installing \"persistent-volumes\" ...", "I0118 05:55:42.134169    2280 interface.go:26] Installing \"sample-templates/sample pipeline\" ...", "I0118 05:55:42.134174    2280 interface.go:26] Installing \"sample-templates/mongodb\" ...", "I0118 05:55:42.134178    2280 interface.go:26] Installing \"sample-templates/mysql\" ...", "I0118 05:55:42.134182    2280 interface.go:26] Installing \"sample-templates/postgresql\" ...", "I0118 05:55:42.134186    2280 interface.go:26] Installing \"sample-templates/cakephp quickstart\" ...", "I0118 05:55:42.134190    2280 interface.go:26] Installing \"sample-templates/dancer quickstart\" ...", "I0118 05:55:42.134195    2280 interface.go:26] Installing \"sample-templates/django quickstart\" ...", "I0118 05:55:42.134198    2280 interface.go:26] Installing \"sample-templates/mariadb\" ...", "I0118 05:55:42.134203    2280 interface.go:26] Installing \"sample-templates/nodejs quickstart\" ...", "I0118 05:55:42.134207    2280 interface.go:26] Installing \"sample-templates/rails quickstart\" ...", "I0118 05:55:42.134211    2280 interface.go:26] Installing \"sample-templates/jenkins pipeline ephemeral\" ...", "I0118 05:55:42.134256    2280 apply_list.go:67] Installing \"sample-templates/jenkins pipeline ephemeral\"", "I0118 05:55:42.134543    2280 apply_template.go:81] Installing \"openshift-web-console-operator\"", "I0118 05:55:42.134689    2280 apply_list.go:67] Installing \"sample-templates/dancer quickstart\"", "I0118 05:55:42.134698    2280 apply_list.go:67] Installing \"sample-templates/cakephp quickstart\"", "I0118 05:55:42.134794    2280 apply_list.go:67] Installing \"sample-templates/django quickstart\"", "I0118 05:55:42.134821    2280 apply_list.go:67] Installing \"centos-imagestreams\"", "I0118 05:55:42.134881    2280 apply_list.go:67] Installing \"sample-templates/mariadb\"", "I0118 05:55:42.134959    2280 apply_list.go:67] Installing \"sample-templates/nodejs quickstart\"", "I0118 05:55:42.135035    2280 apply_list.go:67] Installing \"sample-templates/rails quickstart\"", "I0118 05:55:42.135121    2280 apply_list.go:67] Installing \"sample-templates/sample pipeline\"", "I0118 05:55:42.135210    2280 apply_list.go:67] Installing \"sample-templates/mongodb\"", "I0118 05:55:42.135263    2280 apply_list.go:67] Installing \"sample-templates/mysql\"", "I0118 05:55:42.135297    2280 apply_list.go:67] Installing \"sample-templates/postgresql\"", "I0118 05:55:47.557836    2280 interface.go:41] Finished installing \"sample-templates/sample pipeline\" \"sample-templates/mongodb\" \"sample-templates/mysql\" \"sample-templates/postgresql\" \"sample-templates/cakephp quickstart\" \"sample-templates/dancer quickstart\" \"sample-templates/django quickstart\" \"sample-templates/mariadb\" \"sample-templates/nodejs quickstart\" \"sample-templates/rails quickstart\" \"sample-templates/jenkins pipeline ephemeral\"", "I0118 05:56:13.757288    2280 interface.go:41] Finished installing \"openshift-image-registry\" \"sample-templates\" \"openshift-web-console-operator\" \"centos-imagestreams\" \"openshift-router\" \"persistent-volumes\"", "Login to server ...", "Creating initial project \"myproject\" ...", "Server Information ...", "OpenShift server started.", "", "The server is accessible via web console at:", "    https://192.168.42.212:8443/console", "", "You are logged in as:", "    User:     developer", "    Password: <any value>", "", "To login as administrator:", "    oc login -u system:admin"]}
	to retry, use: --limit @/home/goldmann/git/redhat/contra-env-setup/playbooks/setup.retry

It looks like OKD itself is running fine and it may be that the failure above is a red-herring causing the Ansible playbook to fail.

Any hint?

The new-app command uses template label instead of template name

The new-app command in build_new_app.yml uses template label obtained in setup_os_templates.yml instead of template's name, e.g. in ansible-buildconfig-template.yaml.
The label doesn't have to be the same as the template's name, this could prove to be a problem in future usage with templates with different/shared labels or without them altogether.

Since calling oc process on the template omits template's metadata, I have gone over a couple of alternative solutions neither of which I found completely satisfactory:

Using yq to extract the template's name - this requires pip and installing the yq package
Using a fancy one-liner to extract it directly from the file - doesn't feel very reliable to me

Anyone have any ideas how to go about this?

Don't do `git config --global http.sslVerify false`

The setup currently globally disables git SSL verification with git config --global http.sslVerify false. That is potentially dangerous as users may not expect this being setup globally (at least I wasn't expecting that). Perhaps just setting this per repo would be be enough?

Build container images in parallel?

Hello,
IIUC correctly all the container images are currently build serially. Would it make sense to queue all the builds in parallel and then just wait for all of them to finish? This should speed up the setup quite a bit. Also, in lot of cases the build are IO bound, not CPU bound, so this should not even require more CPU power or so.

Make code more secure when using remote clusters

Right now when using remote OpenShift clusters the code here will by default:

Disable all TLS checking
Require one to specify the OpenShift username and password on the command line

We should:

Enable TLS checking by default and have a parameter to disable it
Enable using a token instead of a password
Enable assuming the local user had already logged into the OpenShift cluster.

Setup failed on machines with AMD cpus

Ansible playbook checked just virtualisation support just for intel VT-x. But qemu-kvm should work well even with AMD SVM.

sh$ ansible-playbook -i "hp-bl465cg8-1.example.com," playbooks/setup.yml
PLAY [all] ********************************************************************

TASK [Gathering Facts] ********************************************************
ok: [hp-bl465cg8-1.example.com]

TASK [prereqs : Check virtualization settings] ********************************
fatal: [hp-bl465cg8-1.example.com]: FAILED! => {"changed": true, "cmd": "cat /proc/cpuinfo | grep vmx", "delta": "0:00:00.019848", "end": "2018-03-13 08:32:56.536636", "msg": "non-zero return code", "rc": 1, "start": "2018-03-13 08:32:56.516788", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []}
...ignoring

TASK [prereqs : Output of cpuinfo from proc] **********************************
ok: [hp-bl465cg8-1.example.com] => {
    "msg": "cpuinfo: "
}

TASK [prereqs : Exit if virtualization is not turned on in the BIOS] **********
fatal: [hp-bl465cg8-1.example.com]: FAILED! => {"changed": false, "msg": "Please enable Intel virtualization (VT) within the host machine's BIOS"}
        to retry, use: --limit @/home/alcik/projects/contra-env-setup/playbooks/setup.retry

PLAY RECAP ********************************************************************
hp-bl465cg8-1.example.com : ok=3    changed=1    unreachable=0    failed=1

https://www.cyberciti.biz/faq/linux-xen-vmware-kvm-intel-vt-amd-v-support/

set run_prereqs=false and setup_minishift=false by default

These are destructive operations and are of no interest if one does not intend to run things locally.
The default should be to switch then off IMO.

Install prereqs fails on Fedora 30 for python-dnf

Looks like the offending line is here:

contra-env-setup/playbooks/roles/prereqs/tasks/install_virtual_reqs.yml

Line 3 in c2f25bb

raw: dnf install -y --allowerasing python-dnf

Fedora 30 doesn't have python-dnf, it has python3-dnf.

The system python is python3 so I'm guessing that for Fedora > 30 the above line should be amended to install python3-dnf?

User "system:serviceaccount:contra-sample-project:jenkins" cannot list pods in the namespace

Spawned a DEV environment with the command:
ansible-playbook -vv -i inventory playbooks/setup.yml -e user=$USER -K

Manually replaced the shared library setting to use contra-hdsl instead the ones configured by the automation.

Running a pipeline which should spawn Contra HDSL pod hangs with the following error in the jenkins log file:

Failed to count the # of live instances on Kubernetes
io.fabric8.kubernetes.client.KubernetesClientException: Failure executing: GET at: https://172.30.0.1/api/v1/namespaces/Contra%20Sample%20Project/pods?labelSelector=jenkins%3Dslave. Message: Forbidden!Configured service account doesn't have access. Service account may have been revoked. pods is forbidden: User "system:serviceaccount:contra-sample-project:jenkins" cannot list pods in the namespace "Contra Sample Project": no RBAC policy matched.

Pull down minishift from GitHub

Pull the latest minishift binary from GitHub
as a replacement to storing a static version in s3/fedora repo.

Can we create 1.0.1 release for the set-project-display-name PR?

so we can use it in CVP?

tmp/durable-9dab65ad/jenkins-log.txt: Permission denied

Spawn a DEV environment.
Jenkinsfile code snippet:

    stage("Execute Tests"){
      try {
        executeTests verbose: true, vars: [ workspace: "${WORKSPACE}" ]
      } finally {
        junit 'junit.xml'
      }
    }

From build console:

[Pipeline] { (Execute Tests)
[Pipeline] readJSON
[Pipeline] withEnv
[Pipeline] {
[Pipeline] container
[Pipeline] {
[Pipeline] sh
sh: /workDir/workspace/PR_submitted_CI_ci-automation-2@tmp/durable-9dab65ad/jenkins-log.txt: Permission denied
touch: cannot touch '/workDir/workspace/PR_submitted_CI_ci-automation-2@tmp/durable-9dab65ad/jenkins-log.txt': Permission denied
sh: /workDir/workspace/PR_submitted_CI_ci-automation-2@tmp/durable-9dab65ad/jenkins-result.txt.tmp: Permission denied
mv: cannot stat '/workDir/workspace/PR_submitted_CI_ci-automation-2@tmp/durable-9dab65ad/jenkins-result.txt.tmp': No such file or directory
touch: cannot touch '/workDir/workspace/PR_submitted_CI_ci-automation-2@tmp/durable-9dab65ad/jenkins-log.txt': Permission denied
touch: cannot touch '/workDir/workspace/PR_submitted_CI_ci-automation-2@tmp/durable-9dab65ad/jenkins-log.txt': Permission denied
touch: cannot touch '/workDir/workspace/PR_submitted_CI_ci-automation-2@tmp/durable-9dab65ad/jenkins-log.txt': Permission denied

Cleanup of failed builds doesn't remove all the created resources

Right now, when a build fails, only the bc, dc, routes, imagestreams, serviceaccounts, pvc, and rolebindings bearing the name of the app are deleted. If anything other than those is defined in the loaded template( e.g. configmaps, secrets, etc.), it won't be deleted and the next build retry with the new-app command will break the playbook as it attempts to create resources that already exist.

Since there is no guarantee that the resources will have name related to the app or the build, I have two ideas for solving this:

Creating the app and starting the build with the new-app command only once (on the first build attempt) with subsequent retries just restarting the builds and using the already created resources
Storing the output of the new-app command in a hash for every specific template (or app) and using the information within it to delete all resources it created if a build fails

Jenkins deployment dynamically set resources

Allow configuring the RAM allocated to the jenkins deployment.
Currently, it hard coded allocates 512 Mib. Jenkins breaks every time the RAM limit is reached.
The manual configurations are done after each crash. Error prone and time consuming process.

If there are failed builds in OpenShift, new ones are not started

It seems that if you ran this playbook against an OpenShift instance, and the builds it created failed for some reason, if you rerun it, it will not start new builds.

Multiple resource creation from template is not idempotent

With the current code resources are created from templates only if:

The image stream created by the template does not exist
The build config created by the template dos not exist
The container created by the template was never built

As a result, the following example situations can occur:

If the image stream exists and the build config does not - it will not get created
If the template defines a deployment config, it will not get created if an image stream or a build already exists

Support to CentOS 7

Create the support to CentOS 7

[Not a issue] Only to register here!

Troubleshooting Developer Deployment

Requirements

ansible
libselinux-python

Cleaning possible errors in a new deployment

Clean any reference to 'minishift' or 'oc':
$ sed -i".bak" '/minishift/d' ~/.bashrc
$ sed -i".bak" '/oc/d' ~/.bashrc
Remove the configs and old deploys of Minishift:
$ rm -rf ~/.minishift
$ rm -rf ~/minishift

NOTE If do you save minishift.iso in another place to copy to ~/minishift/ before start a new deploy could be safe several minutes.

Super cleaner:
$ sed -i".bak" '/minishift/d' ~/.bashrc ; sed -i".bak" '/oc/d' ~/.bashrc ; rm -rf ~/.minishift ; rm -rf ~/minishift ; mkdir ~/minishift ; cp ~/ISOs/minishift.iso ~/minishift/

Deploy to xCI Team

OPTIONAL
$ mkdir ~/minishift
$ cp minishift.iso ~/minishift/