Coder Social home page Coder Social logo

ocp-power-automation / openshift-install-power Goto Github PK

View Code? Open in Web Editor NEW
17.0 8.0 24.0 519 KB

UPI Install helper to deploy OpenShift 4 on IBM Power Systems Virtual Server using Terraform IaC

License: Apache License 2.0

Shell 99.03% Dockerfile 0.97%
ibm powervs bash power openshift4 ibm-cloud openshift infrastructure-as-code openshift-deployment

openshift-install-power's Introduction

Install OpenShift on PowerVS

Introduction

This project contains a bash script to help you deploy OpenShift Container Platform 4.X on IBM® Power Systems™ Virtual Server on IBM Cloud (PowerVS). The Terraform code at ocp4-upi-powervs is used for the deployment process.

Ensure your PowerVS instance is prepped for deploying OpenShift Clusters. Please check this link for more details

Here is quick demo.

Features

  • Simple script based installer to deploy OpenShift (4.5 onwards) cluster on PowerVS leveraging Infrastructure as Code (IaC) pattern
  • Supports multiple platforms including Linux(x86_64/ppc64le), Windows & Mac OSX
  • Sets up the latest IBM Cloud CLI with Power Virtual Servers plugin.
  • Sets up the latest Terraform binary
  • Provides interactive mode to populate required Terraform variables
  • Abstract out the Terraform lifecycle management

Supported Platforms

Only 64bit Operating Systems are supported by the script.

The script requires bash shell and a package manager pre-configured.

MacOS

Catalina (10.15) and above is required.

The script uses Homebrew package manager to install required packages.

Linux (x86_64/ppc64le)

RHEL8/CentOS8 and above or Ubuntu 16.04 and above is required.

The script uses the default package manager (yum/dnf/apt) based on the distribution.

Windows 10 (64-bit)

The script can run on GitBash, Windows Subsystem for Linux and Cygwin terminals.

If using Cygwin, then please ensure curl and unzip packages are installed. You will need to run the Cygwin setup again.

Note: PowerShell is Unsupported.

Firewall Requirements

  1. Ensure inbound access is allowed for the following TCP ports. This is only required when using a Cloud instance or a remote VM so that you can connect to it using SSH and run the installer
    • 22 (SSH)
  2. Ensure outbound access is allowed for the following TCP ports
    • 80 (HTTP)
    • 443 (HTTPS)
    • 6443 (OC CLI)

Usage

  1. Create an install directory where all the configurations, logs and data files will be stored.
$ mkdir ocp-install-dir && cd ocp-install-dir
  1. Download the script on your system and change the permission to execute.
$ curl -sL https://raw.githubusercontent.com/ocp-power-automation/openshift-install-power/main/openshift-install-powervs -o ./openshift-install-powervs
$ chmod +x ./openshift-install-powervs
  1. Run the script.
$ ./openshift-install-powervs

Automation for deploying OpenShift 4.X on PowerVS

Usage:
  openshift-install-powervs [command] [<args> [<value>]]

Available commands:
  setup           Install all the required packages/binaries in current directory
  variables       Interactive way to populate the variables file
  create          Create an OpenShift cluster
  destroy         Destroy an OpenShift cluster
  output          Display the cluster information. Runs terraform output [NAME]
  access-info     Display the access information of installed OpenShift cluster
  help            Display this information

Where <args>:
  -var               Terraform variable to be passed to the create/destroy command
  -var-file          Terraform variable file name in current directory. (By default using var.tfvars)
  -flavor            Cluster compute template to use eg: small, medium, large
  -force-destroy     Not ask for confirmation during destroy command
  -ignore-os-checks  Ignore operating system related checks
  -ignore-warnings   Warning messages will not be displayed. Should be specified first, before any other args.
  -verbose           Enable verbose for terraform console messages
  -all-images        List all the images available during variables prompt
  -trace             Enable tracing of all executed commands
  -version, -v       Display the script version

Environment Variables:
  IBMCLOUD_API_KEY    IBM Cloud API key
  RELEASE_VER         OpenShift release version (Default: 4.15)
  ARTIFACTS_VERSION   Tag or Branch name of ocp4-upi-powervs repository (Default: main)
  RHEL_SUBS_PASSWORD  RHEL subscription password if not provided in variables
  NO_OF_RETRY         Number of retries/attempts to run repeatable actions such as create (Default: 5)

Submit issues at: https://github.com/ocp-power-automation/openshift-install-power/issues

Using the Container Image

To use the images present in the quay.io/powercloud repository, run the following command.

docker run -it -e IBMCLOUD_API_KEY="<key>" -v $(pwd):/data quay.io/powercloud/openshift-install-powervs:<valid-tag> create

OR

podman run -it -e IBMCLOUD_API_KEY="<key>" -v $(pwd):/data quay.io/powercloud/openshift-install-powervs:<valid-tag> create

Tags provide information about the release version, ocp version or supported architecture.

In case you need to build the openshift-install-powervs image, check out the following "Usage with Containers" link

Prerequisites

The script assumes PowerVS prerequisites for OpenShift are already in place.

In case you missed, here is the link to the prerequisites

For running the script you need the following:

  1. IBM Cloud API key: Create the key by following the instructions available in the following link
  2. OpenShift Pull secret: Download the secret from the following link. You'll need to place the file in the install directory and name it as pull-secret.txt
  3. RHEL Subscription ID and Password.

Quickstart

  1. Export the IBM Cloud API Key and RHEL Subscription Password.

    $ set +o history
    $ export IBMCLOUD_API_KEY='<your API key>'
    $ export RHEL_SUBS_PASSWORD='<your RHEL subscription password>'
    $ set -o history
    
  2. Run the create command.

    $ ./openshift-install-powervs create
    

    The script will setup the required tools and run in interactive mode prompting for inputs.

    Once the above command completes successfully it will print the cluster access information.

    Login to bastion: 'ssh -i automation/data/id_rsa [email protected]' and start using the 'oc' command.
    To access the cluster on local system when using 'oc' run: 'export KUBECONFIG=/root/ocp-install-dir/automation/kubeconfig'
    Access the OpenShift web-console here: https://console-openshift-console.apps.test-ocp-6f2c.ibm.com
    Login to the console with user: "kubeadmin", and password: "MHvmI-z5nY8-CBFKF-hmCDJ"
    Add the line on local system 'hosts' file:
    145.48.43.53 api.test-ocp-6f2c.ibm.com console-openshift-console.apps.test-ocp-6f2c.ibm.com integrated-oauth-server-openshift-authentication.apps.test-ocp-6f2c.ibm.com oauth-openshift.apps.test-ocp-6f2c.ibm.com prometheus-k8s-openshift-monitoring.apps.test-ocp-6f2c.ibm.com grafana-openshift-monitoring.apps.test-ocp-6f2c.ibm.com example.apps.test-ocp-6f2c.ibm.com
    
    

Advanced Usage

Before running the script, you may choose to override some environment variables as per your requirement.

Different OpenShift Versions

By default OpenShift version 4.15 is installed.

If you want to install 4.14, then export the variable RELEASE_VER.

$ export RELEASE_VER="4.14"

Non-interactive mode

You can avoid the interactive mode by having the required input files available in the install directory

Required input files

  1. Terraform vars file (filename: var.tfvars)

  2. SSH key files (filename: id_rsa & id_rsa.pub)

    Example var.tfvars file

    ibmcloud_region = "syd"
    ibmcloud_zone = "syd04"
    service_instance_id = "123456abc-xzz-2223434343"
    rhel_image_name = "rhel-83"
    rhcos_image_name = "rhcos-415"
    network_name = "ocp-net"
    openshift_install_tarball = "https://mirror.openshift.com/pub/openshift-v4/ppc64le/clients/ocp/stable-4.15/openshift-install-linux.tar.gz"
    openshift_client_tarball = "https://mirror.openshift.com/pub/openshift-v4/ppc64le/clients/ocp/stable-4.15/openshift-client-linux.tar.gz"
    cluster_id_prefix = "test-ocp"
    cluster_domain = "xip.io"
    storage_type = "nfs"
    volume_size = "300"
    bastion = {memory = "16", processors = "1", "count" = 1}
    bootstrap = {memory = "32", processors = "0.5", "count" = 1}
    master = {memory = "32", processors = "0.5", "count" = 3}
    worker = {memory = "32", processors = "0.5", "count" = 2}
    rhel_subscription_username = "[email protected]"
    rhel_subscription_password = "mysubscriptionPassword"
    

    You can also pass a custom Terraform variables file using the option -var-file <filename> to the script. You can also use the option -var "key=value" to pass a single variable. If the same variable is given more than once then precedence will be from left (low) to right (high).

Tutorials

Check out the following learning path for deploying and using OpenShift on PowerVS

Detailed Explanation of the Core Commands

The following core commands are supported by the script.

Below is a simple flow chart explaining the flow of each command.

Flow Chart

Contributing

Please see the contributing doc for more details. PRs are most welcome !!

openshift-install-power's People

Contributors

abdulhusainhamid1 avatar aishwaryabk avatar bkhadars avatar bpradipt avatar cs-zhang avatar dependabot[bot] avatar dpkshetty avatar elayarajadhanapal avatar mkumatag avatar ppc64le-cloud-bot avatar prajyot-parab avatar prb112 avatar sajauddin avatar sudeeshjohn avatar yussufsh avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

openshift-install-power's Issues

[BUG] path error appears while using https://github.com/ocp-power-automation/openshift-install-power in gitbash terminal for Windows10

Describe the bug
path error appears while using https://github.com/ocp-power-automation/openshift-install-power in gitbash terminal for Windows10

To Reproduce
Steps to reproduce the behavior:

  1. execute openshift-install-power in gitbash terminal for Windows10 in interactive mode.
  2. id_rsa.pub, /id_rsa, /pull-secret.txt are present in same path as install script

Expected behavior
Install should work.

Console Output

Paste the console messages that help explain your problem.
```[apply] Running terraform apply... please wait
Error: Invalid function argument
  on ocp.tf line 94, in module "install":
  94:     pull_secret                     = file(coalesce(var.pull_secret_file, "/dev/null"))
    |----------------
    | var.pull_secret_file is "<Full path of UserDir>/pull-secret.txt"
Invalid value for "path" parameter: no file exists at
<Full path of UserDir>\pull-secret.txt; this
function works only with files that are distributed as part of the
configuration source code, so if this file will be created by a resource in
this configuration you must instead obtain this result from an attribute of
that resource.

**Desktop (please complete the following information):**
  - Windows 10 terminal ,GitBash

Need to have CI for Power systems

For running the script validation on Power systems we need to have a CI for setup command.

We can make use of Travis which has Power support.

[FEATURE] Display version specific RHCOS images in variables

Is your feature request related to a problem? Please describe.
During variables prompt for RHCOS images it displays images of all RHCOS versions.

Describe the solution you'd like
Filter the image names with the OCP version for RHCOS images.

Describe alternatives you've considered
None

Additional context
If the image is still not listed then user needs to run with -all-images argument to the variables command.

v1.8.0 is broken on power

Describe the bug
Latest release - 1.8.0 - is broken on ppc64le.

To Reproduce
Steps to reproduce the behavior:

  1. setup works
# ./openshift-install-powervs -version
Version v1.8.0
# ./openshift-install-powervs setup
Attempt: 1/5
[setup] Running setup command...
[setup_artifacts] Downloading code artifacts main in ./automation
Attempt: 1/5
[setup] SUCCESS: setup command completed!
  1. Create fails
# ./openshift-install-powervs create                                  Attempt: 1/5
[powervs_login] Trying to login with the provided IBMCLOUD_API_KEY...
Listing services under account ISVSynergy2's Account as user [email protected]...
Targeting service crn:v1:bluemix:public:power-iaas:syd05:a/03555c034c9346f3a9528095ee4203f9:23fb57fc-f1ed-4ea3-84d8-4bd73f282831::...
[powervs_login] Targeting 'rdr-ghatwala-ISV-PROD-Syd05-RHP' with Id crn:v1:bluemix:public:power-iaas:syd05:a/03555c034c9346f3a9528095ee4203f9:23fb57fc-f1ed-4ea3-84d8-4bd73f282831::
[init_terraform] Initializing Terraform plugins and validating the code...
╷
│ Error: Failed to install provider
│
│ Error while installing ibm-cloud/ibm v1.49.0: the local package for registry.terraform.io/ibm-cloud/ibm 1.49.0 doesn't match
│ any of the checksums previously recorded in the dependency lock file (this might be because the available checksums are for
│ packages targeting different platforms); for more information:
│ https://www.terraform.io/language/provider-checksum-verification
╵

╷
│ Error: Failed to install provider
│
│ Error while installing hashicorp/random v3.4.3: the local package for registry.terraform.io/hashicorp/random 3.4.3 doesn't
│ match any of the checksums previously recorded in the dependency lock file (this might be because the available checksums are
│ for packages targeting different platforms); for more information:
│ https://www.terraform.io/language/provider-checksum-verification
╵

╷
│ Error: Failed to install provider
│
│ Error while installing hashicorp/null v3.2.1: the local package for registry.terraform.io/hashicorp/null 3.2.1 doesn't match
│ any of the checksums previously recorded in the dependency lock file (this might be because the available checksums are for
│ packages targeting different platforms); for more information:
│ https://www.terraform.io/language/provider-checksum-verification
╵

╷
│ Error: Failed to install provider
│
│ Error while installing community-terraform-providers/ignition v2.1.3: the local package for
│ registry.terraform.io/community-terraform-providers/ignition 2.1.3 doesn't match any of the checksums previously recorded in
│ the dependency lock file (this might be because the available checksums are for packages targeting different platforms); for
│ more information: https://www.terraform.io/language/provider-checksum-verification
╵

╷
│ Error: Failed to install provider
│
│ Error while installing hashicorp/null v3.2.1: the local package for registry.terraform.io/hashicorp/null 3.2.1 doesn't match
│ any of the checksums previously recorded in the dependency lock file (this might be because the available checksums are for
│ packages targeting different platforms); for more information:
│ https://www.terraform.io/language/provider-checksum-verification
╵

╷
│ Error: Failed to install provider
│
│ Error while installing ibm-cloud/ibm v1.49.0: the local package for registry.terraform.io/ibm-cloud/ibm 1.49.0 doesn't match
│ any of the checksums previously recorded in the dependency lock file (this might be because the available checksums are for
│ packages targeting different platforms); for more information:
│ https://www.terraform.io/language/provider-checksum-verification
╵

╷
│ Error: Failed to install provider
│
│ Error while installing hashicorp/random v3.4.3: the local package for registry.terraform.io/hashicorp/random 3.4.3 doesn't
│ match any of the checksums previously recorded in the dependency lock file (this might be because the available checksums are
│ for packages targeting different platforms); for more information:
│ https://www.terraform.io/language/provider-checksum-verification
╵

╷
│ Error: Failed to install provider
│
│ Error while installing community-terraform-providers/ignition v2.1.3: the local package for
│ registry.terraform.io/community-terraform-providers/ignition 2.1.3 doesn't match any of the checksums previously recorded in
│ the dependency lock file (this might be because the available checksums are for packages targeting different platforms); for
│ more information: https://www.terraform.io/language/provider-checksum-verification
╵

╷
│ Error: Failed to install provider
│
│ Error while installing hashicorp/null v3.2.1: the local package for registry.terraform.io/hashicorp/null 3.2.1 doesn't match
│ any of the checksums previously recorded in the dependency lock file (this might be because the available checksums are for
│ packages targeting different platforms); for more information:
│ https://www.terraform.io/language/provider-checksum-verification
╵

╷
│ Error: Failed to install provider
│
│ Error while installing community-terraform-providers/ignition v2.1.3: the local package for
│ registry.terraform.io/community-terraform-providers/ignition 2.1.3 doesn't match any of the checksums previously recorded in
│ the dependency lock file (this might be because the available checksums are for packages targeting different platforms); for
│ more information: https://www.terraform.io/language/provider-checksum-verification
╵

╷
│ Error: Failed to install provider
│
│ Error while installing ibm-cloud/ibm v1.49.0: the local package for registry.terraform.io/ibm-cloud/ibm 1.49.0 doesn't match
│ any of the checksums previously recorded in the dependency lock file (this might be because the available checksums are for
│ packages targeting different platforms); for more information:
│ https://www.terraform.io/language/provider-checksum-verification
╵

╷
│ Error: Failed to install provider
│
│ Error while installing hashicorp/random v3.4.3: the local package for registry.terraform.io/hashicorp/random 3.4.3 doesn't
│ match any of the checksums previously recorded in the dependency lock file (this might be because the available checksums are
│ for packages targeting different platforms); for more information:
│ https://www.terraform.io/language/provider-checksum-verification
╵

╷
│ Error: Failed to install provider
│
│ Error while installing ibm-cloud/ibm v1.49.0: the local package for registry.terraform.io/ibm-cloud/ibm 1.49.0 doesn't match
│ any of the checksums previously recorded in the dependency lock file (this might be because the available checksums are for
│ packages targeting different platforms); for more information:
│ https://www.terraform.io/language/provider-checksum-verification
╵

╷
│ Error: Failed to install provider
│
│ Error while installing hashicorp/random v3.4.3: the local package for registry.terraform.io/hashicorp/random 3.4.3 doesn't
│ match any of the checksums previously recorded in the dependency lock file (this might be because the available checksums are
│ for packages targeting different platforms); for more information:
│ https://www.terraform.io/language/provider-checksum-verification
╵

╷
│ Error: Failed to install provider
│
│ Error while installing hashicorp/null v3.2.1: the local package for registry.terraform.io/hashicorp/null 3.2.1 doesn't match
│ any of the checksums previously recorded in the dependency lock file (this might be because the available checksums are for
│ packages targeting different platforms); for more information:
│ https://www.terraform.io/language/provider-checksum-verification
╵

╷
│ Error: Failed to install provider
│
│ Error while installing community-terraform-providers/ignition v2.1.3: the local package for
│ registry.terraform.io/community-terraform-providers/ignition 2.1.3 doesn't match any of the checksums previously recorded in
│ the dependency lock file (this might be because the available checksums are for packages targeting different platforms); for
│ more information: https://www.terraform.io/language/provider-checksum-verification
╵

╷
│ Error: Failed to install provider
│
│ Error while installing hashicorp/random v3.4.3: the local package for registry.terraform.io/hashicorp/random 3.4.3 doesn't
│ match any of the checksums previously recorded in the dependency lock file (this might be because the available checksums are
│ for packages targeting different platforms); for more information:
│ https://www.terraform.io/language/provider-checksum-verification
╵

╷
│ Error: Failed to install provider
│
│ Error while installing hashicorp/null v3.2.1: the local package for registry.terraform.io/hashicorp/null 3.2.1 doesn't match
│ any of the checksums previously recorded in the dependency lock file (this might be because the available checksums are for
│ packages targeting different platforms); for more information:
│ https://www.terraform.io/language/provider-checksum-verification
╵

╷
│ Error: Failed to install provider
│
│ Error while installing community-terraform-providers/ignition v2.1.3: the local package for
│ registry.terraform.io/community-terraform-providers/ignition 2.1.3 doesn't match any of the checksums previously recorded in
│ the dependency lock file (this might be because the available checksums are for packages targeting different platforms); for
│ more information: https://www.terraform.io/language/provider-checksum-verification
╵
╷
│ Error: Failed to install provider
│
│ Error while installing ibm-cloud/ibm v1.49.0: the local package for registry.terraform.io/ibm-cloud/ibm 1.49.0 doesn't match
│ any of the checksums previously recorded in the dependency lock file (this might be because the available checksums are for
│ packages targeting different platforms); for more information:
│ https://www.terraform.io/language/provider-checksum-verification
╵

Expected behavior
Cluster create should complete.

Desktop (please complete the following information):

  • OS: [e.g. RHEL, Ubuntu] : RHEL
  • Version [e.g. 8.3, 16.04] : 8.7

@yussufsh - can you PTAL ?

Flavor var file is used even if flavor option is not selected

Getting error the following error while running script
Console Logs

Error: Failed to read variables file

Given variables file ./compute-vars/.tfvars does not exist.

To Reproduce
Command used to run ./openshift-install-powervs create -ignore-os-checks -verbose
where I am using default var.tfvars file

Expected result
It should take var.tfvars file as default and not use ./compute-vars/.tfvars if flavor option is not passed

[BUG] 'destroy' doesn't cleanup properly

Describe the bug
openshift-install-powervs destroy doesn't cleanup properly for someone to start afresh second time

To Reproduce
Steps to reproduce the behavior:

  1. openshift-install-powervs destroy
  2. Wait for cluster to get destroyed
  3. delete old COS images & Upload new / updated COS images
  4. openshift-install-powervs create fails with errors that references old image IDs

Expected behavior
'destroy' should cleanup the install dir such that a fresh 'create' works flawlessly

Console Output
forgot to capture the screenshot

Desktop (please complete the following information):

  • OS: [e.g. RHEL, Ubuntu]
  • Version [e.g. 8.3, 16.04]
  • Windows terminal [eg: Cygwin, GitBash]

Additional context

  • I had to cleanup automation/* and vars.tfvar files to make openshift-install-powervs create work again

[FEATURE] Include PowerVS region catalog images while listling images

Is your feature request related to a problem? Please describe.
The image list during interactive prompt does not allow to select catalog images. Only boot images are listed.

Describe the solution you'd like
Should display combined list of catalog and boot images where user can select. This will also help users who did not upload RHEL image to the service images.

Describe alternatives you've considered
As a workaround we can update the var file manually with the catalog image name.

Additional context
NA

[FEATURE] Allow attaching mixed storage

Is your feature request related to a problem? Please describe.
Currently if you deploy a cluster on PowerVS, if the boot volume and extra volume fall on different tiers of storage, the deployment fails with:

Error: failed to provision Failed to Create PVM Instance :[POST /pcloud/v1/cloud-instances/{cloud_instance_id}/pvm-instances][400] pcloudPvminstancesPostBadRequest &{Code:0 Description:bad request: calculated parameter has failed for cloud-instance 3808f2f383f04938a5aeae88e027014c: validate volumes failed, all volumes are not in the same VM deployment storage pool Tier3-IOPS-1 Error:bad request Message:}
on modules/1_prepare/prepare.tf line 77, in resource "ibm_pi_instance" "bastion":
 77: resource "ibm_pi_instance" "bastion" {

Describe the solution you'd like
The PowerVS instances now support mixing storage tiers. Details are provided here: https://cloud.ibm.com/docs/power-iaas?topic=power-iaas-power-iaas-faqs#mixed_storage

To accomplish this you must modify the PVM instance and set the new PVM instance storagePoolAffinity property to false.

Describe alternatives you've considered
Always use the same tier for storage.

Additional context
image (2)

[BUG] Installer fails, when high availability feature is enabled and flavor large, medium or small is chosen

Describe the bug
It seems like the installer is throwing an error when the following constellation is met:
Starting installation with ./openshift-install-powervs create and set High Availability to yes (choose 1) and choose flavor large, medium or small. When the flavor CUSTOM is chosen, there is no error.

Error msg:

[...]
awk: unknown option -i ignored

awk: can't open file { if ( $1 == "bastion" ) $12 = 2 ; print $0 }
 source line number 1

When high availability is not enabled (2) then none of the flavors throw an error.

To Reproduce
Steps to reproduce the behavior:

  1. Download the openshift-installer-power
(base) mgiessing@Marvins-MBP ocp-install-dir % curl -sL https://raw.githubusercontent.com/ocp-power-automation/openshift-install-power/master/openshift-install-powervs -o openshift-install-powervs  && chmod +x openshift-install-powervs

2a. Start installation and enable high availability feature: [This DOES fail]

(base) mgiessing@Marvins-MBP ocp-install-dir % ./openshift-install-powervs create
[setup_ibmcloudcli] Installing the latest version of IBM-Cloud CLI...
Attempt: 1/5
/Users/mgiessing/git/ocp-install-dir/ibmcloud version 2.0.0+5a84b9b-2021-06-23T20:06:11+00:00
[setup_terraform] Installing the latest version of Terraform...
Attempt: 1/5
Terraform v0.13.6
[setup_occli] Installing the latest version of OpenShift CLI...
Attempt: 1/5
Client Version: 4.7.18
[precheck_artifacts] WARN: Cannot find artifacts directory... running setup command
[setup] Running setup command...
[setup_artifacts] Downloading code artifacts release-4.7 in ./automation
Attempt: 1/5
[setup] SUCCESS: setup command completed!
[precheck_input] WARN: No variables specified or var.tfvars does not exist.. running variables command
[variables] Trying to login with the provided IBMCLOUD_API_KEY...
[question] > Select the Service Instance name to use: (ocp-fra-test)
? 
- You have answered: ocp-fra-test
Targeting service crn:v1:bluemix:public:power-iaas:eu-de-1:a/d6e204bdd18e25524a9c4c8ee2da34fe:69a4f3ae-c16b-48b6-89f0-e6d2ec858e5c::...
[variables] Gathering information from the selected Service Instance... Please wait
[question] > Select the RHEL image to use for bastion node:
1) Linux-CentOS-8-3
2) Linux-RHEL-SAP-8-1
3) centos-8.3-12102020
#? 3
- You have answered: centos-8.3-12102020
[question] > Select the RHCOS image to use for cluster nodes:
1) rhcos-47-02072021
#? 1
- You have answered: rhcos-47-02072021
[question] > Select the system type to use for cluster nodes:
1) e980
2) s922
#? 2
- You have answered: s922
[question] > Select the private network to use:
1) test-ocp-2a98-eu-de-1
2) ocp-net
#? 2
- You have answered: ocp-net
[question] > Select the OCP version to use:
1) stable-4.7	    8) 4.7.16	      15) 4.7.9		 22) 4.7.2
2) latest-4.7	    9) 4.7.15	      16) 4.7.8		 23) 4.7.1
3) fast-4.7	   10) 4.7.14	      17) 4.7.7		 24) 4.7.0
4) candidate-4.7   11) 4.7.13	      18) 4.7.6		 25) 4.7.0-rc.3
5) 4.7.19	   12) 4.7.12	      19) 4.7.5		 26) 4.7.0-rc.2
6) 4.7.18	   13) 4.7.11	      20) 4.7.4		 27) 4.7.0-rc.1
7) 4.7.17	   14) 4.7.10	      21) 4.7.3
#? 1
- You have answered: stable-4.7
[question] > Enter a short name to identify the cluster (test-ocp)
? 
- You have answered: test-ocp
[question] > Enter a domain name for the cluster (ibm.com)
? 
- You have answered: ibm.com
[question] > Do you want to configure High Availability for bastion nodes?
1) yes
2) no
#? 1
- You have answered: yes
[question] > Select the flavor for the cluster nodes:
1) large
2) medium
3) small
4) CUSTOM
#? 3
- You have answered: small
awk: unknown option -i ignored

awk: can't open file { if ( $1 == "bastion" ) $12 = 2 ; print $0 }
 source line number 1

In contrast to this see:
2b. Start installation and disable high availability feature: [This does NOT fail]

(base) mgiessing@Marvins-MBP ocp-install-dir % ./openshift-install-powervs create
[precheck_input] WARN: No variables specified or var.tfvars does not exist.. running variables command
[variables] Trying to login with the provided IBMCLOUD_API_KEY...
[question] > Select the Service Instance name to use: (ocp-fra-test)
? 
- You have answered: ocp-fra-test
Targeting service crn:v1:bluemix:public:power-iaas:eu-de-1:a/d6e204bdd18e25524a9c4c8ee2da34fe:69a4f3ae-c16b-48b6-89f0-e6d2ec858e5c::...
[variables] Gathering information from the selected Service Instance... Please wait
[question] > Select the RHEL image to use for bastion node:
1) Linux-CentOS-8-3
2) Linux-RHEL-SAP-8-1
3) centos-8.3-12102020
#? 3
- You have answered: centos-8.3-12102020
[question] > Select the RHCOS image to use for cluster nodes:
1) rhcos-47-02072021
#? 1
- You have answered: rhcos-47-02072021
[question] > Select the system type to use for cluster nodes:
1) e980
2) s922
#? 2
- You have answered: s922
[question] > Select the private network to use:
1) test-ocp-2a98-eu-de-1
2) ocp-net
#? 2
- You have answered: ocp-net
[question] > Select the OCP version to use:
1) stable-4.7	    8) 4.7.16	      15) 4.7.9		 22) 4.7.2
2) latest-4.7	    9) 4.7.15	      16) 4.7.8		 23) 4.7.1
3) fast-4.7	   10) 4.7.14	      17) 4.7.7		 24) 4.7.0
4) candidate-4.7   11) 4.7.13	      18) 4.7.6		 25) 4.7.0-rc.3
5) 4.7.19	   12) 4.7.12	      19) 4.7.5		 26) 4.7.0-rc.2
6) 4.7.18	   13) 4.7.11	      20) 4.7.4		 27) 4.7.0-rc.1
7) 4.7.17	   14) 4.7.10	      21) 4.7.3
#? 1
- You have answered: stable-4.7
[question] > Enter a short name to identify the cluster (test-ocp)
? 
- You have answered: test-ocp
[question] > Enter a domain name for the cluster (ibm.com)
? 
- You have answered: ibm.com
[question] > Do you want to configure High Availability for bastion nodes?
1) yes
2) no
#? 2
- You have answered: no
[question] > Select the flavor for the cluster nodes:
1) large
2) medium
3) small
4) CUSTOM
#? 3
- You have answered: small
[question] > Do you need NFS storage to be configured?
1) yes
2) no
#? 

Expected behavior
I'd expect that everything would work without trowing an error, even when I enable HA

Desktop (please complete the following information):

  • OS: MacOS
  • Version BigSur 11.4

[BUG] Getting Warning: Deprecated Resource and Error: remote-exec provisioner error

Getting Warning: Deprecated Resource and Error: remote-exec provisioner error
Getting above warning and error while deploying OCP cluster.
module.install.null_resource.config (remote-exec): RUNNING HANDLER [restart bind] *************************************************
module.install.null_resource.config (remote-exec): PLAY RECAP *********************************************************************
module.install.null_resource.config (remote-exec): 192.168.17.36 : ok=85 changed=17 unreachable=0 failed=1 skipped=55 rescued=0 ignored=

│ Warning: Deprecated Resource

│ with module.prepare.ibm_pi_network_port.bastion_vip,
│ on modules/1_prepare/prepare.tf line 410, in resource "ibm_pi_network_port" "bastion_vip":
│ 410: resource "ibm_pi_network_port" "bastion_vip" {

│ Resource ibm_pi_network_port is deprecated. Use ibm_pi_network_port_attach
│ to create & attach a network port to a pvm instance

│ (and one more similar warning elsewhere)


│ Error: remote-exec provisioner error

│ with module.install.null_resource.config,
│ on modules/5_install/install.tf line 221, in resource "null_resource" "config":
│ 221: provisioner "remote-exec" {

│ error executing "/tmp/terraform_1760608655.sh": Process exited with status
│ 2

To Reproduce
Steps to reproduce the behavior:

  1. Create var.tfvars file with below details:
    ibmcloud_region = "mon"
    ibmcloud_zone = "mon01"
    service_instance_id = "d7638b97-909b-472a-a82c-2093753afbbe"
    rhel_image_name = "rhel-85-12132021"
    rhcos_image_name = "rhcos-412-86-20221018-tier3"
    system_type = "s922"
    network_name = "ocp-net-montreal-1"
    openshift_install_tarball = "https://mirror.openshift.com/pub/openshift-v4/ppc64le/clients/ocp/stable-4.12/openshift-install-linux.tar.gz"
    openshift_client_tarball = "https://mirror.openshift.com/pub/openshift-v4/ppc64le/clients/ocp/stable-4.12/openshift-client-linux.tar.gz"
    cluster_id_prefix = "hamid412-t8"
    cluster_domain = "nip.io"
    cluster_id = "t8"

Small Configuration Template

bastion = { memory = "16", processors = "0.5", "count" = 1 }
bootstrap = { memory = "32", processors = "0.5", "count" = 1 }
master = { memory = "32", processors = "0.5", "count" = 3 }
worker = { memory = "32", processors = "0.5", "count" = 2 }

storage_type = "nfs"
volume_size = "300"
rhel_subscription_username = "abdulhusain.hamid2"
rhel_subscription_password = "********"
pull_secret_file = "/data/pull-secret.txt"
private_key_file = "/data/id_rsa"
public_key_file = "/data/id_rsa.pub"

setup_squid_proxy = true
setup_snat = false

enable_local_registry = true #Set to true to enable usage of local registry for restricted network install.
local_registry_image = "docker.io/ibmcom/registry-ppc64le:2.6.2.5"
ocp_release_tag = "4.12.5-ppc64le"
ocp_release_name = "ocp-release"
release_image_override = ""
2. Create OCP cluster using below command:
docker run -it --name hamid412-t8-t8 -e IBMCLOUD_API_KEY=<api-key *********> -v $(pwd):/data:z quay.io/powercloud/openshift-install-powervs:ocp4.12 create -all-images
3. It went upto 89%
4. Then gave errors

Expected behavior
OCP cluster should be created successfull.y.

Console Output
module.install.null_resource.config (remote-exec): RUNNING HANDLER [restart bind] *************************************************

module.install.null_resource.config (remote-exec): PLAY RECAP *********************************************************************
module.install.null_resource.config (remote-exec): 192.168.17.36 : ok=85 changed=17 unreachable=0 failed=1 skipped=55 rescued=0 ignored=0


│ Warning: Deprecated Resource

│ with module.prepare.ibm_pi_network_port.bastion_vip,
│ on modules/1_prepare/prepare.tf line 410, in resource "ibm_pi_network_port" "bastion_vip":
│ 410: resource "ibm_pi_network_port" "bastion_vip" {

│ Resource ibm_pi_network_port is deprecated. Use ibm_pi_network_port_attach
│ to create & attach a network port to a pvm instance

│ (and one more similar warning elsewhere)


│ Error: remote-exec provisioner error

│ with module.install.null_resource.config,
│ on modules/5_install/install.tf line 221, in resource "null_resource" "config":
│ 221: provisioner "remote-exec" {

│ error executing "/tmp/terraform_1760608655.sh": Process exited with status
│ 2

Desktop (please complete the following information):

  • OS: RHEL
  • Version: 8.5
  • RHE Linux terminal

Additional context
Giving same error again & again.
CC: @yussufsh

[BUG] Does not work on latest macos

Describe the bug
Because of hard check latest OS version fails with Unsupported OS version error.

To Reproduce
Run the script on the latest 14.1 mac OS.

[FEATURE] Option to install oc client on the terminal where the script is running

Is your feature request related to a problem? Please describe.
User need to download the oc client on his terminal to connect to the OCP cluster deployed using this script. This needs to be as per the client machine type.

Describe the solution you'd like
We can have this as a sub-command or download the oc client binary on the system after successful install.

Describe alternatives you've considered
Manually downloading the client binary.

Additional context
NA

The output is adding some extra text

Running this command printing some extra text ./openshift-install-powervs output bastion_public_ip

To Reproduce
Steps to reproduce the behavior:
1)Deploy cluster using script
2) Run command ./openshift-install-powervs output bastion_public_ip

Expected behavior
It should print only the IP address
192.168.xx.xx

Actual behavior
Its printing
[0;36m[setup_tools] Verifying the latest packages and tools
192.168.xx.xx
Console Output

./openshift-install-powervs output bastion_public_ip

[0;36m[setup_tools] Verifying the latest packages and tools
192.168.xx.xx

Desktop (please complete the following information):

  • OS: Ubuntu x86

Additional context
The same thing is happening with other output variables

Encountering volume not found message in Destroy operation

Describe the bug
While destroying an OCP cluster on PowerVS, the cluster is destroyed on the ibmcloud but we have started observing an error message.

To Reproduce
Command to reproduce the behavior: ./openshift-install-powervs destroy -ignore-os-checks -force-destroy

Expected behavior
A clear and concise description of what you expected to happen.

Console Output

Attempt: 1/5
[retry_terraform] Encountered below errors:
│ Error: failed to perform the Get Volume Operation for volume f22a5a94-8680-4ece-8cfb-9190b5268c10 with error [GET /pcloud/v1/cloud-instances/{cloud_instance_id}/volumes/{volume_id}][404] pcloudCloudinstancesVolumesGetNotFound  &{Code:0 Description:volume does not exist, id: f22a5a94-8680-4ece-8cfb-9190b5268c10 Error:volume not found Message:}
[retry_terraform] WARN: Issues were seen while running the terraform command. Attempting to run again...

[BUG] Bastion terminal connection keeps disconnecting frequently

Describe the bug
Connection to my bastion terminal window keeps disconnecting time to time

To Reproduce
Open a terminal session to Bastion system (using the ssh cmdline in the output post cluster install)
Wait for some time (15-20 mins of no usage)
The terminal is hung.. no echo of any keystroke nor Ctrl-C works!

Expected behavior
Stable connection to Bastion host

Console Output

[root@deepakcs-ocp-psvs-637a-bastion-0 ~]# client_loop: send disconnect: Connection reset by peer
dpkshetty@DESKTOP-L94HV3O:~$

Desktop (please complete the following information):

  • OS: Ubuntu 20.04
  • Version 20.04
  • Windows terminal : WSL Ubuntu

Additional context
Add any other context about the problem here.

[BUG] variables command does not accept pull secret on Mac OS.

Describe the bug
The variables command does not accept pull secret via read prompt on Mac OS.

To Reproduce
Steps to reproduce the behavior:

  1. Run variables command
  2. Give all inputs till pull-secret prompt
  3. On pull-secret prompt paste the content.
  4. Validate the automation/data/pull-secret.txt that does not contain complete pull secret.

Expected behavior
Pull secret given in variables command should create proper file.

Desktop (please complete the following information):

  • OS: Mac

Additional context
https://unix.stackexchange.com/questions/204815/terminal-does-not-accept-pasted-or-typed-lines-of-more-than-1024-characters

[FEATURE] Allow port 80 to be opened by default for running dev/test S2I applications

Is your feature request related to a problem? Please describe.
I can't run basic nodejs and/or pyflask applications using S2I

Describe the solution you'd like
OCP S2I creates a default route at port 80, hence opening up port 80 is needed

Describe alternatives you've considered
The only alternative is to curl from the Bastion host... not elegant and not an alternative for GUI applications

Additional context
Basic pyflask github @ https://github.com/dpkshetty/ocp-ex-pyflask
I started a S2I workflow using "From Git" option under Developer persona.
Everything worked smooth and I got a route URL from OCP but I cannot access my pyflask application as S2I route serves at port 80 which isn't open for PoweVS by default.

image

Also people coming to OCP on PowerVS from OCP on IBM Cloud, will find it strange as this isn't a issue on IBM Cloud.
I have nodejs and pyflask application running successfully on OCP on IBM Cloud, but the same workflow fails on PowerVS!

Given we are targeting dev/test usecases/scenarios, making the OCP on PowerVS cluster work with basic S2I workflow is of utmost importance!

Handle unknown errors while provisining cluster nodes

The cluster nodes would fail with API errors shown below but the nodes are getting created in the backend.

Error: failed to provision unknown error (status 504): {resp:0xc000cf87e0} 

  on modules/4_nodes/nodes.tf line 92, in resource "ibm_pi_instance" "master":
  92: resource "ibm_pi_instance" "master" {

Next apply throws an error:

Error: failed to provision {"description":"bad request: invalid name server name already exists for cloud-instance","error":"bad request"}

  on modules/4_nodes/nodes.tf line 92, in resource "ibm_pi_instance" "master":
  92: resource "ibm_pi_instance" "master" {

Need to add the check for such cases where created resource can be imported again. This will fix the issues where user do not need to go and delete the instances from the UI.

[FEATURE] Allow system_type in input.

Is your feature request related to a problem? Please describe.
PowerVS provides different system types - E980, S922 and currently the install script doesn't provide a mechanism to accept
the system type as an input.

Describe the solution you'd like
When running the script in interactive mode, it should accept the system type as input

Describe alternatives you've considered
Explicitly setting the system type in var file

[BUG] Displaying the versions of Openshift other than the release version provided

Describe the bug
Displaying the versions of Openshift other than the release version provided.

I am also seeing other versions when we grep the RELEASE_VER=4.9.

Expected behavior
It should show only versions related to 4.9. 4.4.9 should not be displayed

Console Output
See below for example it is showing 4.4.9 as well.

GMX+AVRSAF744@DESKTOP-KS58TIV MINGW64 ~
$ ALL_OCP_VERSIONS=$(curl -skL https://mirror.openshift.com/pub/openshift-v4/ppc64le/clients/ocp/| grep "$RELEASE_VER" | cut -f2 -d '>' | cut -f1 -d '<' | tac)

GMX+AVRSAF744@DESKTOP-KS58TIV MINGW64 ~
$ echo $ALL_OCP_VERSIONS
stable-4.9 latest-4.9 fast-4.9 candidate-4.9 4.9.9 4.9.8 4.9.7 4.9.6 4.9.5 4.9.4 4.9.3 4.9.2 4.9.12 4.9.11 4.9.10 4.9.1 4.9.0 4.9.0-rc.8 4.9.0-rc.7 4.9.0-rc.6 4.9.0-rc.5 4.9.0-rc.4 4.9.0-rc.2 4.9.0-rc.1 4.9.0-rc.0 4.4.9

[FEATURE] Ability to backup and restore the key TF and related data

Is your feature request related to a problem? Please describe.
There could be cases where the system used to deploy the cluster goes bad, or the install folder gets accidentally deleted.
In such cases it should be possible to restore the required data from a backup to continue cluster lifecycle management using the install helper

Describe the solution you'd like
Ability to backup and restore key data required for managing the cluster deployed using TF automation

[BUG]Error: Destroy older clusters created with older version of openshift-install-powervs with new version

Describe the bug
Clusters created using older version (v1.7.0) of script openshift-install-powervs is difficult to delete and get error when executed with newer version of script (v1.9.0).

To Reproduce
Steps to reproduce the behavior: ./openshift-install-powervs destroy -ignore-os-checks -force-destroy

Expected behavior
The script should be able to destroy the older cluster without any issue created with previous versions of script

Console Output
Error when executed the newer version (v1.9.0) of script on older clusters:

$ ./openshift-install-powervs destroy -ignore-os-checks -force-destroy
[main] WARN: Ignoring operating system related checks
[main] WARN: Enabling forceful destruction option for terraform destroy command
[setup_occli] Installing the stable version of OpenShift CLI...
Attempt: 1/5
Client Version: 4.12.16
Kustomize Version: v4.5.7
[destroy] Running terraform destroy... please wait
Attempt: 1/5
[retry_terraform] Encountered below errors:
│ Error: Inconsistent dependency lock file
[retry_terraform] WARN: Issues were seen while running the terraform command. Attempting to run again...
Attempt: 2/5
[retry_terraform] Encountered below errors:
│ Error: Inconsistent dependency lock file
[retry_terraform] WARN: Issues were seen while running the terraform command. Attempting to run again...
Attempt: 3/5
[retry_terraform] Encountered below errors:
│ Error: Inconsistent dependency lock file
[retry_terraform] WARN: Issues were seen while running the terraform command. Attempting to run again...
Attempt: 4/5
[retry_terraform] Encountered below errors:
│ Error: Inconsistent dependency lock file
[retry_terraform] WARN: Issues were seen while running the terraform command. Attempting to run again...
Attempt: 5/5
[retry_terraform] Encountered below errors:
│ Error: Inconsistent dependency lock file

Desktop (please complete the following information):

  • OS: fedora
  • Version: 33
  • Windows terminal: ssh (vscode)

Additional context
Add any other context about the problem here.

Output Variable printing extra text[BUG]

Running this command printing some extra text ./openshift-install-powervs output bastion_public_ip

To Reproduce
Steps to reproduce the behavior:
1)Deploy cluster using script
2) Run command ./openshift-install-powervs -ignore-os-checks output bastion_public_ip

Expected behavior
It should print only the IP address
169.57.xx.xx

Actual behavior
Its printing
[1;33m[main] WARN: Ignoring operating system related checks 169.57.xx.xx

This bug is similar to #142

[BUG] nodes are not rebooting even after 30 mins wait

Describe the bug
node reboot feature when ignition is not available or some failure is not working as expected

To Reproduce
Steps to reproduce the behavior:
Deploy normally, install timeout waiting for bootstrap connection.

Expected behavior
Should reboot bootstrap node at least once and retry ignition.

Console Output

NA

Desktop (please complete the following information):
Any

Additional context
NA

[FEATURE] Ability to use default flavor templates

Is your feature request related to a problem? Please describe.
Default flavor templates are provided. This can be selected during the interactive mode or can pass a "-flavor" argument in the command.

[FEATURE] Support for latest OSX and arm64 Arches

Is your feature request related to a problem? Please describe.

Support for the latest OSX and arm64 arches

Describe the solution you'd like

With the latest Apple laptops, arm64 is the only architecture available.

~ arch
arm64

On https://github.com/ocp-power-automation/openshift-install-power/blob/devel/openshift-install-powervs#L1554

if [[ "${ARCH}" != "x86_64" ]] && [[ "${ARCH}" != "ppc64le" ]]; then

Additions of the arm64 is needed on this line

Further, the OSX version is updated to 13.

sw_vers -productVersion 2>/dev/null || echo ""
13.0.1

On https://github.com/ocp-power-automation/openshift-install-power/blob/devel/openshift-install-powervs#L1562

OS_VERSION=$(sw_vers -productVersion 2>/dev/null || echo "")
      if [[ $IGNORE_OS -eq 0 ]] && [[ $OS_VERSION != "10.15"* && $OS_VERSION != "11."* && $OS_VERSION != "12."* ]]; then

Support would be greatly appreciated.

Describe alternatives you've considered
use a jumpbox

Additional context
n/a

[FEATURE] Delete the bootstrap node after install

Is your feature request related to a problem? Please describe.
Even after successful install the bootstrap node does not get deleted automatically. The bootstrap node is of no use after cluster install and should be deleted to save cost. This can be done as soon as the install is complete.

Describe the solution you'd like
Change the bootstrap count to 0 automatically after install and run create again.
Also, need to handle user providing bootstrap count as 1 in subsequent create commands. This variable should be ignored when user provides the variables file.

Describe alternatives you've considered
Need to manually change bootstrap count to 0 and run create again.

Additional context
None

[FEATURE] Ability to run the script from IBM Cloud Shell

Is your feature request related to a problem? Please describe.
I would like to deploy OpenShift on PowerVS using IBM Cloud shell which comes preconfigured with necessary tools and utilities

https://cloud.ibm.com/docs/cloud-shell?topic=cloud-shell-shell-ui

Describe the solution you'd like
IBM Cloud Shell is a temporary environment. So tfstate and other details need to be stored in a persistent storage like IBM Cloud object store or Git.

The solution should handle this aspect

[BUG] Docker image build is not working on ubi8

Describe the bug
The docker build is failing to create the release image [1]. On further debug found that the coreutils install introduced recently is conflicting with curl package install.

[1] - https://prow.ppc64le-cloud.org/view/s3/prow-logs/logs/openshift-install-power-build-and-push-on-release/1551528326256922624

To Reproduce
Steps to reproduce the behavior:

  1. Run docker build command.

Expected behavior
No errors should be thrown and docker image should be created.

Console Output

[root@0f661ad29aa7 data]# yum install -y curl unzip coreutils tar
Last metadata expiration check: 0:07:03 ago on Mon Jul 25 11:51:46 2022.
Package curl-7.61.1-22.el8_6.3.x86_64 is already installed.
Error:
 Problem: problem with installed package coreutils-single-8.30-12.el8.x86_64
  - package coreutils-8.30-12.el8.x86_64 conflicts with coreutils-single provided by coreutils-single-8.30-12.el8.x86_64
  - conflicting requests
(try to add '--allowerasing' to command line to replace conflicting packages or '--skip-broken' to skip uninstallable packages or '--nobest' to use not only best candidate packages)

Desktop (please complete the following information):

  • OS: RHEL UBI
  • Version 8.6

Additional context
Blocking new version Docker image release.

test_overprovision_level_policy_control.py are failing in v4.10.12 on POWER

tests/manage/pv_services/test_overprovision_level_policy_control.py::TestOverProvisionLevelPolicyControl::test_over_provision_level_policy_control[ocs-storagecluster-ceph-rbd-CephBlockPool]
tests/manage/pv_services/test_overprovision_level_policy_control.py::TestOverProvisionLevelPolicyControl::test_over_provision_level_policy_control[ocs-storagecluster-cephfs-CephFileSystem]
tests/manage/pv_services/test_overprovision_level_policy_control.py::TestOverProvisionLevelPolicyControl::test_over_provision_level_policy_control[sc-test-blk-CephBlockPool]
tests/manage/pv_services/test_overprovision_level_policy_control.py::TestOverProvisionLevelPolicyControl::test_over_provision_level_policy_control[sc-test-fs-CephFileSystem]
tests/manage/pv_services/test_overprovision_level_policy_control_with_capacity.py::TestOverProvisionLevelPolicyControlWithCapacity::test_overprovision_level_policy_control_with_capacity

These are the testcases which are failing in the 4.10.12 odf version on Power

FAILURE :

=================================== FAILURES ===================================
_ TestOverProvisionLevelPolicyControl.test_over_provision_level_policy_control[ocs-storagecluster-ceph-rbd-CephBlockPool] _

self = <tests.manage.pv_services.test_overprovision_level_policy_control.TestOverProvisionLevelPolicyControl object at 0x7fff68f0beb0>
setup_sc = {'ocs-storagecluster-ceph-rbd': None, 'ocs-storagecluster-cephfs': None, 'sc-test-blk': <ocs_ci.ocs.resources.ocs.OCS object at 0x7fff6abbde50>, 'sc-test-fs': <ocs_ci.ocs.resources.ocs.OCS object at 0x7fff68ec2280>}
sc_name = 'ocs-storagecluster-ceph-rbd', sc_type = 'CephBlockPool'
teardown_project_factory = <function teardown_project_factory_fixture.<locals>.factory at 0x7fff49a75820>
pvc_factory = <function pvc_factory_fixture.<locals>.factory at 0x7fff49a75a60>
pod_factory = <function pod_factory_fixture.<locals>.factory at 0x7fff49a75b80>
storageclass_factory = <function storageclass_factory_fixture.<locals>.factory at 0x7fff49a75f70>

    @pytest.mark.parametrize(
        argnames=["sc_name", "sc_type"],
        argvalues=[
            pytest.param(*[constants.CEPHBLOCKPOOL_SC, constants.CEPHBLOCKPOOL]),
            pytest.param(*[constants.CEPHFILESYSTEM_SC, constants.CEPHFILESYSTEM]),
            pytest.param(
                *["sc-test-blk", constants.CEPHBLOCKPOOL],
                marks=[skipif_ocs_version("<4.10")],
            ),
            pytest.param(
                *["sc-test-fs", constants.CEPHFILESYSTEM],
                marks=[skipif_ocs_version("<4.10")],
            ),
        ],
    )
    def test_over_provision_level_policy_control(
        self,
        setup_sc,
        sc_name,
        sc_type,
        teardown_project_factory,
        pvc_factory,
        pod_factory,
        storageclass_factory,
    ):
        """
        Test Process:
            1.Create project with “openshift-quota” label
            2.Create new Storage Class or use ceph-rbd or ceph-fs
            3.Add “overprovisionControl” section to storagecluster yaml file [max=8Gi]
            4.Verify storagecluster on Ready state
            5.Create 5Gi pvc on project “ocs-quota-sc-test” with sc “sc-test”
            6.Create new pvc with 6Gi capacity and verify it failed [6Gi + 5Gi > 8Gi]
            7.Resize PVC to 20Gi and verify it failed [20Gi > 8Gi]
            8.Resize the PVC to 6Gi and verify it is working [8Gi > 6Gi]
            9.Create New PVC with 1G capacity and verify it is working [8Gi > 1Gi + 6Gi]

        """
        quota_names = {
            constants.CEPHBLOCKPOOL_SC: "ocs-storagecluster-ceph-rbd-quota-sc-test",
            constants.CEPHFILESYSTEM_SC: "ocs-storagecluster-cephfs-quota-sc-test",
            "sc-test-blk": "sc-test-blk-quota-sc-test",
            "sc-test-fs": "sc-test-fs-quota-sc-test",
        }
        self.quota_name = quota_names[sc_name]
        log.info("Create project with “openshift-quota” label")
        project_name = "ocs-quota-sc-test"
        ocp_project_label = OCP(kind=constants.NAMESPACE)
        ocp_project_label.new_project(project_name=project_name)
        ocp_project_label.add_label(
            resource_name=project_name, label="openshift-quota=quota-sc-test"
        )
        ocp_project_obj = OCP(kind="Project", namespace=project_name)
        teardown_project_factory(ocp_project_obj)

        sc_obj = setup_sc.get(sc_name)

        log.info("Add 'overprovisionControl' section to storagecluster yaml file")
        storagecluster_obj = OCP(
            resource_name=constants.DEFAULT_CLUSTERNAME,
            namespace=defaults.ROOK_CLUSTER_NAMESPACE,
            kind=constants.STORAGECLUSTER,
        )
        sc_name_str = f'"{sc_name}"'
        params = (
            '{"spec": {"overprovisionControl": [{"capacity": "8Gi","storageClassName":'
            + sc_name_str
            + ', "quotaName": '
            '"quota-sc-test", "selector": {"labels": {"matchLabels": {"openshift-quota":"quota-sc-test"}}}}]}}'
        )
        storagecluster_obj.patch(
            params=params,
            format_type="merge",
        )

        log.info("Verify storagecluster on Ready state")
        verify_storage_cluster()

        clusterresourcequota_obj = OCP(kind="clusterresourcequota")
        sample = TimeoutSampler(
            timeout=60,
            sleep=4,
            func=verify_quota_resource_exist,
            quota_name=quota_names[sc_name],
        )
        if not sample.wait_for_func_status(result=True):
            err_str = (
                f"Quota resource {quota_names[sc_name]} does not exist "
                f"after 60 seconds {clusterresourcequota_obj.describe()}"
            )
            log.error(err_str)
            raise TimeoutExpiredError(err_str)

        log.info("Check clusterresourcequota output")
        output_clusterresourcequota = clusterresourcequota_obj.describe(
            resource_name=quota_names[sc_name]
        )
        log.info(f"Output Cluster Resource Quota: {output_clusterresourcequota}")
>       assert self.verify_substrings_in_string(
            output_string=output_clusterresourcequota, expected_strings=["8Gi", "0"]
        ), f"{output_clusterresourcequota}\n expected string does not exist."
E       AssertionError: Name:           ocs-storagecluster-ceph-rbd-quota-sc-test
E         Created:      Less than a second ago
E         Labels:               <none>
E         Annotations:  <none>
E         Namespace Selector: []
E         Label Selector: openshift-quota=quota-sc-test
E         AnnotationSelector: map[]
E         Resource      Used    Hard
E         --------      ----    ----
E
E          expected string does not exist.
E       assert False
E        +  where False = <bound method TestOverProvisionLevelPolicyControl.verify_substrings_in_string of <tests.manage.pv_services.test_overprovision_level_policy_control.TestOverProvisionLevelPolicyControl object at 0x7fff68f0beb0>>(output_string='Name:\t\tocs-storagecluster-ceph-rbd-quota-sc-test\nCreated:\tLess than a second ago\nLabels:\t\t<none>\nAnnotations:...Label Selector: openshift-quota=quota-sc-test\nAnnotationSelector: map[]\nResource\tUsed\tHard\n--------\t----\t----\n', expected_strings=['8Gi', '0'])
E        +    where <bound method TestOverProvisionLevelPolicyControl.verify_substrings_in_string of <tests.manage.pv_services.test_overprovision_level_policy_control.TestOverProvisionLevelPolicyControl object at 0x7fff68f0beb0>> = <tests.manage.pv_services.test_overprovision_level_policy_control.TestOverProvisionLevelPolicyControl object at 0x7fff68f0beb0>.verify_substrings_in_string

tests/manage/pv_services/test_overprovision_level_policy_control.py:171: AssertionError

Then also, done "git checkout release-4.10" and ran the testcase, still it was failing with the below error :
Error :
MainThread - tests.manage.pv_services.test_overprovision_level_policy_control - ERROR - Quota resource ocs-ocs-storagecluster-ceph-rbd-quota-sc-test does not exist after 60 seconds Name: ocs-storagecluster-ceph-rbd-quota-sc-test

=================================== FAILURES ===================================
_ TestOverProvisionLevelPolicyControl.test_over_provision_level_policy_control[ocs-storagecluster-ceph-rbd-CephBlockPool] _

self = <tests.manage.pv_services.test_overprovision_level_policy_control.TestOverProvisionLevelPolicyControl object at 0x7fff98e51940>
setup_sc = {'ocs-storagecluster-ceph-rbd': None, 'ocs-storagecluster-cephfs': None, 'sc-test-blk': <ocs_ci.ocs.resources.ocs.OCS object at 0x7fff9ae61850>, 'sc-test-fs': <ocs_ci.ocs.resources.ocs.OCS object at 0x7fff9a00e4c0>}
sc_name = 'ocs-storagecluster-ceph-rbd', sc_type = 'CephBlockPool'
teardown_project_factory = <function teardown_project_factory_fixture.<locals>.factory at 0x7fff70b7fd30>
pvc_factory = <function pvc_factory_fixture.<locals>.factory at 0x7fff70b7ff70>
pod_factory = <function pod_factory_fixture.<locals>.factory at 0x7fff98ee40d0>
storageclass_factory = <function storageclass_factory_fixture.<locals>.factory at 0x7fff98ee44c0>

    @pytest.mark.parametrize(
        argnames=["sc_name", "sc_type"],
        argvalues=[
            pytest.param(*[constants.CEPHBLOCKPOOL_SC, constants.CEPHBLOCKPOOL]),
            pytest.param(*[constants.CEPHFILESYSTEM_SC, constants.CEPHFILESYSTEM]),
            pytest.param(
                *["sc-test-blk", constants.CEPHBLOCKPOOL],
                marks=[skipif_ocs_version("<4.10")],
            ),
            pytest.param(
                *["sc-test-fs", constants.CEPHFILESYSTEM],
                marks=[skipif_ocs_version("<4.10")],
            ),
        ],
    )
    def test_over_provision_level_policy_control(
        self,
        setup_sc,
        sc_name,
        sc_type,
        teardown_project_factory,
        pvc_factory,
        pod_factory,
        storageclass_factory,
    ):
        """
        Test Process:
            1.Create project with “openshift-quota” label
            2.Create new Storage Class or use ceph-rbd or ceph-fs
            3.Add “overprovisionControl” section to storagecluster yaml file [max=8Gi]
            4.Verify storagecluster on Ready state
            5.Create 5Gi pvc on project “ocs-quota-sc-test” with sc “sc-test”
            6.Create new pvc with 6Gi capacity and verify it failed [6Gi + 5Gi > 8Gi]
            7.Resize PVC to 20Gi and verify it failed [20Gi > 8Gi]
            8.Resize the PVC to 6Gi and verify it is working [8Gi > 6Gi]
            9.Create New PVC with 1G capacity and verify it is working [8Gi > 1Gi + 6Gi]

        """
        quota_names = {
            constants.CEPHBLOCKPOOL_SC: "ocs-storagecluster-ceph-rbd-quota-sc-test",
            constants.CEPHFILESYSTEM_SC: "ocs-storagecluster-cephfs-quota-sc-test",
            "sc-test-blk": "sc-test-blk-quota-sc-test",
            "sc-test-fs": "sc-test-fs-quota-sc-test",
        }
        ocs_version = version.get_semantic_ocs_version_from_config()
        # Bug fixed on ODF4.11 https://bugzilla.redhat.com/show_bug.cgi?id=2158277
        if ocs_version == version.VERSION_4_10:
            for sc in quota_names:
                quota_names[sc] = f"ocs-{quota_names[sc]}"
        self.quota_name = quota_names[sc_name]
        log.info("Create project with “openshift-quota” label")
        project_name = "ocs-quota-sc-test"
        ocp_project_label = OCP(kind=constants.NAMESPACE)
        ocp_project_label.new_project(project_name=project_name)
        ocp_project_label.add_label(
            resource_name=project_name, label="openshift-quota=quota-sc-test"
        )
        ocp_project_obj = OCP(kind="Project", namespace=project_name)
        teardown_project_factory(ocp_project_obj)

        sc_obj = setup_sc.get(sc_name)

        log.info("Add 'overprovisionControl' section to storagecluster yaml file")
        storagecluster_obj = OCP(
            resource_name=constants.DEFAULT_CLUSTERNAME,
            namespace=defaults.ROOK_CLUSTER_NAMESPACE,
            kind=constants.STORAGECLUSTER,
        )
        sc_name_str = f'"{sc_name}"'
        params = (
            '{"spec": {"overprovisionControl": [{"capacity": "8Gi","storageClassName":'
            + sc_name_str
            + ', "quotaName": '
            '"quota-sc-test", "selector": {"labels": {"matchLabels": {"openshift-quota":"quota-sc-test"}}}}]}}'
        )
        storagecluster_obj.patch(
            params=params,
            format_type="merge",
        )

        log.info("Verify storagecluster on Ready state")
        verify_storage_cluster()

        clusterresourcequota_obj = OCP(kind="clusterresourcequota")
        sample = TimeoutSampler(
            timeout=60,
            sleep=4,
            func=verify_quota_resource_exist,
            quota_name=quota_names[sc_name],
        )
        if not sample.wait_for_func_status(result=True):
            err_str = (
                f"Quota resource {quota_names[sc_name]} does not exist "
                f"after 60 seconds {clusterresourcequota_obj.describe()}"
            )
            log.error(err_str)
>           raise TimeoutExpiredError(err_str)
E           ocs_ci.ocs.exceptions.TimeoutExpiredError: Timed Out: Quota resource ocs-ocs-storagecluster-ceph-rbd-quota-sc-test does not exist after 60 seconds Name:              ocs-storagecluster-ceph-rbd-quota-sc-test
E           Created:    About a minute ago
E           Labels:             <none>
E           Annotations:        <none>
E           Namespace Selector: []
E           Label Selector: openshift-quota=quota-sc-test
E           AnnotationSelector: map[]
E           Resource    Used    Hard
E           --------    ----    ----
E
E
E           Name:               ocs-storagecluster-ceph-rbd-storagequota
E           Created:    5 days ago
E           Labels:             <none>
E           Annotations:        <none>
E           Namespace Selector: []
E           Label Selector: storagequota=storagequota
E           AnnotationSelector: map[]
E           Resource    Used    Hard
E           --------    ----    ----
E
E
E           Name:               sc-test-blk-quota-sc-test
E           Created:    2 days ago
E           Labels:             <none>
E           Annotations:        <none>
E           Namespace Selector: []
E           Label Selector: openshift-quota=quota-sc-test
E           AnnotationSelector: map[]
E           Resource    Used    Hard
E           --------    ----    ----

tests/manage/pv_services/test_overprovision_level_policy_control.py:171: TimeoutExpiredError

Cluster version is : Cluster version is 4.9.0-0.nightly-ppc64le-2023-04-17-220850
ODF version : full_version: 4.10.12-1

[BUG] script stop unexpectedly when using -all-images option

Describe the bug
openshift-install-powervs create -all-images script stop unexpectedly
I am following the steps mentioned here...
https://developer.ibm.com/tutorials/install-ocp-on-power-vs/

Altho' rhel and rhcos images are present under "boot images" for my PowerVS service, it fails to detect rhcos image.
Hence I tried with -all-images but it stop unexpectedly!

Expected behavior
It should list me the images for rhel and rhcos to choose

Console Output

dpkshetty@DESKTOP-L94HV3O:~/ocp_on_psvs/ocp-install-dir$ openshift-install-powervs create
[setup_tools] Verifying the latest packages and tools
[precheck_input] WARN: No variables specified or var.tfvars does not exist.. running variables command
[setup_tools] Verifying the latest packages and tools
[variables] Trying to login with the provided IBMCLOUD_API_KEY...
[question] > Select the Service Instance name to use: (Power Systems Virtual Server-Techline)
?
- You have answered: Power Systems Virtual Server-Techline
Targeting service crn:v1:bluemix:public:power-iaas:us-south:a/a39ac7399e414b87e56898b0c3dcda59:bd64dd0b-7a6e-4ce0-b97b-eb2edc606581::...
[variables] Gathering information from the selected Service Instance... Please wait
[variables] ERROR: Cannot find RHCOS image for OCP 4.6! Please use option '-all-images' if you have already imported the image


dpkshetty@DESKTOP-L94HV3O:~/ocp_on_psvs/ocp-install-dir$ openshift-install-powervs create -all-images
[setup_tools] Verifying the latest packages and tools
[precheck_input] WARN: No variables specified or var.tfvars does not exist.. running variables command
[setup_tools] Verifying the latest packages and tools
[variables] Trying to login with the provided IBMCLOUD_API_KEY...
[question] > Select the Service Instance name to use: (Power Systems Virtual Server-Techline)
?
- You have answered: Power Systems Virtual Server-Techline
Targeting service crn:v1:bluemix:public:power-iaas:us-south:a/a39ac7399e414b87e56898b0c3dcda59:bd64dd0b-7a6e-4ce0-b97b-eb2edc606581::...
[variables] Gathering information from the selected Service Instance... Please wait
dpkshetty@DESKTOP-L94HV3O:~/ocp_on_psvs/ocp-install-dir$

Desktop (please complete the following information):

  • OS: Win10
  • Version Win10
  • Windows terminal WSL Ubuntu 20.04

Additional context

[FEATURE] Able to run multiple deployments on a client machine using different install dir

Is your feature request related to a problem? Please describe.
The script is not able to on multiple install directory in parallel. I am not able to do two deploys at a time.

Describe the solution you'd like
I should be able to do more than 1 deployment using different install/config directory.

Describe alternatives you've considered
The problem is the ibmcloud CLI config is placed in HOME directory of current user. We can change that using environment variable IBMCLOUD_HOME=<install-dir>.

Additional context
The ibmcloud CLI config stores the current target CRN in ~/.bluemix/plugins/power-iaas/config.json making it diffult to run more than one deployment on different regions or service instances.

[BUG] Accessing cluster using local `oc` doesn't work as documented

Describe the bug
After succesfull completion of the openshift-install-powervs create script, we get the final msg, part of which says...
To access the cluster on local system when using 'oc' run: export KUBECONFIG=/home/dpkshetty/ocp_on_psvs/ocp-install-dir/automation/kubeconfig
But that didn't work for me!

To Reproduce

dpkshetty@DESKTOP-L94HV3O:~$ export KUBECONFIG=/home/dpkshetty/ocp_on_psvs/ocp-install-dir/automation/kubeconfig
dpkshetty@DESKTOP-L94HV3O:~$ less $KUBECONFIG
dpkshetty@DESKTOP-L94HV3O:~$ oc get nodes
The connection to the server api.deepakcs-ocp-psvs-637a.ocp.deepakcs:6443 was refused - did you specify the right host or port?
dpkshetty@DESKTOP-L94HV3O:~$ ping api.deepakcs-ocp-psvs-637a.ocp.deepakcs
PING api.deepakcs-ocp-psvs-637a.ocp.deepakcs (52.117.41.83) 56(84) bytes of data.
64 bytes from api.deepakcs-ocp-psvs-637a.ocp.deepakcs (52.117.41.83): icmp_seq=1 ttl=48 time=285 ms
64 bytes from api.deepakcs-ocp-psvs-637a.ocp.deepakcs (52.117.41.83): icmp_seq=2 ttl=48 time=303 ms
^C
--- api.deepakcs-ocp-psvs-637a.ocp.deepakcs ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 1000ms
rtt min/avg/max/mdev = 284.732/294.057/303.382/9.325 ms
dpkshetty@DESKTOP-L94HV3O:~$

Expected behavior
oc from my local terminal should work as documented

Console Output
See above

Desktop (please complete the following information):

  • OS: Ubuntu
  • Version: 20.04
  • Windows terminal: WSL

Additional context
None

unable to fetch plugins from power nodes

hi @yussufsh , on power node getting below failures caused by latest - https://github.com/ocp-power-automation/terraform-providers-power/releases ?

# ./openshift-install-powervs setup
[setup_ibmcloudcli] Installing the latest version of IBM-Cloud CLI...
Attempt: 1/5
/root/powervs-clusters/ocp-install-test/ibmcloud version 2.15.0+79fb346-2023-02-21T22:06:52+00:00
[setup_poweriaas] Installing power-iaas plugin...
[setup_terraform] Installing the latest version of Terraform...
Attempt: 1/5
Terraform v1.3.9
Attempt: 1/5
[setup_occli] Installing the stable version of OpenShift CLI...
Attempt: 1/5
Client Version: 4.12.3
Kustomize Version: v4.5.7
[setup] Running setup command...
[setup_artifacts] Downloading code artifacts main in ./automation
Attempt: 1/5
[setup] SUCCESS: setup command completed!

# cd automation/
# ../terraform init
Initializing modules...
- ibmcloud in modules/7_ibmcloud
- install in modules/5_install
- nodes in modules/4_nodes
- prepare in modules/1_prepare

Initializing the backend...

Initializing provider plugins...
- Reusing previous version of community-terraform-providers/ignition from the dependency lock file
- Reusing previous version of hashicorp/random from the dependency lock file
- Reusing previous version of ibm-cloud/ibm from the dependency lock file
- Reusing previous version of hashicorp/null from the dependency lock file
╷
│ Error: Incompatible provider version
│
│ Provider registry.terraform.io/hashicorp/random v3.4.3 does not have a package available for your current platform,
│ linux_ppc64le.
│
│ Provider releases are separate from Terraform CLI releases, so not all providers are available for all platforms. Other
│ versions of this provider may have different platforms supported.
╵

╷
│ Error: Incompatible provider version
│
│ Provider registry.terraform.io/ibm-cloud/ibm v1.49.0 does not have a package available for your current platform,
│ linux_ppc64le.
│
│ Provider releases are separate from Terraform CLI releases, so not all providers are available for all platforms. Other
│ versions of this provider may have different platforms supported.
╵

╷
│ Error: Incompatible provider version
│
│ Provider registry.terraform.io/hashicorp/null v3.2.1 does not have a package available for your current platform,
│ linux_ppc64le.
│
│ Provider releases are separate from Terraform CLI releases, so not all providers are available for all platforms. Other
│ versions of this provider may have different platforms supported.
╵

╷
│ Error: Incompatible provider version
│
│ Provider registry.terraform.io/community-terraform-providers/ignition v2.1.3 does not have a package available for your
│ current platform, linux_ppc64le.
│
│ Provider releases are separate from Terraform CLI releases, so not all providers are available for all platforms. Other
│ versions of this provider may have different platforms supported.
╵

# ../openshift-install-powervs -version
Version v1.8.0

[FEATURE] Create container image with the script and dependencies

Is your feature request related to a problem? Please describe.
A container image with the script and the dependencies including the terraform code will provide additional flexibility and also enable additional use cases. For example deploying a cluster as part of a CI/CD workflow.

Describe the solution you'd like
A container image having the following

  1. installer script
  2. dependencies - terraform, ibmcloud, oc binaries
  3. TF automation code

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.