karmab / aicli Goto Github PK

Hi @karmab

could you consider decreasing the amount of "Waiting" for host/cluster messages, maybe put them only when user run with debug
or single message "Waiting ${TIMEOUT}s for cluster to reach state installed"

https://github.com/karmab/aicli/blob/main/ailib/__init__.py#L960-L1001

Hi,

Environment Details:

AI version : 2.5.2
RHOCP : 4.10.20
Single Node OCP - disconnected Installation

Problem Description:

SNO installation fails with below error as soon as preparation starts:
time="2022-07-07T14:23:58Z" level=warning msg="Cluster installation initialization failed" func="github.com/openshift/assisted-service/internal/bminventory.(*bareMetalInventory).InstallClusterInternal.func3.1" file="/go/src/github.com/openshift/origin/internal/bminventory/inventory.go:1126" cluster_id=962f5c7b-6a5e-4742-8f9a-89eaf357d2be error="failed generating install config for cluster 962f5c7b-6a5e-4742-8f9a-89eaf357d2be: error running openshift-install manifests, level=fatal msg=failed to fetch Master Machines: failed to load asset "Install Config": invalid "install-config.yaml" file: platform.baremetal.Hosts: Required value: not enough hosts found (0) to support all the configured ControlPlane replicas (1)\n: exit status 1" go-id=10936 pkg=Inventory request_id=fd2af407-8cc8-4ffa-a095-bf0494e82571
*time="2022-07-07T14:23:58Z" level=warning msg="Failed to prepare installation of cluster 962f5c7b-6a5e-4742-8f9a-89eaf357d2be" func="github.com/openshift/assisted-service/internal/cluster.(Manager).HandlePreInstallError" file="/go/src/github.com/openshift/origin/internal/cluster/cluster.go:885" cluster_id=962f5c7b-6a5e-4742-8f9a-89eaf357d2be error="failed generating install config for cluster 962f5c7b-6a5e-4742-8f9a-89eaf357d2be: error running openshift-install manifests, level=fatal msg=failed to fetch Master Machines: failed to load asset "Install Config": invalid "install-config.yaml" file: platform.baremetal.Hosts: Required value: not enough hosts found (0) to support all the configured ControlPlane replicas (1)\n: exit status 1" go-id=11714 pkg=cluster-state request_id=
time="2022-07-07T14:23:58Z" level=info msg="Successfully handled pre-installation error, cluster 962f5c7b-6a5e-4742-8f9a-89eaf357d2be" func="github.com/openshift/assisted-service/internal/cluster.(*Manager).HandlePreInstallError" file="/go/src/github.com/openshift/origin/internal/cluster/cluster.go:892" cluster_id=962f5c7b-6a5e-4742-8f9a-89eaf357d2be go-id=11714 pkg=cluster-state request_id=
....

Procedure followed:

Prepared the cluster_parameters.yaml and static_network_config.yaml required to create the cluster and Infra env.
All steps executed using aicli commands.

• cluster was created successfully
• Infr env was updated successfully
• Discovery ISO was generated
• Host was booted successfully with discovery.iso
• Triggered installation after host discovery

Installation failed.

Entered the shell of assisted-service to check the generated install-config.yaml.
- cat /data/install-config-generate/7604e951-76fe-47dc-af6c-280da34394f1-failed/install-config.yaml
Attached the install-config.yaml
install-config.zip
...
compute:

• hyperthreading: Enabled
  name: worker
  replicas: 0
  controlPlane:
  hyperthreading: Enabled
  name: master
  replicas: 1
  platform:
  baremetal:
  provisioningNetwork: ""
  apiVIP: ""
  ingressVIP: ""
  hosts: []
  ...

platform.baremetal.hosts is filled empty. Host data is not collected and prepared as expected.

Note:-
Attached cluster_parameters.yaml & static_network_config.yaml used to create the cluster & infra in "input_files.zip"

input_files.zip

Regards,
Venkat B

Seems that commit f535797 broke access to the API.

Traceback (most recent call last):
  File "/usr/local/bin/aicli", line 11, in <module>
    sys.exit(cli())
  File "/usr/local/lib/python3.6/site-packages/ailib/cli.py", line 552, in cli
    args.func(args)
  File "/usr/local/lib/python3.6/site-packages/ailib/cli.py", line 118, in list_cluster
    clusters = ai.list_clusters()
  File "/usr/local/lib/python3.6/site-packages/ailib/__init__.py", line 241, in list_clusters
    return self.client.list_clusters()
  File "/usr/local/lib/python3.6/site-packages/assisted_service_client/api/installer_api.py", line 3085, in list_clusters
    (data) = self.list_clusters_with_http_info(**kwargs)  # noqa: E501
  File "/usr/local/lib/python3.6/site-packages/assisted_service_client/api/installer_api.py", line 3169, in list_clusters_with_http_info
    collection_formats=collection_formats)
  File "/usr/local/lib/python3.6/site-packages/assisted_service_client/api_client.py", line 330, in call_api
    _preload_content, _request_timeout)
  File "/usr/local/lib/python3.6/site-packages/assisted_service_client/api_client.py", line 161, in __call_api
    _request_timeout=_request_timeout)
  File "/usr/local/lib/python3.6/site-packages/assisted_service_client/api_client.py", line 351, in request
    headers=headers)
  File "/usr/local/lib/python3.6/site-packages/assisted_service_client/rest.py", line 238, in GET
    query_params=query_params)
  File "/usr/local/lib/python3.6/site-packages/assisted_service_client/rest.py", line 228, in request
    raise ApiException(http_resp=r)
assisted_service_client.rest.ApiException: (404)
Reason: Not Found
HTTP response headers: HTTPHeaderDict({'content-type': 'application/json', 'vary': 'Origin', 'date': 'Thu, 22 Jul 2021 08:24:21 GMT', 'content-length': '80', 'set-cookie': '30a266cfd9eb7f5fab315abc251db761=f5ebeebafc5560636ba2f17fa53f61eb; path=/; HttpOnly; Secure; SameSite=None', 'cache-control': 'private', 'x-envoy-upstream-service-time': '1', 'server': 'envoy'})
HTTP response body: {"code":404,"message":"path /api/assisted-install/v1/v1/clusters was not found"}

After removing config.host += "/v1 in line 31 from __init__.py the API connection is working well.

We have a cluster with 5 hosts (3 Control and 2 Compute). OCP version is 4.8.
Once we trigger the installation we tried to use the following command but it ends up with AttributeError as below =>

$ aicli wait cluster sprintlab696
Waiting 5s for cluster sprintlab696 to reach state installed
Traceback (most recent call last):
File "/home/admin/.local/bin/aicli", line 11, in
sys.exit(cli())
File "/home/admin/.local/lib/python3.6/site-packages/ailib/cli.py", line 1042, in cli
args.func(args)
File "/home/admin/.local/lib/python3.6/site-packages/ailib/cli.py", line 445, in wait_cluster
ai.wait_cluster(args.cluster, args.status)
File "/home/admin/.local/lib/python3.6/site-packages/ailib/init.py", line 731, in wait_cluster
self.refresh_token(self.token, self.offlinetoken)
AttributeError: 'AssistedClient' object has no attribute 'token'

AI offline is used (latest AI container images are in use).
Its a disconnected installation.

Any hints why it ends up in that error?

We are trying to install using Assisted Installer offline, OCP 4.10.26 cluster with 'pure IPv4' network (no dual stack) and the following command always fails =>

aicli --url=http://localhost:8090/ create cluster test --paramfile /var/cluster_parameters.yaml

Creating cluster test
Traceback (most recent call last):
File "/home/admin/.local/bin/aicli", line 11, in
sys.exit(cli())
File "/home/admin/.local/lib/python3.6/site-packages/ailib/cli.py", line 1229, in cli
args.func(args)
File "/home/admin/.local/lib/python3.6/site-packages/ailib/cli.py", line 69, in create_cluster
ai.create_cluster(args.cluster, overrides.copy())
File "/home/admin/.local/lib/python3.6/site-packages/ailib/init.py", line 410, in create_cluster
self.update_cluster(name, extra_overrides)
File "/home/admin/.local/lib/python3.6/site-packages/ailib/init.py", line 878, in update_cluster
self.client.v2_update_cluster(cluster_id=cluster_id, cluster_update_params=cluster_update_params)
File "/home/admin/.local/lib/python3.6/site-packages/assisted_service_client/api/installer_api.py", line 4982, in v2_update_cluster
(data) = self.v2_update_cluster_with_http_info(cluster_id, cluster_update_params, **kwargs) # noqa: E501
File "/home/admin/.local/lib/python3.6/site-packages/assisted_service_client/api/installer_api.py", line 5067, in v2_update_cluster_with_http_info
collection_formats=collection_formats)
File "/home/admin/.local/lib/python3.6/site-packages/assisted_service_client/api_client.py", line 330, in call_api
_preload_content, _request_timeout)
File "/home/admin/.local/lib/python3.6/site-packages/assisted_service_client/api_client.py", line 161, in __call_api
_request_timeout=_request_timeout)
File "/home/admin/.local/lib/python3.6/site-packages/assisted_service_client/api_client.py", line 389, in request
body=body)
File "/home/admin/.local/lib/python3.6/site-packages/assisted_service_client/rest.py", line 295, in PATCH
body=body)
File "/home/admin/.local/lib/python3.6/site-packages/assisted_service_client/rest.py", line 228, in request
raise ApiException(http_resp=r)
assisted_service_client.rest.ApiException: (400)
Reason: Bad Request
HTTP response headers: HTTPHeaderDict({'Content-Type': 'application/json', 'Date': 'Fri, 09 Sep 2022 06:13:07 GMT', 'Content-Length': '150'})
HTTP response body: {"code":"400","href":"","id":400,"kind":"Error","reason":"Setting Machine network CIDR is forbidden when cluster is not in vip-dhcp-allocation mode"}

The input file cluster_parameters.yaml is as follows =>

pull_secret: /var/pull-secret-update.txt
additional_ntp_source: 10.18.17.40
high_availability_mode: Full
schedulable_masters: false
api_vip: 10.91.129.194
ingress_vip: 10.91.129.195
vip_dhcp_allocation: false
openshift_version: "4.10"
base_dns_domain: abc.com
ssh_public_key: "ecdsa-sha2-nistp521 AAAAE2VjZHNhLXNoYTItbmllzdHA1MjEAAACFBACW1MMoFAuG9msxyg/63b4mM5V1MAzTCefZ5Q9pc0nU7ZLds1KzJdoDzUSPI9XPziUgAQx2+6UgY5FFt30Q+SbEzAHoqvgFOcbjz2srEmg7aH+Np/ZuJQycGYzNlROztyDCxAm2qoFDnoE2KKMzzaSMsi0RNyurFzZxpINDOQ== admin@test"
network_type: OVNKubernetes

machine_networks:

• cidr: 10.91.128.0/20
  service_networks:
• 172.30.0.0/16
  cluster_networks:
• cidr: 10.128.0.0/14
  hostPrefix: 23

installconfig:
platform:
baremetal:
clusterosimage: http://myimgsrv:8080/rhcos-4.10.16-x86_64-openstack.x86_64.qcow2.gz?sha256=e0a1d8a99c5869150a56b8de475ea7952ca2fa3aacad7ca48533d1176df503ab
ignition_config_override: '{"ignition": {"version":"3.2.0"},"storage": {"files": [{"path": "/etc/pki/ca-trust/source/anchors/reg_ca.pem","mode": 420,"overwrite": true,"user": {"name": "root"},"contents": {"source": "data:text/plain;base64,<...>="}},{"contents": {"source": "data:text/plain;charset=utf-8;base64,ZGVmYXVsdHMgewogICAgdXNlcl9mcmllbmRseV9uYW1lcyB5ZXMKICAgIGZpbmRfbXVsdGlwYXRocyB5ZXMKICAgIGVuYWJsZV9mb3JlaWduICJeJCIKfQpibGFja2xpc3RfZXhjZXB0aW9ucyB7CiAgICBwcm9wZXJ0eSAiKFNDU0lfSURFTlRffElEX1dXTikiCn0KYmxhY2tsaXN0IHsKfQo="},"filesystem": "root","mode": 420,"path" :"/etc/multipath.conf"}]},"systemd": {"units": [{"name": "multipathd.service","enabled": true}]}}'

fips: true

If you see in the above yaml, the machine_networks is specified. This leads to =>
"Setting Machine network CIDR is forbidden when cluster is not in vip-dhcp-allocation mode"

I have attached the tcpdump network capture showing the final PATCH command failing.

But when we remove the machine_networks entry (with its cidr) from the file, the cluster creation is successful.

Question: machine_networks is necessary or not necessary to be specified?

Additional Information: We then created the same cluster using JSON file and using curl command to fire it to the AI Service. This JSON has the machine_networks entry and the cluster created successfully. JSON used here (cluster_configv4.json) =>

{
"additional_ntp_source": "10.18.17.40",
"name": "test",
"high_availability_mode": "Full",
"schedulable_masters": true,
"api_vip": "10.92.82.170",
"ingress_vip": "10.92.82.171",
"vip_dhcp_allocation": false,
"openshift_version": "4.10",
"base_dns_domain": "netact.nsn-rdnet.net",
"pull_secret": "{"auths": {"test.com:5000": {"auth": "cmVyMw=="}}}",
"ssh_public_key": "ecdsa-sha2-nistp521 AAAAE2VjZHNhLXNoYTItbmlzdHA1MjEAAAAIbmlzdHA1MjEAAACFBAEyCQqywxcBhNuAuYoYGt+ktrzcNQ+PSZ+qnL/wrKJiXeXIQTLWyfyLT0G/HjhbINtTpnmSkAiiSB5FHRzx262KKQFag92QFUHTpmAmgNUs3SID1Xu3hzHkdKhBXLnA== admin@test",
"network_type": "OVNKubernetes",
"machine_networks": [
{
"cidr": "10.92.80.0/20"
}
],
"service_networks": [
{
"cidr": "172.30.0.0/16"
}
],
"cluster_networks": [
{
"cidr": "10.128.0.0/14",
"host_prefix": 23
}
]
}

Then doing =>

curl -H "Content-Type: application/json" -d @/var/tmp/cluster_configv4.json -X POST http://localhost:8090/api/assisted-install/v2/clusters

WORKS OK!

What could be going wrong here?

In order to enable multipath at first boot of AI discovery image iso, we are trying to to use the aicli's extra_args option.

Following is the command we used to update the host =>
[admin@host ~]$ aicli update host ocphost.aibox.my.net -P extra_args="rd.multipath=default root=/dev/disk/by-label/dm-mpath-root"
Updating Host ocphost.aibox.my.net

Retrieving the values, gives =>
[admin@host ~]$ aicli info host ocphost.aibox.my.net | grep installer_args
installer_args: ["rd.multipath","default","root","/dev/disk/by-label/dm-mpath-root"]

Its splitting that into 4 different array elements.

Now, when we continue with this, the remote host coreos-installer command line arguments are incorrectly formed.

It goes like this =>

[root@ocphost ~]# coreos-installer install --insecure -i /opt/install-dir/master-0d7d1d77-6f44-641a-9611-5190160223cc.ign rd.multipath default root /dev/disk/by-label/dm-mpath-root --append-karg ip=bond0:dhcp --copy-network /dev/sda

coreos-installer exists with an error =>

Journalctl log entries =>

Jun 04 12:19:47 ocphost.aibox.my.net installer[15682]: time="2022-06-04T12:19:47Z" level=info msg="error: Found argument 'default' which wasn't expected, or isn't valid in this context\n\nUSAGE:\n coreos-installer install --ignition-file --insecure\n\nFor more information try --help\n"
Jun 04 12:19:47 ocphost.aibox.my.net installer[15682]: time="2022-06-04T12:19:47Z" level=info msg="failed executing nsenter [--target 1 --cgroup --mount --ipc --pid -- coreos-installer install --insecure -i /opt/install-dir/master-0d7d1d77-6f44-641a-9611-5190160223cc.ign rd.multipath default root /dev/disk/by-label/dm-mpath-root --append-karg ip=bond0:dhcp --copy-network /dev/sda], env vars [PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin TERM=xterm container=oci HTTPS_PROXY= OPENSHIFT_BUILD_NAMESPACE=ci-op-fgqbhiwx OPENSHIFT_BUILD_NAME=assisted-installer https_proxy= no_proxy= NO_PROXY= BUILD_LOGLEVEL=0 http_proxy= HTTP_PROXY= HOME=/root HOSTNAME=ocphost.aibox.my.net], error exit status 1, waitStatus 1, Output "error: Found argument 'default' which wasn't expected, or isn't valid in this context\n\nUSAGE:\n coreos-installer install --ignition-file --insecure\n\nFor more information try --help""

Any idea how can this be fixed?

is that possible to add parameter
http_proxy
https_proxy
no_proxy
in overrides when create iso ?

It would be great if aicli could support passing the openshift_cluster_id and ams_subscription_ids filters to ListClusters. This would help cases where we have one of those IDs but not the Assisted ID. Thanks!

As per the AI interface definitions =>
https://raw.githubusercontent.com/openshift/assisted-service/master/swagger.yaml

following is said =>
install_config_overrides:
x-go-custom-tag: gorm:"type:text"
type: string
description: JSON-formatted string containing the user overrides for the install-config.yaml file.
example: '{"networking":{"networkType": "OVNKubernetes"},"fips":true}'

In our installations, we want to set fips=true.
How can this be set via aicli?

We tried with the following cluster_parameters.yaml while cluster creation, but FIPS is not getting enabled.

pull_secret: /home/pull-secret.txt
additional_ntp_source: 110.18.4.4
high_availability_mode: Full
schedulable_masters: true
api_vip: 100.193.65.196
ingress_vip: 100.193.65.197
vip_dhcp_allocation: false
openshift_version: "4.10"
base_dns_domain: dyn..net
ssh_public_key: "ecdsa-sha2-nistp521 AAAAE2VjZH"
network_type: OVNKubernetes

machine_networks:

• cidr: 100.193.65.192/27
• cidr: 5A00:8A00:4000:020c:0000:0000:0146:0000/112
  service_networks:
• 172.30.0.0/16
• fd03::/112
  cluster_networks:
• cidr: 10.128.0.0/14
  hostPrefix: 23
• cidr: fd02::/48
  hostPrefix: 64
  installconfig:
  platform:
  none: {}
  ignition_config_override: '{"ignition": {"version": "3.2.0"}, "storage": {"files":[{"path": "/etc/pki/ca-trust/source/anchors/reg_ca.pem", "mode": 420, "overwrite":true,"user": { "name": "root"},"contents": {"source": "data:text/plain;base64,xyz'
  fips: true

Is there support in aicli to view the cluster events in "realtime" when the installation is started?
In the AI GUI, there is the button 'View Cluster Events', that shows the ongoing progress. We are checking if such a function is there already via aicli?

AI Backend Change

Some days ago we have introduced a new syntax for setting networks in the AI backend. It changes the behaviour so that instead of currently used fields

• machine_network_cidr
• cluster_network_cidr
• cluster_network_host_prefix
• service_network_cidr

the following are used

• machine_networks
• cluster_networks
• service_networks

The syntax is shown in the example cURL request below

-H 'Content-Type: application/json' -X PATCH --data '{
  "cluster_networks":[{"cidr":"10.128.0.0/14","host_prefix":23}],
  "service_networks":[{"cidr":"172.30.0.0/16"}],
  "machine_networks":[{"cidr":"192.168.145.0/24"}]
}'

Backwards Compatibility

Backwards compatibility is preserved.

GET

Responses to GET requests contain both old and new fields organized in the following way

• {...}_network_cidr contains the CIDR from the first entry from respective {...}_networks
• cluster_network_host_prefix contains the host prefix from the first entry from cluster_networks

Hi,

When adding an additionalTrustBundle to the aicli parameters file and creating the cluster, the installation fails at the preparing phase with the following error message:

Container images availability: Failed to fetch container images needed for installation from quay.io/openshift-release-dev/ocp-release:4.13.9-x86_64,quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:6367174e22dca6a79d2aca3de974ed38499fb9cd10b7d845143cb82211b7bb02,quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:eba22f67551d60674a8c9550b9284f2a0540b2a69f5e3c12b7cb2d943684b2a3. This may be due to a network hiccup. Retry to install again. If this problem persists, check your network settings to make sure you’re not blocked.

This issue appears to be very similar to the RedHat bugzilla 2038013 referenced HERE

The parameters file I am using to create the cluster is as follows (temporary ca cert so I will show the full cert in the configuration):

openshift_version: "4.13"
base_dns_domain: "home.ie"
network_type: "OVNKubernetes"
cluster_network_cidr: "10.128.0.0/14"
service_network_cidr: "172.30.0.0/16"
cluster_network_host_prefix: 23
vip_dhcp_allocation: false
pull_secret: "/data/openshift/cluster_config/ocp/openshift_pull.json"
proxy:
  http_proxy: "http://172.16.17.3:3128/"
  https_proxy: "http://172.16.17.3:3128/"
  no_proxy: ".home.ie,172.16.17.0/24"
installconfig:
  additionalTrustBundle: |
    -----BEGIN CERTIFICATE-----
    MIIGCjCCA/KgAwIBAgIUTTqxlEyLu6n4T8H6rBvQNRwkVAgwDQYJKoZIhvcNAQEL
    BQAwgZwxCzAJBgNVBAYTAk5MMRYwFAYDVQQIDA1Ob29yZCBIb2xsYW5kMRIwEAYD
    VQQHDAlBbXN0ZXJkYW0xCzAJBgNVBAoMAk1FMQswCQYDVQQLDAJJVDFHMEUGA1UE
    Aww+RG9ja2VyTWlycm9yQm94IENBIFJvb3QgZG9ja2VyLXJlZ2lzdHJ5LXByb3h5
    IDIwMjMuMDguMzEgMDg6NDUwHhcNMjMwODMxMDg0NTE0WhcNMjcwMzIzMDg0NTE0
    WjCBnDELMAkGA1UEBhMCTkwxFjAUBgNVBAgMDU5vb3JkIEhvbGxhbmQxEjAQBgNV
    BAcMCUFtc3RlcmRhbTELMAkGA1UECgwCTUUxCzAJBgNVBAsMAklUMUcwRQYDVQQD
    DD5Eb2NrZXJNaXJyb3JCb3ggQ0EgUm9vdCBkb2NrZXItcmVnaXN0cnktcHJveHkg
    MjAyMy4wOC4zMSAwODo0NTCCAiIwDQYJKoZIhvcNAQEBBQADggIPADCCAgoCggIB
    AMYex0XOLcNnv0GDT5uUH4KmGMnYvzPq6T8kO7a735QnIdVaDJHVNe8tjUaF1yPU
    DP6pOYORAJJ6FGOgu0QSr2NHRlEQ+QcL8ZWQqVcQ+FnIN0w7nDrhk5iQu/2Wh4e5
    y1+Fy9Le4QkgTdJU7h8+9uLQ9Zf7+TQkC+5S2BjrP4vBoz4tj0dxt1hrrJJH+ksh
    B32TL17f1Fd2nireVbTgidrnl+CEThZIpfxEPDKF7ms2TfjAcYH8hLfXCSKsobvy
    Zk8lTnj6UN7bryuzgV2mdywB1CXq1mlRnU97JWfzcuaS1i5HnhmOY5mfMKAY6Z5a
    xXJrult93Bn8ExkqH5aikq/7hvAv5cu7rjPZgPEQwFECCZRwQgTn1OvVcvXLTrUt
    onkdEqanuPGyaMfKk5WknDnMpGVLMw0fptjz3f4bsctIC7zzqVaiIsUOcUNkDWzL
    /KkuV7+8cxd7cbBRUGMx/elyGDBH+Gd9UVytgl6DaGkXcsM5ExQ14osvf+F1sSpy
    evyp6Cn7VqdQtOAMCMXfJh33+eTL4pVc8HzxBRqNj2wjIEG0BcvJ6AAqglVWrTOM
    4n2YtG5qQZdjMU4rtwDaLPplHMu3Z06g9AkbbyME+ryltr5Zp70updXaeFWg1BCl
    t1Lf/58wBBdXgc+37mpKCSBr80FCvnkCIB5uqVN0j8PBAgMBAAGjQjBAMA8GA1Ud
    EwEB/wQFMAMBAf8wDgYDVR0PAQH/BAQDAgGGMB0GA1UdDgQWBBQuX2KcirLT5pS1
    R4lMYJvxBdPTaDANBgkqhkiG9w0BAQsFAAOCAgEAVyMQfM6LBIAVrH2Nb44B8P/X
    bebaVtTq7d8lpXlqLDApOxh31J24c4Ik/FZOZMmnTAS9EX5NejpaCX+RFWBTSrso
    LHUzIlB5j6a9kAMTl0HCnSIcVXuJEplaTxliSQwr0pxLvm9oa/SMcArVfs0kRsfc
    JOxIfc49skhXJMtT8aSn7m6mK6OTMdlxx4t1x7dYPYHl45LRUczqDiJQ1F7UpbTx
    YwzJeayeWOR67wOSvHCB/hdFman5nwZty3/kj2kCciC/R+Yke+1qDQmyXpmWzOKi
    EvzqZj3EoYI7Lr12mxy8bGBPKOFy2Yj57mALqlE9QupWzJNqmQZp7caNi4wMKm8z
    FWRi01esOgJF6RXUaiuXDbi1nRtgst0RKnWMJceGDytcVGEla9g/PmTezATinJgl
    RwPdvsDDQM3wOaebvxvyJqjFPSNRSjkppH/TQhoBt+TYOloFDuhaVpLwpojPy/SD
    cfM/nBFujkT9B+OTPT+lpb37BPc28labRKjkCCS81bu90hqD1BVhO6tgd5sbKh42
    pqSlktBbMeF/dEsBXHAF2zdv5qYPx5OluMAWqQ6+YPHhQcjPeZcF+hOfValZAGmM
    /mOzAO/3N6W187k58/cDDXE23yLAperR4yhvEmOaW/vT5cEMrA/tb5ZhJ7Qf+57/
    th16emzhfSSoucwqDFk=
    -----END CERTIFICATE-----
additional_ntp_source: "172.16.17.1"
minimal: true
ssh_public_key: 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIFfwtJOvjAOZxAfAQ+hDI1DTDMqwzQ2qF1JeVV1H7h9X [email protected]'
user_managed_networking: true
static_network_config:
  - interfaces:
    - name: eth0
      mac-address: 00:0c:29:c8:f9:d4
      ipv4:
        dhcp: true
        auto-dns: true
        auto-gateway: true
        auto-routes: true
        enabled: true
      ipv6:
        enabled: false
      state: up
      type: ethernet
  - interfaces:
    - name: eth0
      mac-address: 00:0c:29:a1:dc:08
      ipv4:
        dhcp: true
        auto-dns: true
        auto-gateway: true
        auto-routes: true
        enabled: true
      ipv6:
        enabled: false
      state: up
      type: ethernet
  - interfaces:
    - name: eth0
      mac-address: 00:0c:29:c4:bc:04
      ipv4:
        dhcp: true
        auto-dns: true
        auto-gateway: true
        auto-routes: true
        enabled: true
      ipv6:
        enabled: false
      state: up
      type: ethernet
  - interfaces:
    - name: eth0
      mac-address: 92:b6:91:2a:f1:31
      ipv4:
        dhcp: true
        auto-dns: true
        auto-gateway: true
        auto-routes: true
        enabled: true
      ipv6:
        enabled: false
      state: up
      type: ethernet
  - interfaces:
    - name: eth0
      mac-address: 92:b6:91:2a:f1:32
      ipv4:
        dhcp: true
        auto-dns: true
        auto-gateway: true
        auto-routes: true
        enabled: true
      ipv6:
        enabled: false
      state: up
      type: ethernet
  - interfaces:
    - name: eth0
      mac-address: 92:b6:91:2a:f1:33
      ipv4:
        dhcp: true
        auto-dns: true
        auto-gateway: true
        auto-routes: true
        enabled: true
      ipv6:
        enabled: false
      state: up
      type: ethernet

The installconfig for this parameters file, pulled down using 'aicli download installconfig ocp', pull-secret removed, is as follows:

apiVersion: v1
baseDomain: home.ie
proxy:
  httpProxy: http://172.16.17.3:3128/
  httpsProxy: http://172.16.17.3:3128/
  noProxy: .home.ie,172.16.17.0/24
networking:
  networkType: OVNKubernetes
  clusterNetwork:
  - cidr: 10.128.0.0/14
    hostPrefix: 23
  machineNetwork:
  - cidr: 172.16.17.0/24
  serviceNetwork:
  - 172.30.0.0/16
metadata:
  name: ocp
compute:
- hyperthreading: Enabled
  name: worker
  replicas: 3
controlPlane:
  hyperthreading: Enabled
  name: master
  replicas: 3
platform:
  none: {}
fips: false
pullSecret: ''
sshKey: ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIFfwtJOvjAOZxAfAQ+hDI1DTDMqwzQ2qF1JeVV1H7h9X
  [email protected]
additionalTrustBundle: |-
  -----BEGIN CERTIFICATE-----
  MIIGCjCCA/KgAwIBAgIUTTqxlEyLu6n4T8H6rBvQNRwkVAgwDQYJKoZIhvcNAQEL
  BQAwgZwxCzAJBgNVBAYTAk5MMRYwFAYDVQQIDA1Ob29yZCBIb2xsYW5kMRIwEAYD
  VQQHDAlBbXN0ZXJkYW0xCzAJBgNVBAoMAk1FMQswCQYDVQQLDAJJVDFHMEUGA1UE
  Aww+RG9ja2VyTWlycm9yQm94IENBIFJvb3QgZG9ja2VyLXJlZ2lzdHJ5LXByb3h5
  IDIwMjMuMDguMzEgMDg6NDUwHhcNMjMwODMxMDg0NTE0WhcNMjcwMzIzMDg0NTE0
  WjCBnDELMAkGA1UEBhMCTkwxFjAUBgNVBAgMDU5vb3JkIEhvbGxhbmQxEjAQBgNV
  BAcMCUFtc3RlcmRhbTELMAkGA1UECgwCTUUxCzAJBgNVBAsMAklUMUcwRQYDVQQD
  DD5Eb2NrZXJNaXJyb3JCb3ggQ0EgUm9vdCBkb2NrZXItcmVnaXN0cnktcHJveHkg
  MjAyMy4wOC4zMSAwODo0NTCCAiIwDQYJKoZIhvcNAQEBBQADggIPADCCAgoCggIB
  AMYex0XOLcNnv0GDT5uUH4KmGMnYvzPq6T8kO7a735QnIdVaDJHVNe8tjUaF1yPU
  DP6pOYORAJJ6FGOgu0QSr2NHRlEQ+QcL8ZWQqVcQ+FnIN0w7nDrhk5iQu/2Wh4e5
  y1+Fy9Le4QkgTdJU7h8+9uLQ9Zf7+TQkC+5S2BjrP4vBoz4tj0dxt1hrrJJH+ksh
  B32TL17f1Fd2nireVbTgidrnl+CEThZIpfxEPDKF7ms2TfjAcYH8hLfXCSKsobvy
  Zk8lTnj6UN7bryuzgV2mdywB1CXq1mlRnU97JWfzcuaS1i5HnhmOY5mfMKAY6Z5a
  xXJrult93Bn8ExkqH5aikq/7hvAv5cu7rjPZgPEQwFECCZRwQgTn1OvVcvXLTrUt
  onkdEqanuPGyaMfKk5WknDnMpGVLMw0fptjz3f4bsctIC7zzqVaiIsUOcUNkDWzL
  /KkuV7+8cxd7cbBRUGMx/elyGDBH+Gd9UVytgl6DaGkXcsM5ExQ14osvf+F1sSpy
  evyp6Cn7VqdQtOAMCMXfJh33+eTL4pVc8HzxBRqNj2wjIEG0BcvJ6AAqglVWrTOM
  4n2YtG5qQZdjMU4rtwDaLPplHMu3Z06g9AkbbyME+ryltr5Zp70updXaeFWg1BCl
  t1Lf/58wBBdXgc+37mpKCSBr80FCvnkCIB5uqVN0j8PBAgMBAAGjQjBAMA8GA1Ud
  EwEB/wQFMAMBAf8wDgYDVR0PAQH/BAQDAgGGMB0GA1UdDgQWBBQuX2KcirLT5pS1
  R4lMYJvxBdPTaDANBgkqhkiG9w0BAQsFAAOCAgEAVyMQfM6LBIAVrH2Nb44B8P/X
  bebaVtTq7d8lpXlqLDApOxh31J24c4Ik/FZOZMmnTAS9EX5NejpaCX+RFWBTSrso
  LHUzIlB5j6a9kAMTl0HCnSIcVXuJEplaTxliSQwr0pxLvm9oa/SMcArVfs0kRsfc
  JOxIfc49skhXJMtT8aSn7m6mK6OTMdlxx4t1x7dYPYHl45LRUczqDiJQ1F7UpbTx
  YwzJeayeWOR67wOSvHCB/hdFman5nwZty3/kj2kCciC/R+Yke+1qDQmyXpmWzOKi
  EvzqZj3EoYI7Lr12mxy8bGBPKOFy2Yj57mALqlE9QupWzJNqmQZp7caNi4wMKm8z
  FWRi01esOgJF6RXUaiuXDbi1nRtgst0RKnWMJceGDytcVGEla9g/PmTezATinJgl
  RwPdvsDDQM3wOaebvxvyJqjFPSNRSjkppH/TQhoBt+TYOloFDuhaVpLwpojPy/SD
  cfM/nBFujkT9B+OTPT+lpb37BPc28labRKjkCCS81bu90hqD1BVhO6tgd5sbKh42
  pqSlktBbMeF/dEsBXHAF2zdv5qYPx5OluMAWqQ6+YPHhQcjPeZcF+hOfValZAGmM
  /mOzAO/3N6W187k58/cDDXE23yLAperR4yhvEmOaW/vT5cEMrA/tb5ZhJ7Qf+57/
  th16emzhfSSoucwqDFk=
  -----END CERTIFICATE-----

It looks like the additionalTrustBundle is applied to the installconfig, however, I'm wondering if there is something else that needs to be done for the discovery iso to trust this ca cert. Do we need to apply an ignition configuration to the discovery iso as well for this to apply properly?

One observation, if I manually add the ca cert via the "Host Discovery" -> "Add Host" -> "Configure cluster-wide trusted certificates" option in the cluster console, regenerate the iso and boot off of it, then the installation proceeds properly, the installconfig with the above process then has two copies of the same certificate (duplicated) in the additionalTrustBundle section.

Is this a possible bug, or, is it something I may be doing wrong here?

Does aicli support https protocol when talking to AI Services?
If it does support, any documentation about it?

Not sure if overkill but there is an official RedFish python ilbrary... https://github.com/DMTF/python-redfish-library

Hello,

I created an OCP cluster using aicli, it worked perfect.

I need to do the installation again due to some misconfiguration.

I deleted the cluster executing:

aicli delete cluster my-cluster

Then, I deleted the aicli image from podman and recreated it:

podman rmi quay.io/karmab/aicli:latest 

alias aicli='podman run --rm -e AI_OFFLINETOKEN=${TOKEN} -v ${HOME}/.ssh/:/root/.ssh/:Z -v $HOME/.aicli:/root/.aicli:Z -v ${HOME}/.aicli/:/workdir:Z quay.io/karmab/aicli:latest'

I just exported my token and created the file aicli-parameters.yaml and OpenShift pull secret (as I did the first time I installed the cluster).

Now, when I try to create the new cluster, the command stops the executions with no returns (Creating cluster message was expected)

[my-user@my-bastion .aicli]$ aicli create cluster --paramfile aicli-parameters.yaml my-cluster
[my-user@ my-bastion .aicli]$ aicli list clusters
Removing old offlinetoken file
+---------+----+--------+------------+
| Cluster | Id | Status | Dns Domain |
+---------+----+--------+------------+
+---------+----+--------+------------+
[my-user@ my-bastion .aicli]$ aicli list infraenv
+----------+----+---------+-------------------+----------+
| Infraenv | Id | Cluster | Openshift Version | Iso Type |
+----------+----+---------+-------------------+----------+
+----------+----+---------+-------------------+----------+

Do you know why I am not able to recreate the cluster once it is deleted?

I need to say also that the cluster I first created is still working and it is a one node cluster.

Any help on this issue will be extremely appreciated.

Best regards.

This could be an RFE, but could we include creating VLAN interfaces as part of static_network_config. I have tried it myself as a workaround with VLAN interface name starting with "bond" and omitting mac address param. I had to also add the parent interface with ipv4 set to disabled.

Probably in the following line we could add "vlan" or "." as part of the "if" statement.
https://github.com/karmab/assisted-installer-cli/blob/48fdcc7c693b55f454ea3f4765fb3f26f2e1cb59/ailib/__init__.py#L243

Trying this on Fedora 36 with podman results in the below error:

alias

alias aicli='podman run --net host -it --rm -e AI_OFFLINETOKEN=$AI_OFFLINETOKEN -v $HOME/.aicli:/root/.aicli -v $PWD:/workdir quay.io/karmab/aicli'

Error

aicli list cluster
Traceback (most recent call last):
  File "/usr/bin/aicli", line 11, in <module>
    load_entry_point('aicli', 'console_scripts', 'aicli')()
  File "/root/aicli/ailib/cli.py", line 1042, in cli
    args.func(args)
  File "/root/aicli/ailib/cli.py", line 105, in list_cluster
    ai = AssistedClient(args.url, token=args.token, offlinetoken=args.offlinetoken, debug=args.debug)
  File "/root/aicli/ailib/__init__.py", line 64, in __init__
    with open(offlinetokenpath, 'w') as f:
PermissionError: [Errno 13] Permission denied: '/root/.aicli/offlinetoken.txt'

Here is an extract of our cluster_parameters.yaml (partial content only is put here) =>

pull_secret: /home/admin/pull-secret.txt
additional_ntp_source: 192.18.147.40
high_availability_mode: Full
schedulable_masters: false
api_vip: 192.92.82.170
ingress_vip: 192.92.82.171
vip_dhcp_allocation: false
openshift_version: 4.10 <== This the version we want to use

When we do =>

aicli create cluster mycluster --paramfile ./cluster_parameters.yaml

Creating cluster mycluster
Traceback (most recent call last):
File "/home/admin/.local/bin/aicli", line 11, in
sys.exit(cli())
File "/home/admin/.local/lib/python3.6/site-packages/ailib/cli.py", line 1098, in cli
args.func(args)
File "/home/admin/.local/lib/python3.6/site-packages/ailib/cli.py", line 68, in create_cluster
ai.create_cluster(args.cluster, overrides.copy())
File "/home/admin/.local/lib/python3.6/site-packages/ailib/init.py", line 369, in create_cluster
self.client.v2_register_cluster(new_cluster_params=cluster_params)
File "/home/admin/.local/lib/python3.6/site-packages/assisted_service_client/api/installer_api.py", line 4331, in v2_register_cluster
(data) = self.v2_register_cluster_with_http_info(new_cluster_params, **kwargs) # noqa: E501
File "/home/admin/.local/lib/python3.6/site-packages/assisted_service_client/api/installer_api.py", line 4409, in v2_register_cluster_with_http_info
collection_formats=collection_formats)
File "/home/admin/.local/lib/python3.6/site-packages/assisted_service_client/api_client.py", line 330, in call_api
_preload_content, _request_timeout)
File "/home/admin/.local/lib/python3.6/site-packages/assisted_service_client/api_client.py", line 161, in __call_api
_request_timeout=_request_timeout)
File "/home/admin/.local/lib/python3.6/site-packages/assisted_service_client/api_client.py", line 373, in request
body=body)
File "/home/admin/.local/lib/python3.6/site-packages/assisted_service_client/rest.py", line 275, in POST
body=body)
File "/home/admin/.local/lib/python3.6/site-packages/assisted_service_client/rest.py", line 228, in request
raise ApiException(http_resp=r)
assisted_service_client.rest.ApiException: (400)
Reason: Bad Request
HTTP response headers: HTTPHeaderDict({'Content-Type': 'application/json', 'Date': 'Wed, 29 Jun 2022 16:43:09 GMT', 'Content-Length': '243'})
HTTP response body: {"code":"400","href":"","id":400,"kind":"Error","reason":"Openshift version 4.1 for CPU architecture x86_64 is not supported: The requested release image for version (4.1) and CPU architecture (x86_64) isn't specified in release images list"}

Its not considering openshift_version: 4.10.

tcpdump packet capture (from aicli to aisvc) shows this =>

.{"name": "mycluster", "high_availability_mode": "Full", "openshift_version": "4.1", "base_dns_domain": "netact.nsn-rdnet.net", "cluster_network_cidr": "192.128.0.0/14", "service_network_cidr": "172.30.0.0/16", "api_vip": "192.92.82.170", "ingress_vip": "192.92.82.171", "pull_secret": "{"auths":{"registry:5000":{"auth":"cweh45sdWEsdfasa34h==","email":"[email protected]"}}}", "ssh_public_key": "ecdsa-sha2-nistp521 uBeRwq1OR/tQG0LzYRDq2qOQiMWUxBkhDzQxMUZJrfPqQETc6Bu7km39kF25059AkqOZPUJ6Bx5jRvVEGW2U65WNZR8BUCtvOhg7IQ== admin@test", "vip_dhcp_allocation": false, "user_managed_networking": false, "additional_ntp_source": "192.8.147.40", "hyperthreading": "all", "network_type": "OVNKubernetes", "schedulable_masters": false, "cpu_architecture": "x86_64"}

Next we tried with this (with double quotes) =>
openshift_version: "4.10"

With double quotes, it works correctly.

What is the reason for this?
Earlier we use 4.8 OCP version and we did not see this issue. Now with 4.10 seems like some handling is converting 4.10 to 4.1?