ppc64le / charts

Helm Charts for ISV and Open Source apps for Kubernetes and IBM Cloud Private

IBM Cloud Private

ISV HELM Chart Repository

Overview

This repository gives IBM Cloud Private users access to technology from non-IBM software providers. Working with IBM, these companies have verified their components run on IBM Cloud Private and meet basic operational standards. Some of the components in this repository will require you contact the respective company to obtain license keys as needed.

If you are interested in having your technology made available via this repository an IBM Cloud Private "on-boarding guide" is available.

Want to learn more about IBM Cloud Private?

Repository Structure

• repo/stable - master branch serves as the landing ground for the published HELM charts.
• repo/beta - master branch serves as the landing ground for charts that are in a pre-releases stage and or under test and verfication.
• beta/{component} - master branch for beta level HELM chart source
• stable/{component} - master branch for production level HELM chart source

charts's Issues

Add DriverlessAI helm chart to the repo

Issues that still need to be worked out:
Ability to modify amount of memory allocated to the image
dataset storage
easy access to logs in case of an issue (mainly for AutoVis and model interpretation)

H2O Driverless AI Helm (GPU version) - unable to mount volume

Helm h2o-dai V 1.5.4
ICP 3.2.0 Enterprise Edition, ppc64le

At deployment, the below is observed in the events:

Unable to mount volumes for pod "h2o-gpu-h2o-dai-7ccbf6d757-zjf2w_h2o-dai(89738ff1-8e5c-11e9-8249-506b8df7ca6a)": timeout expired waiting for volumes to attach or mount for pod "h2o-dai"/"h2o-gpu-h2o-dai-7ccbf6d757-zjf2w". list of unmounted volumes=[config-vol]. list of unattached volumes=[logs-volume nvidia-pers-indicator config-vol default-token-tnl8m]

H2O Driverless AI Helm - Error when deploying on ICP 3.1.1 / Kubernetes version 1.11.3 . Need Helm Chart Update

Hello,
Here is the error I get when deploying on ICP 3.1.1 (no pb with ICP 2.x) due to the GPU management by K8s which changed with the latest versions of K8s. Symptom: POD scheduling error at helm install.
With ICP 3.1 & 3.1.1 (and K8s version >= 11) nvidia.com/gpu should be used instead of alpha.kubernetes.io/nvidia-gpu,
Here is a modified helm chart that works on my environment:
The critical part is in the Helm templates/deployment.yaml file in the requests/limits lines:

resources:
            limits:
            {{- if and (eq (.Capabilities.KubeVersion.Major|int) 1) (lt (.Capabilities.KubeVersion.Minor|int) 11) }}
              alpha.kubernetes.io/nvidia-gpu: {{ .Values.resources.limits.gpu }}
            {{- else }}
              nvidia.com/gpu: {{ .Values.resources.limits.gpu }}
            {{- end }}
              memory: {{ .Values.resources.limits.memory }}
            requests:
            {{- if and (eq (.Capabilities.KubeVersion.Major|int) 1) (lt (.Capabilities.KubeVersion.Minor|int) 11) }}
              alpha.kubernetes.io/nvidia-gpu: {{ .Values.resources.requests.gpu }}
            {{- else }}
              nvidia.com/gpu: {{ .Values.resources.requests.gpu }}
            {{- end }}
              memory: {{ .Values.resources.requests.memory }}

Here is the modified file to be placed in the templates folder of the helm chart, as an example:
deployment.zip

H2O Driveless AI Helm - Error message - Tensorflow

I got some errors at pod / dai-gpu container startup when running on a POWER8 w/ GPU and Cuda/Nvidia drivers installed. DAI is still functional though. Don't know if it is a normal behavior when running on ppc64le as it is related to the Tensorflow part. Please advise.

~/dai-gpu# kubectl logs daimop-dai-gpu-69dbf597d-mjbcd -c dai-gpu --namespace default
---------------------------------
Welcome to H2O.ai's Driverless AI
---------------------------------
     version: Unable to dynamically import either GPU or CPU version of tensorflow: expt_gpu=[Errno 2] No such file or directory: '/opt/h2oai/dai/cuda-9.0/lib/python3.6/site-packages/tensorflow_gpu/__init__.py' expt_cpu=[Errno 2] No such file or directory: '/opt/h2oai/dai/cuda-9.0/lib/python3.6/site-packages/tensorflow_cpu/__init__.py'

Could not import tensorflow

1.2.2

- Put data in the volume mounted at /data
- Logs are written to the volume mounted at /log
- Connect to Driverless AI on port 12345 inside the container
- Connect to Jupyter notebook on port 8888 inside the container
/opt/h2oai/dai/run-dai.sh: line 38:   146 Killed                  nohup ./dai-env.sh ./start-h2o.sh &>${RUN_DAI_LOG_DIR}/h2o.out