ddl-ebrown / distributed-compute-operator

:robot: Kubernetes operator providing Ray|Spark|Dask|MPI clusters on-demand

Distributed Compute Operator

Kubernetes operator providing Ray|Spark|Dask clusters on-demand via Custom Resource Definitions.

Overview

TODO

Installation

The easiest way to install DCO is to use the provided Helm chart.

Prerequisites

Before you get started using the DCO, you need to have a running Kubernetes cluster.

• Access to a Kubernetes cluster version 1.16 or above. This version of the CRD API is stable and supports our required features.
• Install helm client version 3.0.0 or above.
• Install the cert-manager operator. DCO makes extensive use of webhooks which require TLS.

Install

$ helm install distributed-compute-operator ./deploy/helm/distributed-compute-operator

Development

The following instructions will help you create a local Kubernetes environment that can be used to test every feature supported by this operator.

1. Install minikube and create a new cluster.

   # tested using minikube v1.17.1 and k8s v1.20.2
   $ minikube start \
       --cpus=6 --memory=16384 --driver=hyperkit \
       --extra-config=apiserver.enable-admission-plugins=PodSecurityPolicy \
       --addons=pod-security-policy

2. Install cert-manager

3. Install metrics-server

4. Launch operator