oracle-quickstart / oci-sas-viya

Terraform module to deploy SAS Viya on Oracle Cloud Infrastructure (OCI)

SAS Viya 4 Infrastructure as Code (IaC) for OCI

Overview

This project contains Terraform scripts to provision Oracle Cloud Infrastructure (OCI) resources required to deploy SAS Viya 4 products. Here is a list of resources this project will create -

• A VCN with subnets (optionally)
• An OKE cluster and 5 nodepools
• An NFS share using the OCI FSaaS
• A jump box

Prerequisites

Operational knowledge of:

• Terraform
• Docker
• OCI
• Kubernetes

OCI

First off you'll need to do some pre deploy setup to configure terraform detailed here.

You will also need the OCI CLI installed and configured by following these instructions

SAS Viya

You will need your order information and to generate an API key with the links below:

• https://my.sas.com/en/my-orders.html
• https://apiportal.sas.com/get-started

Draft Deploy Steps

Clone

git clone https://github.com/oracle-quickstart/oci-sas-viya.git
cd ./oci-sas-viya

Deploy IaaS

All Terraform variables have defaults. All resources will be prefixed with var.prefix which by default is viya Simply:

terraform plan #optional, to see what resources will be created
terraform apply

Once complete the kubernetes config will be written to viya-oke-kubeconfig.conf. Any kubectl commands below assume you've copied this file to ~/.kube/config.

Install Viya

The install of Viya 4 is done by following the steps detailed here. Clone that repo and complete the prerequisite steps for docker here. The docker method is recomended over ansible. If you are connecting to AutonomousDB you will need to run commands outside of docker, binary version requirements are here.

Currently the OCI CLI is not in the default docker build. Replace the Dockerfile from viya-deployment with the one in this repo in ./deployment/Dockerfile

An example/minimal ansible-vars.yaml can be found in ./deployment/ansible-vars.yaml.example. Replace all XXX values with your values and copy that file to viya-deployment.

The ansible var V4_CFG_INGRESS_FQDN: 'viya.viya.internal' given in the example is intended to be set in you local hosts file, eg:

cat /etc/hosts | grep viya
# viya
129.158.X.Y viya.viya.internal # not actual ip

This can be set to the public ip of the load balancer creating during the viya install below after it completes. Other DNS options are described here.

You can now run the commands below, or similar docker commands. Note the flag --volume $HOME/.oci:/viya4-deployment/.oci \ is OCI specific and allows the OCI CLI inside docker to auth using your user/credentials. Also the paths defined in .oci/config should be relative, eg key_file=~/.oci/oci_api_key.pem

docker build -t viya4-deployment .

docker run --rm \
  --group-add root \
  --user $(id -u):$(id -g) \
  --env GIT_CEILING_DIRECTORIES=/viya4-deployment \
  --volume $HOME:/data \
  --volume $HOME/repos/oci-sas-viya/viya-oke-kubeconfig.conf:/config/kubeconfig \
  --volume $HOME/repos/viya4-deployment/ansible-vars.yaml:/config/config \
  --volume $HOME/repos/oci-sas-viya/terraform.tfstate:/config/tfstate \
  --volume $HOME/.ssh/oci:/config/jump_svr_private_key \
  --volume $HOME/.oci:/viya4-deployment/.oci \
  viya4-deployment --tags "baseline,viya,install"

Once the docker run command finishes you can query the readiness pod by running:

kubectl wait \
  --for=condition=ready pod \
  --selector="app.kubernetes.io/name=sas-readiness" \
  --timeout=1800s

If this returns success, you can log in as the sasboot user at: https://viya.viya.internal/SASLogon/login

Connection to AutonomousDB

Connecting Viya to an instance of ADW requires using kustomize and overlays outside of docker. Be sure to pay attention to the requiired versions of binarires here , and the yq helper commands use v4 syntax. Doc for overlays with Viya in general is here

General documentation can be found in doc downloaded at install at path file://$HOME/viya-oke/viyanamespace/sas-bases/docs/configuring_sasaccess_and_data_connectors_for_sas_viya_4.htm.

Example commands are included below for conveinence, but fundamentally what we're doing is:

• mounting the NFS on a pod
• setting required env vars
• making sure the ADW FQDN is resolvable

cd $HOME/viya-oke/viyanamespace
cp kustomization.yaml kustomization.yaml.bak
cp site.yaml site.yaml.bak

# pwd -> $HOME/viya-oke/viyanamespace/
mkdir site-config/data-access
cp sas-bases/examples/data-access/data-mounts-cas.sample.yaml ~/viya-oke/viyanamespace/site-config/data-access/data-mounts-cas.yaml
cp sas-bases/examples/data-access/data-mounts-job.sample.yaml ~/viya-oke/viyanamespace/site-config/data-access/data-mounts-job.yaml
cp sas-bases/examples/data-access/data-mounts-deployment.sample.yaml ~/viya-oke/viyanamespace/site-config/data-access/data-mounts-deployment.yaml

chmod 644 site-config/data-access/data*.yaml

# edit each file to add NFS example with correct hostname,
# correctly formatted examples with dummy FQDNs are in oci-sas-viya/deployment
nano site-config/data-access/data-mounts-cas.yaml
nano site-config/data-access/data-mounts-job.yaml
nano site-config/data-access/data-mounts-deployment.yaml

yq -i eval '.transformers += ["site-config/data-access/data-mounts-cas.yaml"]' kustomization.yaml
yq -i eval '.transformers += ["site-config/data-access/data-mounts-deployment.yaml"]' kustomization.yaml
yq -i eval '.transformers += ["site-config/data-access/data-mounts-job.yaml"]' kustomization.yaml

# needed env vars
# ORACLE=$(PATH_TO_ORACLE_LIBS)
# ORACLE_BIN=$(PATH_TO_ORACLE_BIN)
# ORACLE_HOME=$(PATH_TO_ORACLE_HOME)

echo "ORACLE=/access-clients/oracle/instantclient_21_6" \
 > ./site-config/data-access/sas-access.properties

echo "ORACLE_HOME=/access-clients/oracle/instantclient_21_6" \
 >> ./site-config/data-access/sas-access.properties

echo "ORACLE_BIN=/access-clients/oracle/instantclient_21_6" \
 >> ./site-config/data-access/sas-access.properties

# Just append
printf "configMapGenerator:
  - name: sas-access-config
    behavior: merge
    envs:
      - site-config/data-access/sas-access.properties
" >> kustomization.yaml

yq -i eval '.transformers += ["sas-bases/overlays/data-access/data-env.yaml"]' kustomization.yaml

mkdir -p site-config/network

# BOTH ip and FQDN of the ADW need to be defined in the files below
# The ips or FQDNS with XXX, X, or Y below are placeholders to be replaced wiith your values

printf 'apiVersion: builtin
kind: PatchTransformer
metadata:
  name: etc-hosts-cas
patch: |-
  - op: add
    path: /spec/controllerTemplate/spec/hostAliases
    value:
      - ip: 192.168.X.Y
        hostnames:
        - "XXX.adb.us-ashburn-1.oraclecloud.com"
target:
  kind: CASDeployment
  annotationSelector: sas.com/sas-access-config=true
' > site-config/network/etc-host-cas.yaml


printf 'apiVersion: builtin
kind: PatchTransformer
metadata:
  name: etc-hosts-job
patch: |-
  - op: add
    path: /template/spec/hostAliases
    value:
      - ip: 192.168.X.Y
        hostnames:
        - "XXX.adb.us-ashburn-1.oraclecloud.com"
target:
  kind: PodTemplate
  annotationSelector: sas.com/sas-access-config=true
' > site-config/network/etc-host-job.yaml


# check yaml
yq e site-config/network/etc-host-cas.yaml && echo "\n\n\n" && yq e site-config/network/etc-host-job.yaml

yq -i eval '.transformers += ["site-config/network/etc-host-cas.yaml"]' kustomization.yaml
yq -i eval '.transformers += ["site-config/network/etc-host-job.yaml"]' kustomization.yaml


# generate site.yaml to include
kustomize build -o site.yaml

# apply site.yaml
kubectl apply --kubeconfig=$HOME/repos/oci-sas-viya/viya-oke-kubeconfig.conf --selector="sas.com/admin=namespace" -f site.yaml --prune

Once these customizations have been applied you can sanity check hostnames/mounts with the below commands:

# sanity check mount
kubectl -n viyanamespace exec -it sas-cas-server-default-controller -- df -k
kubectl -n viyanamespace exec -it sas-cas-server-default-controller -- ls -al /access-clients/oracle

# sanity check db in /etc/hosts
kubectl -n viyanamespace exec -it sas-cas-server-default-controller -- cat /etc/hosts

# sanity check for $ORACLE_XXX in env?
kubectl -n viyanamespace exec -it sas-cas-server-default-controller -- env | grep 'ORACLE\|LIBRARY'

Additionally, a user that is not sasboot should be created if one does not exist for SAS Studio connections to ADW. In Studio the libname statment uses the coonnection string from you ADW is a form like (replacing XXX values):

libname autodb oracle user="admin" password="XXX" path="(description= (retry_count=20)(retry_delay=3)(address=(protocol=tcps)(port=1521)(host=XXX.adb.us-ashburn-1.oraclecloud.com))(connect_data=(service_name=XXX.adb.oraclecloud.com))(security=(ssl_server_dn_match=yes)))";

oci-sas-viya's Issues

Test docker in ORM

Theoretically the docker container can be built on the ORM host. Kubectl (inside ansible inside docker) commands currently depend on a local OCI CLI config file for auth. Should be able to set env vars for delegation token inside container and it just works?

Repo Update / Versions / etc.

I would suggest this repo be updated to reflect changes made on the SAS side of things regarding our IAC repos. Also there have been several changes regarding terraform versions with respect to terraform itself, modules, resources, etc.

Remove compartment creation

This should probably be done and TF assuming target compartment is the desired destination. This is more in line with ORM/mkpl expected behavior.

Add logic for existing VCN

Pull in boilerplate.

Apply taints to nodepools

Currently taints are unset. The OKE nodepool TF resource does not expose taints as a variable. Using the vanilla k8 provider should work.

Configure/test in ORM

Should work transparently, schema.yaml should be set up. Additional vars needed

• expose Flex core count for each nodepool
• expose Flex mem for all (currently not set -> default/core)

Configure DenseIO nvme on nodes

Comparable AWS config is here: https://github.com/sassoftware/viya4-iac-aws/blob/main/files/custom-data/additional_userdata.sh

Flow is:

• skip if exists
• mkfs.ext4 if 1 disk, mdadm is multiple
• block on raid resyncing
• mount
• symlink to /nvme/disk/, 777, nobody:nobody owner

This is at the IAC step, not a conditional in ansible at deploy.

Remove TF warnings

│ Warning: Empty provider configuration blocks are not required
│ 
│   on modules/oci_compartment/main.tf line 2:
│    2: provider "oci" {}
│ 
│ Remove the oci provider block from module.oci_compartment.
│ To ensure the correct provider configuration is used, add oci to the required_providers configuration
│ 
│ (and one more similar warning elsewhere)

Add initial pw info to readme.

Ansible var(s) setable here: https://github.com/sassoftware/viya4-deployment/blob/main/examples/sitedefault.yaml