Support running Nova in-cluster? about nova

[sudermanjr]

Nova should definitely work in-cluster. We actually do this as part of our commercial software that utilizes nova reports. You can reference how we do this in our chart - https://github.com/FairwindsOps/charts/tree/master/stable/insights-agent. Here's an example of a templated cronjob from that chart.

apiVersion: batch/v1
kind: CronJob
metadata:
  annotations:
    linkerd.io/inject: disabled
    polaris.fairwinds.com/cpuLimitsMissing-exempt: "true"
    polaris.fairwinds.com/memoryLimitsMissing-exempt: "true"
    sidecar.istio.io/inject: "false"
  labels:
    app: insights-agent
  name: nova
spec:
  concurrencyPolicy: Forbid
  failedJobsHistoryLimit: 1
  jobTemplate:
    spec:
      activeDeadlineSeconds: 300
      backoffLimit: 1
      template:
        metadata:
          annotations:
            cluster-autoscaler.kubernetes.io/safe-to-evict: "true"
          labels:
            app.kubernetes.io/name: nova
            app.kubernetes.io/part-of: insights-agent
        spec:
          containers:
            - command:
                - /nova
                - find
                - --helm
                - --containers
                - --config=/config/nova.yaml
                - -v3
              env: null
              image: quay.io/fairwinds/nova:v3.6
              imagePullPolicy: Always
              name: nova
              resources:
                requests:
                  cpu: 100m
                  memory: 128Mi
              securityContext:
                allowPrivilegeEscalation: false
                capabilities:
                  drop:
                    - ALL
                privileged: false
                readOnlyRootFilesystem: true
                runAsGroup: 10324
                runAsNonRoot: true
                runAsUser: 10324
              volumeMounts:
                - mountPath: /output
                  name: output
                - mountPath: /config
                  name: config
                - mountPath: /tmp
                  name: tmp
            - command:
                - ./uploader.sh
                - --datatype
                - nova
                - --timeout
                - "300"
                - --organization
                - research-and-development
                - --cluster
                - sandbox
                - --host
                - https://insights.fairwinds.com
                - --version
                - v3.6
                - --file
                - /output/nova.json
              env:
                - name: FAIRWINDS_TOKEN
                  valueFrom:
                    secretKeyRef:
                      key: token
                      name: insights-agent-token
                - name: FAIRWINDS_AGENT_CHART_VERSION
                  value: 2.23.2
                - name: SEND_FAILURES
                  value: "true"
                - name: POD_NAME
                  valueFrom:
                    fieldRef:
                      fieldPath: metadata.name
              image: quay.io/fairwinds/insights-uploader:0.5.1
              imagePullPolicy: Always
              name: insights-uploader
              resources:
                limits:
                  cpu: 250m
                  memory: 512Mi
                requests:
                  cpu: 100m
                  memory: 128Mi
              securityContext:
                allowPrivilegeEscalation: false
                capabilities:
                  drop:
                    - ALL
                privileged: false
                readOnlyRootFilesystem: true
                runAsGroup: 1000
                runAsNonRoot: true
                runAsUser: 1000
              volumeMounts:
                - mountPath: /output
                  name: output
          restartPolicy: Never
          serviceAccountName: insights-agent-nova
          volumes:
            - emptyDir: {}
              name: output
            - configMap:
                name: insights-agent-nova-config
              name: config
            - emptyDir: {}
              name: tmp
  schedule: 27 * * * *
  successfulJobsHistoryLimit: 1

[evandam]

@sudermanjr you're 100% right, user error as usual 😂

Appreciate the quick response!

[sudermanjr]

Also, is it possible you're not automounting the serviceAccountToken for that service account? Nova definitely needs that to access the kube api