elisapolystar / poc-k8s-robottests-nginx

PoC to run robot tests in Kubernetes context

⚠️ Only for demonstation purposes.

Solution

• Nginx containers are tested with simple tests/nginx/nginx.robot tests which checks that the default homepage is shown on addres specified in env WEBSITE_URL
• Tests are run as a kind: CronJob and kind: Job inside the namespace
• report.html (the output html of robot framework) is pushed to Minio object storage
• The report is sered on another nginx container with sidecar updating the report all the time with cronTab

Details

TL;DR

Commands work on default namespace in K8

1. Start kubernetes locally and switch context to it, e.g. with Docker
2. Add bitnami registry with `make add-registrY``
3. Install Minio make install-minio
4. Install Nginx make install-nginx
5. Install jobs make create-jobs
6. Install Nginx to serve report.html make install-nginx-reports

You can view the cluster with Lens or whatever.

To view Minio bucket use kubectl port-forward deployments/minio 9001:9001and then navigate to https://localhost:9001 and accept the scary window of insecure certificate. Credentials can be found from secrets with name minio.

To view the report.htmlrobot report you can use kubectl port-forward deployment/nginx-reports-deployment 8080:80and navigate to http://localhost:8080.

Install Minio

helmfolder contains some default values to run Minio with helm template. Commands can be run with make add-registry and make install-minio to default namespace.

Install the nginx to be tested

You can use make install-nginx to install the default nginx to be tested with robot to the default namespace.

Install the tester jobs and uploader

Jobs can be installed with make create-jobs which installs a batchjob and a cronjob to test previously installed nginx. The container uses run.sh as a baseline which calls robot .to run robot testes and then runs a python script called uploader.py from the root of the project to upload the generated robot.htmlto Minio.

Report is generated to the workdir WORKDIR /home/pwuser/ defined in Dockerfile. This is then uploaded to a bucket called robot-reports on Minio with the filename report.html.

Nginx-report with html downloading from Minio

Idea is to serve the previously generated report.htmlas a static web page which is updated with a sidecar container downloading it from Minio service. It has a simple Nginx container configured with files under /page/ and Dockerfile Reportpage.Dockerfile and the downloader sidecar is defined with Downloader.Dockerfile and downloader.pyscript.

Downloader is a Apline container running the script as a crontab. It connects to minio within downloader.py script and downloads the defined value from there and writes it into a volumeMount workdir. The same volume is mounted to Nginx path /usr/share/nginx/html/ which is served with maintenance.conf configuration.

Misc

Self-signed certificates with Minio

The default minio chart is configured to use TLS under helm/minio-bitnami-values.yaml with

tls:
  enabled: true
  autoGenerated: true

and it is mounted with

    volumeMounts:
      - name: cert
        mountPath: "/certs/ca.crt"
        subPath: ca.crt

and for the Minio clients used in uploader.py and downloader.pyscripts it needs to be handled with customizing the httpClient` as there seems to be no other way to hande self-signed certs with the library. With correctly signed certificates the code is not needed.

httpClient = urllib3.PoolManager(
            timeout=urllib3.Timeout.DEFAULT_TIMEOUT,
                    cert_reqs='CERT_REQUIRED',
                    ca_certs='/certs/ca.crt', # Mounted here
                    retries=urllib3.Retry(
                        total=5,
                        backoff_factor=0.2,
                        status_forcelist=[500, 502, 503, 504]
                    )
        )