This repository contains a sample Python Flask application, along with the various glue to mimic an end-to-end workflow for a containerized application to run in a production Kubernetes cluster, with support from a central platform organization.
The tooling utilized incldues:
It utilizes two additional repos:
- kubecamp-env for providing a separate environment focused repository around candidate Kubernetes manifests as well as applied Kubernetes manifests.
- kubecamp-blueprints for providing upstream Kubernetes manifests outside of an individual team's app repository.
The overall workflow is as follows:
- Developer will work in
kubecamp-app
repo on any branch that is notmain
, utilizingskaffold dev
to iterate with a development Kubernetes cluster, either locally or in the cloud.skaffold dev
will utilize Kustomize as a deployer, with base inkubecamp-blueprints/team-a/resources
and applyingkubecamp-app/app/config
to generate a ConfigMap where they can change an environment variable.
- Once the developer is happy with the state of
kubecamp-app
they will push to remote on their feature branch. - This will trigger
kubecamp-app/builds/create-pr-main.yaml
, a Cloud Build that will run unit tests defined inkubecamp-app/test_main.py
, triggered off of any push to a branch that is notmain
. You will need to create the Cloud Build trigger yourself should you want to replicate this. - Assuming tests pass, the developer can then merge their feature branch into
main
, ready to prepare their application to run in production. - This merge (you will need to create the Cloud Build trigger yourself should you want to replicate this.) will trigger a Cloud Build defined in
kubecamp-app/builds/merge-into-main.yaml
, which will do two things:- It will run
skaffold render -p prod
, which will build, tag, and push the container images forkubecamp-app
. Then it will render Kubernetes manifests inresources.yaml
after utilizing the Kustomize stanza defined in the Skaffold Profileprod
to pull from overlays that contain a new ConfigMap for production and will exposekubecamp-app
via a LoadBalancer rather than ClusterIP. - Then, it will run the
gcloud
container as a build step defined inkubecamp-blueprints/shared/git
, which will take the rendered manifest, and push it to a candidate branch forkubeecamp-env
, tagging the candidate branch with theSHORT_SHA
from the commit inkubecamp-app
that triggered this build. That way we can identify candidate manifests with the specific code changes made in the app repo. You will need to configure Secret Manager to use an SSH key you generated for your GitHub, customizeconfigure_git.sh
and then build a gcloud container usingbuild_git.yaml
with your custom script.
- It will run
- Once a candidate branch is opened in
kubecamp-env
, this will trigger a Cloud Build defined inkubecamp-env/tests/integration_test.yaml
, which mocks automation for integration tests you may want to run in a pre-prod environment (it doesn't actually do this, rather demonstrates where you could do it). You will need to create the Cloud Build trigger yourself should you want to replicate this. - Finally, with integration tests passing, you can then merge your candidate branch into
main
forkubecamp-env
, which will trigger the final Cloud Build defined indeploy.yaml
This will useskaffold apply
with your rendered manifests to your production Kubernetes cluster. You will need to update this with your proper Kubernetes cluster.
TODO
- Add automated creation of build triggers.
- Add cluster creation instructions.
- Add IAM requirements.
- Add specific instructions for customizing
configure_git.sh
. - Add workflow diagram.
- Use Cloud Build support for integration with Secret Manager, rather than hacking it together.