microsoft / bedrock Goto Github PK
View Code? Open in Web Editor NEWAutomation for Production Kubernetes Clusters with a GitOps Workflow
License: MIT License
Automation for Production Kubernetes Clusters with a GitOps Workflow
License: MIT License
Suggest source repo to setup github repo rules that require at least 1 approver and checks to pass before pull request is allowed to merge.
Investigate the github API which might potentially allow setting these rules with the help of a personal access token. Currently it seems that a password is required to do this via the GUI. Reference
As a person in an operations role, I'd like to be able to spin up the Kubernetes cluster itself in an automated manner.
We should be able to validate that the project works end to end before and after each commit with a CI system.
Some ideas on how to test a Terraform deployment here: https://devops.stackexchange.com/questions/863/how-to-test-a-terraform-configuration
As a developer, I want to focus on my application code and have all of the supporting infrastructure scaffolded out for me:
Kube-Monkey may be the main options
We need a repo similar to Getting Started directory that defines the high level definition deployments. This repo will be what our CI/CD will trigger off of.
For moment create an Azure Pipelines instance that is triggered on commits to the HLDD repo. The CI/CD should run Fabrikate on the the components on the HLDD repo
Look at kubediff (and similar solutions) and come up with a recommendation on how to:
Also so: microsoft/fabrikate#29
Read here, and utilize azure pipelines to push to destination repo to avoid using ssh keys and personal access tokens.
The null resource provisioning requires bash shell. This should be at least called out in the readme or refactored to support both Windows and Linux.
Pin to a particular git commit in the Flux repo.
Provide configuration as an input to the cluster creation script for the git repo that you want Flux to pull from.
Add scripts to enable deploying multiple clusters and have them managed by a common ingress (traffic manager).
The helm chart in https://github.com/helm/charts/tree/master/incubator/jaeger is several versions behind now and has been superseded by the Jaeger project maintained Jaeger Operator.
in the setup section of cluster deployment, you need to install the Helm client... I had an error until I did so (I know its probably obvious - but I forgot to do it first, and got an error that helm was not found).
Looked at a few options for developing unit tests, and it appears the using a shell script may be the simplest approach for now. Stumbled upon shUnit2, which is a unit test framework, and could be using it to aid in developing the unit test.
After discussing with the team, it appears that the following steps will take place:
Refactor build.sh so that it is broken up into functions. That way, the unit test can source build.sh and reference reusable blocks of code from the script for testing.
Develop unit test shell script that will specify default environment variables and call functions from build.sh to run for testing. This should be able to run locally and report results.
Refactor build.sh so that it is broken up into functions. That way, the unit test can source build.sh and reference reusable blocks of code from the script for testing.
Develop unit test shell script that will specify default environment variables and call functions from build.sh to run for testing. This should be able to run locally and report results.
Dropping terraform's state in the current directory is not a scalable solution, even if checked into git. Instead, persist this .tfstate in blob storage.
This should run fab generation as a verification to make sure components are valid and build passes before it's merged into master
Figure out what is currently available (to work with Azure)
What tests are possible
How would it integrate
Commit #21 fixed the Jaeger UI not showing up but no services show up in the UI. The correct way to pull the Istio based Jaeger UI is
kubectl port-forward -n istio-system $(kubectl get pod -n istio-system -l app=jaeger -o jsonpath='{.items[0].metadata.name}') 16686:16686 &
As referenced here.
After making this change I see a trace from the simple-service application with custom user-agent referenced. I only see one reference. This could be due to how tracing is implemented in simple-service. I'm going to investigate further to try other jaeger-client operations.
Related to #27 we need to take the generated yam artifacts and git commit/push this files to a source of truth repo. For the moment we may be able to utilize the Azure Pipelines GitHub App to authenticate the git push command.
is there a minimum terraform version required?
and have you considered providing a docker container with binaries and scripts included?
If there is a need for this, i can send a PR.
All the best,
Benjamin
Running ./init
asks for the var.container_repo variable, which looks to be docker.io/timfpark
. What's that repo for?
The deployment of the simple service is IMHO not working correctly.
the repo name refers to @timfpark and i cannot push to this docker repo.
https://github.com/Microsoft/bedrock/blob/master/services/common/deploy-service#L26
possbile way to go:
we change the values in the file:https://github.com/Microsoft/bedrock/blob/master/services/modules/simple-service/deploy-simple-service
or update the README with an overwrite of the default values.
or we could stop building it and relying on a prebuilt docker image.
The Dockerfile is currently missing the docker package and we would have to mount the docker socket into the container while starting it.
Steps done:
tried to test it via minikube:
docker build -t bedrock:latest .
docker run --rm -v /var/run/docker.sock:/var/run/docker.sock --net=host -it -v ~/.minikube:/.minikube -v ~/.kube/config:/.kube/config -e TF_VAR_grafana_admin_password="SECRETpass" bedrock:latest /bin/bash
cd services/environments/dev
./init
./apply
@timfpark what do you think?
All the best,
Benjamin
Create a GitHub repo that will be the location that #28 publishes yam artifacts to
Allow for the deployment of a Kubernetes / pod level chaos solution.
Implementation should include documenting available tests as well as implementing knobs to enable/disable tests.
Ala https://docs.microsoft.com/en-us/azure/aks/aad-integration for AD for AKS
Adjust Traefik configuration to provide TLS via Let's Encrypt
Investigate and prototype.
Flux uses SSH based "deploy keys" to communicate with GitHub repos. It's unclear how we would authenticate with Azure DevOps git repos. Flux provides a guide on how to set up SSH keys on non GitHub repos here.
This will help unit tests run on any machine
Per the research in #40, implement chaos for node/infra level.
Document available tests and how to enable/disable those tests.
As a person with operations responsibilities, I'd like to be able to:
Ideas for implementation include Istio, Linkerd2, or using a higher level management platform around these like supergloo
Ala Azure Key Vault or Hashicorp Vault
Currently Flux requires a key that has read/write access to a GitHub repository. Need to investigate if this is actually needed, how to mitigate the security risk.
Currently you can not enable RBAC without a backing AAD service principal. In the meantime, use the az command line.
TODO: This support is expected within a couple of releases -- switch back when its available.
Lines 27 - 74 of providers/azure-aks/aks.tf
Briefly describe what GitOps is and the main principles we want to adhere by in bedrock regardless of the CI/CD technology used this README.
We follow a release flow
We believe Git to be a source of truth.
We believe Git is enables collaboration at scale
We leverage operational features of the CI/CD platform to
We align to the patterns of
etc
We're planning to use deploy keys for flux and ssh keys for pushing to the second repo from a script, but we would like to refresh these keys periodically and when we do that, it would need to be automatically updated on the github repo via a script. Explore the options for automating it (see github API)
Expand on the README with detail instruction for steps that we currently can't automate.
The expectation here is that from scratch one can:
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.