kapicorp / tesoro Goto Github PK
View Code? Open in Web Editor NEWKapitan Admission Controller Webhook for secrets mgmt
License: Apache License 2.0
tesoro's People
Contributors
![dependabot[bot] avatar](https://avatars.githubusercontent.com/in/29110?v=4 "dependabot[bot]")
Stargazers
Watchers
tesoro's Issues
Occasionally content of secrets is added to the wrong key.
Hello,
I have observed the following behaviour already several times:
./compiled/dev/scripts/kubectl.sh apply -f compiled/dev/manifests/simcontrol-secret.yml
secret/simcontrol created
postgres_password should be a plain password string, instead:
./compiled/dev/scripts/kubectl.sh get secret simcontrol -o yaml | yq -r .data.postgres_password | base64 --decode
{
"keys": [
{
"use": "sig",
"kty": "OKP",
"crv": "Ed25519",
"alg": "EdDSA",
"x": "7NmW7vO2_ipJ8Miavm3rxBJaJqnmDjDnvFRBOaR-VhA",
"d": "L0DSyGASSVNwDdgutkpI8qGPlOk1o09jf8u4-o8J7Gw"
},
{
"use": "sig",
"kty": "EC",
"kid": "23ccf53f-fd72-461c-adb0-c5c4a60e37b7",
"crv": "P-256",
"alg": "ES256",
"x": "N2rXfvFT4S5GfJYyAFoyakFpMpLHzFUlwZMuy1x1O8E",
"y": "rdoijXCFISYI4lX9xwqu-dzK_Y6RurhZewgiQpdZkxk",
"d": "w-3cl7rJC_OZE_dzfCnrZw6UyUPKu21TMbSRpJDn5Es"
}
]
Note that the above value actually comes from another key, jwks_pub:
./compiled/dev/scripts/kubectl.sh get secret simcontrol -o yaml | yq -r .data.jwks_pub | base64 --decode
{
"keys": [
{
"use": "sig",
"kty": "OKP",
"crv": "Ed25519",
"alg": "EdDSA",
"x": "7NmW7vO2_ipJ8Miavm3rxBJaJqnmDjDnvFRBOaR-VhA"
},
{ "use": "sig",
"kty": "EC", "kid": "23ccf53f-fd72-461c-adb0-c5c4a60e37b7",
"crv": "P-256",
"alg": "ES256",
"x": "N2rXfvFT4S5GfJYyAFoyakFpMpLHzFUlwZMuy1x1O8E",
"y": "rdoijXCFISYI4lX9xwqu-dzK_Y6RurhZewgiQpdZkxk"
}
]
}
Also, revealing the secret manifest file directly with kapitan shows that the file has the correct content:
./kapitan.sh refs --reveal -f compiled/dev/manifests/simcontrol-secret.yml | yq -r .stringData.postgres_password | base64 --decode
REDACTED_CORRECT_PASSWORD
Restarting Tesoro (side effect of adding --verbose to collect logs) and re-applying made it work correctly.
Versions
kubectl version
Client Version: version.Info{Major:"1", Minor:"16+", GitVersion:"v1.16.6-beta.0", GitCommit:"e7f962ba86f4ce7033828210ca3556393c377bcc", GitTreeState:"clean", BuildDate:"2020-01-15T08:26:26Z", GoVersion:"go1.13.5", Compiler:"gc", Platform:"darwin/amd64"}
Server Version: version.Info{Major:"1", Minor:"16+", GitVersion:"v1.16.9-gke.6", GitCommit:"14bc8ad5f8c245f1240a8e4eab128c4d51bfeffe", GitTreeState:"clean", BuildDate:"2020-05-30T02:07:03Z", GoVersion:"go1.13.9b4", Compiler:"gc", Platform:"linux/amd64"}
Logs do not show reference to the object being handled (i.e. secret/config map name)
Describe the bug/feature
While looking at the logs, it is difficult to understand which actual kubernetes resource is being handled by kubernetes
To Reproduce
Steps to reproduce the behavior:
- Look at logs
- Try to understand which Kubernetes resource it is referring to
Expected behavior
a log line like
Namespace: alpha, name: my-secret, Kind: {'group': '', 'version': 'v1', 'kind': 'Secret'} Resource: {'group': '', 'version': 'v1', 'resource': 'secrets'}
x509: certificate relies on legacy Common Name field
Describe the bug/feature
Mutating webhook throws the following error when submitting the valid nginx example:
$ kubectl apply -f tests/k8s/nginx_deployment.yml
Error from server (InternalError): error when creating "tests/k8s/nginx_deployment.yml": Internal error occurred: failed calling webhook "tesoro-admission-controller.tesoro.svc": Post "https://tesoro-admission-controller.tesoro.svc:443/mutate?timeout=30s": x509: certificate relies on legacy Common Name field, use SANs or temporarily enable Common Name matching with GODEBUG=x509ignoreCN=0
To Reproduce
I am evaluating Tesoro so was following the instructions:
- Use latest version of minikube to start a cluster
$ minikube start
😄 minikube v1.15.1 on Ubuntu 18.04
✨ Using the docker driver based on user configuration
👍 Starting control plane node minikube in cluster minikube
🔥 Creating docker container (CPUs=2, Memory=3900MB) ...
🐳 Preparing Kubernetes v1.19.4 on Docker 19.03.13 ...
🔎 Verifying Kubernetes components...
🌟 Enabled addons: storage-provisioner, default-storageclass
🏄 Done! kubectl is now configured to use "minikube" cluster and "default" namespace by default
- Clone the tesoro
git clone [email protected]:kapicorp/tesoro.git
- Run the commands in the instructions
kubectl apply -f k8s/clusterrole.yaml
kubectl apply -f k8s/clusterrolebinding.yaml
kubectl apply -f k8s/tesoro_namespace.yaml
kubectl -n tesoro apply -f k8s/tesoro_secret.yaml
kubectl -n tesoro apply -f k8s/tesoro_service.yaml
kubectl -n tesoro apply -f k8s/tesoro_deployment.yaml
Wait for pods to start
kubectl apply -f k8s/tesoro_mutatingwebhook.yaml
Test failed
$ kubectl apply -f tests/k8s/nginx_deployment.yml
Error from server (InternalError): error when creating "tests/k8s/nginx_deployment.yml": Internal error occurred: failed calling webhook "tesoro-admission-controller.tesoro.svc": Post "https://tesoro-admission-controller.tesoro.svc:443/mutate?timeout=30s": x509: certificate relies on legacy Common Name field, use SANs or temporarily enable Common Name matching with GODEBUG=x509ignoreCN=0
Expected behavior
Expected example to work
Make --verbose more informative and safe
Currently --verbose prints out decrypted secrets to stdout/logs.
This is dangerous because secrets would then leak into logs and would all need to be rotated.
I suggest the default version of --verbose should attempt to redact secrets, and perhaps we should have an explicit redact_secrets=false to enable printing out the secrets.
I believe encrypted should be allowed into logs, but this is also another one worth discussing given that, especially for non *KMS backends where the key rotates automatically, it could have security implications
Also,
2020-07-08 18:19:29 DEBUG Secret transformation: decoded_ref: ?{
would be more helpful with a key name
Tesoro forced to run as root user
-
TL;DR
Although the design oftesorowould allow it to run as non-root user, we are forced to run as root for a number of reasons. -
Reason
Ideally, we would want to runtesoroon port8443(unprivileged) and then user the service to expose it as port443.
This works on both KIND and GKE "non private" clusters.
Unfortunately on private GKE clusters, although the apiserver is meant to connect to the tesoro webhook using the service IP on port 443, the connection seems to then reach the pod IP directly, expecting tesoro to be running on the same port.
Logs:
- service configuration
$ kubectl get services
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
tesoro ClusterIP 10.15.5.59 <none> 443/TCP,9095/TCP 52s
- actual pod running on port 8443
$ kubectl apply -f example-gkms-secret.yml
Error from server (InternalError): error when creating "example-gkms-secret.yml": Internal error occurred: failed calling webhook "tesoro.kapicorp.com
": Post https://tesoro.tesoro.svc:443/mutate?timeout=30s: no service port '\u20fb' found for service "tesoro"
We've also experienced other kind of issues like
$ kubectl apply -f example-gkms-secret.yml
Error from server (InternalError): error when creating "example-gkms-secret.yml": Internal error occurred: failed calling webhook "tesoro.kapicorp.com": Post https://tesoro.tesoro.svc:443/mutate?timeout=5s: dial tcp 10.16.0.18:8443: i/
o timeout
while this is probably because of firewall rules: https://gitlab.com/gitlab-org/charts/gitlab/-/issues/1809
Error from server (InternalError): error when creating "example-gkms-secret.yml": Internal error occurred: failed calling webhook "tesoro.kapicorp.com": Post https://tesoro.tesoro.svc:443/mutate?timeout=30s: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)
Because of this kubernetes/kubernetes#61510 although the api supports it, we cannot run both service and pod on 8443 either.
The only solution that seems to work is to have the actual tesoro pod also running on port 443, which makes it work instantly :(
Tesoro fails to reveal references that are inside a file
Describe the bug/feature
Tesoro fails to detect and reveal secrets embedded into other strings
kind: Secret
metadata:
labels:
name: bug-report
tesoro.kapicorp.com: enabled
name: bug-report
stringData:
not_work: does not work ?{base64:eyJkYXRhIjogIk4wOTRlbE5XTlZWalFVZzRWblJKVTBscVpWWXpTVkZYV2pGWFVVOVViV1JSWlRFeFFYWlNhbVJuU1E9PSIsICJlbmNvZGluZyI6ICJvcmlnaW5hbCIsICJ0eXBlIjogImJhc2U2NCJ9:embedded}
works: ?{base64:eyJkYXRhIjogIk4wOTRlbE5XTlZWalFVZzRWblJKVTBscVpWWXpTVkZYV2pGWFVVOVViV1JSWlRFeFFYWlNhbVJuU1E9PSIsICJlbmNvZGluZyI6ICJvcmlnaW5hbCIsICJ0eXBlIjogImJhc2U2NCJ9:embedded}
results in:
{
"not_work": "ZG9lcyBub3Qgd29yayA/e2Jhc2U2NDpleUprWVhSaElqb2dJazR3T1RSbGJFNVhUbFpXYWxGVlp6UldibEpLVlRCc2NWcFdXWHBUVmtaWVYycEdXRlZWT1ZWaVYxSlNXbFJGZUZGWVdsTmhiVkp1VTFFOVBTSXNJQ0psYm1OdlpHbHVaeUk2SUNKdmNtbG5hVzVoYkNJc0lDSjBlWEJsSWpvZ0ltSmhjMlUyTkNKOTplbWJlZGRlZH0=",
"works": "N094elNWNVVjQUg4VnRJU0lqZVYzSVFXWjFXUU9UbWRRZTExQXZSamRnSQ=="
}
with annotations
k get secret/bug-report -o yaml | yq -r .metadata.annotations.'"tesoro.kapicorp.com/revealed"'
["/data/works"]
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.