cert-manager / webhook-example Goto Github PK
View Code? Open in Web Editor NEWA cert-manager sample repository for creating an ACME DNS01 solver webhook
License: Apache License 2.0
A cert-manager sample repository for creating an ACME DNS01 solver webhook
License: Apache License 2.0
This is an incredibly minor thing but GitHub's template repository functionality would be perfectly suited for this repo. It'll allow an easy alternative to forking or setting up a new remote: https://docs.github.com/en/repositories/creating-and-managing-repositories/creating-a-repository-from-a-template
Right now if you run go test
within this repository, the tests will fail.
They fail for a few reasons:
panic
on Present/CleanUp(1) can be fixed by adding a fetch-binaries.sh
script (probably taken straight from sigs.k8s.io/controller-tools).
(2) and (3) can be most easily fixed by embedding a 'fake dns server' into the webhook, which will listen for DNS queries and serve TXT records accordingly. This is how cert-manager itself handles tests for the RFC2136 DNS01 provider, and there is a reusable library that we can use to build it easily: https://github.com/jetstack/cert-manager/tree/master/test/acme/dns/server
The readme mentions its plugged into the apiserver for security reasons. I can see that resource, but I don't see anything on how trust is established/enforced between the apiserver and plugin. Is there a document somewhere that describes this?
This test probably doesn't need cert-manager itself running for the time being.
I think the test should cover:
In future we may want to explore having some form of extended e2e that actually deploys and runs a version of cert-manager too, but let's keep the scope small for now π
See this Slack conversation for context- it would be great if we could show users how to get log output with the example webhook (i.e replace fmt.Printf
here with a log line
Any plan to update it?
go test
As the helm chart dont contain a namespace for several resources to be deployed excepted by example here - https://gist.github.com/cmoulliard/0aaa99004f930516c57902cefae2f89a#file-gadaddy-webhook-yaml-L1-L36, where should the resources be deployed (using default namespace, ...) ?
Examples:
What should be the namespace of the this value namespace: {{ .Release.Namespace | quote }}
which is not defined within the helm values file - values.yaml for issuer, certificate
apiVersion: certmanager.k8s.io/v1alpha1
kind: Issuer
metadata:
name: {{ include "example-webhook.selfSignedIssuer" . }}
namespace: {{ .Release.Namespace | quote }}
Why this RoleBinding is installed under kube-system
?
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: RoleBinding
metadata:
name: {{ include "example-webhook.fullname" . }}:webhook-authentication-reader
namespace: kube-system
Another example
apiVersion: apiregistration.k8s.io/v1beta1
kind: APIService
metadata:
name: v1alpha1.{{ .Values.groupName }}
labels:
app: {{ include "example-webhook.name" . }}
chart: {{ include "example-webhook.chart" . }}
release: {{ .Release.Name }}
heritage: {{ .Release.Service }}
annotations:
certmanager.k8s.io/inject-ca-from: "{{ .Release.Namespace }}/{{ include "example-webhook.servingCertificate" . }}"
spec:
group: {{ .Values.groupName }}
groupPriorityMinimum: 1000
versionPriority: 15
service:
name: {{ include "example-webhook.fullname" . }}
namespace: {{ .Release.Namespace }}
Is there a reason you use sometimes | quote
and sometimes not to generate the namespace name ?
Can this be better documented please and helm templates updated ?
My webhook doesn't work if there are no _acme-challenge
TXT records in the DNS record initially. Cert-manager constantly tries to get this entry with this message until I manually add the _acme-challenge
entry with any value.
When querying the SOA record for the domain '_acme-challenge.kube.example.com.' using nameservers [10.96.0.10: ββ 53], rcode was expected to be 'NOERROR' or 'NXDOMAIN', but got 'SERVFAIL'" logger="cert-manager.challenges" key="cert-manager/wildcard-kube-example-com-2-3963891295-1304023134
The solution to my problem, as I see it, is to call the Present function when starting the certificate Ρhallenges, but the call only happens if I add the entry _acme-challenge
manually with any value
I've been following along with #3 and trying my best to build a solver using this example as a starting point. I think I've done ok and based on what the tests are doing it seems to be working perfectly. I'm now just a little stuck on how exactly to make use of this solver in my actual kubernetes cluster π€
I figured because the solvers are Go libraries we will need to install them somehow, but I don't really have a clue how to proceed here. I'm going to go and look at some examples to try and divine how to get this in production but I thought I would open this issue to discuss adding some guidance to the readme or a partial example to the repo.
let me know what you think π
I have upgraded my webhook to cert-manager v1.11 and have the following errors. I had also to change RBAC to support flowcontrol.apiserver.k8s.io
group
W0307 09:07:34.843358 1 reflector.go:424] pkg/mod/k8s.io/[email protected]/tools/cache/reflector.go:169: failed to list *v1beta3.FlowSchema: the server could not find the requested resource
E0307 09:07:34.843372 1 reflector.go:140] pkg/mod/k8s.io/[email protected]/tools/cache/reflector.go:169: Failed to watch *v1beta3.FlowSchema: failed to list *v1beta3.FlowSchema: the server could not find the requested resource
W0307 09:07:46.213292 1 reflector.go:424] pkg/mod/k8s.io/[email protected]/tools/cache/reflector.go:169: failed to list *v1beta3.PriorityLevelConfiguration: the server could not find the requested resource
E0307 09:07:46.213356 1 reflector.go:140] pkg/mod/k8s.io/[email protected]/tools/cache/reflector.go:169: Failed to watch *v1beta3.PriorityLevelConfiguration: failed to list *v1beta3.PriorityLevelConfiguration: the server could not find the requested resource
Hi, I'm trying to implement Hetzner DNS API based on this sample, Everything is working as expected and the webhook issue the certificates normally, however, I'm getting this log error constantly:
controller.go:129] OpenAPI AggregationController: action for item v1alpha1.acme.mycompany.com: Rate Limited Requeue.
controller.go:116] loading OpenAPI spec for "v1alpha1.acme.mycompany.com" failed with: OpenAPI spec does not exist
any idea how to fix it?
Lines 36 to 37 in 9ea71ae
It looks like the pull request cert-manager/cert-manager#4835 was closed automatically. Is this comment to be removed and the code to be uncommented, or should the pull-request be re-opened ?
The apiversion
is outdated since the release 0.12.0
as the apiversion to be used is
apiVersion: cert-manager.io/v1alpha2
and not anymore
apiVersion: certmanager.k8s.io/v1alpha1
To be fixed within the helm template resource and documented
: https://github.com/jetstack/cert-manager-webhook-example/blob/master/deploy/example-webhook/templates/pki.yaml#L4
Remark: I propose to create a branch in order to handle the apiversion change
I have forked this repo to implement a custom solver for desec, but found myself unable to deploy a working instance on my up-to-date Debian system. The issue has been pinpointed as the go version used for implementing the hook, as an updated version will necessitate an update in the Dockerfile as well. The same problem was encountered by another user in #27
Steps to reproduce:
FROM golang:1.21-alpine3.18 AS build_dep
Expected Result:
The hook is deployed and the relevant cert-manager resources are up and running.
Actual Result:
The webhook pod does not start and ends in a CrashLoopBackOff state. Reviewing its logs, the following error shows up:
cert-manager: error executing command" err="error installing APIGroup for solvers: unable to get openapi models: OpenAPIV3 config must not be nil
Outcome:
Developers are forced to use either one of two ugly workarounds:
macos monterey, go 17.2
got an error when running tests:
go test -v .
=== RUN TestRunsSuite
fixture.go:115: error starting apiserver: timeout waiting for process etcd to start
To problem cause is etcd - it is not starting:
V:cert-manager-webhook-vkcloud v$ _test/kubebuilder/bin/etcd
fatal error: runtime: bsdthread_register error
runtime stack:
runtime.throw(0x1bed3fa, 0x21)
/usr/local/go/src/runtime/panic.go:616 +0x81 fp=0x7ff7bfeff718 sp=0x7ff7bfeff6f8 pc=0x102a871
runtime.goenvs()
/usr/local/go/src/runtime/os_darwin.go:129 +0x83 fp=0x7ff7bfeff748 sp=0x7ff7bfeff718 pc=0x10283f3
runtime.schedinit()
/usr/local/go/src/runtime/proc.go:501 +0xd6 fp=0x7ff7bfeff7b0 sp=0x7ff7bfeff748 pc=0x102d166
runtime.rt0_go(0x7ff7bfeff7e8, 0x1, 0x7ff7bfeff7e8, 0x0, 0x1000000, 0x1, 0x7ff7bfeff970, 0x0, 0x7ff7bfeff98b, 0x7ff7bfeff9a7, ...)
/usr/local/go/src/runtime/asm_amd64.s:252 +0x1f4 fp=0x7ff7bfeff7b8 sp=0x7ff7bfeff7b0 pc=0x1056474
After some investigation I've found this issue https://github.com/golang/go/wiki/MacOS12BSDThreadRegisterIssue, installed etcd manually, etcd starts but now I got kube-apiserver start timeout error.
I occasionally look onto my k8s metrics (CPU/Mem Usage) and i found that the webhook server uses rather much CPU (up to 20 percent) over the time.
From my understanding, a webhook server is a simple k8s OpenAPI server, an HTTP server, which should be actually sleeping most of the time, is this correct? So the CPU load in my expectation should be at most 5%.
in this case, i'm wondering what is causing that CPU load... π€
I want to have my API credentials for my provider in a secret, as the comment above the structure suggests. However, the test will then fail, because the secret is not present.
Other examples use two config keys, one SecretKeySelector and one plain string, one is used during testing and one on an actual cluster. I don't like this solution as there is the possibility of a less than optimal configuration, and it also doesn't test the secret handling, as it will be executed in production.
Is there a place, where I can hook my own function in and submit my secret to the apiserver before testing?
Describe the bug:
I started to create a new webhook for another DNS provider. You can find it here: github.com/4nx/cert-manager-webook-joker. While building and testing the package under arm64 I found a bug within an old dependency package which is used.
TEST_ZONE_NAME=xxx go test .
panic: runtime error: slice bounds out of range [:9] with length 8
goroutine 1 [running]:
github.com/miekg/dns.ClientConfigFromFile(0x1346942, 0x10, 0x0, 0x0, 0x0)
/home/ubuntu/go/pkg/mod/github.com/miekg/[email protected]/clientconfig.go:86 +0x8b4
github.com/jetstack/cert-manager/pkg/issuer/acme/dns/util.getNameservers(0x1346942, 0x10, 0x229a540, 0x2, 0x2, 0x0, 0x25, 0x21ba808)
/home/ubuntu/go/pkg/mod/github.com/jetstack/[email protected]/pkg/issuer/acme/dns/util/wait.go:51 +0x30
github.com/jetstack/cert-manager/pkg/issuer/acme/dns/util.init()
/home/ubuntu/go/pkg/mod/github.com/jetstack/[email protected]/pkg/issuer/acme/dns/util/wait.go:44 +0x88
FAIL github.com/4nx/cert-manager-webhook-joker 0.122s
FAIL
The problem is that a very old version of github.com/miekg/dns package is still used:
The project had a bug until version 1.0.4:
https://github.com/miekg/dns/blob/5364553f1ee9cddc7ac8b62dce148309c386695b/clientconfig.go#L94
which had been fixed in version 1.0.5 after this pull request miekg/dns#642 through:
The most recent version of the project is 1.1.35.
Expected behaviour:
A working webhook under ARM64.
Steps to reproduce the bug:
Simply following: https://github.com/4nx/cert-manager-webhook-joker#development under ARM64
Anything else we need to know?:
The webhook is working under amd64.
Environment details::
/kind bug
With v1.5.0 the test function dns.SetBinariesPath()
has been removed and therefore the test example does not work anymore.
I have tried to adapt the test, but I have failed so far.
Error presenting challenge: godaddy.acme.amprajin.in is forbidden: User "system:serviceaccount:cert-manager:cert-manager" cannot create resource "godaddy" in API group "acme.amprajin.in" at the cluster scope
apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
name: letsencrypt-prod
spec:
acme:
# ACME Server
# prod : https://acme-v02.api.letsencrypt.org/directory
# staging : https://acme-staging-v02.api.letsencrypt.org/directory
server: https://acme-staging-v02.api.letsencrypt.org/directory
# ACME Email address
email: [email protected]
privateKeySecretRef:
name: letsencrypt-staging # staging or production
solvers:
- selector:
dnsNames:
- '*.amprajin.in'
dns01:
webhook:
config:
apiKeySecretRef:
name: godaddy-api-key
key: token
production: true
ttl: 600
groupName: acme.amprajin.in
solverName: godaddy
I have migrated my webhook to cert-manager v1.13.1.
Now I get the provided apiKey printed, which I do not want to see in automatic test protocols...
$ TEST_ZONE_NAME=my.domain. make test
TEST_ASSET_ETCD=_test/kubebuilder-1.28.0-linux-amd64/etcd \
TEST_ASSET_KUBE_APISERVER=_test/kubebuilder-1.28.0-linux-amd64/kube-apiserver \
TEST_ASSET_KUBECTL=_test/kubebuilder-1.28.0-linux-amd64/kubectl \
/usr/bin/go test -v .
=== RUN TestRunsSuite
=== RUN TestRunsSuite/Basic
=== RUN TestRunsSuite/Basic/PresentRecord
util.go:70: created fixture "basic-present-record"
suite.go:38: Calling Present with ChallengeRequest: &v1alpha1.ChallengeRequest{UID:"", Action:"", Type:"", DNSName:"example.com", Key:"123d==", ResourceNamespace:"basic-present-record", ResolvedFQDN:"cert-manager-dns01-tests.my.domain.", ResolvedZone:"my.domain.", AllowAmbientCredentials:false, Config:(*v1.JSON)(0xc00070e270)}
I0315 12:42:10.574692 285584 main.go:142] call function Present: ResourceNamespace=basic-present-record, ResolvedZone=my.domain., ResolvedFQDN=cert-manager-dns01-tests.my.domain. DNSName=example.com
I0315 12:42:10.575049 285584 main.go:148] **Decoded configuration {dynu-secret}**
I0315 12:42:10.576462 285584 main.go:277] call function getDomainIdFromFQDN: **apiKey=abcABC123456**, ResolvedFQDN=cert-manager-dns01-tests.my.domain.
I0315 12:42:11.415191 285584 main.go:306] Domain Detail: id=9209690, node=cert-manager-dns01-tests
I0315 12:42:11.415221 285584 main.go:179] call function determineBaseRecordName: recordName=cert-manager-dns01-tests
I0315 12:42:12.234369 285584 main.go:375] Added TXT record result: {"statusCode":200,"id":10447515,"domainId":9209690,"domainName":"my.domain","nodeName":"cert-manager-dns01-tests","hostname":"cert-manager-dns01-tests.my.domain","recordType":"TXT","ttl":60,"state":true,"content":"cert-manager-dns01-tests.my.domain. 60 IN TXT \"123d==\"","updatedOn":"2024-03-15T11:42:12.117","textData":"123d=="}
I0315 12:42:12.994260 285584 main.go:375] Added TXT record result: {"statusCode":200,"id":10447516,"domainId":9209690,"domainName":"my.domain","nodeName":"","hostname":"my.domain","recordType":"TXT","ttl":60,"state":true,"content":"my.domain. 60 IN TXT \"123d==\"","updatedOn":"2024-03-15T11:42:12.89","textData":"123d=="}
I0315 12:42:12.994302 285584 main.go:173] Presented txt record cert-manager-dns01-tests.my.domain.
I0315 12:42:45.211687 285584 main.go:277] call function getDomainIdFromFQDN: **apiKey=abcABC123456**, ResolvedFQDN=cert-manager-dns01-tests.my.domain.
I0315 12:42:46.027191 285584 main.go:306] Domain Detail: id=9209690, node=cert-manager-dns01-tests
I0315 12:42:46.724078 285584 main.go:227] TXT entry with content my.domain. 60 IN TXT "123d==" (key value 123d==)
I0315 12:42:47.441375 285584 main.go:233] Deleted TXT record result: {"statusCode":200}
I0315 12:42:47.441414 285584 main.go:227] TXT entry with content cert-manager-dns01-tests.my.domain. 60 IN TXT "123d==" (key value 123d==)
I0315 12:42:48.391805 285584 main.go:233] Deleted TXT record result: {"statusCode":200}
I0315 12:42:48.391836 285584 main.go:227] TXT entry with content (key value 123d==)
I0315 12:42:48.391846 285584 main.go:227] TXT entry with content apps.ocp4.my.domain. 60 IN TXT "07QkfbaBeGOsoMdL3NLxvN7yk4NMzUjhhAa7AhCVONQ" (key value 123d==)
I0315 12:42:48.391856 285584 main.go:227] TXT entry with content my.domain. 120 IN SOA ns1.dynu.com. administrator.dynu.com. 0 3600 900 604800 300 (key value 123d==)
I0315 12:42:48.425179 285584 main.go:277] call function getDomainIdFromFQDN: **apiKey=abcABC123456**, ResolvedFQDN=cert-manager-dns01-tests.my.domain.
I0315 12:42:49.153359 285584 main.go:306] Domain Detail: id=9209690, node=cert-manager-dns01-tests
I0315 12:42:49.901336 285584 main.go:227] TXT entry with content (key value 123d==)
I0315 12:42:49.901363 285584 main.go:227] TXT entry with content apps.ocp4.my.domain. 60 IN TXT "07QkfbaBeGOsoMdL3NLxvN7yk4NMzUjhhAa7AhCVONQ" (key value 123d==)
I0315 12:42:49.901373 285584 main.go:227] TXT entry with content my.domain. 120 IN SOA ns1.dynu.com. administrator.dynu.com. 0 3600 900 604800 300 (key value 123d==)
--- PASS: TestRunsSuite (44.52s)
--- PASS: TestRunsSuite/Basic (40.64s)
--- PASS: TestRunsSuite/Basic/PresentRecord (40.64s)
PASS
ok github.com/Dopingus/cert-manager-webhook-dynu 44.544s
A declarative, efficient, and flexible JavaScript library for building user interfaces.
π Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. πππ
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google β€οΈ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.