What happened:
When following the guide: https://github.com/clusternet/clusternet#visit-managedcluster-with-rbac
It will be a problem about how to pass the user cred to managed cluster.
I suffer from "forbidden: User "system:anonymous" cannot get path "/api"" error .
api-server does not set -anonymous-auth=false
explicitly.
What you expected to happen:
Using curl + token to add Child Cluster API, or use kubectl to access it .
How to reproduce it (as minimally and precisely as possible):
Neither proxy/https
mode or proxy/direct
mode, you will have to find a way to satisfy Auth of **Child Cluster **.
So I tried to specific cert/key in your kube.conf or token in curl header (--header "Authorization: Bearer $TOKEN"
)
It will be looked like (curl to access http proxy child cluster )
curl --header "Authorization: Bearer $TOKEN" \
https://${HUB_CLUSTER_IP}:6443/apis/proxies.clusternet.io/v1alpha1/sockets/7a93727c-6609-45bc-8c3e-6556cb89cc2b/proxy/https/${CHILD_CLUSTER_IP}:6443/api/v1/nodes
At first, I thought the TOKEN should be CHILD-Cluster authorized token . So I picked an admin privilege token of child cluster.
Before sending it to Hub Cluster, I verified the token with directly access the Child cluster without clusterNet. token is good.✅
curl --header "Authorization: Bearer $TOKEN" https://${CHILD_CLUSTER_IP}:6443/api/v1/nodes -k
# the result is good, return the nodes to me
But with clusterNet proxy ,using the same $TOKEN
(Child Cluster token), it failed。❌
curl --header "Authorization: Bearer $TOKEN" https://${HUB_CLUSTER_IP}:6443/apis/proxies.clusternet.io/v1alpha1/sockets/7a93727c-6609-45bc-8c3e-656cb89cc2b/proxy/https/${CHILD_CLUSTER_IP}:6443/api/v1/nodes -k
{
"kind": "Status",
"apiVersion": "v1",
"metadata": {
},
"status": "Failure",
"message": "Unauthorized",
"reason": "Unauthorized",
"code": 401
}
I believe it's due to Hub Cluster reject this token . So the first gate(Hub Cluster AuthZ) blocks request.
Ok, I change the $TOKEN
to Hub Cluster admin token.
Now I can access Child Cluster's /healthz (this API does not require auth), ✅ like below
curl --header "Authorization: Bearer $TOKEN" https://${HUB_CLUSTER_IP}:6443/apis/proxies.clusternet.io/v1alpha1/sockets/7a93727c-6609-45bc-8c3e-656cb89cc2b/proxy/https/${CHILD_CLUSTER_IP}:6443/healthz -k
But it still failed for other API( like /apis/v1/nodes ), ❌ error will be :
"message": "forbidden: User \"system:anonymous\" cannot get path \"/ping\"",
Same problem , I'm also confused the user
config in kube-config.
For Child Cluster configuration in kube.conf:
- The ca cert(
certificate-authority-data
) should be Hub-Cluster cert
- But the
user
, I copied client-certificate-data and client-key-data from child cluster kube config file.
Below is my kubectl config, the failure was
kubectl config use-context k8s-21-child
kubectl get no -v=9
Error:
[] GET https://172.**.**.20:6443/apis/proxies.clusternet.io/v1alpha1/sockets/7a93727c-6609-45bc-8c3e-6556cb89cc2b/proxy/https/172.**.**.21:6443/api?timeout=32s 403 Forbidden in 16 milliseconds
[] Response Body: {"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"forbidden: User \"system:anonymous\" cannot get path \"/api\"","reason":"Forbidden","details":{},"code":403}
[root@my-172-**-**-20 ]# kubectl config view
apiVersion: v1
clusters:
- cluster:
certificate-authority-data: DATA+OMITTED
server: https://172.**.**.20:6443
name: k8s-20-parent
- cluster:
certificate-authority-data: DATA+OMITTED
server: https://172.**.**.20:6443/apis/proxies.clusternet.io/v1alpha1/sockets/7a93727c-6609-45bc-8c3e-6556cb89cc2b/proxy/https/172.**.**.21:6443
name: k8s-21-child
contexts:
- context:
cluster: k8s-20-parent
user: kubernetes-admin20
name: k8s-20-parent
- context:
cluster: k8s-21-child
user: kubernetes-admin20
name: k8s-21-child
current-context: k8s-21-child
kind: Config
preferences: {}
users:
- name: kubernetes-admin20
user:
client-certificate-data: REDACTED
client-key-data: REDACTED
- name: kubernetes-admin21
user:
client-certificate-data: REDACTED
client-key-data: REDACTED
Anything else we need to know?:
Environment:
- Clusternet version:
- Clusternet-agent version (user
clusternet-agent --version=json
): latest . 0.4.0
- Clusternet-hub version (user
clusternet-hub --version=json
): 0.4.0
- Kubernetes version (use
kubectl version
): k8s: 1.19.13(build by kubeadm ). kubectl binary : v1.18.20
- Cloud provider or hardware configuration:
- OS (e.g:
cat /etc/os-release
):
- Kernel (e.g.
uname -a
):
- Others: