azure / kubelogin

A Kubernetes credential (exec) plugin implementing azure authentication

kubelogin

This is a client-go credential (exec) plugin implementing azure authentication. This plugin provides features that are not available in kubectl. It is supported on kubectl v1.11+

Check out the official doc page for more details

Installation

https://azure.github.io/kubelogin/install.html

Quick Start

https://azure.github.io/kubelogin/quick-start.html

Contributing

This project welcomes contributions and suggestions. Most contributions require you to agree to a Contributor License Agreement (CLA) declaring that you have the right to, and actually do, grant us the rights to use your contribution. For details, visit https://cla.opensource.microsoft.com.

When you submit a pull request, a CLA bot will automatically determine whether you need to provide a CLA and decorate the PR appropriately (e.g., status check, comment). Simply follow the instructions provided by the bot. You will only need to do this once across all repos using our CLA.

This project has adopted the Microsoft Open Source Code of Conduct. For more information see the Code of Conduct FAQ or contact [email protected] with any additional questions or comments.

kubelogin's Issues

Unauthorized when authenticate in with kube login service principal.

I am using a main service to authenticate with the Kubernetes API in an environment of continuous integration.
My cluster is set up and enabled to integrate with Azure AD and everything works fine for over a year.
Currently I created a main service that has permission Azure Kubernetes Service Cluster User Role to my AKS.
Right at the azure cli normally and I can get kubeconfig to start interacting with the Kubernetes API.

az login --service-principal -u <spn_id> -p <spn_secret>--tenant XXXXXXX
az aks get-credentials --name xxxx --resource-group xxxx --subscription xxxx --overwrite-existing

Then I call kube login to convert my kubeconfig to the expected model.

kubelogin convert-kubeconfig --client-id <spn_id> --client-secret <spn_secret> --tenant-id XXXXXXX --legacy -l spn

At this point, I already have the ClusterRoleBinding created and using the ObjectId of this SPN.

kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: sp-role-binding
subjects:
  - kind: User
    apiGroup: rbac.authorization.k8s.io
    name: <spn_object_id>
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: cluster-admin

My kubeconfig after conversion is something like:

apiVersion: v1
clusters:
- cluster:
    certificate-authority-data: xxxxxx
    server: https://<uri>:443
  name: xxxxxx
contexts:
- context:
    cluster: xxxxxx
    user: clusterUser_xxxxxx
  name: xxxxxx
current-context: xxxxxx
kind: Config
preferences: {}
users:
- name: clusterUser_xxxxxx
  user:
    exec:
      apiVersion: client.authentication.k8s.io/v1beta1
      args:
      - get-token
      - --environment
      - AzurePublicCloud
      - --server-id
      - <server_id>
      - --client-id
      - <spn_id>
      - --tenant-id
      - XXXXXXX
      - --legacy
      - --client-secret
      - <spn_secret>
      - --login
      - spn
      command: kubelogin
      env: null

After that, having a simple command to search for running pods and object always unauthorized.

I0312 13:23:15.244292   27174 round_trippers.go:423] curl -k -v -XGET  -H "Accept: application/json, */*" -H "User-Agent: kubectl/v1.18.8 (linux/amd64) kubernetes/9f2892a" 'https://<uri>:443/api?timeout=32s'
I0312 13:23:16.993746   27174 round_trippers.go:443] GET https://<uri>o:443/api?timeout=32s 401 Unauthorized in 1749 milliseconds
I0312 13:23:16.993817   27174 round_trippers.go:449] Response Headers:
I0312 13:23:16.993834   27174 round_trippers.go:452]     Audit-Id: xxxxxxx
I0312 13:23:16.993847   27174 round_trippers.go:452]     Cache-Control: no-cache, private
I0312 13:23:16.993859   27174 round_trippers.go:452]     Content-Type: application/json
I0312 13:23:16.993870   27174 round_trippers.go:452]     Content-Length: 129
I0312 13:23:16.993881   27174 round_trippers.go:452]     Date: Fri, 12 Mar 2021 13:23:16 GMT
I0312 13:23:16.996236   27174 request.go:1068] Response Body: {"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"Unauthorized","reason":"Unauthorized","code":401}
I0312 13:23:16.998572   27174 cached_discovery.go:121] skipped caching discovery info due to Unauthorized

In contrast, if I try to generate the token manually by executing the following command, I get the token normally.:

kubelogin get-token --environment AzurePublicCloud --server-id <server_id> --client-id <spn_id> --tenant-id XXXXXXX --client-secret  <spn_secret>  --login spn

Am I doing something wrong? Is there a step I need to take that I didn't take? I am looking for help to solve this problem.

Official container image

An offcial container image would be great. I don't know if kubelogin can be used with AKS like aws-iam-authenticatior is used with EKS (node authentication) but I know use cases where people use container images for packaging their tooling.

Kubelogin Managed Service Identity doesn't work with rolebinding

I have test kubelogin with Managed Service Identity with below scenario:

1. Created ClusterRoleBinding with group as MSI Object ID --> It works as expected
2. Created RoleBinding inside namespace with group as MSI Object ID --> Get below error:
   Error from server (Forbidden): pods is forbidden: User "$OBJECT-ID" cannot list resource "pods" in API group "" in the namespace "default".
   Role and Rolebinding Manifest:
   kind: Role
   apiVersion: rbac.authorization.k8s.io/v1
   metadata:
   name: test-msi-role
   rules:

• apiGroups: [""]
  resources: [""]
  verbs: ["*"]

kind: RoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: test-msi-rolebinding
subjects:

• kind: Group
  name: $OBJECT-ID
  apiGroup: rbac.authorization.k8s.io
  roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: Role
  name: test-msi-role

Am I doing something wrong or it is current behaviour?

Separate downloads per platform

Related to #24, I use Windows primarily and to solve lensapp/lens#649 kubelogin was the perfect solution. I looked at the releases tab and aside from the source, a single zip is present. I almost glossed over this as a solution because there is no indication that the build artifact contains Windows, Linux and Mac binaries. It would be helpful to indicate it is for all platforms, or separate these into platform specific files.

Thanks!

Provide winget package

subj

support multiple cached tokens

support multiple cached tokens based on environment, serverID, clientID, tenantID, and legacy

Unable to use Kubelogin with service principal login flow

Hi! I'm trying to use the Service principal login flow to login to my AAD enabled connected cluster to enable non interactive login for Azure/aks-set-context action. However, I've been getting this error:

server response object: [{
  "metadata": {},
  "status": "Failure",
  "message": "the server has asked for the client to provide credentials",
  "reason": "Unauthorized",
  "details": {
    "causes": [
      {
        "reason": "UnexpectedServerResponse",
        "message": "Security token validation failed."
      }
    ]
  },
  "code": 401
}]

Kubelogin works fine with interactive flow, but doesn't seem to work with service principals.
Also, I could not find any executables in the latest release. So, I used the makefile in the source code to get kubelogin.exe. Could that be a reason for this issue? Or is there anything that I am missing?

SPN login not working using 0.0.4

Hi,
I have a Azure AD enabled AKS cluster. I am trying non interactive login with a SP that has Cluster User role assigned to it. But when I execute via kubectl I get:

I0723 17:27:13.956018 6753 execCredentialPlugin.go:111] acquire new token
I0723 17:27:15.046242 6763 execCredentialPlugin.go:111] acquire new token
I0723 17:27:15.789685 6774 execCredentialPlugin.go:111] acquire new token
I0723 17:27:16.445092 6784 execCredentialPlugin.go:111] acquire new token
I0723 17:27:17.195349 6795 execCredentialPlugin.go:111] acquire new token
I0723 17:27:17.926715 6805 execCredentialPlugin.go:111] acquire new token
error: You must be logged in to the server (Unauthorized)

But if I use kubelogin get-token I get the token

Still Getting Deprecation Warning after installing AKS CLI

I'm nearly certain this is user error so apologies in advance.

I have an AKS cluster running 1.23.5. I can auth to it fine after running az aks get-credentials. However I keep getting the warning when I run any Kubectl commands as follows:

I have installed the AKS CLI and added both the the environment variables to my path. Is this expected to continue to get this warning when I run any command? If so, is there a way to supress it?

Unable to use kubelogin with SPN

Hi,

I create a clusterrolebinding in my cluster for SPN giving it admin role at k8s level.
after that I used kubelogin and ran the below commands
kubectl convert-kubeconfig
set AAD_SERVICE_PRINCIPAL_CLIENT_ID = <>
set AAD_SERVICE_PRINCIPAL_CLIENT_SECRET = <>

after that when I am trying to use
kubectl get nodes
kubectl get pods
its giving below error
error: you must be logged into the server(unauthorized)

please if any guidance what am I doing wrong

Support for AKS cluster created with managed identity

Hi,

Will there be support for AKS cluster created with managed identity (--enable-managed-identity ) instead of service principle ? in this case doesn't exist.

Cluster RollBindings do not work w/ Service principal login flow (non interactive)

I followed the instructions in the Service principal login flow (non interactive) section, and I am able to successfully run kubelogin convert-kubeconfig -l spn and set the environmental variables as instructed. Upon trying to execute any command such as kubectl get no or even kubectl get namespace, I receive the following error:

Error from server (Forbidden): nodes is forbidden: User "<service-principal-object-id>" cannot list resource "nodes" in API group "" at the cluster scope

I created the ClusterRoleBinding as instructed in the readme, and, when I login to kubectl interactively, I can confirm that the CRB is definitely in the cluster:

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: sp-role-binding
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: cluster-admin
subjects:
- apiGroup: rbac.authorization.k8s.io
  kind: User
  name: <service-principal-object-id>

What am I missing here, that is preventing me from accessing the AKS cluster using non-interactive service principal?

convert-kubeconfig -l azurecli : You must be logged in to the server (the server has asked for the client to provide credentials)

I am trying to use kubelogin from CI system and I am facing the following error while trying to authenticate.

2021-04-21 18:48:36.443474: apiVersion: v1
2021-04-21 18:48:36.443524: clusters:
2021-04-21 18:48:36.443613: - cluster:
2021-04-21 18:48:36.443699: certificate-authority-data: LS0tLS1CRUdJTiBDRVJUS....RklDQVRFLS0tLS0K
2021-04-21 18:48:36.443779: server: https://xx-xx-xx-xx.hcp.eastus.azmk8s.io:443
2021-04-21 18:48:36.443800: name: coxxxx-aks-ppe-1
2021-04-21 18:48:36.443809: contexts:
2021-04-21 18:48:36.443865: - context:
2021-04-21 18:48:36.443880: cluster: coxxxx-aks-ppe-1
2021-04-21 18:48:36.443888: user: clusterUser_ConxxxxRG_conxxxx-aks-ppe-1
2021-04-21 18:48:36.443894: name: conxxxx-aks-ppe-1
2021-04-21 18:48:36.443900: current-context: conxxxx-aks-ppe-1
2021-04-21 18:48:36.443955: kind: Config
2021-04-21 18:48:36.443985: preferences: {}
2021-04-21 18:48:36.443994: users:
2021-04-21 18:48:36.444037: - name: clusterUser_ConxxxxRG_conxxxx-aks-ppe-1
2021-04-21 18:48:36.444044: user:
2021-04-21 18:48:36.444050: exec:
2021-04-21 18:48:36.444056: apiVersion: client.authentication.k8s.io/v1beta1
2021-04-21 18:48:36.444065: args:
2021-04-21 18:48:36.444071: - get-token
2021-04-21 18:48:36.444077: - --server-id
2021-04-21 18:48:36.444108: - 6dxxxx-4368-xxxx-94ff-xxxx
2021-04-21 18:48:36.444126: - --login
2021-04-21 18:48:36.444133: - azurecli
2021-04-21 18:48:36.444139: command: kubelogin
2021-04-21 18:48:36.444145: env: null
2021-04-21 18:48:36.444156: helm version
2021-04-21 18:48:36.499299: version.BuildInfo{Version:"v3.4.2", GitCommit:"23dd3af5e19a02d4f4baa5b2f242645a1a3af629", GitTreeState:"clean", GoVersion:"go1.14.13"}
2021-04-21 18:48:36.500722: kubectl version
2021-04-21 18:48:38.130123: Client Version: version.Info{Major:"1", Minor:"20", GitVersion:"v1.20.1", GitCommit:"c4d752765b3bbac2237bf87cf0b1c2e307844666", GitTreeState:"clean", BuildDate:"2020-12-18T12:09:25Z", GoVersion:"go1.15.5", Compiler:"gc", Platform:"linux/amd64"}
2021-04-21 18:48:38.130861: error: You must be logged in to the server (the server has asked for the client to provide credentials)

The cluster is AAD RBAC enabled and has MSI as well.

Here is the script that executes az cli comands followed by kubelogin

n=0
SLEEP_TIME=2
while true; do
    echo "Try # $n"
    az login --identity
    signInExitCode=$?
    if [[ $signInExitCode -eq 0 || $n -ge 5 ]]; then break; fi
    n=$((n + 1))
    echo "sleeping for $SLEEP_TIME"
    sleep $SLEEP_TIME
    SLEEP_TIME=$(( SLEEP_TIME * 2 ))
done

if [ $signInExitCode -eq 0 ]; then
    echo "authenticated to azure"
else
    echo "failed to authenticate to azure"
    exit 1
fi

az account set --subscription $deploymentSubscription
az aks get-credentials --resource-group $deploymentResourceGroup --name $targetAKSCluster


apt update && apt install -y unzip >&-
curl -sSLO https://github.com/Azure/kubelogin/releases/download/v0.0.9/kubelogin-linux-amd64.zip && unzip kubelogin-linux-amd64.zip
mv ./bin/linux_amd64/kubelogin /usr/local/bin/kubelogin

echo "kubelogin install completed"

echo ${KUBECONFIG}

echo "kubelogin version"
kubelogin --version

echo "kube config file"

cat ~/.kube/config

export KUBECONFIG=~/.kube/config
kubelogin convert-kubeconfig -l azurecli

Please publish checksums / sha256 with releases

Hello,
it would be great if there were checksums available directly so we can validate during download.

e.g. additional assets:

• kubelogin-darwin-amd64.zip.sha256
• kubelogin-darwin-arm64.zip.sha256
• kubelogin-linux-amd64.zip.sha256
• kubelogin-win-amd64.zip.sha256
• ...

e.g. compare to oidc-login assets.
https://github.com/int128/kubelogin/releases/tag/v1.24.0

Thank you!

'Kubelogin' project name collisions

Packaging as a krew plugin would solve #25 and streamline the user experience. There are however some existing generic ODIC krew plugins and Homebrew repositories using the same name:

https://github.com/int128/kubelogin
https://github.com/Nordstrom/kubelogin

As this project is specific to Azure OIDC authentication, would it not be more appropriate to name as such? For example:
azure-kubelogin
azure-oidc-login

I would like to help with the krew packaging but need direction on naming first.

Get AAD token using Client Certificate

Kubelogin today supports getting AAD token using client id and client secret. The request here is to get AAD access token using client ID and certificate.

can support this adal api: adal.NewServicePrincipalTokenFromCertificate

"parameter 'identityResourceID' cannot be empty" when using default MSI

Logging in with type "MSI" (i.e. -l msi) later fails with parameter 'identityResourceID' cannot be empty. These commands are being run on an Azure VMSS with a system-assigned identity.

Repro commands:

1. az login --identity
2. az account set -s {subscription}
3. sudo az aks get-credentials --resource-group {resourceGroup} --name {clusterName} --overwrite-existing
4. sudo kubelogin convert-kubeconfig -l msi
5. sudo kubectl get nodes

At this point we are seeing this error consistently when using any kubectl commands that access the cluster.

Error: failed to get token: failed to create service principal from managed identity  for token refresh: parameter 'identityResourceID' cannot be empty

Cached credentials are not checked when in legacy mode

After following kube config the conversion script I have updated kubeconfig:

user:
    exec:
      apiVersion: client.authentication.k8s.io/v1beta1
      args:
      - get-token
      - --environment
      - AzurePublicCloud
      - --server-id
      - xxx
      - --client-id
      - xxx
      - --tenant-id
      - xxx
      - --legacy
      command: kubelogin
      env: null

After running a kubectl command and confirming my identity I have credentials stored in ~/.kube/cache/kubelogin/azure.json.
When I tried to run another kubectl command and it asked me to confirm my identity again.

I tried to not use --legacy and authenticate again and I got from k8s api server (obviously non-legacy type of credentials is not accepted):

error: You must be logged in to the server (the server has asked for the client to provide credentials)

When I tried to switch back to --legacy and still got the same error message.
The error disappeared when I cleared the cache and reauthenticated but I was back to having to authenticate every kubectl execution.

Access is denied to move temporary token to desired output location

I am using Kubelogin plugin and run command kubelogin convert-kubeconfig -l azurecli to refresh aks token. but got the following error:

Any direction on how I can grant the permission or any insight on the issue?

It does successfully refresh the aks token but sometimes the kubectl commands are not working

Support converting a single cluster in a kubeconfig file

It would be nice if kubelogin supported converting a single cluster in a kubeconfig file, and let the other cluster configurations be as-is to reduce the blast radius.

Proposal: kubeConfig contract for mandating PoP access tokens

Would like to propose the following contract in kubeConfig for a cluster that mandates access tokens with proof of posession (PoP) semantics.

kind: Config
preferences: {}
users:
- name: user-name
  user:
    exec:
      apiVersion: client.authentication.k8s.io/v1beta1
      command: kubelogin
      args:
      - get-token
      - ...... specific auth flow parameters
      - --with-pop
      provideClusterInfo: true

--with-pop

To indicate that the returned token should be of type "pop" conforming to SHR format with API server's host name (conveyed via provideClusterInfo) captured as "u" claim.

Inject groups:scr1 objects through graph api call in JWT

Hi, we are currently using kubelogin to authenticate against Azure AD and then use the JWT groups claim to authenticate against an EKS cluster in AWS.

Currently we have no Assignment required restrictions on the enterprise application, this way everyone from our organization can get a valid JWT token by using kubelogin. We do this because this saves us quite the administrative hassle of adding additional new groups to that enterprise application before they can sign in. We only check if a user is part of our domain and has valid credentials and some conditional access policies.

With that valid token you are able to use kubectl and when you know the cluster_ca and the cluster_url you are allowed to do calls to the kubernetes api. On the cluster itself we then have clusterrolebindings & rolebindings to further delegate permissions with regular kubernetes RBAC based of the objectid of a (A)AD group that needs to have permissions on the clusterresource(s).

For some of our users we have the well known 200 groups limit in the JWT token issue. Based of that I've got the following question, is it possible to bypass this by having kubelogin directly query the graph url that's in the JWT and inject the groupid's in the JWT? groups:src1

This in itself doesn't seem like a complex function to add in kubelogin but maybe there is some hidden complexitity that I don't know of.

Azure devops integration

Hello,

thanks for this project. Is it foreseen to integration with "service connections" in Azure devops?
Our pipelines use kubectl tasks: https://docs.microsoft.com/en-us/azure/devops/pipelines/tasks/deploy/kubernetes?view=azure-devops . These tasks use a "service connection". It would be nice if kubelogin becomes one of the "service connection" choices for kubernetes.

Question regarding use-cases

Sorry for opening an issue for a usage question - I'm not sure if there's a better forum? GitHub Discussions don't seem to be enabled here.

In any case, if I wanted to connect kubectl to our AAD enabled AKS cluster I would (naively?) think to create a manged identity, assign it the AKS RBAC Writer role and run the pod using that identity (using the AAD Pod Identity project).

Inside the pod I would connect to the Azure CLI:
az login --identity

...and then use az aks get-credentials to connect kubectl to our AKS.

Is there some problem with doing that? Would that not work? If that does work then where does kubelogin fit into the picture?

Kubectl exec plugin support for spn

It doesn't appear that there is a way to configure the kubectl exec plugin when using spn for authentication. Are there updates required for this plugin to work? If this requires testing, I'm definitely willing to help out as I am building automated processes for spn authentication against AKS Managed AAD clusters.

add support for MI

Support Managed Identity Token with kubelogin

Unable to authenticate the request due to an error: [invalid bearer token, oidc: verify token: failed to verify signature: failed to verify id token signature]

Hello, everything on the kubelogin side looks to be working perfectly.

Not having much luck with the above error which I am getting in the kube-apisever log.

I can see there is a parameter --oidc-ca-file which is probably what I need. Where I am at we have security which rewrites SSL. I have the ca-bundle for those, which had to be installed to worker nodes in order to successfully pull images. I've tried the same ca-bundle via the --oidc-ca-file parameter without success.

Not sure how to resolve this. Since many using kubelogin (azure version) may have also run into this, I thought this would be a good place to post the question as many may later on also come here and search for this error message.

Is there a certificate specific to azure's identity provider which I need to download, and configure via --oidc-ca-file?

Static build for linux

I noticed the linux binary from Releases is dynamically linked which makes it incompatible with some linux distributions such as alpine.

Of course, I can compile my own version but it would be very useful if the release version was already statically linked. Especially for use in lightweight docker containers.

Other Go tools such as kubectl are distributed as fully static binaries, so I assumed kubelogin would be too.

Kubectl authorization failing with timeout error in the Docker based Azure App Service

I'm using Docker based Azure App Service as a Azure Devops deployment agent for reaching our private network. In this app service enabled Identity and I want to use this identity for accessing AKS.

In AKS granted ClusterAdmin role for this identity:

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: <binding name>
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: cluster-admin
subjects:
- apiGroup: rbac.authorization.k8s.io
  kind: User
  name: <managed identity object id>

After configuration I tried to run deployment, but got next error:

W0209 12:49:50.510666    1246 transport.go:260] Unable to cancel request for *exec.roundTripper Error: failed to get token: adal: Failed to execute the refresh request. Error = 'Get http://169.254.169.254/metadata/identity/oauth2/token?api-version=2017-09-01&resource=<id>: dial tcp 169.254.169.254:80: i/o timeout' Unable to connect to the server: getting credentials: exec: executable kubelogin failed with exit code 1 (Client.Timeout exceeded while awaiting headers)

Kubelogin tries to get token from 169.254.169.254, but this endpoint works only under VM's, for App Services there is available "MSI_ENDPOINT" environment variable and kubelogin should use that endpoint. Also in the kubelogin this endpoint is hardcoded: https://github.com/Azure/kubelogin/blob/master/pkg/token/msi.go#L39

Continuing investigation I've found that in a newer version of library adal GetMSIVMEndpoint() is deprecated and NewServicePrincipalTokenFromMSI() can automatically detect correct MSI endpoint: https://github.com/Azure/go-autorest/blob/master/autorest/adal/token.go#L714

[MSAL Migration] device code flow is not supported in Managed Device in Conditional Access

The limitation is documented here.

The fix is to adopt auth code grant flow

Sample code: https://github.com/Azure/azure-sdk-for-go/blob/master/sdk/azidentity/interactive_browser_credential.go

kubelogin not respecting kubeconfig environment variable

kubelogin v0.0.11, psversion 7.2.0 core, windows 10.

I expected that when $env:kubeconfig is set, kubelogin would respect the kubeconfig environment variable in the same way that kubectl does.

Test:

> $env:kubeconfig="C:\Projects\.kube\config"
> az aks get-credentials --resource-group myRG --name myCluster --file $env:kubeconfig
The behavior of this command has been altered by the following extension: aks-preview
Merged "myCluster " as current context in C:\Projects\.kube\config
> kubelogin convert-kubeconfig -l azurecli

This results in a new config file being created at %userprofile%\.kube\config but I was expecting the config file at $env:kubeconfig to be updated.
If $env:kubeconfig is not used/set, kubectl and kubelogin updates the file located at %userprofile%\.kube\config
How can I get kubelogin to update the config file located at a specific location, namely $env:kubeconfig ?

thanks

Kubelogin doesn't work correctly with MSI in Container Instances

We're want to run run Kubernetes deployments from Container Instances with attached managed identities, but are hitting an issue with MSI authentication. We are running kubelogin in the Managed Service Identity (non interactive) setup, but after the call to kubelogin convert-kubeconfig -l msi, any kubectl command returns the error:

error: You must be logged in to the server (the server has asked for the client to provide credentials)

When running the same script from a VM using the same managed identity, the kubectl commands succeed after running kubelogin. This leads us to believe that our cluster/managed identity setup is correct, but that the issue lies with managed identities in container instances.

Undefined reference to: gccgoRealSyscall, gccgoRealSyscallNoError, reflect.typelinks, reflect.resolveTypeOff, reflect.typelinks (gollvm)

Hi.

I caught such bugs:

$ make
rm -f bin/linux_amd64/kubelogin
CGO_ENABLED=0 go build -o bin/linux_amd64/kubelogin -ldflags "-X main.version=v0.0.6/1ee0da905a5b9ce9dceacff7e914b3c3c1e198ea -X main.goVersion=go1.15.2 -X main.buildTime=2020-10-13T09:09:19Z"
go: downloading k8s.io/client-go v0.17.4
go: downloading github.com/spf13/cobra v0.0.6
go: downloading k8s.io/cli-runtime v0.17.4
go: downloading github.com/Azure/go-autorest v14.2.0+incompatible
go: downloading k8s.io/apimachinery v0.17.4
go: downloading github.com/Azure/go-autorest/autorest/adal v0.9.1
go: downloading golang.org/x/crypto v0.0.0-20200622213623-75b288015ac9
go: downloading github.com/Azure/go-autorest/tracing v0.6.0
go: downloading github.com/gogo/protobuf v1.2.2-0.20190723190241-65acae22fc9d
go: downloading k8s.io/api v0.17.4
go: downloading github.com/imdario/mergo v0.3.5
go: downloading k8s.io/utils v0.0.0-20191114184206-e782cd3c129f
go: downloading golang.org/x/sys v0.0.0-20190826190057-c7b8b68b1456
go: downloading github.com/evanphx/json-patch v4.2.0+incompatible
go: downloading k8s.io/kube-openapi v0.0.0-20191107075043-30be4d16710a
go: downloading golang.org/x/net v0.0.0-20191004110552-13f9640d40b9
go: downloading github.com/googleapis/gnostic v0.0.0-20170729233727-0c5108395e2d
go: downloading github.com/Azure/go-autorest/autorest/date v0.3.0
go build: when using gccgo toolchain, please pass linker flags using -gccgoflags, not -ldflags

github.com/Azure/kubelogin

/home/oceanfish81/go/pkg/mod/golang.org/x/[email protected]/unix/gccgo.go:55: error: undefined reference to 'gccgoRealSyscall'
/home/oceanfish81/go/pkg/mod/golang.org/x/[email protected]/unix/gccgo.go:30: error: undefined reference to 'gccgoRealSyscall'
/home/oceanfish81/go/pkg/mod/golang.org/x/[email protected]/unix/gccgo.go:23: error: undefined reference to 'gccgoRealSyscallNoError'
/home/oceanfish81/go/pkg/mod/golang.org/x/[email protected]/unix/gccgo.go:37: error: undefined reference to 'gccgoRealSyscall'
/home/oceanfish81/go/pkg/mod/golang.org/x/[email protected]/unix/gccgo.go:44: error: undefined reference to 'gccgoRealSyscall'
/home/oceanfish81/go/pkg/mod/golang.org/x/[email protected]/unix/gccgo.go:50: error: undefined reference to 'gccgoRealSyscallNoError'
/home/oceanfish81/go/pkg/mod/github.com/modern-go/[email protected]/type_map.go:74: error: undefined reference to 'reflect.typelinks'
/home/oceanfish81/go/pkg/mod/github.com/modern-go/[email protected]/type_map.go:78: error: undefined reference to 'reflect.resolveTypeOff'
/home/oceanfish81/go/pkg/mod/github.com/modern-go/[email protected]/type_map.go:42: error: undefined reference to 'reflect.typelinks'
make: *** [Makefile:33: kubelogin] Error 2

I am using

$ go version
go version go1.15.2 gollvm LLVM 12.0.0git linux/amd64

and my here is my environment:

$ go env
GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/oceanfish81/.cache/go-build"
GOENV="/home/oceanfish81/.config/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOINSECURE=""
GOMODCACHE="/home/oceanfish81/go/pkg/mod"
GONOPROXY=""
GONOSUMDB=""
GOOS="linux"
GOPATH="/home/oceanfish81/go"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/home/oceanfish81/gollvm_dist"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/home/oceanfish81/gollvm_dist/tools"
GCCGO="/home/oceanfish81/gollvm_dist/bin/llvm-goc"
AR="ar"
CC="/usr/bin/clang"
CXX="/usr/bin/clang++"
CGO_ENABLED="1"
GOMOD="/home/oceanfish81/go_projects/kubelogin/go.mod"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build215526650=/tmp/go-build -gno-record-gcc-switches -funwind-tables"

CC @ianlancetaylor @thanm @cherrymui

Why aren't the common SDK environment variable names used?

The environment variables used here are different than those used by the other MSFT SDKs (dotnet, golang, java)

All of them use:

• AZURE_CLIENT_ID
• AZURE_TENANT_ID
• AZURE_CLIENT_SECRET
• AZURE_USERNAME
• AZURE_PASSWORD

Here, you need to define entirely different variables names like AAD_SERVICE_PRINCIPAL_CLIENT_ID. Is there a reason for this inconsistency? Having it means that any execution environment that needs to use both kubelogin and an azure SDK must re-define the same values under two different names.

README is AKS specific

The README has a lot of AKS references but no examples for unmanaged clusters. This is a plugin for all Kubernetes clusters on Azure, not just AKS, README should reflect that.

examples of AKS specific mentions in README:

If you are using kubeconfig from AKS AADv1 clusters

To configure the role binding on Azure Kubernetes Service

myAKSAutomationServicePrincipal

In addition, it would be good to add a generic set up guide that doesn't assume AKS AADv2 doing everything for you, kind of like https://github.com/kubernetes/kubernetes/blob/master/staging/src/k8s.io/client-go/plugin/pkg/client/auth/azure/README.md but for out-of-tree (how to set kubeconfig context for a new cluster, etc.)

Add Installation Instructions for Windows (and everything other than Mac!?!)

Why exactly are there only instructions for brew?

This should be on winget and instructions should be added ASAP. (and az cli should install it automatically)

139 round_trippers.go:174] CancelRequest not implemented by *exec.roundTripper

I try to use Azure CLI token login (non interactive) from jenkins pipeline to login to AKS with non interactive but got the following error:

+ az login --service-principal --username **** --password **** --tenant mytenant -o none
[Pipeline] sh
+ az aks get-credentials --name mycluster -g mykaas
Merged "mycluster" as current context in /root/.kube/config
[Pipeline] sh
+ export 'KUBECONFIG=/root/.kube/config'
[Pipeline] sh
+ kubelogin convert-kubeconfig -l azurecli
[Pipeline] }
[Pipeline] // withCredentials
[Pipeline] sh
+ kubectl get pods -n fm-gtv
E0728 10:11:50.667941     139 round_trippers.go:174] CancelRequest not implemented by *exec.roundTripper
E0728 10:12:22.668147     139 round_trippers.go:174] CancelRequest not implemented by *exec.roundTripper
Unable to connect to the server: context deadline exceeded

Who can help? Is there something missing?

Support password field for PKCS certificates or PEMS

Current implementation decodes PKCS12 with empty password, however there are flows where the PFX might be password protected.

https://github.com/Azure/kubelogin/blob/master/pkg/token/serviceprincipaltoken.go#L89

Another thing thing I've noticied is that it should be pretty straight forward to also support PEMs (just like the Azure CLI supports).

https://github.com/Azure/kubelogin/blob/master/pkg/token/serviceprincipaltoken.go#L200

Build kubelogin for linux/arm64

Helllo:

Would we be able to release this binary for linux/arm64 architecture?

Currently, I see we only have the following:

• kubelogin-darwin-amd64.zip
• kubelogin-darwin-arm64.zip
• kubelogin-linux-amd64.zip
• kubelogin-win-amd64.zip

My team uses this binary as dependency for another project, and we would like to start supporting ARM64 architecture, such that the following URL resolves:
https://github.com/Azure/kubelogin/releases/download/v${KUBELOGIN_VERSION}/kubelogin-linux-arm64.zip

v0.0.10 returns 404 not found

We encountered an issue today in our build pipeline. The build script installs the kubectl as part of az aks install-cli. However, the latest version seems to be not available yet at the time of this posting. The following link returns 404.

https://github.com/Azure/kubelogin/releases/download/v0.0.10/kubelogin.zip

kubelogin may not work with MSI when run in Azure Container Instance

Problem

Although kubelogin addresses kubernetes/kubernetes#86410 to remove spn: prefix in audience claim, you may encounter the same issue while using kubelogin with Managed Service Identity (MSI) in Azure Container Instance (ACI).

For example

az login --identity --username ${MSI_CLIENT_ID}
kubelogin convert-kubeconfig -l azurecli
kubectl get pods
error: You must be logged in to the server (Unauthorized)

## If `guard` log is enabled in AKS control plane, you will see
## failed to verify token for azure: oidc: expected audience "6dae42f8-4368-4678-94ff-3960e28e3630" got ["spn:6dae42f8-4368-4678-94ff-3960e28e3630"]

The issue is in ACI where they have different infrastrucutres such that the returned MSI token on legacy one will include spn: prefix in audience claim. You can verify by examining the AAD token by kubelogin get-token --server-id 6dae42f8-4368-4678-94ff-3960e28e3630 --client-id ${MSI_CLIENT_ID} --login msi and paste the access token to https://jwt.ms and look at the audience claim. If it has spn:, your ACI is running on the legacy infrastructure.

Mitigation

Please open support request to ACI team to migrate your subscription

References

#30 #61

Using the same AAD app for server and client

The old Azure Active Directory plugin describes using two AAD application, and the same approach seems to be applicable to kubelogin too based on the --server-id: <AAD server app ID> and --client-id: <AAD client app ID> parameters.

For AKS clusters one is fixed:

The AAD server app ID of AKS Managed AAD is always 6dae42f8-4368-4678-94ff-3960e28e3630 in any environments.

however for non-AKS clusters (e.g. EKS) is it okay to use the same AAD app for both, or is there some (security) issues with doing so?

Can't get kubelogin to work with MSI

I would like to run kubectl commands using a user-assigned managed identity against an AKS cluster.

What I did

I have created a user-assigned managed identity (az identity create) that I would like to use to call kubectl commands against my AKS cluster (setup with Managed ID and managed AAD enabled, cluster version is 1.18.6). Here the steps I did to get the kubeconfig file updated using kubelogin:

az aks get-credentials ...  // create a kubeconfig file with my user's info

kubelogin convert-kubeconfig -l msi --client-id <clientid-of-my-managed-identity>

kubectl get pods // This fails (see below)

The kubeconfig file looks as follows after running kubelogin:

apiVersion: v1
clusters:
- cluster:
    certificate-authority-data: <cert-blob>
    server: https://mycluster.hcp.westeurope.azmk8s.io:443
  name: mycluster
contexts:
- context:
    cluster: mycluster
    user: clusterUser_myresourcegroup_mycluster
  name: mycluster
current-context: mycluster
kind: Config
preferences: {}
users:
- name: clusterUser_myresourcegroup_mycluster
  user:
    exec:
      apiVersion: client.authentication.k8s.io/v1beta1
      args:
      - get-token
      - --server-id
      - <...>
      - --client-id
      - <clientid-of-my-managed-identity>
      - --login
      - msi
      command: kubelogin
      env: null

Actual Result

When running kubectl get pods command, nothing happens for several seconds and then the following output is printed to the console:

Error: failed to get token: adal: Failed to execute the refresh request. Error = 'Get http://169.254.169.254/metadata/identity/oauth2/token?api-version=2018-02-01&client_id=<...>&resource=<...>: dial tcp 169.254.169.254:80: connectex: A socket operation was attempted to an unreachable network.'
Unable to connect to the server: getting credentials: exec: executable kubelogin failed with exit code 1

I also tried to assign the MSI the following roles:

• Azure Kubernetes Service Cluster User Role AKS role
• cluster-admin role in the cluster as described in the kubelogin README.me

Expected Result

kubectl command executes successfully

Any suggestions what I'm missing? Would be great if you could point me to some E2E sample.

kubelogin convert-kubeconfig -l spn prompts for interactive login

Maybe I'm not following the directions correctly, but I'm prompted for an interactive login when using

$ kubelogin convert-kubeconfig -l spn
$ kubectl get no
To sign in, use a web browser to open the page https://microsoft.com/devicelogin and enter the code FN4N26MMT to authenticate.

Environment is set

$ env |grep AAD
AAD_SERVICE_PRINCIPAL_CLIENT_ID=foo
AAD_SERVICE_PRINCIPAL_CLIENT_SECRET=bar

Kubeconfig

apiVersion: v1
clusters:
- cluster:
    certificate-authority-data: <snip>
    server: <snip>
  name: aks-westus-test
contexts:
- context:
    cluster: aks-westus-test
    user: clusterUser_rg-aks-westus-test_aks-westus-test
  name: aks-westus-test
current-context: aks-westus-test
kind: Config
preferences: {}
users:
- name: clusterUser_rg-aks-westus-test_aks-westus-test
  user:
    exec:
      apiVersion: client.authentication.k8s.io/v1beta1
      args:
      - get-token
      - --environment
      - AzurePublicCloud
      - --server-id
      - <snip>
      - --client-id
      - <snip>
      - --tenant-id
      - <snip>
      command: kubelogin
      env: null

Is there existing work to switch kubelogin from MS ADAL to MSAL library

With the MS ADAL library going out of support in June, is there work already underway by someone to switch from the ADAL to the MSAL libraries?

Homebrew integration

It would be very useful to get a homebrew integration

Publish to a package repository

Can this please be published to packages.microsoft.com so apt install can be used to obtain it?

Linux platform binary in relevant archive type (gz)

It would be nicer if the linux binaries were presented in tar.gz format to avoid having to install zip to extract them as they appear to be multiple entry zip archives.

Signed version of kubelogin.exe

Can we get a signed version of kubelogin.exe, we are only allowed to use signed binaries and exe in our production systems.

add unit tests

let's improve code coverage