auth0 / k8s-pixy-auth

k8s plugin to authenticate against an OIDC compatible issuer using PKCE (pixy) flow

k8s-pixy-auth

A kubectl plugin to authenticate against an OIDC compatible issuer using PKCE (pixy) flow

As of Kubernetes v1.11 there is beta support for a client-go credentials plugin. Using the support it is possible to use an Auth0 application to authenticate users and provide tokens with which a correctly configured Kubernetes cluster can authorize user actions.

Note that while this tool uses PKCE, since it was built the recommendation for CLIs was changed from using PKCE to using Device Authorization Flow. The recommendation change is due to usability as Deivce Authorization Flow allows for CLIs to work where browsers cannot be opened (eg: SSH terminal) and does not require you to open a port on the local machine to handle the callback. At this time there are no plans to support Device Authorization Flow in this tool, however, we welcome PRs.

Installation

At this point in the project installation is manual. In the future this will be automated.

1. Make sure your Kubernetes api service is configured to use OpenID Connect Tokens.
2. Download a release binary or pull down this repo with git clone [email protected]:auth0/k8s-pixy-auth.git
3. If you pulled down the repo, change to the cloned directory and build the binary with go build
4. Initialize your kube config making sure to use the argument values applicable to your cluster k8s-pixy-auth init --context-name "minikube" --issuer-endpoint "https://joncarl.auth0.com" --audience "minikube" --client-id "QXV0aDAgaXMgaGlyaW5nISBhdXRoMC5jb20vY2FyZWVycyAK" --port 8080. If you are using refresh tokens add --with-refresh-token to the command arguments. If you are using the ID Token instead of the Access Token add --use-id-token to the command arguments.
5. Run a command against Kubernetes like kubectl get nodes. Since this is the first time k8s-pixy-auth has been invoked for the context it will open a browser to authenticate you.
6. After authentication is complete, switch back to your terminal and you should see the output of the command. If you don't have permissions it will let you know. Make sure you've correctly set up permissions for your user. After authentication is done k8s-pixy-auth will securely cache your the needed tokens.
7. Future commands will use the cached information from the first time you invoked k8s-pixy-auth for that context and will thus not require a browser to be opened each time. Because the auth tokens are stored securely the secure backend might ask for your credentials from time to time (the backend depends on OS).

Securing the Credentials

Keyring is used in the background to secure the credentials. This allows cross-platform support to securely store the credentials.

How to Configure Your Cluster

The k8s api service needs to be configured in order to use this tool. Checkout Auth0Setup.md for a basic guide on how to setup Auth0 as the token issuer. Using that guide you should be able to set up other OIDC providers as well.

Issue Reporting

If you have found a bug or if you have a feature request, please report them at this repository issues section. Please do not report security vulnerabilities on the public GitHub issue tracker. The Responsible Disclosure Program details the procedure for disclosing security issues.

Author

Auth0

License

This project is licensed under the MIT license. See the LICENSE file for more info.

k8s-pixy-auth's Issues

convert project layout to "standard" layout

https://github.com/golang-standards/project-layout

Convert project to standard layout

state not verified by client when getting the authorization code

The authenticate function creates a challenge and then calls the getCode function. The getCode creates a new state and passes it as a parameter to the /authorize call.

The missing check here is that the authenticate function does not verify that the codeResult contains a state parameter whose value must match with the one the client sent to server.

This is required as per RFC 6749 "state REQUIRED if a "state" parameter was present in the client authorization request. The exact value received from the client."

A detailed explanation can also be found in our docs.

Auth credentials are not preserved on Ubuntu 18.04

When I'm using the system, auth credentials are asked every time (on every kubectl command the browser opens) and are not persisted. Technically, I'm not even using master (cause it's simply broken on Ubuntu), but the code from #18. Apparently, the keyring update makes auth flow not stuck, but it still doesn't work as it's supposed to.
See also: #17

Device Authorization Flow

Hi, any plans to support the Device Authorization Flow (the one with a device code and no local server)?

Maybe in a new repo like auth0/k8s-device-auth?

Anyway, from a security standpoint, why is the Device Authorization Flow now recommended by Auth0 for CLIs vs. PKCE?

Make the listen port configurable

Describe the problem you'd like to have solved

Port 8080 is over used, and only having the k8s-pixy-auth binary listen on that could interfere with local development.

Describe the ideal solution

A subcommand for auth to specify the listen port would be very useful.

Alternatives and current work-arounds

maintain a fork.

Not works with kubectl 1.23.5

Description

Provide a clear and concise description of the issue, including what you expected to happen.

k8s-pixy-auth does not work well with kubectl 1.23.5.

Reproduction

Detail the steps taken to reproduce this error, what was expected, and whether this issue can be reproduced consistently or if it is intermittent.

Where applicable, please include:

• Code sample to reproduce the issue
• Log files (redact/remove sensitive information)
• Application settings (redact/remove sensitive information)
• Screenshots

Following https://github.com/auth0/k8s-pixy-auth/blob/master/docs/Auth0Setup.md, but with the latest minikube (version v1.25.2)

When I execute k8s-pixy-auth init xxxxxx, it reports:

./k8s-pixy-auth init --context-name "minikube" --issuer-endpoint "https://xxx.auth0.com/" --audience "xxx-self-hosted-cluster" --client-id "<CLIENT ID>"
Installing binary...
Updating kube config...
panic: Error saving kube config: converting  to : type names don't match (Unknown, RawExtension), and no conversion 'func (runtime.Unknown, runtime.RawExtension) error' registered.

goroutine 1 [running]:
github.com/auth0/k8s-pixy-auth/cmd.glob..func2(0xe87fe0?, {0xa275de?, 0x8?, 0x8?})
        /home/strrl/playground/github/k8s-pixy-auth/cmd/init.go:39 +0x253
github.com/spf13/cobra.(*Command).execute(0xe87fe0, {0xc000168780, 0x8, 0x8})
        /home/strrl/.gvm/pkgsets/go1.18/global/pkg/mod/github.com/spf13/[email protected]/command.go:766 +0x663
github.com/spf13/cobra.(*Command).ExecuteC(0xe88240)
        /home/strrl/.gvm/pkgsets/go1.18/global/pkg/mod/github.com/spf13/[email protected]/command.go:852 +0x2dc
github.com/spf13/cobra.(*Command).Execute(...)
        /home/strrl/.gvm/pkgsets/go1.18/global/pkg/mod/github.com/spf13/[email protected]/command.go:800
github.com/auth0/k8s-pixy-auth/cmd.Execute()
        /home/strrl/playground/github/k8s-pixy-auth/cmd/root.go:36 +0x25
main.main()
        /home/strrl/playground/github/k8s-pixy-auth/main.go:8 +0x17

But I found a possible solution:

I tried to upgrade the dependency of client-go to v0.23.1, it just works. :D

Environment

Please provide the following:

• Version of this library used:

Both master and v0.4.0

• Version of the platform or framework used, if applicable:

$ minikube version  
minikube version: v1.25.2
commit: 362d5fdc0a3dbee389b3d3f1034e8023e72bd3a7-dirty

$ kubectl version                                                                                              
Client Version: version.Info{Major:"1", Minor:"23", GitVersion:"v1.23.5", GitCommit:"c285e781331a3785a7f436042c65c5641ce8a9e9", GitTreeState:"archive", BuildDate:"2022-03-17T21:14:47Z", GoVersion:"go1.18", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"23", GitVersion:"v1.23.3", GitCommit:"816c97ab8cff8a1c72eccca1026f7820e93e0d25", GitTreeState:"clean", BuildDate:"2022-01-25T21:19:12Z", GoVersion:"go1.17.6", Compiler:"gc", Platform:"linux/amd64"}

• Other relevant versions (language, server software, OS, browser):
• Other modules/plugins/libraries that might be involved:

suggested minikube setup for Auth0 does not work with k8s >= 1.14

Description

minikube with k8s >= 1.14 does not pickup extra-config for apiserver and the suggestion here https://github.com/auth0/k8s-pixy-auth/blob/master/docs/Auth0Setup.md#kubernetes-setup no longer works

see kubernetes/minikube#8661 and kubernetes/minikube#6543 (comment)

Reproduction

1. Follow https://github.com/auth0/k8s-pixy-auth/blob/master/docs/Auth0Setup.md
2. Try kubectl get nodes suggested at the end of the steps and you will get the error below:
   You must be logged in to the server (Unauthorized)
   and in apiserver logs following is seen:
   Unable to authenticate the request due to an error: [invalid bearer token]

Environment

k8s-pixy-auth commit: f86b051

By using command kubectl cluster-info, I am facing this error The connection to the server localhost:8080 was refused - did you specify the right host or port? How can I solve it?

Please do not report security vulnerabilities here. The Responsible Disclosure Program details the procedure for disclosing security issues.

Thank you in advance for helping us to improve this library! Please read through the template below and answer all relevant questions. Your additional work here is greatly appreciated and will help us respond as quickly as possible. For general support or usage questions, use the Auth0 Community or Auth0 Support. Finally, to avoid duplicates, please search existing Issues before submitting one here.

By submitting an Issue to this repository, you agree to the terms within the Auth0 Code of Conduct.

Description

Provide a clear and concise description of the issue, including what you expected to happen.

Reproduction

Detail the steps taken to reproduce this error, what was expected, and whether this issue can be reproduced consistently or if it is intermittent.

Where applicable, please include:

• Code sample to reproduce the issue
• Log files (redact/remove sensitive information)
• Application settings (redact/remove sensitive information)
• Screenshots

Environment

Please provide the following:

• Version of this library used:
• Version of the platform or framework used, if applicable:
• Other relevant versions (language, server software, OS, browser):
• Other modules/plugins/libraries that might be involved:

Debug output support

I have an issue with this tool, and I'd like to have a better way to debug it.
Printing more logs if requested is a viable solution to me.

Support Windows SubSystem for Linux

Please do not report security vulnerabilities here. The Responsible Disclosure Program details the procedure for disclosing security issues.

Thank you in advance for helping us to improve this library! Your attention to detail here is greatly appreciated and will help us respond as quickly as possible. For general support or usage questions, use the Auth0 Community or Auth0 Support. Finally, to avoid duplicates, please search existing Issues before submitting one here.

By submitting an Issue to this repository, you agree to the terms within the Auth0 Code of Conduct.

Describe the problem you'd like to have solved

With Windows SubSystem for Linux becoming very popular, some additional checks would be needed to account for it as the "xdg-open" command does not work. However, cmd.exe /c start ... is supported.

A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]

Describe the ideal solution

A clear and concise description of what you want to happen.

Modify the function OpenURL within os/os_interactor.go line 17, and add a couple helper functions to determine WSL.

// OpenURL opens the passed in url in the default OS browser
func (i *DefaultInteractor) OpenURL(url string) error {
	var cmd string
	var args []string

	switch runtime.GOOS {
	case "windows":
		cmd = "cmd"
		args, url = i.BuildWindowsAgsAndURL(url)
	case "darwin":
		cmd = "open"
	default: // "linux", "freebsd", "openbsd", "netbsd"
		if i.IsWSL() {
			cmd = "cmd.exe"
			args, url = i.BuildWindowsAgsAndURL(url)
		} else {
			cmd = "xdg-open"
		}
	}

	args = append(args, url)

	return exec.Command(cmd, args...).Start()
}

...

// IsWSL runs the "uname -a" command to see if the output contains "microsoft" and returns true if it does, else false.
// This is so we can check if the Linux OS is actually Windows SubSystem for Linux, which requires
// A different command to open a browser.
func (i DefaultInteractor) IsWSL() bool {
	b, err := exec.Command("uname", "-a").Output()
	if err != nil {
		fmt.Printf("\nWarning: %s\n", err.Error())
		return false
	}

	if strings.Contains(strings.ToLower(string(b)), "microsoft") {
		return true
	}

	return false
}

// BuildWindowsAgsAndURL builds the command args and escapes the url for the Windows command to open a browser.
// The character & is treated as running a separate command in Windows
// cmd /c start "http://domain.com?param1&param2" results in trying to run cmd /c "start http://domain.com?parm1" & param2
// Also, the " char is used as the delimiter to escape special characters, so "&" would become \&\
// cmd /c start 'http://domain.com?param1=value with space"&"param2=value2' works when inputting directly to the command prompt,
// but the "&" is escaped by \"&\" when passed from code, which becomes \&\, resulting in cmd /c start 'http://domain.com?param1\&\param2'
// The start command uses ^ to escape special characters
func (i DefaultInteractor) BuildWindowsAgsAndURL(url string) (args []string, escapedURL string) {
	return []string{"/c", "start"},
		strings.ReplaceAll(strings.ReplaceAll(url, " ", "%20"), "&", `^&`)
}

Alternatives and current work-arounds

A clear and concise description of any alternatives you've considered or any work-arounds that are currently in place.

I'm unsure of any other solutions aside from installing/requiring 3rd party scripts/tools.

Additional context

Add any other context or screenshots about the feature request here.

I tested the code and it works well on WSL (some real values taken out):

/mnt/c$ ./k8s-pixy-auth auth   --issuer-endpoint "https://ISSUER.COM"   --audience "AUDIENCE"   --client-id "CLIENTID"   --use-id-token
in the future you can set KEYRING_K8S_PIXY_AUTH_PASSWORD to bypass this prompt
Enter passphrase to unlock /home/traviisd/.k8s-pixy-auth: {"kind":"ExecCredential","apiVersion":"client.authentication.k8s.io/v1beta1","spec":{},"status":{"token":"THE JWT YAY!"}}

The connection to the server localhost:8080 was refused

Hello! I have the following behavior:

$ kubectl get nodes
The connection to the server localhost:8080 was refused - did you specify the right host or port?

What should I do to make it work?

OS: Ubuntu 18.04

Which client id？

Description

There are two questions.

13.Make sure you've put in the client ID you saved from step 5 and save the rule.
I'm having trouble, so I want to check.
Is step5 the Client ID of "minikube-login Native"?

--client-id "QXV0aDAgaXMgaGlyaW5nISBhdXRoMC5jb20vY2FyZWVycyAK"
Client ID of "minikube-login Native"?

Distribute via krew?

👋 Hello, maintainer of the kubectl plugin manager krew here.

What do you think about making your plugin available via krew? You would benefit from

• Greater visibility due krew's large user basis.
• Easier installs.
• Seamless upgrades for new plugin versions.

Krew already has over 60 plugins and more are coming in. See our development guide if you're interested. Please also consult our naming guide to see if your plugin already fulfills our naming requirements.

Browser to authenticate does not open/work on Windows 10

Please do not report security vulnerabilities here. The Responsible Disclosure Program details the procedure for disclosing security issues.

Thank you in advance for helping us to improve this library! Please read through the template below and answer all relevant questions. Your additional work here is greatly appreciated and will help us respond as quickly as possible. For general support or usage questions, use the Auth0 Community or Auth0 Support. Finally, to avoid duplicates, please search existing Issues before submitting one here.

By submitting an Issue to this repository, you agree to the terms within the Auth0 Code of Conduct.

Description

Our organization has users on both Mac & Windows, the Mac version works perfectly, but Windows doesn't seem to work correctly..

• The character & is treated as running a separate command in Windows
• cmd /c start "http://domain.com?param1&param2" results in trying to run cmd /c "start http://domain.com?parm1" & param2
• Also, the " char is used as the delimiter to escape special characters, so "&" would become &\
• cmd /c start 'http://domain.com?param1"&"param2' works when inputting directly to the command prompt, but the "&" is escaped by "&" when passed from code, which becomes &, resulting in cmd /c start 'http://domain.com?param1\&\param2'
• And spaces will not work within the string we need to encode only the whitespace.
• This code creates the correct URL string, adds the start command to a Powershell file and executes Powershell to allow the browser to be opened for auth.

Reproduction

Detail the steps taken to reproduce this error, what was expected, and whether this issue can be reproduced consistently or if it is intermittent.

On Windows (replace with actual issuer, audience, and client id):

1. First run the init command

k8s-pixy-auth init `
>>   --context-name "auth-test" `
>>   --issuer-endpoint "https://the-issuer-endpoint" `
>>   --audience "api://default" `
>>   --client-id "SOMECLIENTID" `
>>   --use-id-token

• Results in

Installing binary...
Updating kube config... 

2. Ensure the correct context kubectl config use-context auth-test then kubectl get pods

• The code just hangs and eventually times out with the error:
  W0805 15:32:28.169535 5516 transport.go:243] Unable to cancel request for *exec.roundTripper

3. Try running the auth command

k8s-pixy-auth auth `
--issuer-endpoint "https://the-issuer-endpoint" `
--audience "api://default" `
--client-id "SOMECLIENTID" `
--use-id-token

• Hangs indefinitely
• I added code to output the command with the args, ran it in a terminal, which lead to the error (and this rabbit hole):

At line:1 char:96
+ ... .com/oauth2/default/v1/authorize?audience=api://default&scope=op ...
+                                                                 ~
The ampersand (&) character is not allowed. The & operator is reserved for future use; wrap an ampersand in double quotation marks ("&") to pass it
as part of a string.
At line:1 char:115

Where applicable, please include:

• Code sample to reproduce the issue
• Log files (redact/remove sensitive information)
• Application settings (redact/remove sensitive information)
• Screenshots

Environment

Please provide the following:

• Version of this library used:
  v0.3.3
• Version of the platform or framework used, if applicable:
• Other relevant versions (language, server software, OS, browser):
  Windows 10
• Other modules/plugins/libraries that might be involved:

Code that fixes it

I tried many, many ways and I could only get it to work correctly by creating a ps1 file and running Powershell.

Updated os/os_interpreter.go line 18

// OpenURL opens the passed in url in the default OS browser
func (i *DefaultInteractor) OpenURL(url string) error {
	var cmd string
	var args []string

	switch runtime.GOOS {
	case "windows":
		// The character & is treated as running a separate command in Windows
		// cmd /c start "http://domain.com?param1&param2" results in trying to run cmd /c "start http://domain.com?parm1" & param2
		// Also, the " char is used as the delimiter to escape special characters, so "&" would become \&\
		// cmd /c start 'http://domain.com?param1=value with space"&"param2=value2' works when inputting directly to the command prompt,
		// but the "&" is escaped by \"&\" when passed from code, which becomes \&\, resulting in cmd /c start 'http://domain.com?param1\&\param2'
		// And spaces will not work within the string we need to encode only the whitespace.
		// SO, this code creates the correct URL string, adds the start command to a powershell file and executes powershell
		// to allow the browser to be opened for auth.
		tmpfile := []byte(fmt.Sprintf("start '%s'", strings.ReplaceAll(strings.ReplaceAll(url, " ", "%20"), "&", `"&"`)))
		if err := ioutil.WriteFile("browser-signin.ps1", tmpfile, 0644); err != nil {
			return err
		}
		cmd = "cmd"
		args = []string{"/c", "Powershell", ".\\browser-signin.ps1"}
	case "darwin":
		cmd = "open"
		args = append(args, url)
	default: // "linux", "freebsd", "openbsd", "netbsd"
		cmd = "xdg-open"
		args = append(args, url)
	}

	return exec.Command(cmd, args...).Start()
}

Support for application/x-www-form-urlencoded for token API requests

I was trying to use k8s-pixy-auth with the auth provider we use in our organisation and found that it expects token API requests to be submitted with the content type application/x-www-form-urlencoded, so it fails with 400 Bad Request when trying to login. Had a look through the OAuth 2.0 spec, and it says to use application/x-www-form-urlencoded here: https://tools.ietf.org/html/rfc6749#section-4.3.2

Looking at authorization_tokenretriever.go, k8s-pixy-auth is using application/json which I assume something that works with most auth providers? Will you consider supporting application/x-www-form-urlencoded token API requests as well possibly via specifying a command line flag to indicate which content type to use? Or switch to application/x-www-form-urlencoded without using application/json if it makes this more compliant with the spec?

Unable to use cached token using "freedesktop.secretservice" backend

Please do not report security vulnerabilities here. The Responsible Disclosure Program details the procedure for disclosing security issues.

Thank you in advance for helping us to improve this library! Please read through the template below and answer all relevant questions. Your additional work here is greatly appreciated and will help us respond as quickly as possible. For general support or usage questions, use the Auth0 Community or Auth0 Support. Finally, to avoid duplicates, please search existing Issues before submitting one here.

By submitting an Issue to this repository, you agree to the terms within the Auth0 Code of Conduct.

Description

When executing kubectl via k8s-pixy-auth binary using freedesktop secret service backend, it is possible to get tokens from the K8s apiserver and properly cached.(This is visible using seahorse), and the kubectl command works.

However,when executing the kubectl again, it's not possible to use the cached token, instead it initialize the Authn journey again with the browser.

Reproduction

Detail the steps taken to reproduce this error, what was expected, and whether this issue can be reproduced consistently or if it is intermittent.

Where applicable, please include:

Using freedesktop secret service backend in a Linux OS,

configure the K8S cluster according to the guide,

configure the kubeconfig file according the guide,

execute kubectl get nodes ---> browser auth -> successful result

execute kubectl get nodes ---> browser auth -> successful result

The desired behaviour is supposed to use the cached token in the freedesktop.secretservice and the second time authentication is not needed.

I will open a PR about this tiny fix.

Environment

Please provide the following:

• **Version of this library used: master
• **Version of the platform or framework used, if applicable: go.1.12
• **Other relevant versions (language, server software, OS, browser): Fedora 30, Fedora 31, Chromium browser
• **Other modules/plugins/libraries that might be involved: keyring, libsecret, go-dbus.