jetstack / jsctl Goto Github PK

jsctl registry auth output --format=dockerconfig produces a JSON object with four attributes: username, password, email and auth. Both password and auth are derived from the contents of ~/.jsctl/eu.gcr.io--jetstack-secure-enterprise.json making the resultant JSON look more complicated than it actually is. That would be fine if this complexity served an identifiable purpose.

Given that auth and password are equivalent, I suspected that one was not strictly needed to gain access to the private registry. This suspicion appears to be correct and the password attribute is indeed surplus to requirements.

You can (IMO should) remove the password attribute from the result of jsctl registry auth output --format=dockerconfig.

https://platform.jetstack.io/documentation/installation/enterprise-registry is out of date, we should add a note suggesting users run:

jsctl registry auth init
jsctl registry auth status 
...

Etc. instead of going via the UI.

It'd also be nice to be able to run:

jsctl registry auth show --format=secret
jsctl registry auth show --format=dockercfg 

This would mean we have feature parity with the UI.

The standard behaviour is that the user dances around a browser connected to TLSPK to complete the auth. This is not appropriate in non-interactive sessions where we'd like the sequence to be automatable. To this aim the jsctl auth login --credentials ${creds_file} syntax was introduced. To obtain a creds file, the user needs to create a TLSPK service account from here (https://platform.jetstack.io/org/<ORG_NAME>/manage/service_accounts) which looks something like this:

{
  "user_id": "[email protected]",
  "user_secret": "S5wF+IP......o@7g1\\Ty/"
}

So far so good, but we (@inteon) believe the service accounts are insufficiently privileged because the following happens:

$ jsctl auth login --credentials creds.json
Login succeeded
$ jsctl config set organization gallant-wright
Your organization has been changed to gallant-wright
$ jsctl clusters connect kind_2302201811
failed to create service account: missing some roles in organization "gallant-wright": admin (403)

Perhaps, what we really want is the ability to download equivalent credentials file for an org "Member", rather than a "Service Account".

I can see no reason why this shouldn't be possible since this is no different to how, for example, an AWS user obtains their AccessKeyID/SecretAccessKey pair. NOTE that AWS provides two such pairs per user at any time to enable safe credential rotation which might be a requirement.

Users who are unable to run the CLI and a browser on the same machine need a means to login with jsctl auth login.

I think it makes sense to have them copy the URL into a browser, then copy their redirected URL containing the callback state and code into the terminal. The CLI can then use these to request the token as usual.

I executed

jsctl auth login
jsctl config set organization xxx

as a result, I see that the config and auth data are store on the file system with 644 permissions. This is insecure. At most, the permissions must be 600.

Also, consider

• opening/writing files with symbolic link check. This is to avoid symbolic link attacks.
• do not load configuration/auth data if the file permissions are too open.

Authentication jsctl works well when it is installed on a machine with browser. In many cases, users are not having direct access from their workstations to the K8s clusters. Instead, they need to use bastion/jump hosts. Such hosts are usually headless (no browser), so the current authentication flow will not work. OAuth 2.0 Device Authorization Grant is a flow which can help.

PS: https://github.com/cli/oauth is a nice Go library from the Github.com developers.

jsctl auth login
redirects me to the browser, i use my jetstack email - the browser displays a message Unauthorized, where as the terminal shows Login succeeded. User gets two conflicting messages

Locations where the jsctl auth token is stored as follows:
Linux: ~/.config/jsctl/token.json
MacOS: ~/Library/Application Support/jsctl/token.json

The CLI tools I use most frequently (e.g. aws and kubectl) have consistent config locations across different OS's.
Unless there's a good reason for this unusual discrepancy, may I suggest we standardise on the Linux location (i.e. ~/.config/jsctl/) ?

Found possible bug with jsctl-0.1.2-darwin-arm64 binary. After successful logon, I try the following …
jsctl operator deploy
… and get a mysterious response …
no key file exists at
However if I extract the agent-credentials secret and apply the --credentials credentials.json flag it works.
Is the --credentials flag meant to be mandatory or is something else going on here?

Could jsctl return just the secret credential needed, like:

jsctl clusters connect example --credential-only --stdout

Help something like this:

Creates a new cluster in the control plane and deploys the agent in your current kubenetes context

Usage:
  jsctl clusters connect [name] [flags]

Flags:
  -h, --help              help for connect
      --registry string   Specifies an alternative image registry to use for the agent image "quay.io/jetstack")
      --credential-only   Outputs only the require agent credential for population in your secrets management solution

The reasoning is that most customers, once they are up and running will likely re-use existing manifests (due to custom tweaks), and the only thing they really need is a new credential. This would help fit a little better into existing deployment solutions if the secret / SaaS credential can be returned to be pre-populated before app deployment.

Had just deleted the cluster and there was no kubeconfig to use. Running jsctl clusters connect mycluster causes a panic.

panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x2 addr=0x0 pc=0x105bd9f20]

goroutine 1 [running]:
k8s.io/client-go/kubernetes.NewForConfig(0x105cab861?)
	k8s.io/[email protected]/kubernetes/clientset.go:422 +0x20
github.com/jetstack/jsctl/internal/kubernetes.NewKubeConfigApplier({0x105cab861?, 0x14000448510?})
	github.com/jetstack/jsctl/internal/kubernetes/kubernetes.go:84 +0x2c
github.com/jetstack/jsctl/internal/command.clustersConnect.func1({0x1062001a0, 0x14000448510}, {0x14000512290?, 0x106c26e00?, 0x105fa41c0?})
	github.com/jetstack/jsctl/internal/command/clusters.go:68 +0x138
github.com/jetstack/jsctl/internal/command.run.func1(0x1400039ca00?, {0x14000512290, 0x1, 0x1})
	github.com/jetstack/jsctl/internal/command/command.go:73 +0x1d8
github.com/spf13/cobra.(*Command).execute(0x1400039ca00, {0x14000512270, 0x1, 0x1})
	github.com/spf13/[email protected]/command.go:876 +0x4b8
github.com/spf13/cobra.(*Command).ExecuteC(0x1400002d900)
	github.com/spf13/[email protected]/command.go:990 +0x354
github.com/spf13/cobra.(*Command).Execute(...)
	github.com/spf13/[email protected]/command.go:918
github.com/spf13/cobra.(*Command).ExecuteContext(...)
	github.com/spf13/[email protected]/command.go:911
main.main()
	github.com/jetstack/jsctl/main.go:24 +0x11c

Ideally, we have a script that we can use to install jsctl.
This should allow us to install jsctl using one line: curl -fsSL https://github.com/jetstack/jsctl/install.sh | sh

Currently if the latest release of jsctl is downloaded from a release and then executed on macOS, it'll fail to open because we've not signed the binary (as described here: https://medium.com/anchore-engineering/developers-need-to-handle-macos-binary-signing-how-we-automated-the-solution-part-1-4433b32ae311).

There's an easy workaround but it's not a great developer experience. Would be better if we could provide a signed binary!

When installing jsctl using the install.sh, the script successfully picks up the use of zsh, suggests to add the exports to ~/.zshrc, but then instructs to source ~/.bashrc.

At the moment jsctl x clusters backup command cleans up most fields that could cause issues during restore such as UUID.

It does not however clean up owner references on resources.

I am not sure in what cases users would be migrating owned resources (issuers etc). I assume that if resources were owned (i.e by another controller) the migration would require extra custom steps.

There's a couple extra steps we could do:

• jsctl x clusters uninstall verify command could print a warning if owner references found either on custom resources or CSI driver etc deployments
• jsctl clusters status command could output how a resource/compoent installation was deployed (on basis of helm, kubectl annotations, owner refs etc)
• jsctl x clusters backup command could optionally clean up owner references

Perhaps we could wait to see if any such scenario comes up to determine what would be the best action to take.

Same result for MacOS as well as Linux:

panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0xd195b0]

goroutine 1 [running]:
k8s.io/client-go/kubernetes.NewForConfig(0x10a0650?)
k8s.io/[email protected]/kubernetes/clientset.go:422 +0x20
github.com/jetstack/jsctl/internal/kubernetes.NewKubeConfigApplier({0x10a0650?, 0x1c91b28?})
github.com/jetstack/jsctl/internal/kubernetes/kubernetes.go:84 +0x2c
github.com/jetstack/jsctl/internal/command.operatorDeploy.func2({0x1308e60, 0x4000092e80}, {0x12f2540?, 0x400004ef00?, 0x0?})
github.com/jetstack/jsctl/internal/command/operator.go:82 +0xd0
github.com/jetstack/jsctl/internal/command.run.func1(0x40000f4c80?, {0x1d4f0c0, 0x0, 0x0})
github.com/jetstack/jsctl/internal/command/command.go:74 +0x1f0
github.com/spf13/cobra.(*Command).execute(0x40000f4c80, {0x1d4f0c0, 0x0, 0x0})
github.com/spf13/[email protected]/command.go:876 +0x4d0
github.com/spf13/cobra.(*Command).ExecuteC(0x4000558f00)
github.com/spf13/[email protected]/command.go:990 +0x360
github.com/spf13/cobra.(*Command).Execute(...)
github.com/spf13/[email protected]/command.go:918
github.com/spf13/cobra.(*Command).ExecuteContext(...)
github.com/spf13/[email protected]/command.go:911
main.main()
github.com/jetstack/jsctl/main.go:26 +0x1d0

I am trying to connect a cluster to JSS , in the new instructions - Install the CLI - I copied the commands and ran it on my mac , i get the following error tar: Error opening archive: Unrecognized archive format
I then went to the github release page and did a wget to get the archive and ran tar -xvf jsctl-0.1.9-darwin-x86_64.tar.gz on the downloaded archive and everything worked fine .

I use jsctl 0.1.15. I got confused when running jsctl registry:

$ jsctl registry auth output --format=dockerconfig
no organization must be set

Not sure what that means; we guessed that it actually meant "an organization must be set". The message comes from there:

Preferably, the message should be:

$ jsctl registry auth output --format=dockerconfig
No organization has been configured. To select an organization, run:

     jsctl config set organization jetstack

(here, the command proposes one of the available organizations)

Also, it would be nice if jsctl auth login would automatically select an organization if there is a single org available.

It might be interesting to include ‘cmctl’ as a sub command for jsctl. This is both good for convince and cross pollination between the two clis.

We would like to be able to help users understand what they're running in their clusters prior to onboarding to Jetstack Secure using jsctl.

In order to do this, we'd like to have a command that the user can run to get an overview of the Jetstack tooling and cert-manager components in the cluster in YAML.

The proposal is something like this:

components: # things 'running' in the cluster which are Jetstack or cert-manager related.
  certManager:
    namespace: cert-manager
    version: v1.9.0
    installationMechanism: Helm
    apiVersion: v1
  venafiEnhancedIssuer:
    namespace: jetstack-secure
    version: v0.0.1
    installationMechanism: Helm

crds: # matching *cert-manager.io or *jetstack.io CRDs
  jetstack.io:
  - installations.operator.jetstack.io
  cert-manager.io:
  - certificaterequests.cert-manager.io   
  - certificates.cert-manager.io
  - challenges.acme.cert-manager.io       
  - clusterissuers.cert-manager.io
  - issuers.cert-manager.io              
  - orders.acme.cert-manager.io

issuers: # a list of all issuers and cluster issuers in the cluster
- kind: VenafiIssuer
  name: my-tpp-issuer
  namespace: my-tpp
- kind: ClusterIssuer
  name: my-ca-issuer

ingresses:
- name: foo
  namespace: foo
  ingressClass: nginx
  issuer: my-tpp-issuer
         
namespaces:
- cert-manager
- jetstack-secure

The command might look like this:

jsctl cluster info --format=[json|yaml]

While reviewing #94 I decided to try the Windows installation instructions, in PowerShell, but it didn't work at all:

PS C:\Users\Richard Wall> irm https://raw.githubusercontent.com/jetstack/jsctl/main/install.ps1 | iex
iex : At line:35 char:46
+ ... l jsctl to "$JsctlExe", but is already installed at "$CurrentInstall" ...
+                 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Unexpected token '$JsctlExe", but is already installed at "$CurrentInstall"."' in expression or statement.
At line:35 char:55
+ ... o install jsctl to "$JsctlExe", but is already installed at "$Current ...
+                                  ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Unexpected token '", but is already installed at "' in expression or statement.
At line:35 char:87
+ ... jsctl to "$JsctlExe", but is already installed at "$CurrentInstall"."
+                                                        ~~~~~~~~~~~~~~~~~~
Unexpected token '$CurrentInstall"."' in expression or statement.
At line:35 char:102
+ ... jsctl to "$JsctlExe", but is already installed at "$CurrentInstall"."
+                                                                       ~~~
Unexpected token '"."' in expression or statement.
At line:1 char:73
+ ... tps://raw.githubusercontent.com/jetstack/jsctl/main/install.ps1 | iex
+                                                                       ~~~
    + CategoryInfo          : ParserError: (:) [Invoke-Expression], ParseException
    + FullyQualifiedErrorId : UnexpectedToken,Microsoft.PowerShell.Commands.InvokeExpressionCommand

SystemInfo

PS C:\Users\Richard Wall> systeminfo

OS Name:                   Microsoft Windows 11 Pro
OS Version:                10.0.22621 N/A Build 22621
OS Manufacturer:           Microsoft Corporation
OS Configuration:          Standalone Workstation
OS Build Type:             Multiprocessor Free
...

/kind bug

make docs-gen will regenerate the docs, when commands are changed this needs to be re-run. We often forget and it'd be nice to check this automatically.

Instructions at https://platform.jetstack.io/org/gallant-wright/connect-cluster do not work from Ubuntu VM running on MacBook Pro M1.

1. "uname -m" from MacOS M1 returns "arm64" (works).
2. "uname -m" on Ubuntu VM hosted on MacOS M1 returns "aarch64" (fails).
   In case 2) there is no such Linux binary incorporating this name so the instructions fail.
   Lowest impact solution may be to release copies of the Linux "arm64" binaries renamed as "aarch64".

Alternatively change:
ASSET_REF=$(echo "jsctl-$VERSION-$(uname -s)-$(uname -m)" | tr '[:upper:]' '[:lower:]')
to:
ASSET_REF=$(echo "jsctl-$VERSION-$(uname -s)-$(uname -m | sed s/aarch64/arm64/g)" | tr '[:upper:]' '[:lower:]')

Similar to what we're doing here. However, we'll need to work out what to do about the operator code access.

https://github.com/jetstack/homebrew-jetstack/blob/main/Formula/paranoia.rb

While testing #97 I noticed that jsctl.exe prints multiple warnings every time I execute it:

PS C:\Users\Richard Wall> jsctl organizations list
warning: config file "C:\\Users\\Richard Wall\\.jsctl\\token.json" has insecure file permissions, correcting them
warning: config file "C:\\Users\\Richard Wall\\.jsctl\\config.json" has insecure file permissions, correcting them
NAME                ROLES
jetstack            admin, member
jetstack-demo       admin, member
jetstack-richard    admin, member

PS C:\Users\Richard Wall> jsctl config set organization jetstack-richard
warning: config file "C:\\Users\\Richard Wall\\.jsctl\\token.json" has insecure file permissions, correcting them
warning: config file "C:\\Users\\Richard Wall\\.jsctl\\config.json" has insecure file permissions, correcting them
warning: config file "C:\\Users\\Richard Wall\\.jsctl\\config.json" has insecure file permissions, correcting them
Your organization has been changed to jetstack-richard

PS C:\Users\Richard Wall> jsctl clusters list
warning: config file "C:\\Users\\Richard Wall\\.jsctl\\token.json" has insecure file permissions, correcting them
warning: config file "C:\\Users\\Richard Wall\\.jsctl\\config.json" has insecure file permissions, correcting them
NAME      LAST UPDATED
test11    2023-03-08 11:52:41

PS C:\Users\Richard Wall> ls ~/.jsctl


    Directory: C:\Users\Richard Wall\.jsctl


Mode                 LastWriteTime         Length Name
----                 -------------         ------ ----
d-----        08/03/2023     11:56                bin
-a----        08/03/2023     11:52             40 config.json
-a----        08/03/2023     11:51           1720 token.json

I understand that the agent component materialised long before the operator component but, to this day, the agent is expected to be there first (if only to create the jetstack-secure namespace).

I think committed JSS customers may question why the two are not installed as a pair since they can be considered to be two halves of the same coin.

I think the behaviour of jsctl clusters connect should be to deploy the operator as necessary and then have the operator (by default) deploy the agent.

To put this another way "Jetstack Secure Agent" should appear as a JSS Component in this table:
https://platform.jetstack.io/documentation/installation/js-operator#jss-components

With this arrangement in place the CLI could (perhaps) be simplified by deprecating all operator commands. This would amount to a breaking change so it'll be less painful to do this now than later when the user base will likely be larger.