Coder Social home page Coder Social logo

replicatedhq / replicated Goto Github PK

View Code? Open in Web Editor NEW
34.0 23.0 23.0 30.61 MB

A CLI to create, edit and promote releases in Replicated

Home Page: https://vendor.replicated.com

License: MIT License

Makefile 0.62% Go 99.10% Shell 0.20% Dockerfile 0.08%
replicated kots

replicated's Introduction

Replicated Vendor CLI

This repository provides a Go client and CLI for interacting with the Replicated Vendor API.

CLI

Mac Install

brew install replicatedhq/replicated/cli

Linux Install

curl -o install.sh -sSL https://raw.githubusercontent.com/replicatedhq/replicated/master/install.sh
sudo bash ./install.sh

Getting Started

You can use the replicated CLI command (i.e. $ replicated --help).

(Pre-requirement) Config the CLI to have access to the Replicated Vendor portal

To use the CLI you need a user API Token to connect. You can create one by clicking on New User API Token in your Account Settings.

Then, you can set the following environment variable to avoid passing it as an argument to each command

$ export REPLICATED_API_TOKEN=<Your new User API Token>

Now, let's check an example

We will check all Applications available and the distribution channels:

To list the applications available run replicated app ls, i.e.:

$ replicated app ls  
ID                             NAME     SLUG              SCHEDULER
2FOfwth3fHauBqCvsZ1OaBAr7MU    test     test-rodent       kots
2FOvdw6IR0oewVPVCcmH12tSRoL    nginx    nginx-sheepdog    kots

Then, to check the channels run replicated channel ls --app <Your APP SLUG OR ID>, i.e.:

$ replicated channel ls --app 2FOfwth3fHauBqCvsZ1OaBAr7MU
ID                             NAME        RELEASE    VERSION
2FOfwru7Rq1plkqyZFLH6MLR1fk    Stable      1          0.0.1
2FOfwu2zcDbqR24BVPSjjnkVwIe    Beta        1          0.0.1
2FOfwreTFbmkXtf9bukwh8s1ewb    Unstable    1          0.0.1

Notes:

  • If you do not export the environment variable above then, you must pass your User API token via the flag --token <Your new User API Token>
  • You can also, set via environment variable your app slug or ID (i.e. export REPLICATED_APP=<Your APP SLUG OR ID>). So that, the above command would be simply replicated channel ls.

CI Example

Creating a new release for every tagged build is a common use of the replicated command.

Assume the app's yaml config is checked in at replicated.yaml and you have configured TravisCI or CircleCI with your REPLICATED_APP and REPLICATED_API_TOKEN environment variables.

Then add a release.sh script to your project something like this:

#!/bin/bash

# Create a new release from replicated.yaml and promote the Unstable channel to use it.
# Aborts if version tag is empty.

set -e

VERSION=$1
INSTALL_SCRIPT=https://raw.githubusercontent.com/replicatedhq/replicated/master/install.sh
CHANNEL=Unstable

if [ -z "$VERSION" ]; then
echo "No version; skipping replicated release"
  exit
fi

# install replicated
curl -sSL "$INSTALL_SCRIPT" > install.sh
sudo bash ./install.sh

cat replicated.yaml | replicated release create --yaml - --promote Unstable --version "$VERSION"
# Channel ee9d99e87b4a5acc2863f68cb2a0c390 successfully set to release 15

Now you can automate tagged releases in TravisCI or CircleCI:

# .travis.yml
sudo: required
after_success:
  - ./release.sh "$TRAVIS_TAG"
# circle.yml
deployment:
  tag:
    tag: /v.*/
    owner: replicatedcom
    commands:
      - ./release.sh "$CIRCLE_TAG"

Client

GoDoc

package main

import (
	"fmt"
	"log"
	"os"

	"github.com/replicatedhq/replicated/pkg/platformclient"
)

func main() {
	token := os.Getenv("REPLICATED_API_TOKEN")
	appSlugOrID := os.Getenv("REPLICATED_APP")

	api := platformclient.New(token)

	app, err := api.GetApp(appSlugOrID)
	if err != nil {
		log.Fatal(err)
	}

	channels, err := api.ListChannels(app.Id)
	if err != nil {
		log.Fatal(err)
	}
	for _, c := range channels {
		fmt.Printf("channel %s is on release %d\n", c.Name, c.ReleaseSequence)
	}
}

Development

make build installs the binary to $GOPATH/bin The models are generated from the API's swagger spec.

Tests

Environment

  • REPLICATED_API_ORIGIN may be set to override the API endpoint
  • VENDOR_USER_EMAIL and VENDOR_USER_PASSWORD should be set to delete apps created for testing
  • REPLICATED_REGISTRY_ORIGIN may be set to overrride the registry endpoint

Releases

Releases are created on Travis when a tag is pushed. This will also update the docs container.

git tag -a v0.1.0 -m "First release" && git push upstream v0.1.0

Regenerating Client Code

When the swagger definitions change, you can regenerate the Client code from the swagger spec with

make get-spec-prod gen-models

models for the v2 api isn't really working yet, need to find the URL for that OpenAPI spec.

Usage Recipes

Make a new release by editing another

replicated release inspect 130 | sed 1,4d > config.yaml
vim config.yaml
cat config.yaml | replicated release create --yaml -
# SEQUENCE: 131

replicated's People

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

replicated's Issues

Using single ip and configuring paths to access different services.

I have multiple services and deployments running, for example:
backed, frontend, prometheus

Is there a way I could provide client with a single ip and configure paths to all other services.
Example: base-ip/backend -> backend (takes the user to service backend)
base-ip/monitoring -> prometheus (take the user to service prometheus)

Push Docker image via CLI

It would be great if we could push the Docker image via CLI. Hence we could use tokens instead of using username/password. Is anything like this on the roadmap?

There should be an easy way to pass multiline markdown notes on command line

replicated release create and promote commands take a --release-notes argument:

--release-notes string   When used with --promote <channel>, sets the **markdown** release notes

However this string must be escaped properly in order to work in bash. There has to be an easier way, for example loading release notes from a file. --release-notes-file parameter could be added.

Frequency of cleanup

Hi team 👋

When updating an instance, what is the frequency of the cleanup of older images?
We are aware that when the cleanup of old images does occur, the logs for the command docker logs replicated should output the following:
Removing unused image ...
But unsure how frequent this automated task occurs. Could you please provide us with some clarity?

Thanks!

"installer create" doesn't return error message

If I have a bad key or value in installer YAML, the Replicated Vendor Portal returns the error message, but all the CLI does is return an error code.

Suggested fix: have CLI return error messages from Replicated API to ease troubleshooting.

To reproduce:

  1. Create an installer YAML where the version key is missing for a kurl addon. There is no linter (afaik) for kurl yaml, so I try to create a release by pasting it in Vendor Portal and get an error spec/kotsadm must have required property 'version'.

    apiVersion: "cluster.kurl.sh/v1beta1"
    kind: "Installer"
    metadata: 
      name: "latest"
    spec: 
      kubernetes: 
        #    version: "latest"
      containerd: 
        version: "latest"
    
  2. Do the same thing with CLI using something like replicated installer create --yaml-file kurl-installer.yaml

Result:

  • Creating Installer ✗  
Error: POST https://api.replicated.com/vendor/v3/app/xxxxx/installer 500:

No error message is displayed.

List customers command only returns the first 20 for kots apps

func (c *VendorV3Client) ListCustomers(appID string) ([]types.Customer, error) {
path := fmt.Sprintf("/v3/app/%s/customers", appID)
var resp CustomerListResponse
err := c.DoJSON("GET", path, http.StatusOK, nil, &resp)
if err != nil {
return nil, errors.Wrapf(err, "list customers")
}
return resp.Customers, nil
}
does not include a pageSize query parameter, but https://replicated-vendor-api.readme.io/v3.0/reference#listappcustomers-1 expects one.

Support More Flexible Config File Path

configFilePath is currently hardcoded to ${HOME:-$USERPROFILE}/.replicated/config.yaml which isn't very flexible and doesn't follow expectations of modern programs. In sandboxed environments these directories may not be writable or appropriate.

func configFilePath() string {
      return path.Join(homeDir(), ".replicated", "config.yaml")
}

https://github.com/replicatedhq/replicated/blob/main/pkg/credentials/credentials.go#L100

It would be useful if this supported other directories. For example, if replicated followed the XDG Base Directory specification standard.

This doesn't need to be done in-house, there are already packages in go which "do the right thing" for all platforms, like arg which makes this line something like:

func configFilePath() string {
      return xdg.ConfigFile("replicated/config.yaml")
}

Which would (by default) evaluate to ~/.config/replicated/config.yaml on Linux and ~/Library/Application Support/replicated/config.yaml on MacOS.

Failing that, adding a REPLICATED_CONFIG_HOME env var would make it easy to change this directory, but seeing as the only thing it currently contains is the REPLICATED_API_TOKEN that might not be a desirable solution.

Auto-increment semver version when promoting a release to a channel

Problem to Solve

The vendor portal makes it pretty easy to auto-increment semver versions when promoting. The CLI is not as straightforward. Right now, to promote a release to the stable channel, my default is:

  1. get the most recent release on the Stable channel's current semver (e.g. 1.0.2)
  2. increment it by a minor or patch version e.g. (1.1.0)
  3. Use that to promote it:
replicated release promote 35 Stable --version 1.1.0

To do this in a one liner, you can use a tool like semver from the NPM registry:

replicated release promote 35 Stable --version $(npx semver -i minor $(replicated channel ls |grep Stable | awk '{print $4}'))

Proposed solution

It would be great to just have

replicated release promote 35 Stable --version-increment=minor
# or
replicated release promote 35 Stable --version-increment=patch

Bug: the web ui generates clobbered cli

I was wondering why my cluster only had a TTL of 1h instead of the requested 12h. I copy&pasted the CLI command from the web UI and missed that there is a whitespace missing before the --ttl argument.

The generated CLI looks like this:

replicated cluster create  --distribution openshift --instance-type r1.large  --version 4.14.0-okd--ttl 12h  --disk 50

Seems that version parsing succeedes but then a default TTL of 1h is set.

Automate updating homebrew-replicated on release

The latest release of this hasn't been updated in https://github.com/replicatedhq/homebrew-replicated. Some automation to update on release could help.

I have this setup on one of my projects at https://github.com/puppetlabs/wash/blob/master/.github/workflows/release.yml. It's not a perfect fit because publishing release artifacts happens as part of a Travis CI job, and the GitHub Action can run before that. You'd probably want to combine publishing release artifacts and updating homebrew-replicated.

Release notes lines beginning with hashes are stripped from the output

If I try and put the release notes directly into the application.yaml file like this -

apiVersion: kots.io/v1beta1
kind: Application
metadata:
  name: redgate-clone
spec:
  title: Redgate Clone
  icon: https://cloningcapability.z33.web.core.windows.net/redgate.png
  releaseNotes: |
    ## 1.0.54848
    #### Released on 2023-06-13

    ### Warnings

    ### Features

    ### Fixes
    * Reduced default CPU request for Oracle clones
    * Do stuff like `abc | def` or '
  • and then I run a replicated release create, all the lines apart from the last two are stripped when I look at it in the vendor portal.
    image

CLI command to download installer config

I'm currently managing several different versions of installer config as we beta test containerd but also need to roll out other installer changes. It'd be useful to have something like replicated installer download <sequence> so I can simply compare different sequences.

split channel inspect into smaller commands

  • channel adoption prints the channel's adoption by license type
  • channel counts prints the channel's license counts
  • channel releases prints the channel's release history
  • channel inspect only prints channel attributes

replicated release lint sometimes times out

Error: failed to execute request: Post "https://lint.replicated.com/v1/lint": unexpected EOF

The tar | curl method works great still

tar cf - manifests | \         
                curl -sSL -XPOST --data-binary @- https://lint.replicated.com/v1/lint | \
                jq .

Beta linter encounters nil pointer dereference with LocalRegistryImagePullSecret

Linting

apiVersion: v1
kind: Secret
metadata:
  annotations:
    kots.io/exclude: '{{repl or (eq Namespace (or (ConfigOption "namespace") Namespace)) (not HasLocalRegistry) }}'
  name: kotsadm-replicated-registry
  namespace: '{{repl or (ConfigOption "namespace") Namespace }}'
type: kubernetes.io/dockerconfigjson
data:
  .dockerconfigjson: '{{repl LocalRegistryImagePullSecret}}'

with --beta-linter produces

RULE                TYPE     FILENAME                       LINE    MESSAGE
unable-to-render    error    cd4pe/regcred.yaml                                                                       at <LocalRegistryImagePullSecret>: error calling LocalRegistryImagePullSecret: runtime error: invalid memory address or nil pointer dereference

setup-replicated-cli GitHub Action

Not sure if this should go here or replicatedhq/replicated-actions, but it'd be nice to have a setup-replicated-cli GitHub Action. I don't have strong opinions about the features, but it'd be nice if it supported version and fingerprint arguments.

lmk if you're up for a PR, b.c. I'm already implementing the simple script for this on a private action, and wouldn't mind the logic having community eyes on it

Getting "Not found" when app is not specified

If I forget to specify the app with --app or REPLICATED_APP environment variable, I get a Not found error. It makes sense because the server returns 404, but it's confusing. I always have to think a little before figuring out what happened. It'd be great if it could specifically say the app name is missing.

replicated channel inspect returns no output

From version 0.46 of the CLI, replicated channel inspect <my-channel> doesn't seem to return any output unless you specify a value for --output.

The help text says that --output=text is supposed to be the default, but you need to set it explicitly to get any output. Both text and json seem to work, it's when you don't specify it that you get no output.

Beta linter line numbers are off by one

Using https://github.com/MikaelSmith/cd4pe-k8s to repro. Some errors hidden for brevity

Using the tar endpoint:

tar cf - manifests | \           
                curl -sSL -XPOST --data-binary @- https://lint.replicated.com/v1/lint | \
                jq --raw-output '.lintExpressions[] | select(.positions != null) | "\(.type)\t\(.path)\t\(.positions[] | .start.line)\t\(.rule)\t\(.message)"' | \
                cat <(printf "TYPE\tPATH\tLINE\tRULE\tMESSAGE\n") - | \
                column -t -s $'\t'
TYPE  PATH                         LINE  RULE                 MESSAGE
warn  manifests/config.yaml        76    invalid_type         Invalid type. Expected: string, given: boolean

but using the CLI its off by 1

$ replicated release lint --beta-linter --yaml-dir manifests/
RULE                   TYPE     FILENAME                       LINE    MESSAGE
invalid_type           warn     manifests/config.yaml          77      Invalid type. Expected: string, given: boolean    
Error: One or more errors of severity "error" or higher were found
$ nl manifests/config.yaml
     1  apiVersion: kots.io/v1beta1
     2  kind: Config
     3  metadata:
     4    name: cd4pe-config
     5  spec:
     6    groups:
     7    - name: application_setup
     8      title: Application Setup
     9      items:
    10      - name: namespace
    11        title: Kubernetes Namespace
    12        help_text: |
    13          The namespace to deploy in. The namespace needs to exist before deploying to it. Set blank to deploy to the same namespace as the admin console.
    14        type: text
    15        value: cd4pe
    16      - name: pfi_secret_key
    17        title: Database Encryption Secret Key
    18        help_text: |
    19          A 16-byte secret key used for AES encryption of secrets (such as PE access tokens) supplied to Continuous Delivery for PE.
    20          The default is not cryptographically-generated. Generate a new key by running: `dd bs=1 if=/dev/urandom count=16 2>/dev/null | base64`.
    21          Warning: changing this after initial rollout may make data in CD4PE inaccessible.
    22        type: password
    23        required: true
    24         # Uses Sprig functions to generate a (mostly) random 16-byte key
    25        value: '{{repl randAscii 16 | b64enc}}'
    26      # MinIO/PostgreSQL credentials are here to persist generated values but hidden to prevent changing them.
    27      # They should only be changed in the event of a security incident; doing so requires changing values
    28      # here (so CD4PE knows how to connect) and issuing commands to minio and postgres to change passwords.
    29      - name: postgres_user
    30        type: text
    31        hidden: true
    32        value: cd4pe
    33      - name: postgres_password
    34        type: password
    35        hidden: true
    36        value: '{{repl RandomString 24}}'
    37      - name: minio_access_key
    38        type: text
    39        hidden: true
    40        value: minio
    41      - name: minio_secret_key
    42        type: password
    43        hidden: true
    44        value: '{{repl RandomString 24}}'
    45    - name: ingress_setup
    46      title: Ingress Setup
    47      description: |
    48        Setup how the application is exposed externally.
    49      items:
    50      - name: enable_ingress
    51        type: bool
    52        title: Enable Kubernetes Ingress
    53        help_text: |
    54          Uncheck this box to disable the Kubernetes Ingress resource.
    55        default: "1"
    56      - name: hostname
    57        type: text
    58        title: Hostname
    59        help_text: |
    60          Use this field to provide a hostname for your CD4PE Application installation.
    61        required: true
    62        when: repl{{ ConfigOptionEquals "enable_ingress" "1" }}
    63      - name: allow_http
    64        type: bool
    65        title: Allow Unsecured Access through HTTP
    66        help_text: |
    67          Uncheck this box to disable HTTP traffic between the client and the load balancer.
    68        default: "1"
    69        when: repl{{ ConfigOptionEquals "enable_ingress" "1" }}
    70      - name: annotations
    71        type: textarea
    72        title: Annotations
    73        help_text: |
    74          Use this textarea to provide annotations specific to your ingress controller.
    75          For example, `kubernetes.io/ingress.class: alb` when using the ALB ingress controller.
    76        when: repl{{ ConfigOptionEquals "enable_ingress" "1" }}

"brew upgrade" suggestion can fail if user has not done "brew update"

I don't think Casks get updated as part of the regular auto-update that occurs during a brew upgrade - I have several times had to explicitly issue a brew update && brew upgrade replicatedhq/replicated/cli to get the CLI package to upgrade.

The suggestion from the binary should probably include something to that effect.

Today:

$ replicated cluster kubeconfig 8d9ae092
Update available: v0.70.0
To upgrade, run "brew upgrade replicatedhq/replicated/cli"

$ brew upgrade replicatedhq/replicated/cli
Warning: replicatedhq/replicated/cli 0.69.0 already installed

Should probably read:

To upgrade, run "brew update && brew upgrade replicatedhq/replicated/cli"

Linux CLI install supports Homebrew

README suggests the Linux CLI isn't available for Homebrew (but is for MacOS) but that isn't accurate. Suggest making Homebrew the recommended install path for Linux as well.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.