redhat-et / copilot-ops Goto Github PK

View Code? Open in Web Editor NEW

30.0 30.0 12.0 1.26 MB

copilot-ops is a CLI tool that boosts up any DevOps repo to a ninja level of Artificially Intelligent Ops Repo

License: Apache License 2.0

Jupyter Notebook 56.69% JavaScript 0.05% TypeScript 5.32% Go 36.35% Makefile 1.11% Dockerfile 0.47%

ai devops git gitops hacktoberfest

copilot-ops's People

Contributors

Stargazers

Watchers

Forkers

osilkin98 robotsail schwesig guymguym suppathak djach7 sallyom samz4ra xstp bazhang87 eichnera doytsujin

copilot-ops's Issues

Wrong key variable name in README.md

The variable name is described as OPENAI_API_TOKEN in the README.md, but actually it is OPENAI_API_KEY.

Output and errors standardization

Errors for copilot-ops should be well-defined and explicit so that the user does not receive unambiguous responses.

For example, if copilot-ops exceeds the maximum response length for a request, the user should know that this was the cause of error.

Default Path For Newly-Generated Files

When generating new files, it'll always be a guessing game for the CLI as to where newly-generated files should be saved to.
There are a few solutions to this:

default / user-specified output directory
attempting to extract generated metadata to infer where file should be saved to, based on defined .copilot-ops.yml
user-specified commands within the content describing where the requested generated files should be saved to

Connect to AI Backend

copilot-ops should send the prompt to OpenAI & receive a completion

add more tests for expected success

copilot-ops/pkg/cmd/generate_test.go

Line 44 in 2dd6622

// TODO: add more tests for expected success

package cmd_test

import (
	"net/http/httptest"

	. "github.com/onsi/ginkgo/v2"
	. "github.com/onsi/gomega"
	"github.com/spf13/cobra"

	"github.com/redhat-et/copilot-ops/pkg/cmd"
)

var _ = Describe("Generate command", func() {
	var c *cobra.Command
	var ts *httptest.Server

	BeforeEach(func() {
		c = cmd.NewGenerateCmd()
	})

	When("server is created", func() {
		BeforeEach(func() {
			ts = OpenAITestServer()

			Expect(c).NotTo(BeNil())
			err := c.Flags().Set(cmd.FlagNTokensFull, "1")
			Expect(err).To(BeNil())
		})

		JustBeforeEach(func() {
			ts.Start()
			err := c.Flags().Set(cmd.FlagOpenAIURLFull, ts.URL)
			Expect(err).To(BeNil())
		})
		AfterEach(func() {
			ts.Close()
		})

		It("executes properly", func() {
			err := cmd.RunGenerate(c, []string{})
			// use the minimum amount of tokens from OpenAI
			Expect(err).To(BeNil())
		})
		// TODO: add more tests for expected success
	})

	When("OpenAI server is down", func() {
		BeforeEach(func() {
			// set a port that isn't taken
			err := c.Flags().Set(cmd.FlagOpenAIURLFull, "http://localhost:23423")
			Expect(err).To(BeNil())
		})
		It("fails", func() {
			err := cmd.RunGenerate(c, []string{})
			Expect(err).To(HaveOccurred())
		})
		// TODO: add more cases that should fail
	})
})

Write option fails in edit command

Sample file:
echo "test:
row:
value1: 100" > foo.yaml

Edit works fine:
copilot-ops edit -f foo.yaml -r 'Update value1 to 123'

Edit and write fails:
copilot-ops edit -f foo.yaml -r 'Update value1 to 123' --write
...
Error: open foo.yaml/foo.yaml.yaml: not a directory

Tested on both main and origin/guy-branch-upstream branches

AI Configurability

Enable the user to configure the AI module's parameters, including which backend to be using.

OpenAI currently supports the following parameters per API call: Completion API docs

Update readme with instructions on installing cli tool

It would be helpful if there are instructions shared on installing the command line tool copilot-ops and if there are any pre-requisites needed (such as golang?).

Prompt formatting

copilot-ops should take the given file data + user's input and format it accordingly to create a desired prompt

Improve the github bot UX

Currently the bot only comments back with stdout and stderr of executing copilot-ops, see for example - #51 (comment).

We can improve it the elephant salami method:

The bot should use --write to apply the generate/edit changes to the checked out repo, and then create a git diff output that will be sent as the answer to the issuer.
The bot should create a PR with a fixed name, e.g copilot-ops-fix-<issuenumber>, so that subsequent requests on the same issue would checkout the same PR, and every request will then push a new commit to this PR.
Publish the bot action as a github action in the marketplace so that any repo can embed it and use it (bringing its own openai api key).

change to 1.18 when most linters support it, see https://github.com/golangci/gol...

All possible options can be found here https://github.com/golangci/golangci-lint/blob/master/.golangci.reference.yml

Default: 10

If it's higher than 0.0 (float) the check is enabled

Default: 0.0

Such cases aren't reported by default.

Default: false

If lower than 0, disable the check.

Default: 60

If lower than 0, disable the check.

Default: 40

Default: 30 (but we recommend 10-20)

The settings key is the name of a supported gocritic checker.

The list of supported checkers can be find in https://go-critic.github.io/overview.

Default: true

Values always ignored: time.Date

Default: []

Default: false

Run go tool vet help to see all analyzers.

Default: []

Settings per analyzer.

Default: false

Default: 30

Default: []

Default: false

Default: []

Otherwise, only methods that take *testing.T, *testing.B, and testing.TB as arguments are checked.

Default: false

copilot-ops/.golangci.yaml

Line 15 in b2e1212

    
           go: "1.17" # TODO: change to 1.18 when most linters support it, see https://github.com/golangci/golangci-lint/issues/2649

## Golden config for golangci-lint v1.46.2
#
# This is the best config for golangci-lint based on my experience and opinion.
# It is very strict, but not extremely strict.
# Feel free to adopt and change it for your needs.

run:
  # Timeout for analysis, e.g. 30s, 5m.
  # Default: 1m
  timeout: 30m

  # Define the Go version limit.
  # Mainly related to generics support in go1.18.
  # Default: use Go version from the go.mod file, fallback on the env var `GOVERSION`, fallback on 1.17
  go: "1.17" # TODO: change to 1.18 when most linters support it, see https://github.com/golangci/golangci-lint/issues/2649


# This file contains only configs which differ from defaults.
# All possible options can be found here https://github.com/golangci/golangci-lint/blob/master/.golangci.reference.yml
linters-settings:
  cyclop:
    # The maximal code complexity to report.
    # Default: 10
    max-complexity: 30
    # The maximal average package complexity.
    # If it's higher than 0.0 (float) the check is enabled
    # Default: 0.0
    package-average: 10.0

  errcheck:
    # Report about not checking of errors in type assertions: `a := b.(MyStruct)`.
    # Such cases aren't reported by default.
    # Default: false
    check-type-assertions: true

  funlen:
    # Checks the number of lines in a function.
    # If lower than 0, disable the check.
    # Default: 60
    lines: 100
    # Checks the number of statements in a function.
    # If lower than 0, disable the check.
    # Default: 40
    statements: 50

  gocognit:
    # Minimal code complexity to report
    # Default: 30 (but we recommend 10-20)
    min-complexity: 20

  gocritic:
    # Settings passed to gocritic.
    # The settings key is the name of a supported gocritic checker.
    # The list of supported checkers can be find in https://go-critic.github.io/overview.
    settings:
      captLocal:
        # Whether to restrict checker to params only.
        # Default: true
        paramsOnly: false
      underef:
        # Whether to skip (*x).method() calls where x is a pointer receiver.
        # Default: true
        skipRecvDeref: false

  gomnd:
    # List of function patterns to exclude from analysis.
    # Values always ignored: `time.Date`
    # Default: []
    ignored-functions:
      - os.Chmod
      - os.Mkdir
      - os.MkdirAll
      - os.OpenFile
      - os.WriteFile
      - prometheus.ExponentialBuckets
      - prometheus.ExponentialBucketsRange
      - prometheus.LinearBuckets
      - strconv.FormatFloat
      - strconv.FormatInt
      - strconv.FormatUint
      - strconv.ParseFloat
      - strconv.ParseInt
      - strconv.ParseUint

  gomodguard:
    blocked:
      # List of blocked modules.
      # Default: []
      modules:
        - github.com/golang/protobuf:
            recommendations:
              - google.golang.org/protobuf
            reason: "see https://developers.google.com/protocol-buffers/docs/reference/go/faq#modules"
        - github.com/satori/go.uuid:
            recommendations:
              - github.com/google/uuid
            reason: "satori's package is not maintained"
        - github.com/gofrs/uuid:
            recommendations:
              - github.com/google/uuid
            reason: "see recommendation from dev-infra team: https://confluence.gtforge.com/x/gQI6Aw"

  govet:
    # Enable all analyzers.
    # Default: false
    enable-all: true
    # Disable analyzers by name.
    # Run `go tool vet help` to see all analyzers.
    # Default: []
    disable:
      - fieldalignment # too strict
    # Settings per analyzer.
    settings:
      shadow:
        # Whether to be strict about shadowing; can be noisy.
        # Default: false
        strict: true

  nakedret:
    # Make an issue if func has more lines of code than this setting, and it has naked returns.
    # Default: 30
    max-func-lines: 0

  nolintlint:
    # Exclude following linters from requiring an explanation.
    # Default: []
    allow-no-explanation: [ funlen, gocognit, lll ]
    # Enable to require an explanation of nonzero length after each nolint directive.
    # Default: false
    require-explanation: true
    # Enable to require nolint directives to mention the specific linter being suppressed.
    # Default: false
    require-specific: true

  rowserrcheck:
    # database/sql is always checked
    # Default: []
    packages:
      - github.com/jmoiron/sqlx

  tenv:
    # The option `all` will run against whole test files (`_test.go`) regardless of method/function signatures.
    # Otherwise, only methods that take `*testing.T`, `*testing.B`, and `testing.TB` as arguments are checked.
    # Default: false
    all: true

  varcheck:
    # Check usage of exported fields and variables.
    # Default: false
    exported-fields: false # default false # TODO: enable after fixing false positives


linters:
  disable-all: true
  enable:
    ## enabled by default
    - deadcode # Finds unused code
    - errcheck # Errcheck is a program for checking for unchecked errors in go programs. These unchecked errors can be critical bugs in some cases
    - gosimple # Linter for Go source code that specializes in simplifying a code
    - govet # Vet examines Go source code and reports suspicious constructs, such as Printf calls whose arguments do not align with the format string
    - ineffassign # Detects when assignments to existing variables are not used
    - staticcheck # Staticcheck is a go vet on steroids, applying a ton of static analysis checks
    - structcheck # Finds unused struct fields
    - typecheck # Like the front-end of a Go compiler, parses and type-checks Go code
    - unused # Checks Go code for unused constants, variables, functions and types
    - varcheck # Finds unused global variables and constants
    ## disabled by default
    - asciicheck # Simple linter to check that your code does not contain non-ASCII identifiers
    - bidichk # Checks for dangerous unicode character sequences
    - bodyclose # checks whether HTTP response body is closed successfully
    - contextcheck # check the function whether use a non-inherited context
    - cyclop # checks function and package cyclomatic complexity
    - dupl # Tool for code clone detection
    - durationcheck # check for two durations multiplied together
    - errname # Checks that sentinel errors are prefixed with the Err and error types are suffixed with the Error.
    - errorlint # errorlint is a linter for that can be used to find code that will cause problems with the error wrapping scheme introduced in Go 1.13.
    - execinquery # execinquery is a linter about query string checker in Query function which reads your Go src files and warning it finds
    - exhaustive # check exhaustiveness of enum switch statements
    - exportloopref # checks for pointers to enclosing loop variables
    - forbidigo # Forbids identifiers
    - funlen # Tool for detection of long functions
    - gochecknoglobals # check that no global variables exist
    - gochecknoinits # Checks that no init functions are present in Go code
    - gocognit # Computes and checks the cognitive complexity of functions
    - goconst # Finds repeated strings that could be replaced by a constant
    - gocritic # Provides diagnostics that check for bugs, performance and style issues.
    - gocyclo # Computes and checks the cyclomatic complexity of functions
    - godot # Check if comments end in a period
    - goimports # In addition to fixing imports, goimports also formats your code in the same style as gofmt.
    - gomnd # An analyzer to detect magic numbers.
    - gomoddirectives # Manage the use of 'replace', 'retract', and 'excludes' directives in go.mod.
    - gomodguard # Allow and block list linter for direct Go module dependencies. This is different from depguard where there are different block types for example version constraints and module recommendations.
    - goprintffuncname # Checks that printf-like functions are named with f at the end
    - gosec # Inspects source code for security problems
    - lll # Reports long lines
    - makezero # Finds slice declarations with non-zero initial length
    - nakedret # Finds naked returns in functions greater than a specified function length
    - nestif # Reports deeply nested if statements
    - nilerr # Finds the code that returns nil even if it checks that the error is not nil.
    - nilnil # Checks that there is no simultaneous return of nil error and an invalid value.
    - noctx # noctx finds sending http request without context.Context
    - nolintlint # Reports ill-formed or insufficient nolint directives
    - nonamedreturns # Reports all named returns
    - nosprintfhostport # Checks for misuse of Sprintf to construct a host with port in a URL.
    - predeclared # find code that shadows one of Go's predeclared identifiers
    - promlinter # Check Prometheus metrics naming via promlint
    - revive # Fast, configurable, extensible, flexible, and beautiful linter for Go. Drop-in replacement of golint.
    - rowserrcheck # checks whether Err of rows is checked successfully
    - sqlclosecheck # Checks that sql.Rows and sql.Stmt are closed.
    - stylecheck # Stylecheck is a replacement for golint
    - tenv # tenv is analyzer that detects using os.Setenv instead of t.Setenv since Go1.17
    - testpackage # linter that makes you use a separate _test package
    - tparallel # tparallel detects inappropriate usage of t.Parallel() method in your Go test codes
    - unconvert # Remove unnecessary type conversions
    - unparam # Reports unused function parameters
    - wastedassign # wastedassign finds wasted assignment statements.
    - whitespace # Tool for detection of leading and trailing whitespace
    ## you may want to enable
    #- decorder # check declaration order and count of types, constants, variables and functions
    #- exhaustruct # Checks if all structure fields are initialized
    #- goheader # Checks is file header matches to pattern
    #- ireturn # Accept Interfaces, Return Concrete Types
    #- prealloc # [premature optimization, but can be used in some cases] Finds slice declarations that could potentially be preallocated
    #- varnamelen # [great idea, but too many false positives] checks that the length of a variable's name matches its scope
    #- wrapcheck # Checks that errors returned from external packages are wrapped
    ## disabled
    #- containedctx # containedctx is a linter that detects struct contained context.Context field
    #- depguard # [replaced by gomodguard] Go linter that checks if package imports are in a list of acceptable packages
    #- dogsled # Checks assignments with too many blank identifiers (e.g. x, _, _, _, := f())
    #- errchkjson # [don't see profit + I'm against of omitting errors like in the first example https://github.com/breml/errchkjson] Checks types passed to the json encoding functions. Reports unsupported types and optionally reports occasions, where the check for the returned error can be omitted.
    #- forcetypeassert # [replaced by errcheck] finds forced type assertions
    #- gci # Gci controls golang package import order and makes it always deterministic.
    #- godox # Tool for detection of FIXME, TODO and other comment keywords
    #- goerr113 # [too strict] Golang linter to check the errors handling expressions
    #- gofmt # [replaced by goimports] Gofmt checks whether code was gofmt-ed. By default this tool runs with -s option to check for code simplification
    #- gofumpt # [replaced by goimports, gofumports is not available yet] Gofumpt checks whether code was gofumpt-ed.
    #- grouper # An analyzer to analyze expression groups.
    #- ifshort # Checks that your code uses short syntax for if-statements whenever possible
    #- importas # Enforces consistent import aliases
    #- maintidx # maintidx measures the maintainability index of each function.
    #- misspell # [useless] Finds commonly misspelled English words in comments
    #- nlreturn # [too strict and mostly code is not more readable] nlreturn checks for a new line before return and branch statements to increase code clarity
    #- paralleltest # [too many false positives] paralleltest detects missing usage of t.Parallel() method in your Go test
    #- tagliatelle # Checks the struct tags.
    #- thelper # thelper detects golang test helpers without t.Helper() call and checks the consistency of test helpers
    #- wsl # [too strict and mostly code is not more readable] Whitespace Linter - Forces you to use empty lines!
    ## deprecated
    #- exhaustivestruct # [deprecated, replaced by exhaustruct] Checks if all struct's fields are initialized
    #- golint # [deprecated, replaced by revive] Golint differs from gofmt. Gofmt reformats Go source code, whereas golint prints out style mistakes
    #- interfacer # [deprecated] Linter that suggests narrower interface types
    #- maligned # [deprecated, replaced by govet fieldalignment] Tool to detect Go structs that would take less memory if their fields were sorted
    #- scopelint # [deprecated, replaced by exportloopref] Scopelint checks for unpinned variables in go programs


issues:
  # Maximum count of issues with the same text.
  # Set to 0 to disable.
  # Default: 3
  max-same-issues: 50

  exclude-rules:
    - source: "^//\\s*go:generate\\s"
      linters: [ lll ]
    - source: "(noinspection|TODO)"
      linters: [ godot ]
    - source: "//noinspection"
      linters: [ gocritic ]
    - source: "^\\s+if _, ok := err\\.\\([^.]+\\.InternalError\\); ok {"
      linters: [ errorlint ]
    - path: "_test\\.go"
      linters:
        - bodyclose
        - dupl
        - funlen
        - goconst
        - gosec
        - noctx
        - wrapcheck

Setup CI Build process

We need to create an automated build process which runs make build on new tags and nightly builds so that copilot-ops can be consumed without cloning the repository and running make build.

copilot-ops Github Bot

We would like to configure a GitHub bot which uses Probot and the Operate-First templates to provide copilot-ops' functionality directly into a GitHub repository.

The steps to get this working are as follows:

Create a GitHub bot from the Operate-First template
Implement the ability for the bot to watch for new issues and extract their contents to be processed by copilot-ops
Implement the ability to generate new pull-requests based on the files generated through copilot-ops
Implement a feature where the issue author can make comments & request changes on the PR, and have the copilot-ops bot make the appropriate changes to the PR
Create a namespace in Operate-First to deploy the bot as a deployment

Move the patch command to edit command and make it work on 1 file

generalize overriding default values via CLI

FIXME: create default config methods for these

copilot-ops/pkg/cmd/utils.go

Line 79 in 63d4c21

// TODO: generalize overriding default values via CLI

	if err := conf.Load(); err != nil {
		return nil, err
	}
	// TODO: generalize overriding default values via CLI
	conf.SetDefaults()
	// override OpenAI URL
	if openAIURL != "" {
		conf.OpenAI.BaseURL = openAIURL
	}

	// load files
	fm := filemap.NewFilemap()
	if err := fm.LoadFiles(files); err != nil {
		log.Fatalf("error loading files: %s\n", err.Error())
	}
	if len(filesets) > 0 {
		log.Printf("loading filesets: %v\n", filesets)
	}
	if err := fm.LoadFilesets(filesets, conf, config.ConfigFile); err != nil {
		log.Fatalf("error loading filesets: %s\n", err.Error())
	}
	filemapText := fm.EncodeToInputText()

	// select backend type
	selectedBackend := ai.Backend(aiBackend)
	if selectedBackend == "" {
		selectedBackend = conf.Backend
	}

	// configure backends
	// FIXME: create default config methods for these
	r := Request{
		Config:       conf,
		Filemap:      fm,
		FilemapText:  filemapText,
		UserRequest:  request,
		IsWrite:      write,
		OutputType:   outputType,
		NTokens:      nTokens,
		NCompletions: nCompletions,
		Backend:      selectedBackend,
	}

	return &r, nil

copilot-ops as a serverless function

This is partly related to #67, but anyone interested in using copilot-ops at the moment would need to either have an account with access to the GPT-3 Codex private beta, or access to an API key that has it. In order to scale copilot-ops long-term, it would be helpful to create a serverless function which can run the necessary operations by being accessed through a given endpoint.

Project Branding

Good projects succeed through stellar marketing.
This project needs the following:

Logo
Website

(NOT A BUG) Testing the BOT action

First thing to try -

/copilot-ops --help

Specifying `edit-only` vs. `generate` mode

Related to #16, the Codex /edits endpoint will only edit files in-place based on the provided input, but it does not provide new additions in terms of new files.

For example, the following prompt + input works as expected:

files:

# @kustomization
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
    - namespace.yaml
    - resourcequota.yaml
components:
    - ../../../../components/project-admin-rolebindings/octo-training-model
    - ../../../../components/limitranges/default
namespace: training-model

input:

@kustomization needs to add `globalresources.yaml` to its `.resources` field.

Produces the output:

# @kustomization
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
    - namespace.yaml
    - resourcequota.yaml
    - globalresources.yaml
components:
    - ../../../../components/project-admin-rolebindings/octo-training-model
    - ../../../../components/limitranges/default
namespace: training-model

However, specifying the following only produces changes to @kustomization and doesn't generate the newly requested resource:

files:

# @kustomization
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
    - namespace.yaml
    - resourcequota.yaml
components:
    - ../../../../components/project-admin-rolebindings/octo-training-model
    - ../../../../components/limitranges/default
namespace: training-model

input

@kustomization needs to add `globalresources.yaml` to its `.resources` field.

We also need to create a new ConfigMap tagged `@configmap` containing a username and password for the model to pull from and use as authentication for the dataset

Produces:

# @kustomization
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
    - namespace.yaml
    - resourcequota.yaml
    - globalresources.yaml
components:
    - ../../../../components/project-admin-rolebindings/octo-training-model
    - ../../../../components/limitranges/default
namespace: training-model

Parameters used in completion:

{
  "temperature": 0.16,
  "top_p": 1
}

add more cases that should fail

copilot-ops/pkg/cmd/generate_test.go

Line 57 in 2dd6622

// TODO: add more cases that should fail

package cmd_test

import (
	"net/http/httptest"

	. "github.com/onsi/ginkgo/v2"
	. "github.com/onsi/gomega"
	"github.com/spf13/cobra"

	"github.com/redhat-et/copilot-ops/pkg/cmd"
)

var _ = Describe("Generate command", func() {
	var c *cobra.Command
	var ts *httptest.Server

	BeforeEach(func() {
		c = cmd.NewGenerateCmd()
	})

	When("server is created", func() {
		BeforeEach(func() {
			ts = OpenAITestServer()

			Expect(c).NotTo(BeNil())
			err := c.Flags().Set(cmd.FlagNTokensFull, "1")
			Expect(err).To(BeNil())
		})

		JustBeforeEach(func() {
			ts.Start()
			err := c.Flags().Set(cmd.FlagOpenAIURLFull, ts.URL)
			Expect(err).To(BeNil())
		})
		AfterEach(func() {
			ts.Close()
		})

		It("executes properly", func() {
			err := cmd.RunGenerate(c, []string{})
			// use the minimum amount of tokens from OpenAI
			Expect(err).To(BeNil())
		})
		// TODO: add more tests for expected success
	})

	When("OpenAI server is down", func() {
		BeforeEach(func() {
			// set a port that isn't taken
			err := c.Flags().Set(cmd.FlagOpenAIURLFull, "http://localhost:23423")
			Expect(err).To(BeNil())
		})
		It("fails", func() {
			err := cmd.RunGenerate(c, []string{})
			Expect(err).To(HaveOccurred())
		})
		// TODO: add more cases that should fail
	})
})

Output Formatting

At the time of writing, the copilot-ops CLI only has one way of printing the data to STDOUT: plaintext.
This is not sufficient for integrating the CLI into other services since processes will need to be able to extract data efficiently.
Copilot-ops should be able to format its output as JSON in addition to the current plaintext output.

Override API URL

Provide the ability to override the OpenAI URL so that we can perform tests with Ginkgo without burning through our credits.

Potentially use go-gpt3 as a client for OpenAI

To reduce the amount of maintained code in our repository, we should switch over to using go-gpt3 instead of our own. If we need to add features, we can simply PR them to that repository.

and other comment keywords

goerr113 # [too strict] Golang linter to check the errors handling expressions
gofmt # [replaced by goimports] Gofmt checks whether code was gofmt-ed. By default this tool runs with -s option to check for code simplification
gofumpt # [replaced by goimports, gofumports is not available yet] Gofumpt checks whether code was gofumpt-ed.
grouper # An analyzer to analyze expression groups.
ifshort # Checks that your code uses short syntax for if-statements whenever possible
importas # Enforces consistent import aliases
maintidx # maintidx measures the maintainability index of each function.
misspell # [useless] Finds commonly misspelled English words in comments
nlreturn # [too strict and mostly code is not more readable] nlreturn checks for a new line before return and branch statements to increase code clarity
paralleltest # [too many false positives] paralleltest detects missing usage of t.Parallel() method in your Go test
tagliatelle # Checks the struct tags.
thelper # thelper detects golang test helpers without t.Helper() call and checks the consistency of test helpers
wsl # [too strict and mostly code is not more readable] Whitespace Linter - Forces you to use empty lines!

deprecated

exhaustivestruct # [deprecated, replaced by exhaustruct] Checks if all struct's fields are initialized
golint # [deprecated, replaced by revive] Golint differs from gofmt. Gofmt reformats Go source code, whereas golint prints out style mistakes
interfacer # [deprecated] Linter that suggests narrower interface types
maligned # [deprecated, replaced by govet fieldalignment] Tool to detect Go structs that would take less memory if their fields were sorted
scopelint # [deprecated, replaced by exportloopref] Scopelint checks for unpinned variables in go programs

Set to 0 to disable.

Default: 3

copilot-ops/.golangci.yaml

Line 233 in b2e1212

#- godox # Tool for detection of FIXME, TODO and other comment keywords

## Golden config for golangci-lint v1.46.2
#
# This is the best config for golangci-lint based on my experience and opinion.
# It is very strict, but not extremely strict.
# Feel free to adopt and change it for your needs.

run:
  # Timeout for analysis, e.g. 30s, 5m.
  # Default: 1m
  timeout: 30m

  # Define the Go version limit.
  # Mainly related to generics support in go1.18.
  # Default: use Go version from the go.mod file, fallback on the env var `GOVERSION`, fallback on 1.17
  go: "1.17" # TODO: change to 1.18 when most linters support it, see https://github.com/golangci/golangci-lint/issues/2649


# This file contains only configs which differ from defaults.
# All possible options can be found here https://github.com/golangci/golangci-lint/blob/master/.golangci.reference.yml
linters-settings:
  cyclop:
    # The maximal code complexity to report.
    # Default: 10
    max-complexity: 30
    # The maximal average package complexity.
    # If it's higher than 0.0 (float) the check is enabled
    # Default: 0.0
    package-average: 10.0

  errcheck:
    # Report about not checking of errors in type assertions: `a := b.(MyStruct)`.
    # Such cases aren't reported by default.
    # Default: false
    check-type-assertions: true

  funlen:
    # Checks the number of lines in a function.
    # If lower than 0, disable the check.
    # Default: 60
    lines: 100
    # Checks the number of statements in a function.
    # If lower than 0, disable the check.
    # Default: 40
    statements: 50

  gocognit:
    # Minimal code complexity to report
    # Default: 30 (but we recommend 10-20)
    min-complexity: 20

  gocritic:
    # Settings passed to gocritic.
    # The settings key is the name of a supported gocritic checker.
    # The list of supported checkers can be find in https://go-critic.github.io/overview.
    settings:
      captLocal:
        # Whether to restrict checker to params only.
        # Default: true
        paramsOnly: false
      underef:
        # Whether to skip (*x).method() calls where x is a pointer receiver.
        # Default: true
        skipRecvDeref: false

  gomnd:
    # List of function patterns to exclude from analysis.
    # Values always ignored: `time.Date`
    # Default: []
    ignored-functions:
      - os.Chmod
      - os.Mkdir
      - os.MkdirAll
      - os.OpenFile
      - os.WriteFile
      - prometheus.ExponentialBuckets
      - prometheus.ExponentialBucketsRange
      - prometheus.LinearBuckets
      - strconv.FormatFloat
      - strconv.FormatInt
      - strconv.FormatUint
      - strconv.ParseFloat
      - strconv.ParseInt
      - strconv.ParseUint

  gomodguard:
    blocked:
      # List of blocked modules.
      # Default: []
      modules:
        - github.com/golang/protobuf:
            recommendations:
              - google.golang.org/protobuf
            reason: "see https://developers.google.com/protocol-buffers/docs/reference/go/faq#modules"
        - github.com/satori/go.uuid:
            recommendations:
              - github.com/google/uuid
            reason: "satori's package is not maintained"
        - github.com/gofrs/uuid:
            recommendations:
              - github.com/google/uuid
            reason: "see recommendation from dev-infra team: https://confluence.gtforge.com/x/gQI6Aw"

  govet:
    # Enable all analyzers.
    # Default: false
    enable-all: true
    # Disable analyzers by name.
    # Run `go tool vet help` to see all analyzers.
    # Default: []
    disable:
      - fieldalignment # too strict
    # Settings per analyzer.
    settings:
      shadow:
        # Whether to be strict about shadowing; can be noisy.
        # Default: false
        strict: true

  nakedret:
    # Make an issue if func has more lines of code than this setting, and it has naked returns.
    # Default: 30
    max-func-lines: 0

  nolintlint:
    # Exclude following linters from requiring an explanation.
    # Default: []
    allow-no-explanation: [ funlen, gocognit, lll ]
    # Enable to require an explanation of nonzero length after each nolint directive.
    # Default: false
    require-explanation: true
    # Enable to require nolint directives to mention the specific linter being suppressed.
    # Default: false
    require-specific: true

  rowserrcheck:
    # database/sql is always checked
    # Default: []
    packages:
      - github.com/jmoiron/sqlx

  tenv:
    # The option `all` will run against whole test files (`_test.go`) regardless of method/function signatures.
    # Otherwise, only methods that take `*testing.T`, `*testing.B`, and `testing.TB` as arguments are checked.
    # Default: false
    all: true

  varcheck:
    # Check usage of exported fields and variables.
    # Default: false
    exported-fields: false # default false # TODO: enable after fixing false positives


linters:
  disable-all: true
  enable:
    ## enabled by default
    - deadcode # Finds unused code
    - errcheck # Errcheck is a program for checking for unchecked errors in go programs. These unchecked errors can be critical bugs in some cases
    - gosimple # Linter for Go source code that specializes in simplifying a code
    - govet # Vet examines Go source code and reports suspicious constructs, such as Printf calls whose arguments do not align with the format string
    - ineffassign # Detects when assignments to existing variables are not used
    - staticcheck # Staticcheck is a go vet on steroids, applying a ton of static analysis checks
    - structcheck # Finds unused struct fields
    - typecheck # Like the front-end of a Go compiler, parses and type-checks Go code
    - unused # Checks Go code for unused constants, variables, functions and types
    - varcheck # Finds unused global variables and constants
    ## disabled by default
    - asciicheck # Simple linter to check that your code does not contain non-ASCII identifiers
    - bidichk # Checks for dangerous unicode character sequences
    - bodyclose # checks whether HTTP response body is closed successfully
    - contextcheck # check the function whether use a non-inherited context
    - cyclop # checks function and package cyclomatic complexity
    - dupl # Tool for code clone detection
    - durationcheck # check for two durations multiplied together
    - errname # Checks that sentinel errors are prefixed with the Err and error types are suffixed with the Error.
    - errorlint # errorlint is a linter for that can be used to find code that will cause problems with the error wrapping scheme introduced in Go 1.13.
    - execinquery # execinquery is a linter about query string checker in Query function which reads your Go src files and warning it finds
    - exhaustive # check exhaustiveness of enum switch statements
    - exportloopref # checks for pointers to enclosing loop variables
    - forbidigo # Forbids identifiers
    - funlen # Tool for detection of long functions
    - gochecknoglobals # check that no global variables exist
    - gochecknoinits # Checks that no init functions are present in Go code
    - gocognit # Computes and checks the cognitive complexity of functions
    - goconst # Finds repeated strings that could be replaced by a constant
    - gocritic # Provides diagnostics that check for bugs, performance and style issues.
    - gocyclo # Computes and checks the cyclomatic complexity of functions
    - godot # Check if comments end in a period
    - goimports # In addition to fixing imports, goimports also formats your code in the same style as gofmt.
    - gomnd # An analyzer to detect magic numbers.
    - gomoddirectives # Manage the use of 'replace', 'retract', and 'excludes' directives in go.mod.
    - gomodguard # Allow and block list linter for direct Go module dependencies. This is different from depguard where there are different block types for example version constraints and module recommendations.
    - goprintffuncname # Checks that printf-like functions are named with f at the end
    - gosec # Inspects source code for security problems
    - lll # Reports long lines
    - makezero # Finds slice declarations with non-zero initial length
    - nakedret # Finds naked returns in functions greater than a specified function length
    - nestif # Reports deeply nested if statements
    - nilerr # Finds the code that returns nil even if it checks that the error is not nil.
    - nilnil # Checks that there is no simultaneous return of nil error and an invalid value.
    - noctx # noctx finds sending http request without context.Context
    - nolintlint # Reports ill-formed or insufficient nolint directives
    - nonamedreturns # Reports all named returns
    - nosprintfhostport # Checks for misuse of Sprintf to construct a host with port in a URL.
    - predeclared # find code that shadows one of Go's predeclared identifiers
    - promlinter # Check Prometheus metrics naming via promlint
    - revive # Fast, configurable, extensible, flexible, and beautiful linter for Go. Drop-in replacement of golint.
    - rowserrcheck # checks whether Err of rows is checked successfully
    - sqlclosecheck # Checks that sql.Rows and sql.Stmt are closed.
    - stylecheck # Stylecheck is a replacement for golint
    - tenv # tenv is analyzer that detects using os.Setenv instead of t.Setenv since Go1.17
    - testpackage # linter that makes you use a separate _test package
    - tparallel # tparallel detects inappropriate usage of t.Parallel() method in your Go test codes
    - unconvert # Remove unnecessary type conversions
    - unparam # Reports unused function parameters
    - wastedassign # wastedassign finds wasted assignment statements.
    - whitespace # Tool for detection of leading and trailing whitespace
    ## you may want to enable
    #- decorder # check declaration order and count of types, constants, variables and functions
    #- exhaustruct # Checks if all structure fields are initialized
    #- goheader # Checks is file header matches to pattern
    #- ireturn # Accept Interfaces, Return Concrete Types
    #- prealloc # [premature optimization, but can be used in some cases] Finds slice declarations that could potentially be preallocated
    #- varnamelen # [great idea, but too many false positives] checks that the length of a variable's name matches its scope
    #- wrapcheck # Checks that errors returned from external packages are wrapped
    ## disabled
    #- containedctx # containedctx is a linter that detects struct contained context.Context field
    #- depguard # [replaced by gomodguard] Go linter that checks if package imports are in a list of acceptable packages
    #- dogsled # Checks assignments with too many blank identifiers (e.g. x, _, _, _, := f())
    #- errchkjson # [don't see profit + I'm against of omitting errors like in the first example https://github.com/breml/errchkjson] Checks types passed to the json encoding functions. Reports unsupported types and optionally reports occasions, where the check for the returned error can be omitted.
    #- forcetypeassert # [replaced by errcheck] finds forced type assertions
    #- gci # Gci controls golang package import order and makes it always deterministic.
    #- godox # Tool for detection of FIXME, TODO and other comment keywords
    #- goerr113 # [too strict] Golang linter to check the errors handling expressions
    #- gofmt # [replaced by goimports] Gofmt checks whether code was gofmt-ed. By default this tool runs with -s option to check for code simplification
    #- gofumpt # [replaced by goimports, gofumports is not available yet] Gofumpt checks whether code was gofumpt-ed.
    #- grouper # An analyzer to analyze expression groups.
    #- ifshort # Checks that your code uses short syntax for if-statements whenever possible
    #- importas # Enforces consistent import aliases
    #- maintidx # maintidx measures the maintainability index of each function.
    #- misspell # [useless] Finds commonly misspelled English words in comments
    #- nlreturn # [too strict and mostly code is not more readable] nlreturn checks for a new line before return and branch statements to increase code clarity
    #- paralleltest # [too many false positives] paralleltest detects missing usage of t.Parallel() method in your Go test
    #- tagliatelle # Checks the struct tags.
    #- thelper # thelper detects golang test helpers without t.Helper() call and checks the consistency of test helpers
    #- wsl # [too strict and mostly code is not more readable] Whitespace Linter - Forces you to use empty lines!
    ## deprecated
    #- exhaustivestruct # [deprecated, replaced by exhaustruct] Checks if all struct's fields are initialized
    #- golint # [deprecated, replaced by revive] Golint differs from gofmt. Gofmt reformats Go source code, whereas golint prints out style mistakes
    #- interfacer # [deprecated] Linter that suggests narrower interface types
    #- maligned # [deprecated, replaced by govet fieldalignment] Tool to detect Go structs that would take less memory if their fields were sorted
    #- scopelint # [deprecated, replaced by exportloopref] Scopelint checks for unpinned variables in go programs


issues:
  # Maximum count of issues with the same text.
  # Set to 0 to disable.
  # Default: 3
  max-same-issues: 50

  exclude-rules:
    - source: "^//\\s*go:generate\\s"
      linters: [ lll ]
    - source: "(noinspection|TODO)"
      linters: [ godot ]
    - source: "//noinspection"
      linters: [ gocritic ]
    - source: "^\\s+if _, ok := err\\.\\([^.]+\\.InternalError\\); ok {"
      linters: [ errorlint ]
    - path: "_test\\.go"
      linters:
        - bodyclose
        - dupl
        - funlen
        - goconst
        - gosec
        - noctx
        - wrapcheck

Builds docker image for the bot

Example:

https://github.com/AICoE/meteor-operator/blob/main/Dockerfile
Perhaps https://github.com/google/ko can be useful

Release process for CI

Copilot-ops needs to be built as a binary so that we can package it into various formats (node-module, pypi) and distribute it for other projects to use.

Code Coverage & Unit Testing

Generate should expose a flag to ask for multiple completions returned

copilot-ops/pkg/openai/types.go

Lines 42 to 43 in 9fe09c4

    
           // N Is a number indicating how many completions should be generated for the given prompt. 
        
           N int32 `json:"n,omitempty"`

AI Output Processing

Given a response from the AI backend, we must process the generated response and map the new data back into the input files

Set up this repo to show case how to use `copilot-ops patch` automation with CI/CD

We suggested the following:

Use the examples folder in this repo (also thought of a hidden branch, but it's ok in main too for now) to host a set of devops configurations for a cluster - start with somthing simple and it will grow as we use it to demo.
Deploy a cluster that will pull any changes and deploy automatically with argocd.
Set up github actions to run copilot-ops patch on new issues.
Filter and parse only issues with a specific label that are intended to operate on the example cluster configuration.
Run the CLI to patch the cloned code.
Compose a PR from the patch.

CC @RobotSail @cooktheryan

Switch to Codex insert-and-update mode

The prompt for generation is currently built on a force-to-generate-updates mode, meaning that the entire document being sent between copilot-ops and OpenAI consists of the following structure:

# existing file 1
# existing file 2
# ...
--- 
# updated file 1
# updated file 2
# ...

OpenAI recently updated added a new feature to edit files in-place to obtain generated updates within them. We need to investigate this feature so we can restructure our design accordingly.

Generate arbitrary BASH commands

Would be nice to say copilot-ops generate cmd "search for files recursively and select those with the r+w attribute" and immediately obtain a pluggable command.

enable after fixing false positives

deadcode # Finds unused code
errcheck # Errcheck is a program for checking for unchecked errors in go programs. These unchecked errors can be critical bugs in some cases
gosimple # Linter for Go source code that specializes in simplifying a code
govet # Vet examines Go source code and reports suspicious constructs, such as Printf calls whose arguments do not align with the format string
ineffassign # Detects when assignments to existing variables are not used
staticcheck # Staticcheck is a go vet on steroids, applying a ton of static analysis checks
structcheck # Finds unused struct fields
typecheck # Like the front-end of a Go compiler, parses and type-checks Go code
unused # Checks Go code for unused constants, variables, functions and types
varcheck # Finds unused global variables and constants

disabled by default

asciicheck # Simple linter to check that your code does not contain non-ASCII identifiers
bidichk # Checks for dangerous unicode character sequences
bodyclose # checks whether HTTP response body is closed successfully
contextcheck # check the function whether use a non-inherited context
cyclop # checks function and package cyclomatic complexity
dupl # Tool for code clone detection
durationcheck # check for two durations multiplied together
errname # Checks that sentinel errors are prefixed with the Err and error types are suffixed with the Error.
errorlint # errorlint is a linter for that can be used to find code that will cause problems with the error wrapping scheme introduced in Go 1.13.
execinquery # execinquery is a linter about query string checker in Query function which reads your Go src files and warning it finds
exhaustive # check exhaustiveness of enum switch statements
exportloopref # checks for pointers to enclosing loop variables
forbidigo # Forbids identifiers
funlen # Tool for detection of long functions
gochecknoglobals # check that no global variables exist
gochecknoinits # Checks that no init functions are present in Go code
gocognit # Computes and checks the cognitive complexity of functions
goconst # Finds repeated strings that could be replaced by a constant
gocritic # Provides diagnostics that check for bugs, performance and style issues.
gocyclo # Computes and checks the cyclomatic complexity of functions
godot # Check if comments end in a period
goimports # In addition to fixing imports, goimports also formats your code in the same style as gofmt.
gomnd # An analyzer to detect magic numbers.
gomoddirectives # Manage the use of 'replace', 'retract', and 'excludes' directives in go.mod.
gomodguard # Allow and block list linter for direct Go module dependencies. This is different from depguard where there are different block types for example version constraints and module recommendations.
goprintffuncname # Checks that printf-like functions are named with f at the end
gosec # Inspects source code for security problems
lll # Reports long lines
makezero # Finds slice declarations with non-zero initial length
nakedret # Finds naked returns in functions greater than a specified function length
nestif # Reports deeply nested if statements
nilerr # Finds the code that returns nil even if it checks that the error is not nil.
nilnil # Checks that there is no simultaneous return of nil error and an invalid value.
noctx # noctx finds sending http request without context.Context
nolintlint # Reports ill-formed or insufficient nolint directives
nonamedreturns # Reports all named returns
nosprintfhostport # Checks for misuse of Sprintf to construct a host with port in a URL.
predeclared # find code that shadows one of Go's predeclared identifiers
promlinter # Check Prometheus metrics naming via promlint
revive # Fast, configurable, extensible, flexible, and beautiful linter for Go. Drop-in replacement of golint.
rowserrcheck # checks whether Err of rows is checked successfully
sqlclosecheck # Checks that sql.Rows and sql.Stmt are closed.
stylecheck # Stylecheck is a replacement for golint
tenv # tenv is analyzer that detects using os.Setenv instead of t.Setenv since Go1.17
testpackage # linter that makes you use a separate _test package
tparallel # tparallel detects inappropriate usage of t.Parallel() method in your Go test codes
unconvert # Remove unnecessary type conversions
unparam # Reports unused function parameters
wastedassign # wastedassign finds wasted assignment statements.
whitespace # Tool for detection of leading and trailing whitespace

you may want to enable

decorder # check declaration order and count of types, constants, variables and functions
exhaustruct # Checks if all structure fields are initialized
goheader # Checks is file header matches to pattern
ireturn # Accept Interfaces, Return Concrete Types
prealloc # [premature optimization, but can be used in some cases] Finds slice declarations that could potentially be preallocated
varnamelen # [great idea, but too many false positives] checks that the length of a variable's name matches its scope
wrapcheck # Checks that errors returned from external packages are wrapped

disabled

containedctx # containedctx is a linter that detects struct contained context.Context field
depguard # [replaced by gomodguard] Go linter that checks if package imports are in a list of acceptable packages
dogsled # Checks assignments with too many blank identifiers (e.g. x, _, _, _, := f())
errchkjson # [don't see profit + I'm against of omitting errors like in the first example https://github.com/breml/errchkjson] Checks types passed to the json encoding functions. Reports unsupported types and optionally reports occasions, where the check for the returned error can be omitted.
forcetypeassert # [replaced by errcheck] finds forced type assertions
gci # Gci controls golang package import order and makes it always deterministic.

copilot-ops/.golangci.yaml

Line 150 in b2e1212

    
           exported-fields: false # default false # TODO: enable after fixing false positives

## Golden config for golangci-lint v1.46.2
#
# This is the best config for golangci-lint based on my experience and opinion.
# It is very strict, but not extremely strict.
# Feel free to adopt and change it for your needs.

run:
  # Timeout for analysis, e.g. 30s, 5m.
  # Default: 1m
  timeout: 30m

  # Define the Go version limit.
  # Mainly related to generics support in go1.18.
  # Default: use Go version from the go.mod file, fallback on the env var `GOVERSION`, fallback on 1.17
  go: "1.17" # TODO: change to 1.18 when most linters support it, see https://github.com/golangci/golangci-lint/issues/2649


# This file contains only configs which differ from defaults.
# All possible options can be found here https://github.com/golangci/golangci-lint/blob/master/.golangci.reference.yml
linters-settings:
  cyclop:
    # The maximal code complexity to report.
    # Default: 10
    max-complexity: 30
    # The maximal average package complexity.
    # If it's higher than 0.0 (float) the check is enabled
    # Default: 0.0
    package-average: 10.0

  errcheck:
    # Report about not checking of errors in type assertions: `a := b.(MyStruct)`.
    # Such cases aren't reported by default.
    # Default: false
    check-type-assertions: true

  funlen:
    # Checks the number of lines in a function.
    # If lower than 0, disable the check.
    # Default: 60
    lines: 100
    # Checks the number of statements in a function.
    # If lower than 0, disable the check.
    # Default: 40
    statements: 50

  gocognit:
    # Minimal code complexity to report
    # Default: 30 (but we recommend 10-20)
    min-complexity: 20

  gocritic:
    # Settings passed to gocritic.
    # The settings key is the name of a supported gocritic checker.
    # The list of supported checkers can be find in https://go-critic.github.io/overview.
    settings:
      captLocal:
        # Whether to restrict checker to params only.
        # Default: true
        paramsOnly: false
      underef:
        # Whether to skip (*x).method() calls where x is a pointer receiver.
        # Default: true
        skipRecvDeref: false

  gomnd:
    # List of function patterns to exclude from analysis.
    # Values always ignored: `time.Date`
    # Default: []
    ignored-functions:
      - os.Chmod
      - os.Mkdir
      - os.MkdirAll
      - os.OpenFile
      - os.WriteFile
      - prometheus.ExponentialBuckets
      - prometheus.ExponentialBucketsRange
      - prometheus.LinearBuckets
      - strconv.FormatFloat
      - strconv.FormatInt
      - strconv.FormatUint
      - strconv.ParseFloat
      - strconv.ParseInt
      - strconv.ParseUint

  gomodguard:
    blocked:
      # List of blocked modules.
      # Default: []
      modules:
        - github.com/golang/protobuf:
            recommendations:
              - google.golang.org/protobuf
            reason: "see https://developers.google.com/protocol-buffers/docs/reference/go/faq#modules"
        - github.com/satori/go.uuid:
            recommendations:
              - github.com/google/uuid
            reason: "satori's package is not maintained"
        - github.com/gofrs/uuid:
            recommendations:
              - github.com/google/uuid
            reason: "see recommendation from dev-infra team: https://confluence.gtforge.com/x/gQI6Aw"

  govet:
    # Enable all analyzers.
    # Default: false
    enable-all: true
    # Disable analyzers by name.
    # Run `go tool vet help` to see all analyzers.
    # Default: []
    disable:
      - fieldalignment # too strict
    # Settings per analyzer.
    settings:
      shadow:
        # Whether to be strict about shadowing; can be noisy.
        # Default: false
        strict: true

  nakedret:
    # Make an issue if func has more lines of code than this setting, and it has naked returns.
    # Default: 30
    max-func-lines: 0

  nolintlint:
    # Exclude following linters from requiring an explanation.
    # Default: []
    allow-no-explanation: [ funlen, gocognit, lll ]
    # Enable to require an explanation of nonzero length after each nolint directive.
    # Default: false
    require-explanation: true
    # Enable to require nolint directives to mention the specific linter being suppressed.
    # Default: false
    require-specific: true

  rowserrcheck:
    # database/sql is always checked
    # Default: []
    packages:
      - github.com/jmoiron/sqlx

  tenv:
    # The option `all` will run against whole test files (`_test.go`) regardless of method/function signatures.
    # Otherwise, only methods that take `*testing.T`, `*testing.B`, and `testing.TB` as arguments are checked.
    # Default: false
    all: true

  varcheck:
    # Check usage of exported fields and variables.
    # Default: false
    exported-fields: false # default false # TODO: enable after fixing false positives


linters:
  disable-all: true
  enable:
    ## enabled by default
    - deadcode # Finds unused code
    - errcheck # Errcheck is a program for checking for unchecked errors in go programs. These unchecked errors can be critical bugs in some cases
    - gosimple # Linter for Go source code that specializes in simplifying a code
    - govet # Vet examines Go source code and reports suspicious constructs, such as Printf calls whose arguments do not align with the format string
    - ineffassign # Detects when assignments to existing variables are not used
    - staticcheck # Staticcheck is a go vet on steroids, applying a ton of static analysis checks
    - structcheck # Finds unused struct fields
    - typecheck # Like the front-end of a Go compiler, parses and type-checks Go code
    - unused # Checks Go code for unused constants, variables, functions and types
    - varcheck # Finds unused global variables and constants
    ## disabled by default
    - asciicheck # Simple linter to check that your code does not contain non-ASCII identifiers
    - bidichk # Checks for dangerous unicode character sequences
    - bodyclose # checks whether HTTP response body is closed successfully
    - contextcheck # check the function whether use a non-inherited context
    - cyclop # checks function and package cyclomatic complexity
    - dupl # Tool for code clone detection
    - durationcheck # check for two durations multiplied together
    - errname # Checks that sentinel errors are prefixed with the Err and error types are suffixed with the Error.
    - errorlint # errorlint is a linter for that can be used to find code that will cause problems with the error wrapping scheme introduced in Go 1.13.
    - execinquery # execinquery is a linter about query string checker in Query function which reads your Go src files and warning it finds
    - exhaustive # check exhaustiveness of enum switch statements
    - exportloopref # checks for pointers to enclosing loop variables
    - forbidigo # Forbids identifiers
    - funlen # Tool for detection of long functions
    - gochecknoglobals # check that no global variables exist
    - gochecknoinits # Checks that no init functions are present in Go code
    - gocognit # Computes and checks the cognitive complexity of functions
    - goconst # Finds repeated strings that could be replaced by a constant
    - gocritic # Provides diagnostics that check for bugs, performance and style issues.
    - gocyclo # Computes and checks the cyclomatic complexity of functions
    - godot # Check if comments end in a period
    - goimports # In addition to fixing imports, goimports also formats your code in the same style as gofmt.
    - gomnd # An analyzer to detect magic numbers.
    - gomoddirectives # Manage the use of 'replace', 'retract', and 'excludes' directives in go.mod.
    - gomodguard # Allow and block list linter for direct Go module dependencies. This is different from depguard where there are different block types for example version constraints and module recommendations.
    - goprintffuncname # Checks that printf-like functions are named with f at the end
    - gosec # Inspects source code for security problems
    - lll # Reports long lines
    - makezero # Finds slice declarations with non-zero initial length
    - nakedret # Finds naked returns in functions greater than a specified function length
    - nestif # Reports deeply nested if statements
    - nilerr # Finds the code that returns nil even if it checks that the error is not nil.
    - nilnil # Checks that there is no simultaneous return of nil error and an invalid value.
    - noctx # noctx finds sending http request without context.Context
    - nolintlint # Reports ill-formed or insufficient nolint directives
    - nonamedreturns # Reports all named returns
    - nosprintfhostport # Checks for misuse of Sprintf to construct a host with port in a URL.
    - predeclared # find code that shadows one of Go's predeclared identifiers
    - promlinter # Check Prometheus metrics naming via promlint
    - revive # Fast, configurable, extensible, flexible, and beautiful linter for Go. Drop-in replacement of golint.
    - rowserrcheck # checks whether Err of rows is checked successfully
    - sqlclosecheck # Checks that sql.Rows and sql.Stmt are closed.
    - stylecheck # Stylecheck is a replacement for golint
    - tenv # tenv is analyzer that detects using os.Setenv instead of t.Setenv since Go1.17
    - testpackage # linter that makes you use a separate _test package
    - tparallel # tparallel detects inappropriate usage of t.Parallel() method in your Go test codes
    - unconvert # Remove unnecessary type conversions
    - unparam # Reports unused function parameters
    - wastedassign # wastedassign finds wasted assignment statements.
    - whitespace # Tool for detection of leading and trailing whitespace
    ## you may want to enable
    #- decorder # check declaration order and count of types, constants, variables and functions
    #- exhaustruct # Checks if all structure fields are initialized
    #- goheader # Checks is file header matches to pattern
    #- ireturn # Accept Interfaces, Return Concrete Types
    #- prealloc # [premature optimization, but can be used in some cases] Finds slice declarations that could potentially be preallocated
    #- varnamelen # [great idea, but too many false positives] checks that the length of a variable's name matches its scope
    #- wrapcheck # Checks that errors returned from external packages are wrapped
    ## disabled
    #- containedctx # containedctx is a linter that detects struct contained context.Context field
    #- depguard # [replaced by gomodguard] Go linter that checks if package imports are in a list of acceptable packages
    #- dogsled # Checks assignments with too many blank identifiers (e.g. x, _, _, _, := f())
    #- errchkjson # [don't see profit + I'm against of omitting errors like in the first example https://github.com/breml/errchkjson] Checks types passed to the json encoding functions. Reports unsupported types and optionally reports occasions, where the check for the returned error can be omitted.
    #- forcetypeassert # [replaced by errcheck] finds forced type assertions
    #- gci # Gci controls golang package import order and makes it always deterministic.
    #- godox # Tool for detection of FIXME, TODO and other comment keywords
    #- goerr113 # [too strict] Golang linter to check the errors handling expressions
    #- gofmt # [replaced by goimports] Gofmt checks whether code was gofmt-ed. By default this tool runs with -s option to check for code simplification
    #- gofumpt # [replaced by goimports, gofumports is not available yet] Gofumpt checks whether code was gofumpt-ed.
    #- grouper # An analyzer to analyze expression groups.
    #- ifshort # Checks that your code uses short syntax for if-statements whenever possible
    #- importas # Enforces consistent import aliases
    #- maintidx # maintidx measures the maintainability index of each function.
    #- misspell # [useless] Finds commonly misspelled English words in comments
    #- nlreturn # [too strict and mostly code is not more readable] nlreturn checks for a new line before return and branch statements to increase code clarity
    #- paralleltest # [too many false positives] paralleltest detects missing usage of t.Parallel() method in your Go test
    #- tagliatelle # Checks the struct tags.
    #- thelper # thelper detects golang test helpers without t.Helper() call and checks the consistency of test helpers
    #- wsl # [too strict and mostly code is not more readable] Whitespace Linter - Forces you to use empty lines!
    ## deprecated
    #- exhaustivestruct # [deprecated, replaced by exhaustruct] Checks if all struct's fields are initialized
    #- golint # [deprecated, replaced by revive] Golint differs from gofmt. Gofmt reformats Go source code, whereas golint prints out style mistakes
    #- interfacer # [deprecated] Linter that suggests narrower interface types
    #- maligned # [deprecated, replaced by govet fieldalignment] Tool to detect Go structs that would take less memory if their fields were sorted
    #- scopelint # [deprecated, replaced by exportloopref] Scopelint checks for unpinned variables in go programs


issues:
  # Maximum count of issues with the same text.
  # Set to 0 to disable.
  # Default: 3
  max-same-issues: 50

  exclude-rules:
    - source: "^//\\s*go:generate\\s"
      linters: [ lll ]
    - source: "(noinspection|TODO)"
      linters: [ godot ]
    - source: "//noinspection"
      linters: [ gocritic ]
    - source: "^\\s+if _, ok := err\\.\\([^.]+\\.InternalError\\); ok {"
      linters: [ errorlint ]
    - path: "_test\\.go"
      linters:
        - bodyclose
        - dupl
        - funlen
        - goconst
        - gosec
        - noctx
        - wrapcheck

Hosting copilot-ops as a service, hiding AI engine subscription internally

We would like copilot-ops to be offered as a managed service, something that can be productized in the future.
This includes user authentication and management, providing the service to OpenShift, RHEL users, along with as a GitHub bot that has its own pricing model.

Include instructions to download the binary

README.md should also include instructions on how the binary can be downloaded via the copilot-ops releases

maybe enable after https://github.com/sivchari/nosnakecase/issues/14

paralleltest # [too many false positives] paralleltest detects missing usage of t.Parallel() method in your Go test
tagliatelle # Checks the struct tags.
thelper # thelper detects golang test helpers without t.Helper() call and checks the consistency of test helpers

copilot-ops/.golangci.yaml

Line 239 in 4f0ad96

    
           #- nosnakecase # Detects snake case of variable naming and function name. # TODO: maybe enable after https://github.com/sivchari/nosnakecase/issues/14

    #- maintidx # maintidx measures the maintainability index of each function.
    #- misspell # [useless] Finds commonly misspelled English words in comments
    #- nlreturn # [too strict and mostly code is not more readable] nlreturn checks for a new line before return and branch statements to increase code clarity
    #- nosnakecase # Detects snake case of variable naming and function name. # TODO: maybe enable after https://github.com/sivchari/nosnakecase/issues/14
    #- paralleltest # [too many false positives] paralleltest detects missing usage of t.Parallel() method in your Go test
    #- tagliatelle # Checks the struct tags.
    #- thelper # thelper detects golang test helpers without t.Helper() call and checks the consistency of test helpers

Define I/O Process

Given input data, e.g. prompt="generate 3 node deployments to host a gameserver", there must be output data written to a predictable directory / output stream.

Test the loading/writing mechanisms

Fine tuning the model using our collection of devops request/responses

We want to collect fine tuning examples for our devops use cases in the repo.
Then we want to use the copilot-ops cli to "push" the fine tuning examples to the openai endpoint.
https://beta.openai.com/docs/guides/fine-tuning
But we will need to wait for Codex fine tuning to be available.

OpenAI Codex tokens limit - identify and help the user workaround the limit by reducing the file set

With the OpenAI Codex model, we are only able to use 4096 tokens in total (prompt + generation), so we need to either develop a scheme to chain multiple completions together (maybe some sort of stitching mechanism), or error out when the #tokens(input files) > (MAX_TOKENS - #tokens(prompt)) / 2

Summer 2022 Internship Roadmap

Goals

Copilot-ops

Integrate copilot-ops into the existing workflows used by Operate-First to allow developers to quickly iterate on their work with the help of AI-assisted tooling. Learn about integrating code into the OperateFirst workflow process as well as leveraging GitHub automation tools. Evaluate the process of user interaction and make adjustments where necessary in order to improve the overall flow. Make note of incorrect generations from OpenAI and adjust the prompt or train the model in order to increase accuracy (quantified by number of iterations taken to merge a PR).

FetchIt

Become familiar with FetchIt. Understand the difference between running an application in Kubernetes and directly on a system, and the reasons why you’d choose one over the other. Learn how to manage applications as systemd services.

Tasks

Integrate copilot-ops with OperateFirst

Integrate copilot-ops with OperateFirst tekton
Present copilot-ops with operate first community

Copilot-ops CLI

Control the number of completions made
Document installation process
Output control with JSON/Text/YAML

Copilot-ops as a Service (CaaS)

Integrate with OperateFirst workflows
- Tekton pipelines
- Probot

Improve copilot-ops experience

Engineer better prompts to improve OpenAI accuracy
Document pain points as you use copilot-ops

Copilot-ops test modules

Create Golang tests for copilot-ops with Ginkgo
Mock OpenAI server for testing outbound functionality

WASM Starter Guide

Learn the basics of WebAssembly
Create a guide on using AssemblyScript in order to author WASM
Develop a simple project using AssemblyScript (ideas)
- Fermyon
- Hello world Smart contract
- Game

Objectives to Meet Alongside Tasks

FetchIt

Attend FetchIt meetings to better understand the development process
Work alongside engineers to fill in gaps
Deploy WASM/WASI with FetchIt

Gain experience using various technologies

Community Involvement/Building

Find at least 1 project to become involved with
Ideas:
- WASM/WASI
- KCP

Stretch Goals

Copilot-ops CLI

Improve code readability
Rework OpenAI client to specify model in body
Prompt engineering
Lunch with Daniel Walsh

AI Consistency Testing

Testing that an AI provides the correct response on a given prompt is difficult.

We should develop some system of unit-testing which applies to a stochastic system where the output is never a guarantee.
One idea could be setting the temperature parameter to 0 and seeing how often a given response yields certain properties.

Since we are also paying per request, this also becomes a costly endeavor.

Define a Concrete File Referencing Scheme

We need a format for users to be able to provide the files they would like to pull in, including provide an alias so that they can be referenced in relation to the issue, and the path from where they could be located by copilot-ops.

The scheme should pick characters that are distinguishable by copilot-ops from anything else that we could appear in YAML, Bash, or other commonly inlined scripts that are found in cloud-native projects.

Create a constant enumeration of engines so that we can include

their character limits.

OrganizationID is the ID of the organization used to authenticate with

the OpenAI API.

NTokens Is an integer representing the maximum number of tokens to

generate in the completion.

NCompletions is an integer representing the maximum number of completions

to generate from a prompt.

copilot-ops/pkg/openai/types.go

Line 129 in b2e1212

// TODO: Create a constant enumeration of engines so that we can include

	Object string `json:"object"`
}

// Client Defines the client for interacting with the OpenAI API.
type Client struct {
	// Client is the HTTP client used to perform requests to the OpenAI API.
	Client *http.Client
	// APIUrl Defines the endpoint that the client will use to interact with
	// the OpenAI API.
	APIUrl string
	// Engine represents the engine used when getting completions from OpenAI.
	// TODO: Create a constant enumeration of engines so that we can include
	// their character limits.
	Engine string
	// AuthToken is the token used to authenticate with the OpenAI API.
	AuthToken string
	// OrganizationID is the ID of the organization used to authenticate with
	// the OpenAI API.
	OrganizationID string
	// NTokens Is an integer representing the maximum number of tokens to
	// generate in the completion.
	NTokens int32
	// NCompletions is an integer representing the maximum number of completions
	// to generate from a prompt.
	NCompletions int32
}

	// N Is a number indicating how many completions should be generated for the given prompt.
	N int32 `json:"n,omitempty"`

redhat-et / copilot-ops Goto Github PK

copilot-ops's People

Contributors

Stargazers

Watchers

Forkers

copilot-ops's Issues

Parameters used in completion:

deprecated

disabled by default

you may want to enable

disabled

Goals

Copilot-ops

FetchIt

Tasks

Objectives to Meet Alongside Tasks

FetchIt

Gain experience using various technologies

Community Involvement/Building

Stretch Goals

Copilot-ops CLI

Recommend Projects

Recommend Topics

Recommend Org