enterprise-contract / ec-cli

Enterprise Contract command line interface

ec a command line client for evaluating the Enterprise Contract

The ec tool is used to evaluate Enterprise Contract policies for Software Supply Chain. Various sub-commands can be used to assert facts about an artifact such as:

• Validating container image signature
• Validating container image provenance
• Evaluating Enterprise Contract policies over the container image provenance
• Fetching artifact authorization

Consult the documentation for available sub-commands, descriptions and examples of use.

Building

Run make build from the root directory and use the dist/ec executable, or run make dist to build for all supported architectures.

Testing

Run make test to run the unit tests, and make acceptance to run the acceptance tests.

Linting

Run make lint to check for linting issues, and make lint-fix to fix linting issues (formatting, import order, ...).

Demo

Run hack/demo.sh to evaluate the policy against images that have been built ahead of time.

To regenerate those images, say in case of change in the attestation data, run hack/rebuild.sh.

Troubleshooting

The --debug parameter enables debug logging. Setting EC_DEBUG environment variable can be set to prevent deletion of temporary ec-work-* directories so that the attestations, policy and data files can be examined.

When running acceptance tests you may experience issues with starting enough Docker containers to successfullyl complete testing. These issues may appear as repeated failures, such as seen below, and a failed acceptance test run:

time="2024-03-08T09:10:50-05:00" level=warning msg="Failed, retrying in 1s ... (3/3). Error: trying to reuse blob sha256:b5976a979c30628edfeee0a1f1797362b0c84cf6cb4760776aa64ec8e3e4c2b3 at destination: pinging container registry localhost:37837: Get \"http://localhost:37837/v2/\": read tcp 127.0.0.1:34090->127.0.0.1:37837: read: connection reset by peer"

This issue may be resolved by increasing the total number of fs.inotify.max_user_watches by executing the following on: Red Hat / Fedora systems (other systems may need modifications to this)

$ echo fs.inotify.max_user_watches=524288 | sudo tee -a /etc/sysctl.conf && sudo sysctl -p

ec-cli's Issues

[BUG] PURL_functions acceptance test asserts over log files with zoned timestamp

Describe the Bug

The snapshot differs based on the time zone of the computer the acceptance tests are running on.

Steps to Reproduce

1. cd acceptance
2. go test -test.run 'TestFeatures/PURL_functions

Expected Behavior

A description of what you expected to happen.

Actual Behavior

--- FAIL: TestFeatures/PURL_functions (6.75s)
    suite.go:527: 1 error occurred:
        	* 
        - Snapshot - 0
        + Received + 1
        
        - time="${TIMESTAMP}" level=error msg="Parsing PURL \"this-is-not-a-valid-purl\" failed: purl scheme is not \"pkg\": \"\""
        + time="${TIMESTAMP}+01:00" level=error msg="Parsing PURL \"this-is-not-a-valid-purl\" failed: purl scheme is not \"pkg\": \"\""
          Error: success criteria not met
          ↵
        
        at ../../../../../../../../../../../../../tmp/tmp.GHLYxwb0uo/features/__snapshots__/validate_image.snap:3342

Possible Solution

Looks like custom formatter should be implemented.

Cache is not thread safe

The EC cache is used to avoid fetching the same image layers from the registry more than once. However, this is not thread safe.

A user reported random redhat_manifests.redhat_manifests_missing violations that should've not occurred. This can only be reproduced when validating multiple images that share layers.

Disabling the cache avoids the issue altogether.

Let's either make the cache safe, or pull the plug on it.

Increase unit test coverage

For the most part, the important code paths in this repo are well covered. However, there are a few statements in a handful of files that could benefit from additional testing.

NOTE: The goal here is not to achieve 100% test coverage since this can be quite daunting in Go due how errors are handled.

Acceptance Criteria

• Increase unit test coverage of the non-error error handling statements in the following files:
  • https://app.codecov.io/gh/enterprise-contract/ec-cli/blob/main/cmd%2Finspect%2Finspect_policy.go
  • https://app.codecov.io/gh/enterprise-contract/ec-cli/blob/main/internal%2Fattestation%2Fattestation.go
  • https://app.codecov.io/gh/enterprise-contract/ec-cli/blob/main/internal%2Fevaluation_target%2Fapplication_snapshot_image%2Fapplication_snapshot_image.go
  • https://app.codecov.io/gh/enterprise-contract/ec-cli/blob/main/internal%2Foutput%2Foutput.go
  • https://app.codecov.io/gh/enterprise-contract/ec-cli/blob/main/internal%2Fapplicationsnapshot%2Freport.go
  • https://app.codecov.io/gh/enterprise-contract/ec-cli/blob/main/internal%2Ftracker%2Ftracker.go
  • https://app.codecov.io/gh/enterprise-contract/ec-cli/blob/main/internal%2Fopa%2Frule%2Frule.go

ec validate input errs when policy sets publicKey

With a policy.yaml that specifies a publicKey, e.g.:

publicKey: k8s://openshift-pipelines/public-key
sources:
- data:
  - ...
  policy:
  - ...

The ec validate input command always fails:

Error: 1 error occurred:
	* error validating file input.json: no check options or sig verifier configured

With the --debug flag:

DEBU[0000] helpers.go:80 readPolicyConfigurationFile Loaded policy.yaml as policyConfiguration    
DEBU[0000] policy.go:394 parseEffectiveTime Chosen to use effective time of `now`, using current time 2024-04-15T18:44:38Z 
DEBU[0000] policy.go:309 loadPolicy Read EnterpriseContractPolicy as YAML        
DEBU[0000] policy.go:314 loadPolicy Unable to parse EnterpriseContractPolicy from "configuration:\n  # exclude:\n  # - tasks.required_tasks_found:prefetch-dependencies\n  # - hermetic_build_task.build_task_hermetic\n  # - test.no_test_warnings\n  # - redhat_manifests.redhat_manifests_missing\n  # - labels.required_labels\n  # - rpm_ostree_task.builder_image_param\n  # - sbom_cyclonedx.found\n  # - test.no_failed_tests\n  # include:\n  # - '@redhat'\n  include:\n  - attestation_task_bundle\ndescription: Rules for shipping content to registry.redhat.io\npublicKey: k8s://openshift-pipelines/public-key\nsources:\n- data:\n  - github.com/release-engineering/rhtap-ec-policy//data\n  - oci::quay.io/redhat-appstudio-tekton-catalog/data-acceptable-bundles:latest\n  name: Release Policies\n  policy:\n  # - oci::quay.io/enterprise-contract/ec-release-policy:latest\n  - /home/lucarval/src/enterprise-contract/ec-policies/policy/lib\n  - /home/lucarval/src/enterprise-contract/ec-policies/policy/release\n  # ruleData:\n  #   allowed_registry_prefixes:\n  #   - registry.access.redhat.com/\n  #   - registry.redhat.io/\n  #   - quay.io/rh-osbs/ubi9\n  #   - localhost/rhtap-final-image\n  #   allowed_rpm_ostree_builder_image_prefixes:\n  #   - quay.io/centos-bootc/bootc-image-builder\n" 
DEBU[0000] policy.go:315 loadPolicy Attempting to parse as EnterpriseContractPolicySpec 
DEBU[0000] validate.go:37 ValidateInput Current input filePath: "input.json"         
DEBU[0000] validate.go:67 detectInput unable to detect input as JSON               
DEBU[0000] validate.go:75 detectInput unable to detect input as YAML               
DEBU[0000] input.go:53 NewInput policySource: &source.PolicyUrl{Url:"/home/lucarval/src/enterprise-contract/ec-policies/policy/lib", Kind:"policy"} 
DEBU[0000] input.go:53 NewInput policySource: &source.PolicyUrl{Url:"/home/lucarval/src/enterprise-contract/ec-policies/policy/release", Kind:"policy"} 
DEBU[0000] input.go:53 NewInput policySource: &source.PolicyUrl{Url:"github.com/release-engineering/rhtap-ec-policy//data", Kind:"data"} 
DEBU[0000] input.go:53 NewInput policySource: &source.PolicyUrl{Url:"oci::quay.io/redhat-appstudio-tekton-catalog/data-acceptable-bundles:latest", Kind:"data"} 
DEBU[0000] conftest_evaluator.go:277 NewConftestEvaluatorWithNamespace Created work dir /tmp/ec-work-678054144      
DEBU[0000] policy.go:380 EffectiveTime Using effective time: 2024-04-15T18:44:38Z   
DEBU[0000] input.go:58 NewInput Failed to initialize the conftest evaluator! 
DEBU[0000] validate.go:46 ValidateInput Failed to create input!                      
Error: 1 error occurred:
	* error validating file input.json: no check options or sig verifier configured

Git bisect tells me this started with commit 117055d.

Remove publicKey from the policy bypasses the issue.

Create a PoC for moving builtin rules to rego

In the previous community meeting, we talked about exploring the option of making the builtin rules more similar to other policy rules.

This issue is about creating a proof-of-concept so we can evaluate the idea further.

Proposal (as a starting point; PoC can, and likely will, deviate):

• Re-implement the builtin rules in rego.

  • image.signature_check
  • image.accessible
  • attestation.signature_check
  • attestation.syntax_check

• Embed those rego rules in the ec-cli binary.

• Modify ec validate image to inject a new policy source group into the policy config that uses the embedded builtin rules.

Considerations:

• How would this impact the report if one of the builtin policy rules fails? How does it differ from the current behavior? Maybe wider use of depends_on is necessary.
• How could someone create a policy config that explicitly includes the builtin rules? What if someone does it? Can the auto-injection detect and skip injecting them? Could someone use this as mechanism to exclude them?

Acceptance Criteria

• A PoC is created the illustrates the ideas above. This should be working code but it does not have to be perfect nor handle different use cases. Hard code things if it makes it easier.
• Demo the PoC with EC maintainers and maybe a wider audience.
• Help decide next steps. (Abandoning the idea is also a perfectly acceptable next step). File issues accordingly.

Introduce ec.fetch_oci_blob rego function

Description

We want the ability to fetch blobs from an OCI registry from policy rules. Let's introduce a new custom built-in rego function to enable this, see example commit and the docs.

Considerations:

• Memoization. Using the function on the same OCI reference should not cause the SBOM to be fetched more than once from the OCI registry. The rego may provide some utilities for this.
• Error handling. If the golang implementation of the function returns an error, this causes EC to exit with fatal failure, which is bad. Consider adding a supporting "is_valid" function to ensure the reference is a valid OCI reference by digest, and return an empty string if a runtime error occurs, for example.
• Data limit. Use something like io.LimitReader to set some boundaries on how much data EC is fetching. (10 MB should be enough for SBOMs to start with.) Be sure to validate the fetched content has the expected digest.
• Authentication. The function should pick up whatever OCI authentication creds are available on the system. Similarly to how ec-cli does in the rest of the code base.

Acceptance Criteria

• A new rego function can be used to fetch a blob from an OCI registry.

Multiple outputs to stdout appear concatenated

If ec validate --output summary --output json is invoked the two JSON outputs are outputted on the same line.
We should detect that the output goes to stdout and add a newline between the two outputs to stdout.

Per source group configuration might not be included in the reports

Seems probable based on the failure of the e2e tests in EC update.

Use renovatebot to keep .tools-versions up to date

We added the .tools-versions file to this repo in order to better support usage of asdf. The go version listed is in the format MAJOR.MINOR.PATCH. Although updates to MAJOR and MINOR can be done manually, we should ensure the latest value for PATCH is used since it may contain important bug/security fixes.

Let's use renovatebot to automate this process, since it already supports asdf, and the ability to restrict updates to only patch updates.

Let's also be careful to ensure renovatebot doesn't interfere with dependabot already setup in this repo. This is likely possible by simply omitting the extends directive.

Empty policy rule selection does not cause failure

Consider the following policy config:

---
publicKey: |
  -----BEGIN PUBLIC KEY-----
  MFkwEwYHKoZIzj0CAQYIKoZIzj0DAQcDQgAEZP/0htjhVt2y0ohjgtIIgICOtQtA
  naYJRuLprwIv6FDhZ5yFjYUEtsmoNcW7rx2KM6FOXGsCX3BNc7qhHELT+g==
  -----END PUBLIC KEY-----
sources:
 - config:
    include:
    - spam
   data:
   - oci::quay.io/redhat-appstudio-tekton-catalog/data-acceptable-bundles:latest
   - github.com/release-engineering/rhtap-ec-policy//data
   policy:
   - oci::quay.io/enterprise-contract/ec-release-policy:latest

Notice how it includes the non-existing spam package. If we use this policy during validation, ec report success:

🐚 ec validate image --ignore-rekor --policy policy-dummy.yaml --image quay.io/redhat-appstudio/ec-golden-image@sha256:304040ca1911aa4d911bd7c6d6d07193c57dc49dbc43e63828b42ab204fb1b25 --output yaml --show-successes
components:
 - attestations:
  - predicateBuildType: tekton.dev/v1beta1/TaskRun
    predicateType: https://slsa.dev/provenance/v0.2
    signatures:
    - keyid: SHA256:IhiN7gY</ins>Z3uSSd7tmj6w5Zfhqafzdhm3DZjIvGc6iYY
      sig: MEUCIQDcgZIwEkLFqD7U9HrobgEC8Jo7wm<ins>xJ5AoyO3qg</ins>aj8QIgb9xDpjYGRMmpVk<ins>QATeVKlHonzBiu51HtT3J</ins>lQXPXc=
    type: https://in-toto.io/Statement/v0.1
  - predicateBuildType: tekton.dev/v1beta1/PipelineRun
    predicateType: https://slsa.dev/provenance/v0.2
    signatures:
    - keyid: SHA256:IhiN7gY<ins>Z3uSSd7tmj6w5Zfhqafzdhm3DZjIvGc6iYY
      sig: MEYCIQDKSihaAR/zAhJhR5GCqleDvfUUtvRw61vk0YeTBAnOSQIhAKa09B4yEfaSJronmWBFbu5cVPNxm17CMl/PElEz1POa
    type: https://in-toto.io/Statement/v0.1
  containerImage: quay.io/redhat-appstudio/ec-golden-image@sha256:304040ca1911aa4d911bd7c6d6d07193c57dc49dbc43e63828b42ab204fb1b25
  name: Unnamed
  signatures:
  - keyid: ""
    sig: MEUCIQD86lmOqCovYZDPKm0XxxsLgDQcFIFAv</ins>QZxrFSHmCvQAIgTd1I005ox8MfABqsAen6PZEyg2MCEQNBCx1NLS3V0JQ=
  source: {}
  success: true
  successes:
  - metadata:
      code: builtin.attestation.signature_check
    msg: Pass
  - metadata:
      code: builtin.attestation.syntax_check
    msg: Pass
  - metadata:
      code: builtin.image.signature_check
    msg: Pass
ec-version: v0.3.2727-5682f93
effective-time: "2024-04-23T13:28:51.736545442Z"
key: |
  -----BEGIN PUBLIC KEY-----
  MFkwEwYHKoZIzj0CAQYIKoZIzj0DAQcDQgAEZP/0htjhVt2y0ohjgtIIgICOtQtA
  naYJRuLprwIv6FDhZ5yFjYUEtsmoNcW7rx2KM6FOXGsCX3BNc7qhHELT<ins>g==
  -----END PUBLIC KEY-----
policy:
  publicKey: |
    -----BEGIN PUBLIC KEY-----
    MFkwEwYHKoZIzj0CAQYIKoZIzj0DAQcDQgAEZP/0htjhVt2y0ohjgtIIgICOtQtA
    naYJRuLprwIv6FDhZ5yFjYUEtsmoNcW7rx2KM6FOXGsCX3BNc7qhHELT</ins>g==
    -----END PUBLIC KEY-----
  sources:
  - config:
      include:
      - spam
    data:
    - oci::quay.io/redhat-appstudio-tekton-catalog/data-acceptable-bundles:latest
    - github.com/release-engineering/rhtap-ec-policy//data
    name: Default
    policy:
    - oci::quay.io/enterprise-contract/ec-release-policy:latest
success: true

EC should error out if no policy rules were actually selected and the list of source groups is not empty.

Enable grouped dependabot PRs

Dependabot is great. But we've been getting hammered with updates. Let's try grouping those updates.

Allow --policy to accept ECP resources in all cases

The EnterpriseContractPolicy is a custom kubernetes resource. This resource makes it convenient to store EC policy configurations in an kubernetes cluster. The .spec portion of such a resource is the policy configuration.

When using the EC CLI a reference to an ECP can be used, e.g. ec validate image ... --policy my-namespace/my-ecp.

However, in order to load a policy configuration from a git repo, local file, or inline, an ECP is not allowed. In
those cases, you must use just the policy configuration. (The .spec portion of an ECP).

This is confusing!

Let's change the --policy parameter to allow either an ECP definition, or a policy config (as it does today).

Introduce policy rule file fetch annotation

The EC CLI can now fetch files from the image being validated and feed that to the policy-input. That's been helpful in addressing some use cases.

In order to avoid pulling every image that EC validates, certain files are only fetched depending on certain image labels. See docs here.

However, this is not ideal. The files should only be fetched if there's actually a policy rule that needs them.

Let's invert things a little bit. Let's introduce a rego package annotation that states that the policy rules in a certain package want a certain file from the image. Then, let's teach the ec cli to look at those annotations in order to decide which files to fetch. If the file is not found, that's ok. The policy rules are responsible for verifying this.

NOTE: Kudos to Zoran for coming up with this idea.

Acceptance Criteria

• Files are only fetched from an image if there are policy rules that require such files.

Document how to debug antora extensions

This repo defines a couple of antora extensions that are used during the website build, [reference-antora-extension], rego-antora-extension, and tekton-task-antora-extension.

When making a change to these extensions, I don't know how to try out my changes in order to verify they are working properly.

Acceptance Criteria

• Document the process of executing antora extensions locally. Ideally, this includes executing them as part of a local website build.

Improve testing for `ec init policies`

The ec init policies command currently has relatively minimal testing. We should add more unit tests and, perhaps, an acceptance test to better test it.

Some ideas to get started:

• Spit out a _test.rego file
• Spit out a command to run tests
• Look at tests for the ec test command

A/C:

• ec init policies has a fleshed-out test suite.

Place the builtin Rego functions in separate files

We're adding new the builtin functions in internal/evaluator/rego.go. It would be nicer if we could split those into different files, e.g. purl.go, oci.go...

Clean up policy input

Policy input is quite messy at the moment. With the resolution of enterprise-contract/ec-policies#756, our current input format looks like this (credit to @simonbaird):

attestations: [
  {
     ...all the statement key/values
     extra: {
        signatures: [...]
     },
     statement: {
         ...all the statement key/values once more again
     }
     signatures: [...]
  }
]

This includes a lot of unnecessary duplication, and is generally pretty confusing. We want the input format to be a lot cleaner - something more like this:

attestations: [
  {
     statement: {...}
     signatures: [...]
  }
]

We should clean up extraneous policy input.

A/C: ec-cli policy input conforms to the much cleaner proposed format above.

Abandoned documentation removal

With the documentation generation in #1500 and #1493 the .adoc files are generated in docs/modules/ROOT/pages. A document created prior to it's abandonment, for example if a command or a Rego builtin is removed, will not be removed.
The document generation code should keep track of any abandoned components and remove the corresponding Asciidoc files.

Introduce command to initialize Sigstore root

When using a custom Sigstore deployment, the Sigstore root must be explicitly initialized. This is often done by using cosign initialize.

This means that ec on its own cannot support this use case. It relies on another utility, cosign. Users must have both installed. This is the approach taken by EC's verify-enterprise-contract Task.

Let's add a new command to ec, e.g. ec sigstore initialize, to perform this operation.

Acceptance Criteria

• The ec-cli image no longer includes the cosign binary.
• The verify-enterprise-contract Task can still initialize the Sigstore root.

Support installing ec-cli via brew on macOS

I would like to easily install and update ec-cli via homebrew on a mac.

$ brew install ec-cli
Warning: No available formula with the name "ec-cli". Did you mean osc-cli or oci-cli?
==> Searching for similarly named formulae and casks...
==> Formulae
osc-cli                                                                                         oci-cli

To install osc-cli, run:
  brew install osc-cli

Add a custom built-in rego function for fetching a GH workflow

The SLSA Provenance generated on GitHub does not include details about the workflow used to build a container image.

This makes it hard to create a policy rule that checks if a certain GitHub action was included in the process, e.g. did a code scanner run?

The SLSA Provenance does contain a reference to the workflow:

🐚 cosign verify-attestation quay.io/lucarval/festoji:latest --type slsaprovenance --certificate-github-workflow-repository lcarva/festoji --certificate-identity 'https://github.com/slsa-framework/slsa-github-generator/.github/workflows/generator_container_slsa3.yml@refs/tags/v1.7.0' --certificate-oidc-issuer 'https://token.actions.githubusercontent.com' \
  | jq '.payload | @base64d | fromjson | .predicate.invocation.configSource'

{
  "uri": "git+https://github.com/lcarva/festoji@refs/heads/master",
  "digest": {
    "sha1": "848edc452ccbc6d42ec56c2807eef2f49e754c5e"
  },
  "entryPoint": ".github/workflows/package.yaml"
}

Let's introduce a new custom built-in rego function to fetch the workflow, e.g. ec.fetch_slsa_config_source(attestation)

Change `ec validate input` to use positional arguments

The ec validate input should take positional arguments instead of named arguments for the list of files to validate, i.e. ec validate input spam.yaml instead of ec validate input --file spam.yaml

Why? Because of all the arguments that command accepts, the input files are likely the thing to have a variable length. Consider the case where you have a bunch of yaml files in a directory. Assuming a shell is being used, positional arguments would look like ec validate input *.yaml, and named parameter like ec validate input $(find . -name '*.yaml' -printf '--file=%p ') . The second version relies on the find utility which may not be available in some contexts.

See cobra's docs on positional arguments.

Acceptance Criteria

• Change ec validate input to take positional arguments. Each is treated as a file to be validated.
• For backwards compatibility, continue to accept the --file parameter, but be sure to mark it as deprecated. If both --file and positional arguments are provided, combine the values.

Increase coverage of Tekton Task parameters in acceptance tests

Today, 5 out of 11 parameters are not covered by acceptance tests. (The most impactful parameters, e.g. IMAGES and POLICY_CONFIGURATION are tested.)

Acceptance Criteria

• Identify the parameters not covered by acceptance tests.
• Add coverage as needed. (NOTE: Ideally, all of them are covered, but there is room for skipping some if it makes sense.)

Add tests for the antora extensions

Let's make sure we have some basic coverage for the code defined in the antora extensions in this repo.

Acceptance Criteria

• Write basic tests to verify the functionality of the antora extensions in this repo.
• These tests should run as part of the CI in this repo. (If this turns out to be too much of an effort, consider splitting this item into another issue.)

Find a good way to reduce the toil related to refreshing wiremock recordings of Sigstore/TUF used in acceptance tests

Background:

• Using production Sigstore gives 6 month expiration.
• Using stage Sigstore gives shorter expiration.
• Spinning up a local instance does work but it's resource intensive, (could run in GitHub though).
• Could we somehow produce a 100 year expiration for test data and never worry about this again?

See also:

• #1577 and #1622

[BUG] Panic in `ec validate input`: `concurrent map writes`

Happened (once) today on a pull request to build-definitions where ec is run to validate input.

ec-task-checks.log

Add a blurb or link on how to provide feedback in `--help` text

Folk using ec CLI might not be aware on how to best provide feedback, a blurb, link of the sort in the help text shown when --help is provided.

Combine generative, integration, and unit flags of coverage report

Today, the test coverage for this repo is 82.43%. This is a combined score of unit and acceptance tests.

It is possible to split up this value to better understand the amount of unit tests vs acceptance tests via flags. But, three out of four flags (generative, integration, unit) are really all unit tests. Given that there is likely overlap between these three flags, a combined score is not easily achieved.

To make that report more useful, let's combine the three flags into one.

Acceptance Criteria

• codecov reports only two flags, acceptance and unit

test sync2jira

test sync2jira

Create a more useful example rego file for `ec init policies`

The existing Rego included when running ec init policies is fairly basic. We should include something with a little more detail - particularly something that verifies attestations.

Some potential places to look for inspiration include:

• input.image
• input.attestations

A/C:

• Running ec init policies generates a Rego file that checks an image's attestations.

Today ec validate image task accepts policy configuration as CR or inline not the URL for policy

Usecase:

when we use verify-enterprise-contract the only way to provide policy is via CR or inline as per the doc

But if we use CR then there will be a prerequisite for CR availability on that cluster.

So along with CR and inline it would be nice if EC support providing URL for policy yaml
ex: https://gist.githubusercontent.com/savitaashture/574a85f799626d25a36c47c71fbda234/raw/4ef554003eeace9fc5987192a30e92ed36eb138e/policy.yaml

It will be more useful

Implement a type flag for `ec init policies`

As is, ec init policy emits a basic policy, geared towards ec validate image. That said, it might be possible that users will want to try writing their own policies for e.g. ec validate definition.

ec init policies should have an option to allow users to specify what kind of policy they'd like to generate - perhaps something like --type. It could have specific options, or it could take a string.

A/C:

• ec init policies has an option to control the policy type being created.

Restructure `cobra` command hierarchy to allow for more thorough unit testing

Currently, ec-cli's unit tests rely on creating a mock subcommand, not connected to any higher-level root commands, to run unit tests. This has caused spurious issues occasionally when writing unit tests.

A better approach, outlined here, seems to be structuring cobra's command hierarchy to allow the full set of commands to be constructed in unit tests. ec-cli cannot currently do this, however, since all commands are defined inline.

Abstracting command construction out into a separate function could improve modularity and make better unit testing possible.

A/C:

• ec-cli's Cobra commands can be constructed independently.

Document EC-provided custom rego functions

Description

Let's come up with a way that we can easily document custom rego functions introduced by EC, e.g. the one from #1070.

It should be clear to users how to use the function. (Expected parameters, expected return values, how it behaves, etc)

Acceptance Criteria

• The ec.fetch_oci_blob function is documented somewhere on enterprisecontract.dev.

Include `expires_on` in the acceptable bundles list

The acceptable bundles list looks like this today:

task-bundles:
  quay.io/redhat-appstudio-tekton-catalog/task-buildah:
    - digest: sha256:163009699fca1c5c043516d84986b44f8b6ef25418c97eed51d12518a94d552e
      effective_on: "2024-03-09T00:00:00Z"
      tag: "0.1"
    - digest: sha256:5db20956b51725f1b50f02d21413cd7d0c082e4f51364c54e51f78dfa75893a4
      effective_on: "2024-03-09T00:00:00Z"
      tag: "0.1"
    - digest: sha256:1ee1d570b3cf7feaf85a22c828076e798a57266ffcfea41c1aa0e9f85b0a885a
      effective_on: "2024-03-07T00:00:00Z"
      tag: "0.1"
    - digest: sha256:351af2c0e5eeb92a5d6d4083847c1559475b596cda7671f489756d5302a4c847
      effective_on: "2024-02-16T00:00:00Z"
      tag: "0.1"
    - digest: sha256:155ba11fda63c10ac3a824f4c2db385c22299551124db5d239b325b576249839
      effective_on: "2024-02-10T00:00:00Z"
      tag: "0.1"
    - digest: sha256:38f557374d73ce220237c507b60ce653430284c5b81a29ce840fcb48081855d6
      effective_on: "2024-02-04T00:00:00Z"
      tag: "0.1"

As you can see each item has an effective_on value. The data is sorted in descending order by the effective_on date. (Newer records are at the top.)

At a certain effective_on date/time, that particular digest is the oldest acceptable value. Anything newer is also ok to use. Anything older is treated as if it is not even in the list; those are expired items.

In other words, an item becomes expired when the effective_on date of a newer item is reached. The effective_on value of record N is the expires_on value of record N+1.

The most recent item does not have an expiration date because there is no newer item to be used. (This is true today, but may change in the future when we add support for deprecating a bundle repo altogether.)

This story is about enhancing ec track bundle to also include an expires_on date in the acceptable bundles items.

NOTE: See this commit for inpiration.

Acceptance Criteria

• Each item in the list of acceptable bundles includes an expires_on date. It also continue to include an effective_on value to make the transition easier.

Patch version is always set to 1

Seems that all released versions have the patched version set to 1 instead of the number of commits on the default branch.

Include full configuration examples rather than snippets

In https://enterprisecontract.dev/docs/ec-cli/main/configuration.html#_examples the examples are snippets of YAML, yet the example above is an inline JSON. That can be a bit confusing. We should have whole examples and in single format to make the configuration clearer.

Use workflow to refresh TUF recordings for acceptance tests

In #1577, we updated the TUF recordings used in the acceptance tests to use the staging deployment of Sigstore. This made it easier to manually refresh the data (required every 6 months). However, this means the image being tested is associated with my identity.

Let's create a GitHub Workflow that can execute the script to re-generate the test image and the TUF recordings, make it run periodically (monthly?), and submit a PR so all we have to do is merge it.

See this comment for additional context.

Acceptance Criteria

• A workflow exists to keep the expiring TUF data up to date.
• The identity used in the test image is not associated to an individual. Instead, it is the identity of the workflow.

Figure out how to run unit tests in ec-policies that use custom function

Description

The function introduced in #1070 means that a policy rule could write something like this:

ec.fetch_oci_bundle(ref) 

Plain OPA doesn't know anything about that function. If a policy rule in ec-policies wants to use that function, this error will occur in the tests:

rego_unsafe_var_error: var ec is unsafe 

If we try to mock that in the tests by using the with keyword:

rego_type_error: with keyword target must reference existing input, data, or a function

One option could be to wrap the opa command in ec-cli.

This also raises the question of how we can mock those functions. We don't want to actually hit a registry during the unit tests, for example.

Acceptance Criteria

• ec-provided custom rego functions can be used in ec-policies without
  breaking tests.

Fail more gracefully when a rego rule references a custom rego function that isn't supported

Now that we're introducing custom rego functions, we should consider how to avoid hard failures when the function isn't supported in the available verison of ec-cli.

Ideas:

• Interrogate ec to provide a list of available custom functions and handle the case where the function isn't there.

Change the data model for Acceptable Bundles

The acceptable bundles data is modeled as such:

<type>-bundles:
  <repo>:
    - digest: <digest>
      effective_on: <timestamp>
      tag: <tag

For example:

task-bundles:
  quay.io/redhat-appstudio-tekton-catalog/task-buildah:
    - digest: sha256:c37e542031de193398cd54463af146e42539b5f5c9082df56d65354494566db0
      effective_on: "2023-11-06T00:00:00Z"
      tag: "0.1"
    - digest: sha256:97f21661e237735af04b37feeeaedd328424bfa0ebd4cd0f79ac39cde17137f6
      effective_on: "2023-10-25T00:00:00Z"
      tag: "0.2"

Then there's rego that interprets this data to do the right thing.

Sometimes we use the value of tag to determine what is the latest version for that particular tag. In such cases, there are really virtual lists, one for each tag value.

The reality is that there is always a tag value, and we could make that a requirement. This issue is about changing how the data is represented, for example:

task-bundles:
  quay.io/redhat-appstudio-tekton-catalog/task-buildah:
    "0.1":
      - digest: sha256:c37e542031de193398cd54463af146e42539b5f5c9082df56d65354494566db0
        effective_on: "2023-11-06T00:00:00Z"
    "0.2":
      - digest: sha256:97f21661e237735af04b37feeeaedd328424bfa0ebd4cd0f79ac39cde17137f6
        effective_on: "2023-10-25T00:00:00Z"

This would impact ec track bundle (may need to add code to convert things on the fly to allow for an easier transition), and ec-policies (may need to support both formats for a bit).

Reduce memory usage dynamically

We have a hypothesis that the memory usage is proportional to the number of processed images concurrently. We could inspect the memory constraints (e.g. cgroup memory limit) and reduce the concurrency, e.g. process one image at a time.
(Followup from #1168)

Add log field to all custom Rego builtin functions

Similar to how it is done in #1107, something like:

log := log.WithField("rfunc", functionName)

And, if we're not already logging within those functions we should add some to ease troubleshooting.

--freshen takes too long

Running the following command:

ec --debug track bundle \
  --input "quay.io/redhat-appstudio-tekton-catalog/data-acceptable-bundles:latest" \
  --freshen

Takes about 4 1/2 minutes.

It looks like part of the problem is that freshen selects almost 300 image refs to query. It is a large data source so this is somewhat expected.

Let's consider ways that we can speed this up a bit. My computer had low cpu and network usage during the period which suggests we should get some improvements if we check those refs in parallel.

Also, some of the tags represent a git commit. Maybe we can ignore those? That should reduce the amount of queries significantly.

Remove default value of the policy flag

By default, the --policy flag has the value of enterprise-contract-service/default. This works in very particular circumstances (you're logged in to an RHTAP member cluster which only RHTAP devs can do).

As we broaden the usage of EC, this default value gets in the way. For example, if you want to run just the basic signature checks, you must use --policy ''. That's cumbersome.

Let's set the default value to ''.

I don't think anything relies on that default value, but it's worth double checking.

Handle effective* in include/exclude

With enterprise-contract/enterprise-contract-controller#191, the include/exclude list may contain objects instead of just strings. See that issue for more details.

This is about teaching the CLI to handle those correctly.

Consider the following example:

sources:
  - config:
      exclude:
        - step_image_registries
        - name: tasks
          effectiveUntil: "2024-01-01T00:00:00Z"
        - name: tests
          effectiveOn: "2023-10-01T00:00:00Z"
      include:
        - '@redhat'
    data:
      ...
    policy:
      ...

step_image_registries is always excluded. tasks is only excluded until January 1st, 2024. tests is excluded starting on January 1st, 2023.

Eventually, we may want to emit a warning in certain cases. For example, if a certain violation that is being excluded today will be included in a few days (either via exclude entry with effectiveUntil, or include entry with effectiveOn). Let's keep that out of scope for this issue.

Create a custom built-in rego function to parse PURLs

We want to write policies that evaluate the components in a CycloneDX SBOM. Those components are identified by their purl. To facilitate policy authoring, let's introduce two rego functions:

ec.purl.is_valid to evaluate if a given string is a valid purl.

ec.purl.parse to deconstruct a valid purl string into an object that represents its various fields.

We should consider using https://github.com/package-url/packageurl-go.

open-policy-agent/conftest#856 is a good resource for how to write a custom built-in rego function.

Update to OPA 0.57.0

Create a new PR, dependabot keeps closing #1048

Run GitHub workflows only when necessary for the change

We currently run GitHub workflows for all commits, we should be able to determine when running a particular workflow is not needed. For example changing the readme, should not trigger a release.
For this leveraging paths filter is very helpful.

Run `ec-cli`'s unit tests using the full Cobra command hierarchy

Relies on #1010. Once that is complete, unit tests can be restructured to take advantage of the new modularity by constructing a full command hierarchy.

A/C:

• Unit tests consist of a full command hierarchy, instead of mock subcommands.