in-toto / go-witness

Go implementation of witness

go-witness

A client library for Witness, written in Go.

Status

This library is currently pre-1.0 and therefore the API may be subject to breaking changes.

Features

• Creation and signing of in-toto attestations
• Verification of in-toto attestations and associated signatures with:
  • Witness policy engine
  • OPA Rego policy language
• A growing list of attestor types defined under a common interface
• A selection of attestation sources to search for attestation collections

Documentation

For more detail regarding the library itself, we recommend viewing pkg.go.dev. For the documentation of the witness project, please view the main witness repository.

Requirements

In order to effectively contribute to this library, you will need:

• A Unix-compatible Operating System
• GNU Make
• Go 1.19

Running Tests

This repository uses Go tests for testing. You can run these tests by executing make test.

go-witness's Issues

support pushing attestations to multiple destinations

Currently witness can only push attestations to one destination besides writing it to disk, which is Archivista.

It would be useful to expand this to be able to push the location to wherever the user decided it to be most worthwhile (e.g., OCI Registry). On top of this, it would be useful for the user to be able to specify multiple destinations for a witness run command to push the attestation collections that it generates to.

Product include and exclude glob is not working correctly.

After some investigation I found that the Attest part of the Product attestor does not exclude or include items specifically. It is taking into account the items when building the list of subjects that is used for creating a subject collection. But it does not check these products during the creation of the attestation.

I test bash script creating different outputs and using the exclude / include glob arguments. But running the code thru the debugger its not using these values during the creation of the Product attestation.

Better support for container images with OCI and/or Product Attestor

The OCI attestation currently only works for .tar files output from a docker save command. In a common workflow we first build our images with docker build and the image is create typically in /var/lib/docker filesystem and visible in the docker images command output. It doesn't seem like there is currently a way to easily link the product of docker build to the input of docker save for policy enforcement. I'm not sure how this would work since docker build doesn't produce any files but maybe just tracking the tag and hash of an image as a product?

symbolic link in directory where `witness run` executed leads to failure

I added a symbolic link to README.md in this very repository after moving it into the docs directory. By coincidence I ran witness run in the directory and got the following failure.

➜ go run . run -s build -a environment -k buildkey.pem -o build-attestation.json -- \         
bash -c "echo 'hello' > hello.txt"
INFO    Starting environment attestor...             
INFO    Starting material attestor...                
ERROR   Error running material attestor: EvalSymlinks: too many links 
ERROR   failed to run attestors: EvalSymlinks: too many links 
exit status 1

Putting this issue here for me to pick up in a little bit.

Support PKCS#11 Signing

Cosign supports a number of cryptographic signers that witness currently does not. Some examples include Azure Key Vault, PKCS#11, and some hardware tokens. Instead of trying to keep in sync with every signer that cosign supports, is it possible to give the option for witness to make a command line call to cosign for signing with the parameters specified in a configuration file? This way users could configure the cosign call as they desire without witness having to explicitly support each parameter. If this is not possible (or desirable), can support for PKCS#11 be added since it is a generic cryptographic interface (albeit a not-so-modern one)?

Extending Witness Policies: Adding Support for Retained Attestations

Purpose

To enhance the Witness policy framework by incorporating a mechanism for handling attestations that are not tied to specific steps but are essential for validation throughout the lifecycle of an artifact.

Schema Extension

New Field: retainedAttestations

A new field, retainedAttestations, will be added to the policy schema. This field will specify attestations that should be retained and validated, even if they are not associated with a particular step.

Updated Policy Schema Example:

{
  "expires": "2024-12-31T23:59:59Z",
  "roots": {
    "rootKeyId": {
      "certificate": "base64 encoded certificate"
    }
  },
  "publickeys": {
    "publicKeyId": {
      "keyid": "publicKeyId",
      "key": "base64 encoded key"
    }
  },
  "steps": {
    "build": {
      "name": "build",
      "attestations": [
        {
          "type": "https://witness.dev/attestations/command-run/v0.1",
          "regopolicies": [
            {
              "name": "cmd-policy",
              "module": "base64 encoded rego module"
            }
          ]
        }
      ]
    }
  },
  "retainedAttestations": {
    "attestations": [
      {
        "type": "https://in-toto.io/attestation/vulns/v0.1",
        "regopolicies": [
          {
            "name": "vuln-policy",
            "module": "base64 encoded rego module"
          }
        ]
      }
    ],
    "functionaries": [
      {
        "type": "publickey",
        "publickeyid": "publicKeyId"
      }
    ]
  }
}

Field Descriptions

• retainedAttestations: Defines attestations to be validated outside of specific steps.
  • attestations: Lists attestation types and associated Rego policies.
    • type: Specifies the attestation type (e.g., "vulnerability").
    • regopolicies: Array of Rego policies to validate the attestation.
      • name: The name of the policy.
      • module: The base64 encoded Rego module.
  • functionaries: Lists trusted functionaries authorized to sign these attestations.

Implementation Steps

1. Update Policy Parsing:

   • Modify the policy parser to recognize and handle the retainedAttestations field, ensuring it correctly loads and validates the specified attestations and Rego policies.

2. Incorporate Validation Logic:

   • Extend the verification process to include validation of retained attestations according to the criteria set out in the policy. This should be integrated with existing validation workflows. We should inspect the subjects to ensure either the input hash, product, or a backref subject matches the subject on the retained attestation.

3. Revise Documentation and Tools:

   • Update documentation to reflect the new capabilities, providing clear examples and guidance on how to configure and use the retainedAttestations field.
   • Ensure that any tools or utilities used for policy management are updated to support the new field.

Example Use Case

Consider a scenario where vulnerability attestations are required for compliance but are not associated with specific steps such as "build" or "deploy." By defining these as retained attestations, they can be validated independently, ensuring consistent compliance checks throughout the artifact lifecycle.

Support to test AWS KMS signing using Localstack

I have been trying to setup acceptance tests for aws kms signing but I'm running into,

failed to verify certificate: x509: cannot validate certificate

error for the localstack endpoint. Digging a bit deeper I found that if I hardcode a.options.insecureSkipVerify to true here the signing operation works fine.
I also found this function that returns an Option which can set the insecureSkipVerify property to false. But I couldn't figure out how to use it when creating a SignerProvider or a SignerVerifier.
Can someone please help me with configuring this property 🙏
Thanks.

[Feat]: Jenkins CI attestor

Describe the solution you'd like:

Capture Jenkins CI environment variables

User value:
We'll be able to query attestations using Jenkins build data. This will also allow Jenkins data to be used in the SLSA provenance attestor.

Expected behavior:
Jenkins attestor will capture default Jenkins environment variables.

Proposed solution:
Pretty much the same as the Github / GitLab attestors.

Anything else you would like to add:

Testing changes required:

Documentation changes required:
Add the Jenkins attestor to documentation.

Refactor Policy Verification

Currently, the policy signature verification in Verify.go is handled somewhat differently to how the policy itself is verified. While this makes sense to a degree (verifying policy signatures does not currently support the same amount of signing methods), we should probably refactor the code to reduce repetition and avoid drift between the two verification mechanisms.

KMS Support

Allow signing and verification (including policies) with KMS. Code progress can be seen in #120.

Maven attester not resolving versions or scope if not explicitly set

Creating issue in relation to in-toto/witness#282 over here as it is relevant to this repository.

from @garethahealy:

If the pom.xml doesnt explicitly set the scope or version and uses dependencyManagement, witness does not output it in the attestation. i.e.:

<dependencyManagement>
        <dependencies>
            <dependency>
                <groupId>${quarkus.platform.group-id}</groupId>
                <artifactId>${quarkus.platform.artifact-id}</artifactId>
                <version>${quarkus.platform.version}</version>
                <type>pom</type>
                <scope>import</scope>
            </dependency>
            <dependency>
                <groupId>${quarkus.platform.group-id}</groupId>
                <artifactId>quarkus-camel-bom</artifactId>
                <version>${quarkus.platform.version}</version>
                <type>pom</type>
                <scope>import</scope>
            </dependency>
        </dependencies>
</dependencyManagement>

<dependencies>
        <!-- Core -->
        <dependency>
            <groupId>io.quarkus</groupId>
            <artifactId>quarkus-arc</artifactId>
        </dependency>
        <dependency>
            <groupId>org.apache.camel.quarkus</groupId>
            <artifactId>camel-quarkus-core</artifactId>
        </dependency>
        <dependency>
            <groupId>com.google.code.findbugs</groupId>
            <artifactId>jsr305</artifactId>
            <version>3.0.2</version>
        </dependency>
        <dependency>
            <groupId>io.quarkus</groupId>
            <artifactId>quarkus-container-image-docker</artifactId>
        </dependency>
        <dependency>
            <groupId>io.rest-assured</groupId>
            <artifactId>rest-assured</artifactId>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>org.apache.httpcomponents.client5</groupId>
            <artifactId>httpclient5</artifactId>
            <version>5.2.1</version>
            <scope>test</scope>
        </dependency>
    </dependencies>

produces:

{
        "type": "https://witness.dev/attestations/maven/v0.1",
        "attestation": {
          "groupid": "org.acme",
          "artifactid": "cats-and-dogs",
          "version": "1.0.0-SNAPSHOT",
          "projectname": "",
          "dependencies": [
            {
              "groupid": "io.quarkus",
              "artifactid": "quarkus-arc",
              "version": "",
              "scope": ""
            },
            {
              "groupid": "org.apache.camel.quarkus",
              "artifactid": "camel-quarkus-core",
              "version": "",
              "scope": ""
            },
            {
              "groupid": "com.google.code.findbugs",
              "artifactid": "jsr305",
              "version": "3.0.2",
              "scope": ""
            },
            {
              "groupid": "io.quarkus",
              "artifactid": "quarkus-container-image-docker",
              "version": "",
              "scope": ""
            },
            {
              "groupid": "io.rest-assured",
              "artifactid": "rest-assured",
              "version": "",
              "scope": "test"
            },
            {
              "groupid": "org.apache.httpcomponents.client5",
              "artifactid": "httpclient5",
              "version": "5.2.1",
              "scope": "test"
            }
          ]
        },
        "starttime": "2023-09-06T10:48:14.932748+01:00",
        "endtime": "2023-09-06T10:48:14.934408+01:00"
      }

I'd of expected the version/scope to be included.

Time not being unmarshaled correctly in the AttestationCollection

When i try to unmarshal get the time from an attestation collection it is a nil time. PR Incomming

Issue and pull request template

I noticed that the in-toto/witness project had a nice issue and PR template set. Would it be wise to copy those over to the in-toto/go-witness project too? If thats helpful I can create a PR for it.

Add oidc redirect url option for Fulcio

Add ability to specify oidc redirect url for Fulcio configuration from cli.

Allowing dynamic port allocation on the Issuer's side is sometimes a flag to turn on/off. This would better accommodate private Fulcio deployments.

Strange verification behavior

Hello!

I discovered this project and tried to go through Getting Started. I noticed, that when i run verification process i have this 3 weird lines (i highlighted them), which i don't understand:

INFO    Using config file: .witness.yaml             
INFO    Starting verify attestors stage...           
INFO    Starting policyverify attestor...            
INFO    policy signature verified                    
INFO    Finished policyverify attestor... (0.001980459s) 
INFO    Completed verify attestors stage...          
INFO    Verification succeeded                       
INFO    Evidence:                                    
INFO    Step: build                                  
INFO    0: test-att.json  <-- what is it? why it is duplicated 3 times?                            
INFO    1: test-att.json                             
INFO    2: test-att.json  

Then i tried to modify policy to include requirement for attestation in order to see how verification will fail, well error is very misleading and also duplicated 3 times:

ERROR   Verification failed                          
ERROR   Evidence:                                    
ERROR   Step: build                                  
ERROR   verification failure: Reason: collection validation failed:
 - collection verification failed: no collections found for step build,
 - collection verification failed: no verifiers present to validate against collection verifiers,
 - missing attestation in collection for step build: https://witness.dev/attestations/material/v0.1,
 - missing attestation in collection for step build: https://witness.dev/attestations/command-run/v0.1,
 - missing attestation in collection for step build: https://witness.dev/attestations/gitlab/v0.1 
ERROR   verification failure: Reason: collection validation failed:
 - collection verification failed: no collections found for step build,
 - collection verification failed: no verifiers present to validate against collection verifiers,
 - missing attestation in collection for step build: https://witness.dev/attestations/material/v0.1,
 - missing attestation in collection for step build: https://witness.dev/attestations/command-run/v0.1,
 - missing attestation in collection for step build: https://witness.dev/attestations/gitlab/v0.1 
ERROR   verification failure: Reason: collection validation failed:
 - collection verification failed: no collections found for step build,
 - collection verification failed: no verifiers present to validate against collection verifiers,
 - missing attestation in collection for step build: https://witness.dev/attestations/material/v0.1,
 - missing attestation in collection for step build: https://witness.dev/attestations/command-run/v0.1,
 - missing attestation in collection for step build: https://witness.dev/attestations/gitlab/v0.1 
ERROR   verification failure: Reason: failed to verify artifacts for step build: no passed collections present 
ERROR   failed to verify policy: policy verification failed 

Actually only gitlab attestation should be mentioned in error, others are present. And why error just duplicated 3 times? I started digging into code and found this piece of code in https://github.com/in-toto/go-witness/blob/main/policy/policy.go

func (p Policy) Verify(ctx context.Context, opts ...VerifyOption) (bool, map[string]StepResult, error) {
	vo := &verifyOptions{
		searchDepth: 3,
	}

Looks like because of this magic number 3, verification is done 3 times with mostly the same parameters, thats why i see error and success 3 times. Is it expected behavior?

Implement Concurrency and Timeout in Policy Verification Process

https://github.com/testifysec/go-witness/blob/5e567f0823cff91d879d7f4f0691b648af1d27a1/policy/policy.go#L201-L223

Description:

In the current implementation of the Verify function in policy.go, the Search function is called synchronously for each step in the policy. This could potentially block for a significant amount of time, especially if there are many attestations to search through because it will be i/o call.

Proposed Solution:

To improve the performance and responsiveness of the policy verification process, we propose to make the Search function call non-blocking by using Go's built-in concurrency with goroutines and channels. Additionally, we suggest adding a timeout for each search to prevent the verification process from hanging indefinitely in case of slow or unresponsive searches.

Here's a rough example of how the code could be modified:

var wg sync.WaitGroup
results := make(chan PolicyResult)
errors := make(chan error)

for stepName, step := range p.Steps {
	wg.Add(1)
	go func(stepName string, step Step) {
		defer wg.Done()
		ctxWithTimeout, cancel := context.WithTimeout(ctx, 5*time.Second)
		defer cancel()

		statements, err := vo.verifiedSource.Search(ctxWithTimeout, stepName, vo.subjectDigests, attestationsByStep[stepName])
		if err != nil {
			errors <- err
			return
		}

		approvedCollections := step.checkFunctionaries(statements, trustBundles)
		stepResults := step.validateAttestations(approvedCollections)
		results <- PolicyResult{EvidenceByStep: map[string][]source.VerifiedCollection{stepName: stepResults.Passed}}
	}(stepName, step)
}

Benefits:

1. Performance: By making the Search function call non-blocking, the policy verification process can continue with other tasks while the search is being performed, potentially improving the overall performance.

2. Responsiveness: Adding a timeout for each search ensures that the verification process doesn't hang indefinitely in case of slow or unresponsive searches, improving the system's responsiveness.

3. Resource Usage: By limiting the time spent on each search, we can also control the resource usage of the policy verification process, which can be particularly beneficial in resource-constrained environments.

4. Scalability: With the proposed changes, we could potentially increase the searchDepth in the policy verification process. This would allow us to perform a more thorough search over many iterations, potentially uncovering more attestations that satisfy the policy. This can be particularly beneficial for complex policies with many steps or dependencies between steps. By implementing concurrency and timeouts, we can scale the depth of our search without significantly impacting performance or responsiveness.

I wanted to try this code change before proposing it but didn't have the infra or the tests to run it.

[Feat]: Extensibility for the environment attestor

Describe the solution you'd like:
A way to add environment variables to the default denylist, or a way to optionally only attest a specified list of environment variables.

User value:

Some environments may have sensitive information stored in environment variables which are not covered in the denylist now. Using the environment attestor will store them in recoverable plaintext in generated attestations.

Expected behavior:

Environment variables can be selectively denied or allowed to be attested.

Proposed solution:

Add a flag to decide between allowlist / denylist, and another flag to append to either list for later filtering.

Anything else you would like to add:

Testing changes required:

Add tests to make sure environment variables are correctly filtered.

Documentation changes required:

Add flags to the CLI reference and modify the Environment attestor page.

Feature: generate maven effective `pom.xml` as part of attestation process

as mentioned in #98, the pom.xml file is not always full rendered. To aid this, the mvn cli has a help command called effective-pom which renders the final pom that maven uses to execute the final task it performs.

To ensure the fullness of the pom, witness could run this to ensure that it isn't missing any context from the pom.xml that it is using to create the attestation.

[Feat]: CLO Monitor Recommendations

Achieve a score of 💯 on CLOMonitor

Remaning Tasks:

• Dependency Policy
• SBOM
• Security Insights file
• Token Permissions

Support signing attestations by multiple signers

Currently if an attestation is generated with witness run, it cannot be resigned later on by another identity.

The ability for a user to call witness run in reference to an attestation that has already been generated and signed by another identity could be useful as confirmation of some process having been executed against the metadata that the attestation contains.

migrate to supplying x.509 CA certificates in a bundle rather than separate root and intermediate certs

Currently Witness requires that root certs are supplied separately to intermediate certs like so:

  "roots": {
    "test": {
      "certificate": <base64-encoded-root-cert-pem>,
      "intermediates": [
        <base64-encoded-intermediate-cert-pem>
      ]
    }
  },

Instead, allowing the user to supply all the necessary certs (root and intermediate) as a single pem encoded trustbundle would be an easier experience:

  "roots": {
    "test": {
      "trustBundle": <base64-encoded-trust-bundle-pem>
    }
  },

Support all Fulcio Cert Extensions

Fulcio has a publicly export API now for all of the x509 cert extensions. We should support these as constraints for functionaries and policy signatories.

https://github.com/sigstore/fulcio/blob/main/pkg/certificate/extensions.go

Attestors not registered when using the `attestation` package directly

When I used the https://github.com/in-toto/go-witness/tree/main/attestation package for the attestation collection, I wasn't able to unmarshal a collection (with its collectionattestations) because attestors are registered separately in https://github.com/in-toto/go-witness/blob/main/imports.go. After pulling in these imports (and thus invoking their inits), I was able to proceed.

Note that imports.go is missing material / product, I think they're pulled in somewhere else.