kubernetes-sigs / bom Goto Github PK
View Code? Open in Web Editor NEWA utility to generate SPDX-compliant Bill of Materials manifests
Home Page: https://kubernetes-sigs.github.io/bom/
License: Apache License 2.0
A utility to generate SPDX-compliant Bill of Materials manifests
Home Page: https://kubernetes-sigs.github.io/bom/
License: Apache License 2.0
On running bom generate
pointing to a remote, non-public image (in GCR), I receive error:
FATA generating doc: creating SPDX document: generating SPDX package from image ref gcr.io/my-gcp-project/myimage:mytag: while downloading images to archive: fetching remote descriptor: GET https://gcr.io/v2/my-gcp-project/my-gcp-project/myimage/manifests/mytag: UNAUTHORIZED: You don't have the needed permissions to perform this operation, and you may have invalid credentials. To authenticate your request, follow the steps in: https://cloud.google.com/container-registry/docs/advanced-authentication
Yet I am able to docker pull
it.
Appears to originate here:
bom/pkg/spdx/implementation.go
Line 303 in 6434978
The bom utility should be able to fetch external SBOMs.
In order to perform a complete verification of artifacts described in an SBOM, out spdx libraries should provide methods to fetch externally referenced documents.
For example, this Package was generated from source code described in an external SBOM:
Relationship: SPDXRef-Package-k8s.gcr.io-kube-apiserver-s390x-v1.23.0-alpha.3 GENERATED_FROM DocumentRef-kubernetes-v1.23.0-alpha.3:SPDXRef-Package-kubernetes
The external SBOM is defined at the document level:
ExternalDocumentRef:DocumentRef-kubernetes-v1.23.0-alpha.3 https://sbom.k8s.io/v1.23.0-alpha.3/source SHA1: 8fa4c9ccde6d6b8dab3e818cf
f56cd45e66cd463
The Document
object should be able to download, parse and use the SBOM from https://sbom.k8s.io/v1.23.0-alpha.3/source when it needs it. Unless we have a method like this, we cannot validate releases split across several SBOMs.
When extracting image layers, the layer analyzer will treat the blobs as a tar file or as compressed data depending on the file extension. This causes an error when the blobs are extracted and opened for inspection
Layer blobs should be opened and inspected
Trying to generate an SBOM with bom v0.2.0 of this Istio image fails:
wget https://storage.googleapis.com/istio-prerelease/prerelease/1.13.0-beta.0/docker/app-debug.tar.gz
bom generate --image-archive app-debug.tar.gz
INFO Generating SPDX Bill of Materials
INFO Processing tarball app-debug.tar.gz
INFO Generating SPDX package from image tarball app-debug.tar.gz
INFO Successfully extracted 12 files from image tarball app-debug.tar.gz
INFO Package describes istio/app:1.13.0-beta.0-debug image
INFO Image manifest lists 7 layers
FATA generating doc: creating SPDX document: generating tarball package: getting os data from container: reading os type from layer: reading os release: extracting os-release from tar: reading tarfile: archive/tar: invalid tar header
Currently the docs are just sitting idle at https://github.com/kubernetes-sigs/bom/tree/main/docs .
Let's publish them on a website!
EnsureKo
method throws the following error when you type $ go run mage.go BuildImagesLocal
:
Building image with ko for local test...
Checking if `ko` version 0.11.2 is installed
Error: unknown command "0.11.2" for "ko"
Run 'ko --help' for usage.
2022/07/29 21:42:27 error during command execution:unknown command "0.11.2" for "ko"
Error: could not determine the installed version of /Users/batuhan.apaydin/go/bin/ko with '/Users/batuhan.apaydin/go/bin/ko 0.11.2': running "/Users/batuhan.apaydin/go/bin/ko 0.11.2" failed with exit code 1
exit status 1
After some discussions with Carlos, we decided to start staging and promoting the artifacts of the kubernetes-sigs/bom project. This is the tracking issue to track the rollout.
/cc @cpanato
When generating an SBOM for a multi-arch image from the local docker cache, the SBOM is not only invalid but it describes only one arch.
I would expect the SBOM to describe all the architectures, just as with the remote images
Running the following will yield a valid right.spdx
SBOM describing all the images and an invalid wrong.spdx
sbom which not only does not validate but only has information about the local platform image:
# First pulll a multi-arch image to local cache and generate an sbom
docker pull nginx:latest
bom generate -i nginx:latest > wrong.spdx
# Now generate the SBOM from the remote image:
docker rmi nginx:latest
bom generate -i nginx:latest > right.spdx
To mitigate, the local docker cache needs to not have an image in order to generate its SBOM (as shown above)
bom v0.3.0
The PR kubernetes/release#2478 got reverted because it caused troubles with the error wrapping conversion.
We should still do that, because the package github.com/pkg/errors
is now in maintenance mode due to the golang native error wrapping. The goal is to do that on a package-by-package basis, where everyone is welcome to contribute.
errors
import in favor of github.com/pkg/errors
errors.Errorf
to fmt.Errorf
errors.Wrap(err, "…")
to fmt.Errorf("…: %w", err)
and check that err != nil
beforeerrors.Wrapf(err, "… %s: %s", foo, bar)
to fmt.Errorf("… %s: %s: %w", foo, bar err)
and check that err != nil
beforeIn order to release binaries of the tool, we need to establish image promotion. This depends on the ongoing effort to establish the release infrastructure (#35).
The animation on the main project page is very outdated. Some of the flags are no longer current and the tools supports a lot more features.
We need to record a new animation that can show how to generate an SBOM but also how to check the output, etc.
using bom to generate an sbom for a container stored in AWS ECR
output an sbom
na
bom version
______ _____ ___ ___
| ___ \| _ || \/ |
| |_/ /| | | || . . |
| ___ \| | | || |\/| |
| |_/ /\ \_/ /| | | |
\____/ \___/ \_| |_/
bom: A tool for working with SPDX manifests
GitVersion: v0.3.0
GitCommit: unknown
GitTreeState: unknown
BuildDate: unknown
GoVersion: go1.19.1
Compiler: gc
Platform: darwin/arm64
uname -a
Darwin Jamess-MBP-2.localdomain 21.6.0 Darwin Kernel Version 21.6.0: Mon Aug 22 20:19:52 PDT 2022; root:xnu-8020.140.49~2/RELEASE_ARM64_T6000 arm64
Jamess-MBP-2:adobe-images strongjz$
log output
Jamess-MBP-2:adobe-images strongjz$ bom generate -i 123456789012.dkr.ecr.us-east-1.amazonaws.com/cluster-registry-client:ed22c79
INFO bom v0.3.0: Generating SPDX Bill of Materials
INFO Processing image reference: 123456789012.dkr.ecr.us-east-1.amazonaws.com/cluster-registry-client:ed22c79
INFO Adding image tag 123456789012.dkr.ecr.us-east-1.amazonaws.com/cluster-registry-client:ed22c79 from reference
INFO Checking the local image cache for 123456789012.dkr.ecr.us-east-1.amazonaws.com/cluster-registry-client:ed22c79
INFO 123456789012.dkr.ecr.us-east-1.amazonaws.com/cluster-registry-client:ed22c79 was found in the local image cache
panic: runtime error: index out of range [1] with length 1
goroutine 1 [running]:
sigs.k8s.io/bom/pkg/spdx.(*spdxDefaultImplementation).PullImagesToArchive(0x0?, {0x140001b2eb0, 0x4c}, {0x140001b2fa0, 0x45})
/Users/strongjz/Documents/code/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/spdx/implementation.go:422 +0xce8
sigs.k8s.io/bom/pkg/spdx.(*spdxDefaultImplementation).ImageRefToPackage(0x140001ca000?, {0x140001b2eb0, 0x4c}, 0x1e?)
/Users/strongjz/Documents/code/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/spdx/implementation.go:735 +0xf8
sigs.k8s.io/bom/pkg/spdx.(*SPDX).ImageRefToPackage(...)
/Users/strongjz/Documents/code/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/spdx/spdx.go:247
sigs.k8s.io/bom/pkg/spdx.(*defaultDocBuilderImpl).GenerateDoc(0xfa78?, 0x1010871b0, 0x140001c3680)
/Users/strongjz/Documents/code/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/spdx/builder.go:246 +0x924
sigs.k8s.io/bom/pkg/spdx.(*DocBuilder).Generate(0x140001a5530, 0x140001c3680)
/Users/strongjz/Documents/code/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/spdx/builder.go:96 +0xbc
sigs.k8s.io/bom/cmd/bom/cmd.generateBOM(0x140001c3560)
/Users/strongjz/Documents/code/go/pkg/mod/sigs.k8s.io/[email protected]/cmd/bom/cmd/generate.go:341 +0x38c
sigs.k8s.io/bom/cmd/bom/cmd.AddGenerate.func1(0x14000472a00?, {0x14000375fc0?, 0x2?, 0x2?})
/Users/strongjz/Documents/code/go/pkg/mod/sigs.k8s.io/[email protected]/cmd/bom/cmd/generate.go:146 +0xa4
github.com/spf13/cobra.(*Command).execute(0x14000472a00, {0x14000375fa0, 0x2, 0x2})
/Users/strongjz/Documents/code/go/pkg/mod/github.com/spf13/[email protected]/command.go:872 +0x4d0
github.com/spf13/cobra.(*Command).ExecuteC(0x10107e8c0)
/Users/strongjz/Documents/code/go/pkg/mod/github.com/spf13/[email protected]/command.go:990 +0x354
github.com/spf13/cobra.(*Command).Execute(...)
/Users/strongjz/Documents/code/go/pkg/mod/github.com/spf13/[email protected]/command.go:918
sigs.k8s.io/bom/cmd/bom/cmd.Execute()
/Users/strongjz/Documents/code/go/pkg/mod/sigs.k8s.io/[email protected]/cmd/bom/cmd/root.go:71 +0x28
main.main()
/Users/strongjz/Documents/code/go/pkg/mod/sigs.k8s.io/[email protected]/cmd/bom/main.go:24 +0x1c
Jamess-MBP-2:adobe-images strongjz$
cat /etc/os-release
):uname -a
):As per the email sent to kubernetes-dev[1], please create a SECURITY_CONTACTS
file.
The template for the file can be found in the kubernetes-template repository[2].
A description for the file is in the steering-committee docs[3], you might need
to search that page for "Security Contacts".
Please feel free to ping me on the PR when you make it, otherwise I will see when
you close this issue. :)
Thanks so much, let me know if you have any questions.
(This issue was generated from a tool, apologies for any weirdness.)
[1] https://groups.google.com/forum/#!topic/kubernetes-dev/codeiIoQ6QE
[2] https://github.com/kubernetes/kubernetes-template-project/blob/master/SECURITY_CONTACTS
[3] https://github.com/kubernetes/community/blob/master/committee-steering/governance/sig-governance-template-short.md
The SPDX license list should be included with the bom distribution to avoid downloading it.
On first run, bom will download the SPDX license list. When generating a quick SBOM from a cold start like this the license download can be a bit cumbersome. It can also lead to a CI/CD failure when running bom in a pipeline.
I think we should exploring compiling the license list into a distributable form (a tarball or similar) and shipping it with the binary. I think using go embed could be the way to go to do this.
When generating an SBOM for a go module, bom will list the module as a dependency of itself.
The module getting described in the SBOM should not be listed in the dependencies.
bom generate . > bom.spdx
sigs.k8s.io/bom
(2nd line):grep sigs.k8s.io bom.spdx | grep Relationship
Relationship: SPDXRef-Package-bom DEPENDS_ON SPDXRef-Package-sigs.k8s.io-release-utilsC64v0.3.0
Relationship: SPDXRef-Package-bom DEPENDS_ON SPDXRef-Package-sigs.k8s.io-bom
Relationship: SPDXRef-Package-bom DEPENDS_ON SPDXRef-Package-sigs.k8s.io-bom
cat /etc/os-release
):uname -a
):What do you think about adding a feature
bom compose
for merging multiple SBOMs into a top level artefact?
A PoC already exists in sbom-composer, but as bom
already supports wider range of functionalities, it would be great to have such option in order to work as all-in-one solution
Having granular SBOMs is not a new concept and it's inevitable to end-up operating with tens or hundreds of documents for many use-cases. By now this is only resolved by using External Document References
, but it's much harder to work with linked documents, rather than a single composed one. Having the compose
option would increase the usefulness of the bom
tool, resolving this problem.
Bom already has parsing and filtering features to build compose
on top of.
One of the SPDX documents for k8s releases has an incorrect date format:
The "Created" field has a datetime format that is inconsistent with commonly used layouts (it looks like the code seems to want to use YYYY-MM-DD as well, which is why it's a little confusing).
SPDXVersion: SPDX-2.2
DataLicense: CC0-1.0
SPDXID: SPDXRef-DOCUMENT
DocumentName: kubernetes-v1.21.2
DocumentNamespace: https://k8s.io/sbom/source/v1.21.2
Creator: Tool: k8s.io/release/pkg/spdx
Created: 2021-16-06T13:41:24Z
curl -L https://sbom.k8s.io/v1.21.2/source | grep "Created:"
N/A. Let me know if this should be opened elsewhere!
N/A
bom
supports defining complex SBOMs through a YAML configuration file. However, the YAML file is not defined anywhere. An example of the configuration file taken from the bom-installer README:
---
namespace: https://example.com/
license: Apache-2.0
name: myproject
creator:
person: Author Name ([email protected])
artifacts:
- type: directory
source: .
gomodules: true
- type: file
source: bin/linux-binary
- type: file
source: bin/mac-binary
- type: file
source: bin/windows-binary.exe
- type: image
source: ghcr.io/myorg/myrepo/myimage:tag
license: Apache-2.0
If we expect users to use the feature, we should document it somewhere 🙃
We should publish a GitHub action that installs a released binary. We should create the action to install the bom and then if it works well lets donate the repo to the k8s org.
Having a ready to use GitHub action would make life easier for users to simply drop it into their existing workflows.
From https://spdx.dev/resources/use/#documents:
SPDX Documents can use one of five file formats, tag/value (.spdx), JSON(.spdx.json), YAML(.spdx.yml), RDF/xml(spdx.rdf) and spreadsheets (.xls).
On bom generate
, would like to be able to specify --format=json
to get an SPDX document in JSON format.
SPDX defines an optional FileType
field. This field can take any of these values:
SOURCE | BINARY | ARCHIVE | APPLICATION | AUDIO | IMAGE | TEXT | VIDEO | DOCUMENTATION | SPDX | OTHER
We could do a rapid scan of the files and do a quick determination of the type
Ref: https://spdx.github.io/spdx-spec/file-information/#83-file-type-field
Scanning and determining this value results in a richer SBOM.
Currently the purls obtained as external refs for golang packages look like:
grep "ExternalRef: PACKAGE-MANAGER purl" bom.spdx
ExternalRef: PACKAGE-MANAGER purl pkg:golang/github.com/[email protected]
ExternalRef: PACKAGE-MANAGER purl pkg:golang/github.com/[email protected]
ExternalRef: PACKAGE-MANAGER purl pkg:golang/github.com/[email protected]
ExternalRef: PACKAGE-MANAGER purl pkg:golang/github.com/[email protected]
ExternalRef: PACKAGE-MANAGER purl pkg:golang/github.com/[email protected]
These have namespaces like github.com
. It misses the repository owners.
Expected purls would look like:
pkg:golang/github.com/jbenet/[email protected]
I built the image, then tried to generate an SBOM for that image but bom tool tried to pull an image from a registry even if it exists locally.
It should generate an SBOM for an image that exists in the local image cache.
Try to build an image then run bom generate -i <img>
$bom version
______ ________ ___
| ___ \| _ | \/ |
| |_/ /| | | | . . |
| ___ \| | | | |\/| |
| |_/ /\ \_/ / | | |
\____/ \___/\_| |_/
bom: A tool for working with SPDX manifests
GitVersion: devel
GitCommit: unknown
GitTreeState: unknown
BuildDate: unknown
GoVersion: go1.17.8
Compiler: gc
Platform: darwin/amd64
cat /etc/os-release
): Darwinuname -a
): Darwin C02DM1U3MD6R 20.6.0 Darwin Kernel Version 20.6.0: Wed Jun 23 00:26:31 PDT 2021; root:xnu-7195.141.2~5/RELEASE_X86_64 x86_64GitHub reviews should act as /lgtm
+ /approve
for SIG Release repos.
Filing an issue for this...
Original issue: kubernetes/sig-release#1464
Implementation example: kubernetes/test-infra#20934
cc: @kubernetes-sigs/release-engineering
Originally posted by @justaugustus in #27 (comment)
Running bom generate
on the repository https://github.com/lumjjb/sample-golang-prov fails with a runtime error using the latest commit of bom.
The error occurred both when running in a github action and when running locally. Link to the github action logs: https://github.com/lumjjb/sample-golang-prov/runs/7026978132?check_suite_focus=true#:~:text=level%3Dinfo%20msg%3D%22Got,with%20exit%20code%202
Clone the sample-golang-prov
repository and run bom generate -n "http://www.example.com/binary.spdx" -o "binary.spdx" .
.
The error does not occur when using bom at commit 5cfc466ecf53
.
The final SPDX 2.3 spec is expected to land before the end of July '22. bom
should support the new spec.
We want to enable our users to generate SBOMs in the latest version of the spec. Yet, there are still open questions on what to do with 2.2.x support. My first thought was to drop support 2.2 generation altogether but now, with the support of serializers landing last week it is feasible to support both.
This could be useful in case certain tool expects SPDX SBOM of specific version.
The ![terminal demo](/docs/cast.svg "Terminal demo")
part of the README cause extreme performance issues in browsers. On chrome (desktop and mobile) this used 100% CPU and 2gb memory and caused the entire repo page to freeze for 1 minute.
Perhaps https://asciinema.org/ or a GIF could be used instead?
Since Syft v0.40.0, Syft can generate and sign SBOM attestations by using cosign. It brings another chance that we can now attach the attestation generated by Syft to a container image.
To get more detail about the development and the concept, follow the links below:
anchore/syft#510
anchore.com/sbom/creating-sbom-attestations-using-syft-and-sigstore
https://github.com/sigstore/cosign/blob/main/specs/ATTESTATION_SPEC.md
https://github.com/sigstore/cosign/blob/main/doc/cosign_attach_attestation.md
The same concept can be applied here to the bom tool too.
To be able to make SBOM generated via the bom tool tamper-proof.
Parse and recognize SPDX license identifiers.
One big omission in the license scanner is a simple feature to recognize SPDX license identifiers in source code files. This should be quick and easy to implement.
Based on the comment that @puerco did, it'd be better to have a base image for bom tool that includes all the necessary packages such as go, git, etc. The proper way of doing it is that use a tech stack including apko1 and melange2.
cc: @justaugustus @kaniini @imjasonh @cpanato
To provide a proper base image for bom tool that fits its needs.
Currently the external reference for a detected package is only the corresponding package's PURL. We should also provide the CPE for this package.
CPEs in some cases can be used for looking up vulnerabilities through NVD.
When using the SPDX libraries, adding a new file to a document using document.AddFile()
returns an error:
unable to generate file ID, filename not set
even when the files was generated with FileFromPath()
The SPDX file should be added without error
spdxClient := spdx.NewSPDX()
spdxFile, _ := spdxClient.FileFromPath("/path/file.txt")
// spdxFile.BuildID() // Bug in bom, we have to call manually
if err := doc.AddFile(spdxFile); err != nil {
logrus.Errof(err, "adding SPDX file to SBOM")
}
This seems to be because there was a late refactor of the ID building code and probably AddFile
uses and older version. Uncommenting the line above forces the use of the new function BuildID()
which generates the ID correctly and does not throw the error.
cat /etc/os-release
):NAME="Fedora Linux"
VERSION="35 (Workstation Edition)"
ID=fedora
VERSION_ID=35
VERSION_CODENAME=""
PLATFORM_ID="platform:f35"
PRETTY_NAME="Fedora Linux 35 (Workstation Edition)"
ANSI_COLOR="0;38;2;60;110;180"
LOGO=fedora-logo-icon
CPE_NAME="cpe:/o:fedoraproject:fedora:35"
HOME_URL="https://fedoraproject.org/"
DOCUMENTATION_URL="https://docs.fedoraproject.org/en-US/fedora/f35/system-administrators-guide/"
SUPPORT_URL="https://ask.fedoraproject.org/"
BUG_REPORT_URL="https://bugzilla.redhat.com/"
REDHAT_BUGZILLA_PRODUCT="Fedora"
REDHAT_BUGZILLA_PRODUCT_VERSION=35
REDHAT_SUPPORT_PRODUCT="Fedora"
REDHAT_SUPPORT_PRODUCT_VERSION=35
PRIVACY_POLICY_URL="https://fedoraproject.org/wiki/Legal:PrivacyPolicy"
VARIANT="Workstation Edition"
VARIANT_ID=workstation
uname -a
):Linux babieco.home.cochinitos.net 5.15.6-200.fc35.x86_64 #1 SMP Wed Dec 1 13:41:10 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Looking through the kubernetes 1.23.1 SBOM, I noticed that license tags are repeated in the PackageLicenseInfoFromFiles
tags:
##### Package: kubernetes
PackageName: kubernetes
SPDXID: SPDXRef-Package-kubernetes
PackageDownloadLocation: NONE
FilesAnalyzed: true
PackageVerificationCode: 8db6751a05b6ffa6c545d92d89d1aa4c431121fd
PackageLicenseConcluded: Apache-2.0
PackageLicenseInfoFromFiles: Apache-2.0
PackageLicenseInfoFromFiles: BSD-3-Clause
PackageLicenseInfoFromFiles: MIT
PackageLicenseInfoFromFiles: ISC
PackageLicenseInfoFromFiles: BSD-2-Clause
PackageLicenseInfoFromFiles: LGPL-3.0-only
PackageLicenseInfoFromFiles: GPL-2.0-only
PackageLicenseInfoFromFiles: CC-BY-SA-4.0
PackageLicenseInfoFromFiles: Apache-2.0
PackageLicenseInfoFromFiles: BSD-3-Clause
PackageLicenseInfoFromFiles: MIT
PackageLicenseInfoFromFiles: ISC
PackageLicenseInfoFromFiles: BSD-2-Clause
PackageLicenseInfoFromFiles: LGPL-3.0-only
PackageLicenseInfoFromFiles: GPL-2.0-only
PackageLicenseInfoFromFiles: CC-BY-SA-4.0
License tags should be listed just once. These tags collect all the files found in the source code and should only be listed one time for each kind.
Download the sbom from here: https://sbom.k8s.io/v1.23.1/source
and check the output
SPDX has support for referencing elements defined in external SBOMs and building relationships pointing to them, while we already have support in the libraries for it already, the command line utility does not have a flag to define them.
If you want to use Micro SBOMs, there is no way to link them together using only the command line tool.
See https://spdx.github.io/spdx-spec/v2.3/document-creation-information/#67-license-list-version-field .
Currently the models seem to have this. However the generated SBOM, both in tag and json format don't include this field.
Trying to obtain SBOM by using bom
installed via docker image errors with
level=fatal msg="generating doc: creating SPDX document: generating package from directory: scanning go packages: opening new module path: building module package list: unable to get full list of packages, go executable not found "
To reproduce use following invocation:
podman run -v /home/shivam/work/bom:/opt/stuff/ k8s.gcr.io/bom/bom:v0.3.0 generate /opt/stuff/
I'm using bom to generate SBOM for kubearmor project. Here's the issue I'm facing.
I have a config file .sbom.yaml
namespace: https://kubearmor.io # an URI that serves as namespace for the SPDX doc
license: Apache-2.0 # SPDX license identifier to declare in the SBOM
name: kubearmor sbom #name for the document, in contrast to URLs, intended for humans
creator:
person: Anurag
tool: bom
artifacts:
- type: directory # Valid choices are "directory" or "file" or "image"
source: ./KubeArmor # Path to container in registry if type is "image" else path to directory or file
gomodules: true # Boolean. Set it to true if this artifact is a gomodule.
- type: directory
source: ./deployments
gomodules: true
- type: directory
source: ./KubeArmor/BPF/tests
gomodules: true
- type: directory
source: ./pkg/KubeArmorAnnotation
gomodules: true
- type: directory
source: ./pkg/KubeArmorController
gomodules: true
- type: directory
source: ./pkg/KubeArmorHostPolicy
gomodules: true
- type: directory
source: ./pkg/KubeArmorPolicy
gomodules: true
- type: image # Valid choices are "directory" or "file" or "image"
source: kubearmor/kubearmor:latest # Path to container in registry if type is "image" else path to directory or file
license: Apache-2.0 # SPDX identifier of the license
- type: image
source: kubearmor/kubearmor-init:latest
license: Apache-2.0
- type: file # Valid choices are "directory" or "file" or "image"
source: Dockerfile # Path to container in registry if type is "image" else path to directory or file
license: Apache-2.0 # SPDX identifier of the license
bom generate --config .sbom.yaml >> bomsbom.md
Now, when I use bom document outline bomsbom.md
and the error I'm getting is
$ bom document outline bomsbom.md
FATA opening doc: duplicate SPDXID SPDXRef-Package-gomod-k8s.io-utils-v0.0.0-20220823124924-e9cbc92d1a73
cat /etc/os-release
): Ubuntu 20.04uname -a
): 5.4.0-1090-azureProvide support and generate CycloneDX SBOM.
OWASP CycloneDX is a lightweight Software Bill of Materials (SBOM) standard designed to use in the application security contexts and supply chain component analysis. CycloneDX is an OWASP flagship project ( https://owasp.org/www-project-cyclonedx ). The Open Web Application Security Project is a non-profit foundation that works to improve the security of software.
CycloneDX is already supported by many security vendors and projects ( https://cyclonedx.org/about/supporters ). It is also recommended in the Technology Radar Volume 26 ( https://www.thoughtworks.com/radar/platforms?blipid=202203034 )
Related discussion links:
https://app.slack.com/client/T09NY5SBT/C2C40FMNF/thread/C2C40FMNF-1637756090.238600
https://twitter.com/puerco/status/1414786491776212992
A container image per release containing the statically linked binary for convenient usage.
It would make usage of the bom tool easier in environments which support containers but not directly golang.
The following command
bom generate -o sbom_ci_pr_docker-plugin_dee90ec522b992efa8962579f9e0e9be243388cd.spdx \
bom generate -o sbom_ci_pr_docker-plugin_dee90ec522b992efa8962579f9e0e9be243388cd.spdx \
--dirs=. \
--image=quay.io/cilium/docker-plugin-ci@sha256:b6a2934ca71557db04e88e42548e26cf8f907eadc0eb2fd7c0abc232215b54aa
bom generate -o sbom_ci_pr_race_docker-plugin_dee90ec522b992efa8962579f9e0e9be243388cd.spdx \
--dirs=. \
--image=quay.io/cilium/docker-plugin-ci@sha256:8348cf7c457e4261469f7313889b51d0976645e4fdb762910639ad569cca555c
bom generate -o sbom_ci_pr_unstripped_docker-plugin_dee90ec522b992efa8962579f9e0e9be243388cd.spdx \
--dirs=. \
--image=quay.io/cilium/docker-plugin-ci@sha256:2b454f79a725e060602806202126576b45b676e380be502ff4cef24659dab334
results in
panic: interface conversion: name.Reference is name.Digest, not name.Tag
goroutine 1 [running]:
sigs.k8s.io/bom/pkg/spdx.refInfoFromIndex(0xc0020a7540)
sigs.k8s.io/bom/pkg/spdx/implementation.go:253 +0xed3
sigs.k8s.io/bom/pkg/spdx.getImageReferences({0xc0000308c0, 0x67})
sigs.k8s.io/bom/pkg/spdx/implementation.go:211 +0x25d
sigs.k8s.io/bom/pkg/spdx.(*spdxDefaultImplementation).PullImagesToArchive(0x0?, {0xc0000308c0?, 0x973a03?}, {0xc00835f6a0, 0x19})
sigs.k8s.io/bom/pkg/spdx/implementation.go:359 +0x4c
sigs.k8s.io/bom/pkg/spdx.(*spdxDefaultImplementation).ImageRefToPackage(0xc0000a2000?, {0xc0000308c0, 0x67}, 0x1e?)
sigs.k8s.io/bom/pkg/spdx/implementation.go:698 +0x172
sigs.k8s.io/bom/pkg/spdx.(*SPDX).ImageRefToPackage(...)
sigs.k8s.io/bom/pkg/spdx/spdx.go:253
sigs.k8s.io/bom/pkg/spdx.(*defaultDocBuilderImpl).GenerateDoc(0x0?, 0xd82b00, 0xc00009cfc0)
sigs.k8s.io/bom/pkg/spdx/builder.go:234 +0xb6a
sigs.k8s.io/bom/pkg/spdx.(*DocBuilder).Generate(0xc0000b9b50?, 0x1?)
sigs.k8s.io/bom/pkg/spdx/builder.go:96 +0xb5
sigs.k8s.io/bom/cmd/bom/cmd.generateBOM(0xc00009cea0)
sigs.k8s.io/bom/cmd/bom/cmd/generate.go:341 +0x3db
sigs.k8s.io/bom/cmd/bom/cmd.AddGenerate.func1(0xc000178000?, {0xc000167100?, 0x4?, 0x4?})
sigs.k8s.io/bom/cmd/bom/cmd/generate.go:146 +0xe8
github.com/spf13/cobra.(*Command).execute(0xc000178000, {0xc0001670c0, 0x4, 0x4})
github.com/spf13/[email protected]/command.go:916 +0x862
github.com/spf13/cobra.(*Command).ExecuteC(0xd7a3e0)
github.com/spf13/[email protected]/command.go:1044 +0x3bd
github.com/spf13/cobra.(*Command).Execute(...)
github.com/spf13/[email protected]/command.go:968
sigs.k8s.io/bom/cmd/bom/cmd.Execute()
sigs.k8s.io/bom/cmd/bom/cmd/root.go:71 +0x25
main.main()
./main.go:24 +0x17
Version v0.4.1/bom-linux-amd64
as seen in https://github.com/cilium/cilium/actions/runs/3392808891/jobs/5640821590#step:22:202
cc @sandipanpanda @puerco
I'm using v0.4.0.
Following is error I'm getting.
@sbs2001 ➜ /workspaces/gg (main) $ bom generate .
INFO bom v0.4.0: Generating SPDX Bill of Materials
INFO Processing directory .
INFO Loading license data from downloader
INFO Sending GET request to https://spdx.org/licenses/licenses.json
INFO Read data for 496 licenses. Downloading.
INFO Downloaded 496 licenses
INFO Got 496 licenses from downloader
INFO Writing license data to /tmp/spdx/downloadCache
INFO Writing 496 SPDX licenses to /tmp/spdx/licenses
WARN Label Copyright is not an identifier of a known license
FATA generating doc: creating SPDX document: generating package from directory: generating SPDX package from directory: scanning directory for licenses: getting directory license: scanning topmost license file: ID does not correspond to a valid license: 'Copyright'
Running the command from example:
bom generate -n http://example.com/ --image registry.k8s.io/kube-apiserver:v1.21.0
Produces the following crash:
INFO bom (devel): Generating SPDX Bill of Materials
INFO Processing image reference: registry.k8s.io/kube-apiserver:v1.21.0
INFO Reference image index points to 5 manifests
INFO Adding image registry.k8s.io/kube-apiserver@sha256:1435e167151f90b7f4abfd416726751c46b8672cc7288507fab7cfa5a05b866c (amd64/linux)
INFO Adding image registry.k8s.io/kube-apiserver@sha256:90cbf8d0444fe5fad4e06d37d42c6d202bf27edc5d931c337b6aadf749d03550 (arm/linux)
INFO Adding image registry.k8s.io/kube-apiserver@sha256:1a495c62bc2e2f2209e3a8c5fe6c364878d55418c802a7543edadeb1728d987a (arm64/linux)
INFO Adding image registry.k8s.io/kube-apiserver@sha256:a2199aece7bf28ca30233895803984edd45cab59b054eeade5ea398185d995db (ppc64le/linux)
INFO Adding image registry.k8s.io/kube-apiserver@sha256:4dc554858e54ad2ae7cd1990d1fb11e1c516fbffd9f17f8a4f98ead0b5eae148 (s390x/linux)
INFO Downloading registry.k8s.io/kube-apiserver@sha256:1435e167151f90b7f4abfd416726751c46b8672cc7288507fab7cfa5a05b866c
INFO Downloading registry.k8s.io/kube-apiserver@sha256:90cbf8d0444fe5fad4e06d37d42c6d202bf27edc5d931c337b6aadf749d03550
INFO Downloading registry.k8s.io/kube-apiserver@sha256:1a495c62bc2e2f2209e3a8c5fe6c364878d55418c802a7543edadeb1728d987a
INFO Downloading registry.k8s.io/kube-apiserver@sha256:a2199aece7bf28ca30233895803984edd45cab59b054eeade5ea398185d995db
INFO Downloading registry.k8s.io/kube-apiserver@sha256:4dc554858e54ad2ae7cd1990d1fb11e1c516fbffd9f17f8a4f98ead0b5eae148
INFO Generating SPDX package from image tarball /tmp/doc-build-2746872942/1435e167151f90b7f4abfd416726751c46b8672cc7288507fab7cfa5a05b866c.tar
INFO Successfully extracted 5 files from image tarball /tmp/doc-build-2746872942/1435e167151f90b7f4abfd416726751c46b8672cc7288507fab7cfa5a05b866c.tar
INFO Package describes registry.k8s.io/kube-apiserver:1435e167151f90b7f4abfd416726751c46b8672cc7288507fab7cfa5a05b866c image
INFO Image manifest lists 3 layers
INFO Writing etc/os-release to /tmp/os-release-3381435396
INFO Scan of container layers found debian base image
INFO dbdata is blank
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x8 pc=0x81e0c9]
goroutine 1 [running]:
sigs.k8s.io/bom/pkg/osinfo.(*ContainerScanner).ReadOSPackages(0x8a27c0?, {0xc000461d40?, 0x3?, 0x4?})
/home/shivam/work/bom/pkg/osinfo/container_scanner.go:62 +0xe9
sigs.k8s.io/bom/pkg/spdx.(*spdxDefaultImplementation).PackageFromImageTarball(0xc000026d50?, 0xd255e0, {0xc000028960, 0x5e})
/home/shivam/work/bom/pkg/spdx/implementation.go:797 +0x6dc
sigs.k8s.io/bom/pkg/spdx.(*spdxDefaultImplementation).ImageRefToPackage(0xc0000bc000?, {0xc000026d50, 0x26}, 0x1e?)
/home/shivam/work/bom/pkg/spdx/implementation.go:686 +0x87b
sigs.k8s.io/bom/pkg/spdx.(*SPDX).ImageRefToPackage(...)
/home/shivam/work/bom/pkg/spdx/spdx.go:242
sigs.k8s.io/bom/pkg/spdx.(*defaultDocBuilderImpl).GenerateDoc(0x100?, 0xd30f90, 0xc0000b77a0)
/home/shivam/work/bom/pkg/spdx/builder.go:204 +0x8ae
sigs.k8s.io/bom/pkg/spdx.(*DocBuilder).Generate(0xc0003c5c18, 0xc0000b77a0)
/home/shivam/work/bom/pkg/spdx/builder.go:73 +0x73
sigs.k8s.io/bom/cmd/bom/cmd.generateBOM(0xc0000b4900)
/home/shivam/work/bom/cmd/bom/cmd/generate.go:319 +0x375
sigs.k8s.io/bom/cmd/bom/cmd.AddGenerate.func1(0xc000229180?, {0xc0002012c0?, 0x4?, 0x4?})
/home/shivam/work/bom/cmd/bom/cmd/generate.go:134 +0xab
github.com/spf13/cobra.(*Command).execute(0xc000229180, {0xc000201280, 0x4, 0x4})
/home/shivam/.asdf/installs/golang/1.18.1/packages/pkg/mod/github.com/spf13/[email protected]/command.go:856 +0x67c
github.com/spf13/cobra.(*Command).ExecuteC(0xd285c0)
/home/shivam/.asdf/installs/golang/1.18.1/packages/pkg/mod/github.com/spf13/[email protected]/command.go:974 +0x3b4
github.com/spf13/cobra.(*Command).Execute(...)
/home/shivam/.asdf/installs/golang/1.18.1/packages/pkg/mod/github.com/spf13/[email protected]/command.go:902
sigs.k8s.io/bom/cmd/bom/cmd.Execute()
/home/shivam/work/bom/cmd/bom/cmd/root.go:71 +0x25
main.main()
/home/shivam/work/bom/cmd/bom/main.go:24 +0x17
AFAIK, bom tool couldn’t generate an SBOM for an image that has been built from a scratch image. But, here is the idea, we can find the entrypoint of an image then we can basically run go version -m to list the deps for that to generate an SBOM, can’t we?
Thanks to @jonjohnsonjr, he already prepared a gist for it.
👉 https://gist.github.com/jonjohnsonjr/66c3b5bc75a2bf69e0524d823e3f7228
To be able to generate an SBOMs for images that have been built from scratch base image
When signing images, our sign library will check the signature of an image right after signing. This may lead to a consistency error in the image promoter where the newly created tag may not be available right away as evidenced in this canary run
The image promoter should account for the delay in the registry to reach consistency when verifying the signatures
See this canary run run: https://prow.k8s.io/view/gs/kubernetes-jenkins/logs/ci-promo-tools-image-promo-canary/1511496608326356992
I have a global gitignore that ignores .DS_Store
files.
When I ran bom
against a local git repo, it included .DS_Store
in the output.
$ git config --global --get core.excludesFile
~/.config/git/config
$ git check-ignore --verbose .DS_Store
~/.config/git/ignore:2:.DS_Store .DS_Store
$ bom generate -d . -n http://example.com/
[...]
Relationship: SPDXRef-Package-my-repo CONTAINS SPDXRef-File-86f7fc8c194db4803d37c5b2feecf09c3ee135fb
FileName: .DS_Store
SPDXID: SPDXRef-File-1f11d692242833beff175ce40424d9f280dd3c8a
FileChecksum: SHA1: 0a9e0aee4f8f8d9bf492bfc64d896d157045fe23
FileChecksum: SHA256: ec746d722904dc246254c17bb76bc7e8579af01ea124f38b06df02200c1d503e
FileChecksum: SHA512: a6f5b29d97fb892fd3ca6189c589f7d1af178d212eebf0294848eb734ec7dbb13d69a09c83419199a0936295f0c7ca936b78898a0b6c9fba1745fd288296aebd
LicenseConcluded: NOASSERTION
LicenseInfoInFile: NONE
FileCopyrightText: NOASSERTION
[...]
I also confirmed this behavior is present for non-standard gitignore files (i.e., not named .gitignore
) set locally with (core.excludesFile
).
I expected bom
to ignore the .DS_Store
file, and any others excluded by my global gitignore.
$ git init bom-gitignore-test
Initialized empty Git repository in ~/bom-gitignore-test/.git/
$ cd bom-gitignore-test
$ touch some-file
$ echo some-file > ignores
$ git config --global core.excludesFile ignores # or use --local to only affect this repo
$ git check-ignore --verbose some-file
ignores:1:some-file some-file
$ bom generate -d . -n http://example.com/
[...]
Relationship: SPDXRef-Package-bom-gitignore-test CONTAINS SPDXRef-File-cbbba3a3cb11930313b724a71f4775324a43ff61
FileName: some-file
SPDXID: SPDXRef-File-45e075645e9d6d3346b37d8e0cd826a0b203f055
FileChecksum: SHA1: da39a3ee5e6b4b0d3255bfef95601890afd80709
FileChecksum: SHA256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
FileChecksum: SHA512: cf83e1357eefb8bdf1542850d66d8007d620e4050b5715dc83f4a921d36ce9ce47d0d13c5d85f2b0ff8318d2877eec2f63b931bd47417a81a538327af927da3e
LicenseConcluded: NOASSERTION
LicenseInfoInFile: NONE
FileCopyrightText: NOASSERTION
[...]
Global gitignore files can exist at several paths without being set in the user's gitconfig, so the output of git config --global --get core.excludesFile
is not sufficient alone. Here is an excellent gist with some background.
cat /etc/os-release
): MacOS Montereryuname -a
): darwin 21.2.0As per the email sent to kubernetes-dev[1], please create a SECURITY_CONTACTS
file.
The template for the file can be found in the kubernetes-template repository[2].
A description for the file is in the steering-committee docs[3], you might need
to search that page for "Security Contacts".
Please feel free to ping me on the PR when you make it, otherwise I will see when
you close this issue. :)
Thanks so much, let me know if you have any questions.
(This issue was generated from a tool, apologies for any weirdness.)
[1] https://groups.google.com/forum/#!topic/kubernetes-dev/codeiIoQ6QE
[2] https://github.com/kubernetes/kubernetes-template-project/blob/master/SECURITY_CONTACTS
[3] https://github.com/kubernetes/community/blob/master/committee-steering/governance/sig-governance-template-short.md
Generated SBOM for container image (ubuntu:latest).
Expected PackageName to not contain version string
bom generate -a --image ubuntu:latest -o ubuntu.spdx
Nothing
uname -a output:
Linux kali 5.16.0-kali6-amd64 #1 SMP PREEMPT Debian 5.16.14-1kali2 (2022-03-23) x86_64 GNU/Linux
Running the generate outline
subcommand without an argument doesn't show usage info:
$ bom document outline
FATA You should only specify one file
Running the generate command without any args or flags shows the command usage when required commands are not provided, so I would have expected the outline command to do the same, maybe like:
$ bom document outline
bom document outline → Draw structure of a SPDX document",
This subcommand draws a tree-like outline to help the user visualize
the structure of the bom. Even when an SBOM represents a graph structure,
drawing a tree helps a lot to understand what is contained in the document.
You can define a level of depth to limit the expansion of the entities.
For example set --depth=1 to only visualize only the files and packages
attached directly to the root of the document.
bom will try to add useful information to the oultine but, if needed, you can
set the --spdx-ids to only output the IDs of the entities.
Usage:
bom document outline [flags]
Flags:
-d, --depth int recursion level (default -1)
-h, --help help for outline
--spdx-ids use SPDX identifiers in tree nodes instead of names
Global Flags:
--log-level string the logging verbosity, either 'panic', 'fatal', 'error', 'warning', 'info', 'debug', 'trace' (default "info")
FATA You should only specify one file
You can obviously run bom document outline --help
to see this. The usage command also doesn't indicate there is a required positional argument w/ the file name to outline.
Using latest (installed from go install sigs.k8s.io/bom/cmd/bom@latest
), run:
bom document outline
cat /etc/os-release
): macOS Montereyuname -a
): Darwin jeremys-mbp.lan 21.2.0 Darwin Kernel Version 21.2.0: Sun Nov 28 20:28:41 PST 2021; root:xnu-8019.61.5~1/RELEASE_ARM64_T6000 arm64Add a -C
flag that allows the user to specify a directory to read files from. Just like tar has -C we should add one just like it.
Running bom from a directory and adding single files from another is a bit cumbersome. The usual run involves running the utility from the repo root after the build, but usually, files will be stored in a directory, like ./bin or ./output. Adding the files from these results in the paths recorded in the sbom as eg ./output/binary.exe
which is not clean and could be impractical for verification.
It would be really useful to have glob support in the --file
and --dirs
parameters. So instead of
bom generate --file=output/bom-darwin-amd64 \
--file=output/bom-darwin-arm64 \
--file=output/bom-linux-amd64 \
--file=output/bom-linux-arm64 \
--file=output/bom-windows-amd64.exe
`
You could simply do:
bom generate --file=output/bom*
#### Why is this needed:
Currently, those parameters only take a single file but it would be much more practical to point the generator to a dir and add everything in there.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.