paketo-buildpacks / libpak

An opinionated extension to the libcnb Cloud Native Buildpack Library

github.com/paketo-buildpacks/libpak

libpak is a Go library with useful functionality for building Paketo-style buildpacks.

Usage

go get github.com/paketo-buildpacks/libpak

License

This library is released under version 2.0 of the Apache License.

libpak's Issues

Remove http query params from destination file name `release-2.x` branch

Describe the Enhancement

Motivation

See #273

Possible Solution

Port to release-2.x

Redo the interface for SBOM generation

Describe the Enhancement

I would like to see the SBOM generation reworked behind a new interface or abstraction. We had some limits imposed when this functionality was added because we didn't want to break compatibility in v1. That is not a concern with v2, so we should rethink the SBOM interface and abstractions in the library.

In particular, I would like to be able to easily swap in/out different SBOM generators w/out causing changes to core code.

Motivation

SBOM is an area where things are still changing rapidly. We need to be flexible. We also need to support multiple tools, because Syft is not the only tool folks want to use.

Avoid including http query parameters in downloaded (http) artifact's filename

Describe the Enhancement

This request is in the context of Paketo's apache-tomcat buildpack.
We are trying to use a presigned S3 url to download tomcat external configurations, setting the url as the value for the environment variable BP_TOMCAT_EXT_CONF_URI. Internally the buildpack calls libpak. DependencyCache's Artifact function.
This function sets the name of the downloaded artifact based on the value returned by the filepath.Base(uri) function call.
However, if we use an http uri with query parameters this call will try to set the filename with the query parameters included.

In our case, the S3 presigned url we are using has very long query parameter values, and when libpak tries to save the downloaded file with these parameters included in the filename, it runs into file name too long error.

Possible Solution

We can parse the uri using net/url's Parse function and then get the clean uri without the query parameters using the Path property. We can also put checks to ensure this is done only for http/https URIs.

Motivation

This would add support for presigned S3 urls to the Tomcat buildpack to provide external configurations.

We already have a PR drafted, let us know if we can mark it ready for review.

Document usage of libpak executables

Document Usage:

• create-package
  • .netrc file supported
• update-build-image-dependency
• update-buildpack-dependency
• update-lifecycle-dependency
• update-package-dependency

libpak should provide the slices.Contains utility

Describe the Enhancement

Some buildpacks try to reproduce this common utility, it'd be better if it was available in libpak in the first place.

Motivation & Possible Solution

You can get this functionality from "golang.org/x/exp/slices" and even copy it from https://cs.opensource.google/go/x/exp/+/master:slices/slices.go

Add a Makefile

Describe the Enhancement

Add a Makefile.

Possible Solution

Basically, copy what libcnb has.

Motivation

We should have a Makefile. It's standard in a lot of Go projects. It also makes it easier to run test, lint, and CI.

Improve behavior of walking file system

What happened?

• What were you attempting to do?

There are a number of buildpack that use this code to walk the file system and produce a snapshot of what a pushed application looks like. This is done to determine if the application has changed.

Many applications use Git and have a .git folder. A easy win would be to skip the .git folder in this check. It doesn't really matter if Git changed, and if it did change it's very likely that an actual application file changed as well.

I think we might even be able to go one step further and use git to determine if there have been file changes, rather than crawling the app tree. If there is a .git folder, we could look at the most recent commit in the log, if that commit is the same as previous runs, then the app should be the same.

• What did you expect to happen?

N/A

• What was the actual behavior? Please provide log output, if possible.

N/A

Build Configuration

• What platform (pack, kpack, tekton buildpacks plugin, etc.) are you
  using? Please include a version.

All

• What buildpacks are you using? Please include versions.

Gradle buildpack is one specific buildpack that uses this functionality, but there are others too.

• What builder are you using? If custom, can you provide the output from pack inspect-builder <builder>?

N/A

• Can you provide a sample app or relevant configuration (buildpack.yml,
  nginx.conf, etc.)?

Any app that uses git source control.

Checklist

• I have included log output.
• The log output includes an error message.
• I have included steps for reproduction.

Bindings for dependency-mapping should allow local files (in the container or in a volume)

It looks like /platforms/bindings of type dependency-mapping (see here) might be useful for short-circuiting the tiresome cache behaviour in paketo buildpacks. Unfortunately, it doesn't work with a file:// URL:

[INFO]     [creator]       GraalVM JDK 8.0.272: Contributing to layer
[INFO]     [creator]         Downloading from file:///platform/bindings/jdk/graalvm.tgz
[INFO]     [creator]     unable to invoke layer creator
[INFO]     [creator]     unable to get dependency jdk
[INFO]     [creator]     unable to download file:///platform/bindings/jdk/graalvm.tgz
[INFO]     [creator]     unable to request file:///platform/bindings/jdk/graalvm.tgz
[INFO]     [creator]     Get "file:///platform/bindings/jdk/graalvm.tgz": unsupported protocol scheme "file"
[INFO]     [creator]     ERROR: failed to build: exit status 1
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------

Add support for the static stack

Describe the Enhancement

See paketo-buildpacks/procfile#175

Possible Solution

We minimally need to create a new const for the stack id. We should probably also create some helper methods like stackHasShell(). I think that is the reason we actually care about the stack.

Don't shell parse all config values

What happened?

We shouldn't shell parse all configuration values. Some values are not shell words and fail to parse, resulting in failed builds. Instead buildpacks that accept shell word configuration should parse these values.

• What were you attempting to do?
  Using the paketo sample jar app: https://github.com/paketo-buildpacks/samples/tree/main/java/jar

  pack build samples/jar --env BP_OCI_DESCRIPTION="some(example)string" --builder paketobuildpacks/builder:base
  

• What did you expect to happen?

  A successful build with label key org.opencontainers.image.description set to value some(example)string

• What was the actual behavior? Please provide log output, if possible.

  Paketo Image Labels Buildpack 2.0.6
    https://github.com/paketo-buildpacks/image-labels
  
  Paketo Image Labels Buildpack 2.0.6
    unable to create configuration resolver
    unable to parse value
    invalid command line string
  

Build Configuration

• What platform (pack, kpack, tekton buildpacks plugin, etc.) are you
  using? Please include a version.

  pack version
  0.16.0+git-e0f6c50.build-1898
  

• What buildpacks are you using? Please include versions.

  Paketo Image Labels Buildpack 2.0.6
    https://github.com/paketo-buildpacks/image-labels
  

• What builder are you using? If custom, can you provide the output from pack inspect-builder <builder>?

  paketobuildpacks/builder:base
  

• Can you provide a sample app or relevant configuration (buildpack.yml,
  nginx.conf, etc.)?

  https://github.com/paketo-buildpacks/samples/tree/main/java/jar

Checklist

• I have included log output.
• The log output includes an error message.
• I have included steps for reproduction.

Layer contributors should handle when an uncached layer becomes cached

Current Behavior

Given A layer from a previous build that was cache=false and launch=true
Given A buildpack wants to contribute a layer that is identical in every way except that cache=false
When The buildpack tries to use a LayerContributor to provide this layer
Then The resulting layer is empty, resulting in a failed build

===> EXPORTING
Reusing layer 'paketo-buildpacks/ca-certificates:helper'
Adding layer 'paketo-buildpacks/bellsoft-liberica:helper'
Adding layer 'paketo-buildpacks/bellsoft-liberica:java-security-properties'
ERROR: failed to export: layer 'paketo-buildpacks/bellsoft-liberica:jre' is cache=true but has no contents

Expected Behavior

If the expected layer is cached, the LayerContributor should check for layer contents and contribute the layer if none exist.

Clarify default settings versus user specified settings in the config table

Describe the Enhancement

The config table is a little confusing in that we don't specify what's a default setting versus user-supplied. This makes it a little tricky sometimes to understand what value is going to be applied, in particular with BP_JVM_VERSION.

Log without setting BP_JVM_VERSION explicitly:

[INFO]     [creator]     Paketo Buildpack for BellSoft Liberica 10.5.2
[INFO]     [creator]       https://github.com/paketo-buildpacks/bellsoft-liberica
[...]
[INFO]     [creator]         $BP_JVM_TYPE                 JRE                                                          the JVM type - JDK or JRE
[INFO]     [creator]         $BP_JVM_VERSION              17                                                           the Java version
[...]
[INFO]     [creator]         Using Java version 21 extracted from MANIFEST.MF

Log with setting BP_JVM_VERSION to 17 explicitly.

[INFO]     [creator]     Paketo Buildpack for BellSoft Liberica 10.5.2
[INFO]     [creator]       https://github.com/paketo-buildpacks/bellsoft-liberica
[...]
[INFO]     [creator]         $BP_JVM_TYPE                 JRE                                                          the JVM type - JDK or JRE
[INFO]     [creator]         $BP_JVM_VERSION              17                                                           the Java version
[...]
[INFO]     [creator]         Using Java version 17 from BP_JVM_VERSION

Note how the last line shows that they are different because it impacts which version is selected. In both cases $BP_JVM_VERSION is logged as 17, but only if I set it explicitly the variable is used to determine the runtime jdk.

Possible Solution

We need a way to specify when a value is a default value versus user-supplied.

Options:

1. Add an asterisk after default values. Add a line below the table indicating items marked with an asterisk are default values.
2. Both add an asterisk and add some colors.
3. Do nothing, or maybe just indicate below the table that this is the list of combined settings, and don't worry about trying to differentiate.
4. Something else? 🤔

Things I'd prefer not to do:

1. I don't want to add another column to the table, cause the table is already a bit large and hard to read.
2. I don't want to do something that would make existing columns wider, like showing both the default and user-supplied versions. Some of the columns are already very wide, this would make that worse.
3. I don't want to add just colors because that's not friendly for someone that's color blind or that just has colors disabled.

Motivation

Spun out of libjvm issue -> paketo-buildpacks/libjvm#350

FileListing fails if there is a symlinked folder in the root

What happened?

	it("create listing as hash with non-regular file", func() {
		Expect(ioutil.WriteFile(filepath.Join(path, "alpha.txt"), []byte{}, 0644)).To(Succeed())
		Expect(os.MkdirAll(filepath.Join(path, "test-directory"), 0755)).To(Succeed())
		Expect(ioutil.WriteFile(filepath.Join(path, "test-directory", "bravo.txt"), []byte{}, 0644)).To(Succeed())
		Expect(os.Symlink(filepath.Join(path, "test-directory"), filepath.Join(path, "symlink-test-dir")))

		e, err := sherpa.NewFileListing(path)
		Expect(err).NotTo(HaveOccurred())

		Expect(e).To(HaveLen(4))
	})

• What were you attempting to do?

If the root that is traversed contains a symlink to a folder, the traversal will fail.

• What did you expect to happen?

Handle non-regular files like a folder.

• What was the actual behavior? Please provide log output, if possible.

It attempts to read and take a sha256 hash of the symlinked folder which fails.

FAIL |         Unit/libpak/sherpa/FileListing/create_listing_as_hash_with_non-regular_file (0.00s)
     |             file_listing_test.go:67:
     |                 Unexpected error:
     |                     <*fmt.wrapError | 0xc0004a8020>: {
     |                         msg: "unable to create file listing\nunable to hash file /private/var/folders/17/0cr7w_g53klgrg14c2xvkvdh0000gn/T/file-listing694936971/symlink-test-dir\nread /private/var/folders/17/0cr7w_g53klgrg14c2xvkvdh0000gn/T/file-listing694936971/symlink-test-dir: is a directory",
     |                         err: <*fmt.wrapError | 0xc0004a8000>{
     |                             msg: "unable to hash file /private/var/folders/17/0cr7w_g53klgrg14c2xvkvdh0000gn/T/file-listing694936971/symlink-test-dir\nread /private/var/folders/17/0cr7w_g53klgrg14c2xvkvdh0000gn/T/file-listing694936971/symlink-test-dir: is a directory",
     |                             err: <*fs.PathError | 0xc000480030>{
     |                                 Op: "read",
     |                                 Path: "/private/var/folders/17/0cr7w_g53klgrg14c2xvkvdh0000gn/T/file-listing694936971/symlink-test-dir",
     |                                 Err: <syscall.Errno>0x15,
     |                             },
     |                         },
     |                     }
     |                     unable to create file listing
     |                     unable to hash file /private/var/folders/17/0cr7w_g53klgrg14c2xvkvdh0000gn/T/file-listing694936971/symlink-test-dir
     |                     read /private/var/folders/17/0cr7w_g53klgrg14c2xvkvdh0000gn/T/file-listing694936971/symlink-test-dir: is a directory
     |                 occurred

Checklist

• I have included log output.
• The log output includes an error message.
• I have included steps for reproduction.

Image build without previous-image setting fails with interface conversion error

After PR #261 it looks like that image creation without any previous image fails with an error.

Expected Behavior

The image can be built without any previous image.

Current Behavior

The following error occurs during image build:

panic: interface conversion: interface {} is time.Time, not string

goroutine 1 [running]:
github.com/paketo-buildpacks/libpak.(*LayerContributor).Equals(0xc00011cc00?, 0x2c5?, 0x300?)
	/home/runner/go/pkg/mod/github.com/paketo-buildpacks/[email protected]/layer.go:142 +0x4f7
github.com/paketo-buildpacks/libpak.(*LayerContributor).checkIfMetadataMatches(0xc000233fd8, {{0x0, 0x0, 0x1}, 0xc00027a120, {0xc0000b5c68, 0x3}, {0xc00020e960, 0x2f}, 0xc000213800, ...})
	/home/runner/go/pkg/mod/github.com/paketo-buildpacks/[email protected]/layer.go:122 +0x2ab
github.com/paketo-buildpacks/libpak.(*LayerContributor).Contribute(0xc000233fd8, {{0x0, 0x0, 0x1}, 0xc00027a120, {0xc0000b5c68, 0x3}, {0xc00020e960, 0x2f}, 0xc000213800, ...}, ...)
	/home/runner/go/pkg/mod/github.com/paketo-buildpacks/[email protected]/layer.go:75 +0x145
github.com/paketo-buildpacks/libpak.(*DependencyLayerContributor).Contribute(0xc000234280, {{0x0, 0x0, 0x1}, 0xc00027a120, {0xc0000b5c68, 0x3}, {0xc00020e960, 0x2f}, 0xc000213800, ...}, ...)
	/home/runner/go/pkg/mod/github.com/paketo-buildpacks/[email protected]/layer.go:254 +0x2a8
github.com/paketo-buildpacks/libjvm.JRE.Contribute({{0xc0000b5700, 0xa}, {{0x7c0e92, 0x22}, {0x0, 0x0, 0x0}, {0x8467a0, 0xc000154f80}}, 0x1, ...}, ...)
	/home/runner/go/pkg/mod/github.com/paketo-buildpacks/[email protected]/jre.go:76 +0x129
github.com/buildpacks/libcnb.Build({0x8472a0, 0xc0001427e0}, {0xc0000e57b8, 0x4, 0x847320?})
	/home/runner/go/pkg/mod/github.com/buildpacks/[email protected]/build.go:280 +0x1d47
github.com/buildpacks/libcnb.Main({0x8472c0, 0xc0001427d0}, {0x8472a0, 0xc0001427e0}, {0xc0000e57b8, 0x4, 0x4})
	/home/runner/go/pkg/mod/github.com/buildpacks/[email protected]/main.go:47 +0x2bd
github.com/paketo-buildpacks/libpak.Main({0x847260?, 0xa7b2a8?}, {0x847240?, 0xc0001ae500?}, {0x0, 0x0, 0x8467a0?})
	/home/runner/go/pkg/mod/github.com/paketo-buildpacks/[email protected]/main.go:28 +0x3ed
main.main()
	/home/runner/work/bellsoft-liberica/bellsoft-liberica/cmd/main/main.go:35 +0x20e
ERROR: failed to build: exit status 2

Possible Solution

Rollback the base image maximum 0.3.279-base

Steps to Reproduce

Create an image with the following command inside a buildpacks/base based container:

/cnb/lifecycle/creator \
  -no-color \
  -skip-restore \
  -app=/workspace \
  -process-type=web \
  -run-image=${RUN_IMAGE} \
  ${APP_IMAGE}:${TAG_NAME}

Support a platform-dependent dependency mapping

Currently, online packaged buildpacks must download their contributed dependency from the open internet. There are situations where the URIs for those download locations are blocked from within a corporate network. libpack's DependencyCache should noticed the existence of an artifact or artifacts in /platform that defines a mapping between a buildpack's dependency (id/version) and a different URL that it should be downloaded from. This should happen transparently (save for a different log message) to both the buildpack author using DependencyCache and the application developer using a buildpack that uses DependencyCache.

NewLayerContributor does not log expected logs

What happened?

NewLayerContributor does not accept a logger instance. As a result bard.Logger does not have a proper file descriptor to log to. As a result contributors created using this don't show proper logs during the build process around caching etc.

• What were you attempting to do?
  Use NewLayerContributor
• What did you expect to happen?
  See logs around it
• What was the actual behavior? Please provide log output, if possible.
  No logs

Build Configuration

• What platform (pack, kpack, tekton buildpacks plugin, etc.) are you
  using? Please include a version.
  pack and kpack
• What buildpacks are you using? Please include versions.
  Internal
• What builder are you using? If custom, can you provide the output from pack inspect-builder <builder>?

Checklist

• I have included log output.
• The log output includes an error message.
• I have included steps for reproduction.

Split `create-package` out of libpak

Describe the Enhancement

We need to split create-package out into its own repo and have it take a dependency on libpak, instead of being included with libpak. This will make it more flexible. It should also enable us to have actual releases of create-package, so it can be installed w/out requiring a Go SDK. It should also more easily facilitate some documentation on the tool.

Motivation

See above.

FileLists and MavenFileLists Metadata is too big

A user has reported that the file lists and maven file list metadata used for some layer caches is bigger than it needs to be. The suggestion from this user was that we hash the file list once created and store that instead. That will give us the same behavior in a much smaller footprint.

libpak should generate an SBOM based on the buildpack.toml for dependencies

Currently libpak always hardcodes and outputs syft json for dependencies at

libpak/buildpack.go

Line 112 in 5cea674

func (b BuildpackDependency) AsSyftArtifact() (sbom.SyftArtifact, error) {

We should change this to output sbom files based on the sbom media types for a particular buildpack.

Improve the UX around bindings

What happened?

It's super common for users to attach a binding at build time to pack build, but then forget or not understand that it's also required to have the binding present at runtime. When run without the binding at runtime, the service will typically not work because it is missing credentials and other critical info.

• What were you attempting to do?

• What did you expect to happen?

What I think would be reasonable is for libpak to log binding information, like it does with configuration information (the ConfigurationResolver.

Obviously, we wouldn't log the actual value, since that is sensitive information, but we could log the bindings that are visible and perhaps the binding keys that are present, that way a user can see they have bindings set up correctly (getting this set up is tricky).

We can also log a message like, "Buidlpacks do not store bindings in generated images. Please remember to also mount your bindings when you actually run your container images.". Perhaps even link to some docs that explains it more.

• What was the actual behavior? Please provide log output, if possible.

Nothing. Bindings will be present and may or may not be used. It's confusing and difficult to debug when there are issues.

Checklist

• I have included log output.
• The log output includes an error message.
• I have included steps for reproduction.

License information missing from BOM

When generating an sBOM report using Syft for a container image built with the new-relic buildpack, the "New Relic Java Agent" element does not correctly set license information so the Syft tool can export it correctly in an sBOM file.

Expected Behavior

We expect the sBOM file generated by Syft on an image has the licenseConcluded and licenseDeclared elements of the "New Relic Java Agent" element in the "packages" array set correctly.

Current Behavior

Using Syft on an image generated with Paketo with new-relic buildpack included, the "New Relic Java Agent" has incorrect (LicenseRef-) value for the licenseConcluded and licenseDeclared elements.

  {
   "name": "New Relic Java Agent",
   "SPDXID": "SPDXRef-Package-UnknownPackage-New-Relic-Java-Agent-51f555dbd7cae650",
   "versionInfo": "8.2.0",
   "downloadLocation": "NOASSERTION",
   "sourceInfo": "acquired package info from SBOM: /layers/sbom/launch/paketo-buildpacks_new-relic/new-relic-java/sbom.syft.json",
   "licenseConcluded": "LicenseRef-",
   "licenseDeclared": "LicenseRef-",
   "copyrightText": "NOASSERTION",
   "externalRefs": [
    {
     "referenceCategory": "SECURITY",
     "referenceType": "cpe23Type",
     "referenceLocator": "cpe:2.3:a:newrelic:java-agent:8.2.0:*:*:*:*:*:*:*"
    },
    {
     "referenceCategory": "SECURITY",
     "referenceType": "cpe23Type",
     "referenceLocator": "cpe:2.3:a:New_Relic_Java_Agent:New_Relic_Java_Agent:8.2.0:*:*:*:*:*:*:*"
    },
    {
     "referenceCategory": "PACKAGE-MANAGER",
     "referenceType": "purl",
     "referenceLocator": "pkg:generic/[email protected]?arch=amd64"
    }
   ]
  }

Possible Solution

Steps to Reproduce

1. Generate a container image with Paketo Buildpacks following instruction in this repository.
2. Run syft on the generated image: syft <image>
3. Inspect sbom for the "New Relic Java Agent" element in the "packages" array

Motivations

create-package with dependency doesn't work

Expected Behavior

When to use the create-package by

go install -ldflags="-s -w" github.com/paketo-buildpacks/libpak/cmd/[email protected]

The targe folder looks

Current Behavior

When to use the create-package by

go install -ldflags="-s -w" github.com/paketo-buildpacks/libpak/cmd/create-package@latest

Possible Solution

Steps to Reproduce

Motivations

Enable more flexible binding searches

Describe the Enhancement

libpak provides ways to search for bindings based on the name, type and provider (exact match). We should add a more flexible method that takes a callback. If the callback returns true, then it matches. If the callback returns false, then it doesn't match. This would allow buildpacks to have more control over how they match bindings.

Possible Solution

See above.

Motivation

Provide more flexibility to buildpacks when matching bindings.

Don't add file mod times to file lists

We should not store mod times in file lists in layer metadata. On platforms like kpack that clone the repo before each build, mod times will change for every build. This in turn will cause cached layers with file lists in their metadata to be invalidated on every build, even when reusing the layer is the correct/performant behavior.

libpak fails to use cached dependencies when no CPEs are specified.

After creating a buildpack using create-package with --include-dependencies then running pack buildpack package .... The resulting buildpack when included fails to use the cached dependencies and try to download the artifacts.

Expected Behavior

The buildpack continues without trying to download the dependency.

Current Behavior

The buildpack tries to download the dependency but fails in an offline environment.

Steps to Reproduce

Create the buildpack with a dependency, ensuring there are no CPES specified.
Prepare the offline buildpack with create-package and --include-dependencies
Package the buildpack pack buildpack package ...
Include the buildpack for a build. pack build ... -b <thebuildpack> --network none

Motivations

Any dependencies without CPES can not be used offline.

Note

From what we discovered, the CPE list of the libpak.BuildpackDependency retrieved from DependencyResolver hasCPEs: []string{nil} while the result from parsing the cache toml /dependencies/<sha>.toml returned a CPEs: []string{}. Which caused the comparison to fail.

LibPak Version: 1.61.0

Provide Global Mechanism to Disable SBOM Generation

Describe the Enhancement

Implement RFC 0044 by checking for BP_DISABLE_SBOM as one of the first things in build.go and, if set, return early.

Possible Solution

This needs support in both libpak and libbs. Any point where we generate SBOM information or run syft (or other tools), needs to be aware of the opt-out and should skip generating SBOM information.

In addition, the container needs to be flagged as having opted out of SBOM generation so it's clear this was due to a user request.

Motivation

At the moment, this remains unclear. If you find this issue and it is of interest to you. Please post a comment and include some details about why you'd like this setting, your use case and how it impacts you. If we get enough user interest, we can implement this feature.

create-package should support multi arch buildpacks

create-package should support multi arch buildpacks

it should understand the targets entries in buildpack.toml

and then be able to only copy from linux/a(md|rm)64/bin to bin

Possible Solution

after calling pre package, move the proper files under bin/

Motivation

support occam testing fwk that calls pre-package this way:

https://github.com/paketo-buildpacks/occam/blob/main/packagers/libpak.go#L19

which is in turn used this way:

	libpakBuildpackStore := occam.NewBuildpackStore().WithPackager(packagers.NewLibpak())
	watchexecBuildpack, err = libpakBuildpackStore.Get.
		Execute("github.com/paketo-buildpacks/watchexec")

Create some testing utils for working with and asserting against TOML

Describe the Enhancement

Buildpacks have a lot of TOML. Working with it is kind of a pain, especially when trying to do comparisons in tests.

We should take some time to write some helper code for working with and asserting against TOML.

Motivation

I think this will simplify our test cases and make them easier to read.

Can't build twice

What happened?

I am exploring buildpacks. I wanted to build my image again, but the second build breaks.

$ curl -G https://start.spring.io/starter.zip\?type=gradle-project -d dependencies=web -d javaVersion=11 -o demo.zip
$ unzip demo.zip -d .
$ pack config default-builder paketobuildpacks/builder:base # according to https://paketo.io/docs/buildpacks/language-family-buildpacks/java/#about-the-examples
$ pack build demo --path $PWD. # this one works as expected
$ pack build demo --path $PWD
# ...
Paketo Maven Buildpack 5.3.0
  unable to create application layer
  failed to generate expected metadata
  unable to determine java version
  error executing 'javac -version':
   Combined Output: :
  unable to start PTY
  exec: "javac": executable file not found in $PATH
ERROR: failed to build: exit status 1
ERROR: failed to build: executing lifecycle: failed with status code: 145

Forgive me. I have just begun to use buildpacks. I assume that the build is run in complete isolation. The fact that I am using sdkman to install and select the Java version on the host should not get in the way of buildpacks, should it? The first build works as advertised though. 🤔

$ pack version
0.19.0+git-360dbae.build-2550

Support SHA512 for buildpack dependencies

Describe the Enhancement

Make it possible to define checksum = "sha512:..." in the buildpack.toml metadata.dependencies (instead of just {sha256,md5})

Possible Solution

Motivation

Some dependency providers (in our case Apache Spark collected from archive FTP) publish sha512 only, in such case it's not a trival task to use it in buidlpacks.

Consolidate on one TOML library

There are presently two TOML libraries being used:

• github.com/BurntSushi/toml
• github.com/pelletier/go-toml

It is not entirely consistent when they are being used, but it looks like github.com/BurntSushi/toml does not have a Marshal method and some of the code in libpak is calling Marshal. github.com/BurntSushi/toml does have an Encoder though, so this may require switching from calling Marshal to using the Encoder.

Including two libraries increases the size of the binaries that we are generating. If we can consolidate to one library, it'll reduce the size of the binaries we are creating.

The libcnb library uses just github.com/BurntSushi/toml so it would be best if we can consolidate on using this library in libpak & Paketo buildpacks. Otherwise, we'd have to suggest switching in libcnb to github.com/pelletier/go-toml as well to get any benefit.

The risk here is if encoding with github.com/pelletier/go-toml is different than encoding with github.com/BurntSushi/toml. For example, if it handles encoding in a non-compatible way.