paketo-buildpacks / poetry

Poetry Cloud Native Buildpack

gcr.io/paketo-buildpacks/poetry

The Paketo Buildpack for Poetry is a Cloud Native Buildpack that installs Poetry into a layer and places it on the PATH.

The buildpack is published for consumption at gcr.io/paketo-buildpacks/poetry and paketobuildpacks/poetry.

Detection

• Detects when pyproject.toml exists.
• Provides poetry.
• Always requires cpython and pip.
• Optionally requires poetry when BP_POETRY_VERSION is set.

Build

• Contributes the poetry binary to a layer
• Prepends the poetry layer to the PYTHONPATH environment variable
• Adds the newly installed poetry location to the PATH environment variable

Configuration

Environment Variable	Description
$BP_POETRY_VERSION	Configure the version of Poetry to install. Buildpack releases (and the Poetry versions for each release) can be found here.

Integration

The Poetry CNB provides Poetry as a dependency. Downstream buildpacks can require the Poetry dependency by generating a Build Plan TOML file that looks like the following:

[[requires]]

  # The name of the Poetry dependency is "poetry". This value is considered
  # part of the public API for the buildpack and will not change without a plan
  # for deprecation.
  name = "poetry"

  # The Poetry buildpack supports some non-required metadata options.
  [requires.metadata]

    # Optional.
    # When not specified, the buildpack will select the latest supported version from buildpack.toml
    # This buildpack only supports exact version numbers.
    version = "21.0.1"

    # Set to true to ensure that `poetry` is avilable on both `$PATH` and `$PYTHONPATH` for subsequent buildpacks.
    build = true

    # Set to true to ensure that `poetry` is avilable on both `$PATH` and `$PYTHONPATH` for the launch container.
    launch = true

Usage

To package this buildpack for consumption:

$ ./scripts/package.sh --version x.x.x

This will create a buildpackage.cnb file under the build directory which you can use to build your app as follows:

pack build <app-name> \
  --path <path-to-app> \
  --buildpack build/buildpackage.cnb \
  --buildpack <other-buildpacks..>

To run the unit and integration tests for this buildpack:

$ ./scripts/unit.sh && ./scripts/integration.sh

Known issues and limitations

• This buildpack does not work in an offline/air-gapped environment; it requires internet access to install poetry. The impact of this limitation is mitigated by the fact that poetry itself does not support vendoring of dependencies, and so cannot function in an offline/air-gapped environment.

poetry's Issues

Implement Dependency RFC 0004 for poetry

Implement Dependency Management RFC Phase 1 for poetry. Check out the RFC for more details and background.

1. Determine dependency source strategy

Background

When possible, dependencies should be used directly from upstream, rather than undergoing any additional compilation or modifications performed by Paketo-maintained code. For each dependency, the corresponding buildpack maintainer group will decide if the dependency can be used directly from upstream, and must identify the location from which the dependency will be pulled from. Some of the Paketo Java buildpacks perform directory stripping during the buildpack build process itself. This could be a viable alternative to performing directory modifications during the dependency management process for maintainers to consider.

• Determine whether the dependency can be used directly from its upstream, rather than undergoing additional compilation/modification.

• If using directly from upstream, note where dependencies will come from
• If compiled or modified, note the decision for this

• Document decision in a language-family level RFC. This decision can be combined in the RFC with the decision made for other buildpacks in the language family.

2. Version retrieval and metadata generation code

Please refer to the retrieval RFC section on version retrieval for details around the API and background.

• Create retrieval code for the dependency

• Will live in the buildpack under: dependency/retrieval/
• Per the RFC, this will be a combination of (1) discovering new dependency versions based on the buildpack.toml, and (2) generating metadata for each new version.
• Note : Per the RFC caveat - if the dependency is to be compiled, the SHA256 and URI field from the metadata should be omitted in this step.

3. Compilation code (if needed)

If the dependency will be compiled/modified, then refer to the compilation RFC section for API details and background.

• Create compilation action

• Will live in the buildpack under: dependency/actions/compile/

4. Dependency testing (optional)

It's up to maintainer discretion if the dependency will be tested. It's recommended to test the dependency if it's been compiled. Check out the testing RFC section for details.

• Add tests for dependency

• Will live in the buildpack under: dependency/test.

5. Makefile Setup

When using the generalized workflows for dependency management down the line, version retrieval and dependency testing will be executed via a Makefile in order to provide the workflow a standardized way to run the code, regardless of what language it was written in. Check out the RFC section for what this should look like.

• Add Makefile

• Will live in the buildpack at: dependency/Makefile

6. Leveraging the new code

This issue serves to set up all the main logic for the dependency management. The work to actually leverage this code and migrate off of the dep-server will be completed in a separate issue once workflows and infrastructure is set up.

Implement RFC #0007: Python Dependencies

See paketo-buildpacks/rfcs#230

Validate that we can install specific version of poetry

In the pipenv buildpack we observed that we were always installing the latest version of pipenv that is available online. We should validate that the poetry buildpack does not suffer from the same issue.

Confirm to RFC RFC0043: Reproducible builds

To confirm to RFC0043 this buildpack should ensure that builds are reproducible. Specifically it should not include a built_at metadata field. In the tests that leverage this field to assert layer reuse, we should instead compare layer SHA values across rebuilds.

See also the tracking issue: paketo-buildpacks/rfcs#165.

Conform to Paketo RFC0038: Generate SBOM on filesystem as well as label metadata

To implement Paketo RFC0038, this buildpack will need to store SBOM information the filesystem in addition to the layer metadata. The RFC outlines what these files are and what they should contain.

See also paketo-buildpacks/python#399.

See the nodejs buildpack integration tests for an example on how to test this.

Missing label for io.buildpacks.builder.metadata

Expected Behavior

Should allow pack build

Current Behavior

ERROR: failed to build: invalid builder gcr.io/paketo-buildpacks/poetry: builder gcr.io/paketo-buildpacks/poetry:latest missing label io.buildpacks.builder.metadata -- try recreating builder

Possible Solution

Add the label to the build

Steps to Reproduce

Run

pack build --builder=gcr.io/paketo-buildpacks/poetry <anything>

$ pack --version                                                                                         
0.29.0

Motivations

I can't build my Poetry project

Implement Poetry installation

Background

RFC 0006 outlines the proposed addition of support for Poetry in the Python buildpack. In order to conform to the RFC, a new buildpack named poetry should be created.

Proposal

• Create a buildpack using packit that installs poetry.
• The buildpack API should be as follows:
  • provides: poetry
  • requires: cpython and pip during build

Use `tool.poetry.dependencies.python` from `pyproject.toml` to constrain the Python version

The poetry buildpacks do not seem to take the user-requested python version into account.

If you modify this version range, the integration tests fail: https://github.com/paketo-buildpacks/python-start/blob/faec2a13a9a05dc056701fc8f0238abde08bec69/integration/testdata/poetry-dep-only/pyproject.toml#L8.

For example, I might expect python = "3.6" to select an appropriate 3.6.x version from the [email protected] buildpack, but instead it fails even though python 3.6.x versions are available (https://github.com/paketo-buildpacks/cpython/blob/v0.7.5/buildpack.toml).

Note that tool.poetry.dependencies.python is required: https://python-poetry.org/docs/pyproject/#dependencies-and-dev-dependencies

Failure: Go get update workflow

Go get update workflow failed.

Detect is unnecessarily noisy when pyproject.toml file does not exist

Detect should fail silently when pyproject.toml is not present. It should only display an error to the user when the file exists and there is an issue with it.

Example output of running detect against a non-poetry app:

======== Output: paketo-buildpacks/[email protected] ========
pyproject.toml must include [tool.poetry.dependencies.python], see https://python-poetry.org/docs/pyproject/#dependencies-and-dev-dependencies
err:  paketo-buildpacks/[email protected] 

Proposal:

• We should probably return an empty string and nil error from:
  

  poetry/pyproject_parser.go

  Line 30 in 4583111

  _, err = toml.DecodeFile(filepath.Join(path, PyProjectTomlFile), &pyProject)

• In detect.go we should return packit.Fail if there is no error and an empty string.

Poetry 1.2.0 support

Poetry released new major version (with some braking changes inside) that adds new capabilities too. Would be nice pack to support that too.

Describe the Enhancement

I want to be able to reference version 1.2.0 through parameter list. But keep 1.1.13 as a default one for now.

Motivation

Poetry 1.2.0 adds keyring support for external python repositories that enables key-less authentication with various python artifactories.

Failure: Create Draft Release workflow

Create Draft Release workflow failed.

Leverage `scribe.Emitter` functionality

This buildpack uses scribe.NewFormattedMapFromEnvironment to log environment variables, when we could instead use logger.EnvironmentVariables. We should make this change as it is simpler and provides an improved log output.

See also paketo-buildpacks/cpython#337

Support poetry 1.3 and 1.4

The currently supported versions of poetry are 1.3.x and 1.4.x. We should support these.