paketo-buildpacks / pipenv

Pipenv Cloud Native Buildpack

The Paketo Buildpack for Pipenv is a Cloud Native Buildpack that installs pipenv into a layer and makes it available on the PATH.

The buildpack is published for consumption at gcr.io/paketo-buildpacks/pipenv.

Behavior

This buildpack always participates.

The buildpack will do the following:

• At build time:
  • Contributes the pipenv binary to a layer
  • Prepends the pipenv layer to the PYTHONPATH
  • Adds the newly installed pipenv location to PATH
• At run time:
  • Does nothing

Configuration

Environment Variable	Description
$BP_PIPENV_VERSION	Configure the version of pipenv to install. Buildpack releases (and the supported pipenv versions for each release) can be found here.

Integration

The Pipenv CNB provides pipenv as a dependency. Downstream buildpacks can require the pipenv dependency by generating a Build Plan TOML file that looks like the following:

[[requires]]

  # The name of the Pipenv dependency is "pipenv". This value is considered
  # part of the public API for the buildpack and will not change without a plan
  # for deprecation.
  name = "pipenv"

  # The Pipenv buildpack supports some non-required metadata options.
  [requires.metadata]
    
    # Use `version` to request a specific version of `pipenv`.
    # This buildpack supports specifying a semver constraint in the form of "2018.*", "2018.11.*",
    # or even "2018.11.26".
    # Optional, defaults to the latest version of `pipenv` found in the `buildpack.toml` file.
    version = "2018.11.26"

    # When `build` is true, this buildpack will ensure that `pipenv` is available
    # on the `$PATH` for later buildpacks.
    # Optional, default false.
    build = true

    # When `launch` is true, this buildpack will ensure that `pipenv` is available
    # on the `$PATH` for the running application.
    # Optional, default false.
    launch = true

Limitations

This buildpack requires internet connectivity to install pipenv. Installation in an air-gapped environment is not supported.

The dependency contained in buildpack.toml is not actually used, as pipenv is installed directly from the internet. However, the rest of the dependency metadata is used (e.g. for generating an SBOM). This will be addressed in upcoming work which will change the way dependencies are consumed by buildpacks.

Usage

To package this buildpack for consumption:

$ ./scripts/package.sh --version x.x.x

This will create a build/buildpackage.cnb file which you can use to build your app as follows:

pack build <app-name> \
  --path <path-to-app> \
  --buildpack <cpython-buildpack> \
  --buildpack <pip-buildpack> \
  --buildpack build/buildpackage.cnb \
  --buildpack <some-pipenv-consumer-buildpack>

To run the unit and integration tests for this buildpack:

$ ./scripts/unit.sh && ./scripts/integration.sh

pipenv's Issues

Failure: Update Dependencies workflow

Update Dependencies From Metadata workflow failed.

Integration Test fails for v0.0.115 as of date

When I checkout v0.0.115 and run ./scripts/integration.sh, it fails with App failed to start: app failed to start: testdata/pipfile_lock

Implement Dependency RFC 0004 for pipenv

Implement Dependency Management RFC Phase 1 for pipenv. Check out the RFC for more details and background.

1. Determine dependency source strategy

Background

When possible, dependencies should be used directly from upstream, rather than undergoing any additional compilation or modifications performed by Paketo-maintained code. For each dependency, the corresponding buildpack maintainer group will decide if the dependency can be used directly from upstream, and must identify the location from which the dependency will be pulled from. Some of the Paketo Java buildpacks perform directory stripping during the buildpack build process itself. This could be a viable alternative to performing directory modifications during the dependency management process for maintainers to consider.

• Determine whether the dependency can be used directly from its upstream, rather than undergoing additional compilation/modification.

• If using directly from upstream, note where dependencies will come from
• If compiled or modified, note the decision for this

• Document decision in a language-family level RFC. This decision can be combined in the RFC with the decision made for other buildpacks in the language family.

2. Version retrieval and metadata generation code

Please refer to the retrieval RFC section on version retrieval for details around the API and background.

• Create retrieval code for the dependency

• Will live in the buildpack under: dependency/retrieval/
• Per the RFC, this will be a combination of (1) discovering new dependency versions based on the buildpack.toml, and (2) generating metadata for each new version.
• Note : Per the RFC caveat - if the dependency is to be compiled, the SHA256 and URI field from the metadata should be omitted in this step.

3. Compilation code (if needed)

If the dependency will be compiled/modified, then refer to the compilation RFC section for API details and background.

• Create compilation action

• Will live in the buildpack under: dependency/actions/compile/

4. Dependency testing (optional)

It's up to maintainer discretion if the dependency will be tested. It's recommended to test the dependency if it's been compiled. Check out the testing RFC section for details.

• Add tests for dependency

• Will live in the buildpack under: dependency/test.

5. Makefile Setup

When using the generalized workflows for dependency management down the line, version retrieval and dependency testing will be executed via a Makefile in order to provide the workflow a standardized way to run the code, regardless of what language it was written in. Check out the RFC section for what this should look like.

• Add Makefile

• Will live in the buildpack at: dependency/Makefile

6. Leveraging the new code

This issue serves to set up all the main logic for the dependency management. The work to actually leverage this code and migrate off of the dep-server will be completed in a separate issue once workflows and infrastructure is set up.

Failure: Go get update workflow

Go get update workflow failed.
Please take a look to ensure CVE patches can be released. (cc @pivotal-cf/commercial-buildpacks).

Please configure GITBOT

Pivotal provides the Gitbot service to synchronize issues and pull requests made against public GitHub repos with Pivotal Tracker projects.

If you do not want to use Pivotal Tracker to manage this GitHub repo, you do not need to take any action.

If you are a Pivotal employee, you can configure Gitbot to sync your GitHub repo to your Pivotal Tracker project with a pull request.

Steps:

• Add the Toolsmiths-Bots team to have admin access to your repo
• Add the cf-gitbot ([email protected]) user to have owner access to your Pivotal Tracker project
• Create new branch in this repo: cfgitbot-config (an ask+cf@ ticket is the fastest way to get write access if you get a 404)
• Add your new repo and or project to config-production.yml file
• Submit a PR, which will get auto-merged if you've done it right. Detailed steps here

If you are not a pivotal employee, you can request that [email protected] set up the integration for you.

You might also be interested in configuring GitHub's Service Hook for Tracker on your repo so you can link your commits to Tracker stories. You can do this yourself by following the directions at:

https://www.pivotaltracker.com/help/articles/github_integration/

If there are any questions, please reach out to [email protected].

Failure: Create Draft Release workflow

Create Draft Release workflow failed.

Restructure: rewrite pipenv buildpack

Background

See RFC 0001 that outlines the proposed restructure of python family of buildpacks. In order to conform to the RFC, the pipenv buildpack needs to be restructured. The existing pipenv buildpack transforms Pipfile into a requirements.txt file for consumption by downstream buildpacks. With the restructuring, the pipenv buildpack should have the sole responsibility of providing the pipenv tool for downstream users/buildpacks.

Proposal

• Rewrite using packit library.
• The pipenv cnb should just install pipenv using pip
• It must no longer convert Pipfile to requirements.txt
• The buildpack API should be as follows:
  • provides: pipenv
  • requires: cpython and pip during build
• Record in an RFC the behavior of this buildpack

Conform to Paketo RFC0038: Generate SBOM on filesystem as well as label metadata

To implement Paketo RFC0038, this buildpack will need to store SBOM information the filesystem in addition to the layer metadata. The RFC outlines what these files are and what they should contain.

See also paketo-buildpacks/python#399.

See the nodejs buildpack integration tests for an example on how to test this.

Failure: Push Buildpackage workflow

Push Buildpackage workflow failed.

Document pipenv dependency name

Let's add some documentation to the pipenv-cnb that states what dependencies are provided and how to require them.

Failure: Update Dependencies workflow

Update Dependencies From Metadata workflow failed.

Implement RFC #0007: Python Dependencies

paketo-buildpacks/rfcs#230

Consume Pipenv dependency from the dep-server

Per paketo-buildpacks/dep-server#45, we should start consuming the Pipenv dependency from the dep-server (https://api.deps.paketo.io/v1/dependency?name=<dependency>) instead of the https://buildpacks.cloudfoundry.org/dependencies/<dependency>/... location we currently get the dependencies from.

This will make the dependency publishing/consumption process more transparent than the process we use for the dependencies available via the dependency-builds pipeline.

We have already done this switch-over in the Node Engine and Yarn Buildpacks. The outline of what this work will entail can be found in the dep-server issue linked at the top.

Leverage `scribe.Emitter` functionality

This buildpack uses scribe.NewFormattedMapFromEnvironment to log environment variables, when we could instead use logger.EnvironmentVariables. We should make this change as it is simpler and provides an improved log output.

See also paketo-buildpacks/cpython#337

Upgrade to packit v2

Please upgrade to the latest packit v2 release to enable new features and extended support.

Support offline installation of pipenv

Currently unable to run pip install offline because of the same issue we are seeing for pip: paketo-buildpacks/pip#111

The tests have already been written in PR: #88 but are currently in Pend.

Confirm to RFC RFC0043: Reproducible builds

To confirm to RFC0043 this buildpack should ensure that builds are reproducible. Specifically it should not include a built_at metadata field. In the tests that leverage this field to assert layer reuse, we should instead compare layer SHA values across rebuilds.

See also the tracking issue: paketo-buildpacks/rfcs#165.