paketo-buildpacks / pip

PIP Cloud Native Buildpack

The Paketo Buildpack for Pip is a Cloud Native Buildpack that installs pip into a layer and places it on the PATH.

The buildpack is published for consumption at gcr.io/paketo-buildpacks/pip and paketobuildpacks/pip.

Behavior

This buildpack always participates.

The buildpack will do the following:

• At build time:
  • Contributes the pip binary to a layer
  • Prepends the pip layer to the PYTHONPATH
  • Adds the newly installed pip location to PATH
• At run time:
  • Does nothing

Configuration

Environment Variable	Description
$BP_PIP_VERSION	Configure the version of pip to install. Buildpack releases (and the pip versions for each release) can be found here.

Note that Pip releases are of the form X.Y instead of X.Y.0, so providing X.Y will attempt to match that exact version. Providing X.Y.Z will select the exact patch version, and providing X.Y.* or ~X.Y will select the latest patch version.

Integration

The Pip CNB provides pip as a dependency. Downstream buildpacks can require the pip dependency by generating a Build Plan TOML file that looks like the following:

[[requires]]

  # The name of the Pip dependency is "pip". This value is considered
  # part of the public API for the buildpack and will not change without a plan
  # for deprecation.
  name = "pip"

  # The version of the Pip dependency is not required. In the case it
  # is not specified, the buildpack will select the latest supported version in
  # the buildpack.toml.
  # If you wish to request a specific version, the buildpack supports
  # specifying a semver constraint in the form of "21.*", "21.0.*", or even
  # "21.0.1".
  version = "21.0.1"

  # The Pip buildpack supports some non-required metadata options.
  [requires.metadata]

    # Setting the build flag to true will ensure that the Pip dependency is
    # available on the $PATH, and the $PYTHONPATH contains the path to pip for
    # subsequent buildpacks during their build phase. If you are writing a
    # buildpack that needs to run Pip during its build process, this flag should
    # be set to true.
    build = true

    # Setting the launch flag to true will ensure that the Pip
    # dependency is available on the $PATH, and the $PYTHONPATH contains the
    # path to pip for the running application. If you are writing an
    # application that needs to run Pip at runtime, this flag should be set to
    # true.
    launch = true

Usage

To package this buildpack for consumption:

$ ./scripts/package.sh --version x.x.x

This will create a buildpackage.cnb file under the build directory which you can use to build your app as follows: pack build <app-name> -p <path-to-app> -b build/buildpackage.cnb -b <other-buildpacks..>.

To run the unit and integration tests for this buildpack:

$ ./scripts/unit.sh && ./scripts/integration.sh

pip's Issues

Failure: Push Buildpackage workflow

Push Buildpackage workflow failed.

Leverage `scribe.Emitter` functionality

This buildpack uses scribe.NewFormattedMapFromEnvironment to log environment variables, when we could instead use logger.EnvironmentVariables. We should make this change as it is simpler and provides an improved log output.

See also paketo-buildpacks/cpython#337

Failure: Update Dependencies workflow

Update Dependencies From Metadata workflow failed.

Conform to Paketo RFC0038: Generate SBOM on filesystem as well as label metadata

To implement Paketo RFC0038, this buildpack will need to store SBOM information the filesystem in addition to the layer metadata. The RFC outlines what these files are and what they should contain.

See also paketo-buildpacks/python#399.

See the nodejs buildpack integration tests for an example on how to test this.

Restructure: Retire the 'require: {python-runtime, requirements}' contract

Background

See RFC 0001 that outlines the proposed restructure of python family of buildpacks.

In #82, this buildpack was made to support multiple contracts - requiring cpython per RFC 0001, and requiring {python-runtime + requirements} for backward compatibility.

Proposal

• Remove {python-runtime + requirements} requirement to be full compliant to the RFC.
• Buildpack API should be as follows:
  • provides: pip
  • requires: cpython during build

pip v0.3.0 dependency contains duplicate pip src files

See the <dependency.tgz> https://github.com/paketo-community/pip/blob/v0.3.0/buildpack.toml#L11-L27
I see the same pip src files duplicated under pip-<version>.tgz. This should be removed if possible

Failure: Approve bot PR workflow

Approve bot PR workflow failed.

Failure: Update Dependencies workflow

Update Dependencies From Metadata workflow failed.

Failure: Update Dependencies workflow

Update Dependencies From Metadata workflow failed.

Confirm to RFC RFC0043: Reproducible builds

To confirm to RFC0043 this buildpack should ensure that builds are reproducible. Specifically it should not include a built_at metadata field. In the tests that leverage this field to assert layer reuse, we should instead compare layer SHA values across rebuilds.

See also the tracking issue: paketo-buildpacks/rfcs#165.

Compile dependency on actual stack images

The dependency that this buildpack provides is compiled on an image that isn't actually the stack image. This could result in subtle bugs - like features not being enabled or enabled features not being available at run-time - due to differences in configuration and available libraries. We should compile the dependency on the actual stack base image rather than a similar image.

Problem running a build with new version of cpython (1.6.0) and pip (0.16.0) buildpacks

Version 0.16.0 of pip buildpack fails installing pip:

[builder] Running build for buildpack paketo-buildpacks/[email protected]
[builder] Looking up buildpack
[builder] Finding plan
[builder] Running build for buildpack Paketo Buildpack for Pip 0.16.0
[builder] Creating plan directory
[builder] Preparing paths
[builder] Running build command
[builder] Paketo Buildpack for Pip 0.16.0
[builder]   Resolving Pip version
[builder]     Candidate version sources (in priority order):
[builder]       BP_PIP_VERSION -> "22.3.0"
[builder]       <unknown>      -> ""
[builder]
[builder]     Selected Pip version (using BP_PIP_VERSION): 22.3.0
[builder]
[builder]   Executing build process
[builder]     Installing Pip 22.3.0
[builder] failed to configure pip:
[builder] Looking in links: /tmp/pip-source2821721444
[builder] Processing /tmp/pip-source2821721444
[builder]   Installing build dependencies: started
[builder]   Installing build dependencies: finished with status 'done'
[builder]   Getting requirements to build wheel: started
[builder]   Getting requirements to build wheel: finished with status 'done'
[builder]   Preparing metadata (pyproject.toml): started
[builder]   Preparing metadata (pyproject.toml): finished with status 'error'
[builder]   error: subprocess-exited-with-error
[builder]
[builder]   × Preparing metadata (pyproject.toml) did not run successfully.
[builder]   │ exit code: 1
[builder]   ╰─> [82 lines of output]
[builder]       /tmp/pip-build-env-l9nbmere/overlay/lib/python3.10/site-packages/setuptools/config/setupcfg.py:508: SetuptoolsDeprecationWarning: The license_file parameter is deprecated, use license_files instead.
[builder]         warnings.warn(msg, warning_class)
[builder]       running dist_info
[builder]       creating /tmp/pip-modern-metadata-3f_ps_la/pip.egg-info
[builder]       writing /tmp/pip-modern-metadata-3f_ps_la/pip.egg-info/PKG-INFO
[builder]       writing dependency_links to /tmp/pip-modern-metadata-3f_ps_la/pip.egg-info/dependency_links.txt
[builder]       writing entry points to /tmp/pip-modern-metadata-3f_ps_la/pip.egg-info/entry_points.txt
[builder]       writing top-level names to /tmp/pip-modern-metadata-3f_ps_la/pip.egg-info/top_level.txt
[builder]       writing manifest file '/tmp/pip-modern-metadata-3f_ps_la/pip.egg-info/SOURCES.txt'
[builder]       reading manifest file '/tmp/pip-modern-metadata-3f_ps_la/pip.egg-info/SOURCES.txt'
[builder]       reading manifest template 'MANIFEST.in'
[builder]       warning: no files found matching 'src/pip/_vendor/pyparsing/diagram/template.jinja2'
[builder]       warning: no files found matching 'docs/docutils.conf'
[builder]       warning: no previously-included files found matching '.coveragerc'
[builder]       warning: no previously-included files found matching '.mailmap'
[builder]       warning: no previously-included files found matching '.appveyor.yml'
[builder]       warning: no previously-included files found matching '.readthedocs.yml'
[builder]       warning: no previously-included files found matching '.pre-commit-config.yaml'
[builder]       warning: no previously-included files found matching 'tox.ini'
[builder]       warning: no previously-included files found matching 'noxfile.py'
[builder]       warning: no files found matching '*.css' under directory 'docs'
[builder]       warning: no previously-included files found matching 'src/pip/_vendor/six'
[builder]       warning: no previously-included files found matching 'src/pip/_vendor/six/moves'
[builder]       warning: no previously-included files matching '*.pyi' found under directory 'src/pip/_vendor'
[builder]       no previously-included directories found matching '.github'
[builder]       no previously-included directories found matching 'docs/build'
[builder]       no previously-included directories found matching 'news'
[builder]       no previously-included directories found matching 'tasks'
[builder]       no previously-included directories found matching 'tests'
[builder]       no previously-included directories found matching 'tools'
[builder]       adding license file 'LICENSE.txt'
[builder]       writing manifest file '/tmp/pip-modern-metadata-3f_ps_la/pip.egg-info/SOURCES.txt'
[builder]       creating '/tmp/pip-modern-metadata-3f_ps_la/pip-22.3.dist-info'
[builder]       Traceback (most recent call last):
[builder]         File "/layers/paketo-buildpacks_cpython/cpython/lib/python3.10/site-packages/pip/_vendor/pep517/in_process/_in_process.py", line 363, in <module>
[builder]           main()
[builder]         File "/layers/paketo-buildpacks_cpython/cpython/lib/python3.10/site-packages/pip/_vendor/pep517/in_process/_in_process.py", line 345, in main
[builder]           json_out['return_val'] = hook(**hook_input['kwargs'])
[builder]         File "/layers/paketo-buildpacks_cpython/cpython/lib/python3.10/site-packages/pip/_vendor/pep517/in_process/_in_process.py", line 164, in prepare_metadata_for_build_wheel
[builder]           return hook(metadata_directory, config_settings)
[builder]         File "/tmp/pip-build-env-l9nbmere/overlay/lib/python3.10/site-packages/setuptools/build_meta.py", line 377, in prepare_metadata_for_build_wheel
[builder]           self.run_setup()
[builder]         File "/tmp/pip-build-env-l9nbmere/overlay/lib/python3.10/site-packages/setuptools/build_meta.py", line 335, in run_setup
[builder]           exec(code, locals())
[builder]         File "<string>", line 26, in <module>
[builder]         File "/tmp/pip-build-env-l9nbmere/overlay/lib/python3.10/site-packages/setuptools/__init__.py", line 87, in setup
[builder]           return distutils.core.setup(**attrs)
[builder]         File "/tmp/pip-build-env-l9nbmere/overlay/lib/python3.10/site-packages/setuptools/_distutils/core.py", line 185, in setup
[builder]           return run_commands(dist)
[builder]         File "/tmp/pip-build-env-l9nbmere/overlay/lib/python3.10/site-packages/setuptools/_distutils/core.py", line 201, in run_commands
[builder]           dist.run_commands()
[builder]         File "/tmp/pip-build-env-l9nbmere/overlay/lib/python3.10/site-packages/setuptools/_distutils/dist.py", line 968, in run_commands
[builder]           self.run_command(cmd)
[builder]         File "/tmp/pip-build-env-l9nbmere/overlay/lib/python3.10/site-packages/setuptools/dist.py", line 1217, in run_command
[builder]           super().run_command(command)
[builder]         File "/tmp/pip-build-env-l9nbmere/overlay/lib/python3.10/site-packages/setuptools/_distutils/dist.py", line 987, in run_command
[builder]           cmd_obj.run()
[builder]         File "/tmp/pip-build-env-l9nbmere/overlay/lib/python3.10/site-packages/setuptools/command/dist_info.py", line 101, in run
[builder]           bdist_wheel = self.get_finalized_command('bdist_wheel')
[builder]         File "/tmp/pip-build-env-l9nbmere/overlay/lib/python3.10/site-packages/setuptools/_distutils/cmd.py", line 305, in get_finalized_command
[builder]           cmd_obj = self.distribution.get_command_obj(command, create)
[builder]         File "/tmp/pip-build-env-l9nbmere/overlay/lib/python3.10/site-packages/setuptools/_distutils/dist.py", line 859, in get_command_obj
[builder]           klass = self.get_command_class(command)
[builder]         File "/tmp/pip-build-env-l9nbmere/overlay/lib/python3.10/site-packages/setuptools/dist.py", line 954, in get_command_class
[builder]           self.cmdclass[command] = cmdclass = ep.load()
[builder]         File "/layers/paketo-buildpacks_cpython/cpython/lib/python3.10/importlib/metadata/__init__.py", line 171, in load
[builder]           module = import_module(match.group('module'))
[builder]         File "/layers/paketo-buildpacks_cpython/cpython/lib/python3.10/importlib/__init__.py", line 126, in import_module
[builder]           return _bootstrap._gcd_import(name[level:], package, level)
[builder]         File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
[builder]         File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
[builder]         File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
[builder]         File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
[builder]         File "<frozen importlib._bootstrap_external>", line 883, in exec_module
[builder]         File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
[builder]         File "/tmp/pip-build-env-l9nbmere/overlay/lib/python3.10/site-packages/wheel/bdist_wheel.py", line 28, in <module>
[builder]           from .macosx_libfile import calculate_macosx_platform_tag
[builder]         File "/tmp/pip-build-env-l9nbmere/overlay/lib/python3.10/site-packages/wheel/macosx_libfile.py", line 43, in <module>
[builder]           import ctypes
[builder]         File "/layers/paketo-buildpacks_cpython/cpython/lib/python3.10/ctypes/__init__.py", line 8, in <module>
[builder]           from _ctypes import Union, Structure, Array
[builder]       ModuleNotFoundError: No module named '_ctypes'
[builder]       [end of output]
[builder]
[builder]   note: This error originates from a subprocess, and is likely not a problem with pip.
[builder] error: metadata-generation-failed
[builder]
[builder] × Encountered error while generating package metadata.
[builder] ╰─> See above for output.
[builder]
[builder] note: This is an issue with the package mentioned above, not pip.
[builder] hint: See above for details.
[builder]
[builder] [notice] A new release of pip available: 22.2.2 -> 22.3.1
[builder] [notice] To update, run: pip3 install --upgrade pip

pack report output:

Pack:
  Version:  0.27.0+git-f4f5be1.build-3382
  OS/Arch:  darwin/amd64

Default Lifecycle Version:  0.14.1

Supported Platform APIs:  0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9

Config:
  default-builder-image = "[REDACTED]"

  [[trusted-builders]]
    name = "[REDACTED]"

Problem occurs either with pip 22.3.0 and 22.3.1.

Expected Behavior

The pip buildpack runs without a problem and the builds moves to another layer of the build.

Current Behavior

The build fails in pip installation layer.

Possible Solution

Steps to Reproduce

Run build where both cpython v1.6.0 and pip 0.16.0 buildpacks would participate.

Motivations

Failure: Go get update workflow

Go get update workflow failed.

Please configure GITBOT

Pivotal provides the Gitbot service to synchronize issues and pull requests made against public GitHub repos with Pivotal Tracker projects.

If you do not want to use Pivotal Tracker to manage this GitHub repo, you do not need to take any action.

If you are a Pivotal employee, you can configure Gitbot to sync your GitHub repo to your Pivotal Tracker project with a pull request.

Steps:

• Add the Toolsmiths-Bots team to have admin access to your repo
• Add the cf-gitbot ([email protected]) user to have owner access to your Pivotal Tracker project
• Create new branch in this repo: cfgitbot-config (an ask+cf@ ticket is the fastest way to get write access if you get a 404)
• Add your new repo and or project to config-production.yml file
• Submit a PR, which will get auto-merged if you've done it right. Detailed steps here

If you are not a pivotal employee, you can request that [email protected] set up the integration for you.

You might also be interested in configuring GitHub's Service Hook for Tracker on your repo so you can link your commits to Tracker stories. You can do this yourself by following the directions at:

https://www.pivotaltracker.com/help/articles/github_integration/

If there are any questions, please reach out to [email protected].

Implement Dependency RFC 0004 for pip

Implement Dependency Management RFC Phase 1 for pip. Check out the RFC for more details and background.

1. Determine dependency source strategy

Background

When possible, dependencies should be used directly from upstream, rather than undergoing any additional compilation or modifications performed by Paketo-maintained code. For each dependency, the corresponding buildpack maintainer group will decide if the dependency can be used directly from upstream, and must identify the location from which the dependency will be pulled from. Some of the Paketo Java buildpacks perform directory stripping during the buildpack build process itself. This could be a viable alternative to performing directory modifications during the dependency management process for maintainers to consider.

• Determine whether the dependency can be used directly from its upstream, rather than undergoing additional compilation/modification.

• If using directly from upstream, note where dependencies will come from
• If compiled or modified, note the decision for this

• Document decision in a language-family level RFC. This decision can be combined in the RFC with the decision made for other buildpacks in the language family.

2. Version retrieval and metadata generation code

Please refer to the retrieval RFC section on version retrieval for details around the API and background.

• Create retrieval code for the dependency

• Will live in the buildpack under: dependency/retrieval/
• Per the RFC, this will be a combination of (1) discovering new dependency versions based on the buildpack.toml, and (2) generating metadata for each new version.
• Note : Per the RFC caveat - if the dependency is to be compiled, the SHA256 and URI field from the metadata should be omitted in this step.

3. Compilation code (if needed)

If the dependency will be compiled/modified, then refer to the compilation RFC section for API details and background.

• Create compilation action

• Will live in the buildpack under: dependency/actions/compile/

4. Dependency testing (optional)

It's up to maintainer discretion if the dependency will be tested. It's recommended to test the dependency if it's been compiled. Check out the testing RFC section for details.

• Add tests for dependency

• Will live in the buildpack under: dependency/test.

5. Makefile Setup

When using the generalized workflows for dependency management down the line, version retrieval and dependency testing will be executed via a Makefile in order to provide the workflow a standardized way to run the code, regardless of what language it was written in. Check out the RFC section for what this should look like.

• Add Makefile

• Will live in the buildpack at: dependency/Makefile

6. Leveraging the new code

This issue serves to set up all the main logic for the dependency management. The work to actually leverage this code and migrate off of the dep-server will be completed in a separate issue once workflows and infrastructure is set up.

Support offline installation of pip

Problem

pip: v0.1.0

When I try to pack build the default_app with offline cypthon buildpack (v0.3.0) and build-plan buildpack with network turned off, it fails with the following error:

pack build offlinepipimage -b <cpython-cnb> -b <pip-cnb> -b <build-plan-cnb> --network=none
...
[builder]   Executing build process
[builder]     Installing Pip 21.0.1
[builder] failed to configure pip:
[builder] Processing /tmp/pip-source841787572
[builder]   Installing build dependencies: started
[builder]   Installing build dependencies: finished with status 'error'
[builder]   ERROR: Command errored out with exit status 1:
[builder]    command: /layers/paketo-community_cpython/cpython/bin/python /layers/paketo-community_cpython/cpython/lib/python3.8/site-packages/pip install --ignore-installed --n
o-user --prefix /tmp/pip-build-env-pflm9poq/overlay --no-warn-script-location --no-binary :none: --only-binary :none: -i https://pypi.org/simple -- setuptools wheel
[builder]        cwd: None
[builder]   Complete output (7 lines):
[builder]   WARNING: Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<pip._vendor.urllib3.connecti
on.HTTPSConnection object at 0x7f9a8fce0730>: Failed to establish a new connection: [Errno -3] Temporary failure in name resolution')': /simple/setuptools/
[builder]   WARNING: Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<pip._vendor.urllib3.connecti
on.HTTPSConnection object at 0x7f9a8fce0970>: Failed to establish a new connection: [Errno -3] Temporary failure in name resolution')': /simple/setuptools/
[builder]   WARNING: Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<pip._vendor.urllib3.connecti
on.HTTPSConnection object at 0x7f9a8fce0b20>: Failed to establish a new connection: [Errno -3] Temporary failure in name resolution')': /simple/setuptools/
[builder]   WARNING: Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<pip._vendor.urllib3.connecti
on.HTTPSConnection object at 0x7f9a8fce0ca0>: Failed to establish a new connection: [Errno -3] Temporary failure in name resolution')': /simple/setuptools/
[builder]   WARNING: Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<pip._vendor.urllib3.connecti
on.HTTPSConnection object at 0x7f9a8fce0df0>: Failed to establish a new connection: [Errno -3] Temporary failure in name resolution')': /simple/setuptools/
[builder]   ERROR: Could not find a version that satisfies the requirement setuptools (from versions: none)
[builder]   ERROR: No matching distribution found for setuptools
[builder]   ----------------------------------------
[builder] ERROR: Command errored out with exit status 1: /layers/paketo-community_cpython/cpython/bin/python /layers/paketo-community_cpython/cpython/lib/python3.8/site-packages
/pip install --ignore-installed --no-user --prefix /tmp/pip-build-env-pflm9poq/overlay --no-warn-script-location --no-binary :none: --only-binary :none: -i https://pypi.org/simple -- setupt
ools wheel Check the logs for full command output.

Proposal

Pip buildpack should support offline installation.
It looks like the dependency of setuptools is not provided with this buildpack. It should be provided with either this buildpack or a via a buildpack of its own.

Expectation

I expect this offline test to pass.

Restructure: Rewrite and Re-architect the pip Buildpack

In an effort to fulfill the re-architecture plan laid out in Python RFC #0001 the following changes need to be made to the pip Buildpack.

• The pip cnb should just install pip.
• It must no longer run the logic for pip install process.
• It must no longer require procfile, and no longer write a start command.
• It must be rewritten using the packit library.
• The API at this point must require cpython OR {python-runtime + requirements}
  (Note: The {python-runtime + requirements} requirement is temporary to provide compatibility with the unconverted pipenv this will be removed when the re-architecture is complete)

As part of this restructure an RFC should be generate that lays out the exact new behavior of the rewritten pip Buildpack.

Ensure we vendor the minimal, sufficient, set of toolchain dependencies for pip applications

In paketo-buildpacks/pip-install#241 we had to vendor wheel in the list of application dependencies in order to get the integration tests to pass. We also vendor wheel here - we think we shouldn't need to do both.

We don't fully understand the python toolchain well enough to know which parts we need to vendor, so this issue is to understand what we need to vendor for pip, and also see if we can avoid requiring applications (like the pip-install integration tests) to also vendor wheel.

Failure: Create Draft Release workflow

Create Draft Release workflow failed.

Support manylinux10 ABI in 3.6.x buildpack by updating to pip 19.x

Versions

What version of Cloud Foundry and CF CLI are you using? (i.e. What is the output of running cf curl /v2/info && cf version?

{
   "name": "",
   "build": "",
   "support": "",
   "version": 0,
   "description": "Cloud Foundry at SAP Cloud Platform",
   "min_cli_version": null,
   "min_recommended_cli_version": null,
   "api_version": "2.141.0",
   "osbapi_version": "2.15",

}

➜  ~ cf version
cf version 6.46.0+29d6257f1.2019-07-09

What version of the buildpack you are using?

1.6.36

If you were attempting to accomplish a task, what was it you were attempting to do?

I am trying to cf push an app with vendored dependencies. These vendored dependencies were downloaded using pip version 19.x. Among these vendored dependencies was cryptography-2.8-cp34-abi3-manylinux2010_x86_64.whl

What did you expect to happen?

I expected the push to succeed.

What was the actual behavior?

The push failed because the vendored dependencies could not be installed:

[2019-10-20T21:56:08.834Z]        Collecting cryptography (from sap-xssec==2.0.6->meh-commons==2.0.26->-r /tmp/app/requirements.txt (line 14))
[2019-10-20T21:56:08.834Z]          Could not find a version that satisfies the requirement cryptography (from sap-xssec==2.0.6->meh-commons==2.0.26->-r /tmp/app/requirements.txt (line 14)) (from versions: )
[2019-10-20T21:56:08.834Z]        No matching distribution found for cryptography (from sap-xssec==2.0.6->meh-commons==2.0.26->-r /tmp/app/requirements.txt (line 14))

I believe this problem occurs because the buildpack mentioned uses an older version of pip (18.x), which does not have support for the manylinux2010 ABI (see PEP 571. Support for this ABI was introduced in pip 19.0 according to the pip changelog.

Thus, packages vendored with a newer version of pip cannot be used with the current version of the Python buildpack. This recently started occuring ion my project due to 2.8 release of the cryptography package, which now provides manylinux2010 along the old manylinux1 wheels.

Steps to reproduce:

• Use python 3.6 buildpack
• Add manylinux2010 wheel to vendor/
• Attempt to push app and install this dependency

Suggested resolution

Update pip to 19.x.

Please confirm where necessary:

• I have included a log output
• My log includes an error message
• I have included steps for reproduction

Stop Checking PyPI Index during Installation

Describe the Enhancement

The buildpack is checking the PyPI index during installation, even though it has the files it needs for installation locally.

Possible Solution

Add --no-index to the pip install command to ensure only local files are considered.

Relevant pip documentation:

pip looks for packages in a number of places: on PyPI (if not disabled via --no-index), in the local filesystem, and in any additional repositories specified via --find-links or --index-url. There is no ordering in the locations that are searched. Rather they are all checked, and the “best” match for the requirements (in terms of version number - see PEP 440 for details) is selected.

Source

Motivation

In air-gapped environments where unallowed network calls time out rather than immediately reject (ex. a Kubernetes cluster with Network Policies), attempting to read from the PyPI index can result in slow builds.

Upgrade to packit v2

Please upgrade to the latest packit v2 release to enable new features and extended support.

How should we handle specific Major.Minor pip version requests?

While working on #82 we came across a Pip versioning issue. This issue applies to code that hasn't landed on the main branch yet.

Pip versions can either be of the form Major.Minor (ex. 21.0) or Major.Minor.Patch (ex. 21.0.1). See the Pip releases for a full list of versions.

The issue here is that if a user requests version 21.0 (via an environment variable such, for example) after the rewrite work in #82, the dependency resolver we use from packit ends up selecting version 21.0.1 since 21.0 isn't a valid major.minor.patch semver version that the resolver tries to find an "exact" match for. The patch versions would get selected since they are technically a "higher" compatible version. This is a problem because 21.0 is a valid version, that could be exactly matched.

The only way a user could get 21.0 to be used would be if they requested 21.0.0 (which doesn't actually exist).

The question here is, should we allow users to request specific versions such as 21.0, or is it OK to grab the next version that has a patch?
If not, what's the best solution here?

Consume Pip dependency from the dep-server

Per paketo-buildpacks/dep-server#45, we should start consuming the Pip dependency from the dep-server (https://api.deps.paketo.io/v1/dependency?name=<dependency>) instead of the https://buildpacks.cloudfoundry.org/dependencies/<dependency>/... location we currently get the dependencies from.

This will make the dependency publishing/consumption process more transparent than the process we use for the dependencies available via the dependency-builds pipeline.

We have already done this switch-over in the Node Engine and Yarn Buildpacks. The outline of what this work will entail can be found in the dep-server issue linked at the top.

Make buildpack compatible with API 0.5

For the new buildpacks in the Python re-architecture work, we have expressed wanting to use buildpack API v0.5. We should make the Pip Buildpack compatible with Buildpack API v0.5 as well so it's consistent with the rest of the buildpacks in the language family.

Currently, the buildpack breaks when upgraded to v0.5 because the build code modifies the buildpack plan, which is read only in the new API version. In order to support this, we will need to refactor the build code so it does not modify the buildpack plan.

Document pip dependency name

Let's add some documentation to the pip-cnb that states what dependencies are provided and how to require them.