Comments (15)
I think you can test that yourself. Try installing pendulum (same author as Poetry, packaged w/ Poetry) into a Python 3.10 virtualenv.
from python-package-template.
Other packaging and env-management tools recommended during conversations with @hynek and @behnazh, are flit and asdf and pyenv. Also, note this comment for more references.
Another recommended read in this context is the post setup.py vs requirements.txt and Python Application Dependency Management.
from python-package-template.
Consider also the discussion on PEP 665: A file format to list Python dependencies for reproducibility of an application (and ensuing discussion).
from python-package-template.
The PEP 665 makes a lot of sense. It also mentions how npm has solved the reproducibility problem by using package.json.
But for the time being Poetry seems to be doing well enough in dependency management and reproducibility I think.
from python-package-template.
Interesting conversation regarding flit
on python.org.
from python-package-template.
Interesting conversation regarding
flit
on python.org.
Interesting indeed. I didn't find any solid reasoning in the thread for Flit though. In particular, I tend to disagree with this argument:
my default is that any packaging-related project that wants to be part of PyPA should be welcomed. We shouldn’t need to justify including projects, rather we should require justification from people who want to say “no” to a request to join.
Welcoming any project without requiring justifications brings us to where we are now, which might be convenient for some developers but quite chaotic compared to npm.
from python-package-template.
So, more conversations and I’m starting to agree with @hynek’s view: all we need is the pyproject.toml file to declare the dependencies within the [build-system]
section, and if necessary pin/hash the rest.
There are more details in this Python Application Dependency Management blog post, the setuptools docs, and as an example we can take a look at structlog’s pyproject.toml — which is a package, not an application, though.
And in Python’s Python Packaging User Guide we have:
- To learn how to manage dependencies in a version controlled project, see the tutorial on managing application dependencies
- To learn how to package and distribute your projects, see the tutorial on packaging and distributing
from python-package-template.
Here’s another choice by the Stripe folks, and a discussion to consider: Remove Poetry and reinstate setup.py #583
I’d like to know @ob-stripe’s and @koobs’ thoughts on that subject (and this issue) now, two years after that decision, and their experiences since.
from python-package-template.
Hmm… https://github.com/regebro/pyroma 🤔
from python-package-template.
Another interesting thread: Should I be pinning my dependencies?
TL;DR — if we’re building a package, don’t pin because the package needs to live amongst other packages and their dependencies; if we’re building an app, do pin and package accordingly (a somewhat neglected area of the Python ecosystem).
from python-package-template.
Looking at the Remediation
section in Scorecard security analysis results on this repo for pinning dependencies, its recommendation is consistent with the idea of pinning dependencies for apps and not pinning them for packages. It also references another interesting article, which gives practical examples of the consequences of pinning dependencies in packages.
I like to look at this problem from all these perspectives:
- supply chain security
- reproducibility
- conflicts in dependency resolution
As also recommended by Scorecard, checking in lock files to the repo can be used as a signature of the state of dependencies, which is essential for the first two concerns. It can, however, still not be perfect for the third issue. However, using dependency update tools, such as Dependabot can mitigate this problem when dependencies are guaranteed to be the latest, which also pass the CI tests. Or am I missing something?
I personally use poetry
these days and this is how I use it: I keep the un-pinned dependencies in pyproject.toml
and also check in the poetry.lock
to have a snapshot of my dependency graph for supply chain security and reproducibility reasons.
from python-package-template.
Interesting comment:
I think that in general, packaging Python applications is not a well-supported scenario. The packaging ecosystem is mostly focused on packaging libraries. Yes, entry points allow your library to expose a command interface, but the packaging capabilities are still designed around libraries.
from python-package-template.
Today I stumbled over the hashin package which allows to add integrity hashes for packages, much like Node’s package-lock.json file.
from python-package-template.
This conversation is growing lengthy, so I wanted to pick it up again with a few thoughts based on what we’ve learned over the past months about packaging Python projects.
PR #74 adds Makefile support and commit 34b8953 adds an interesting feature to that Makefile support: make requirements
creates a snapshot of the Python virtual environment (aka. pip freeze
) including integrity hashes. Given that new Makefile support, here’s my proposal.
Packages
If a user builds a package then specify only direct package dependencies in setup.py
using standard version range specifications, and install that package using e.g. pip
.
Applications
If a user builds an application then specify only direct package dependencies in setup.py
using standard version range specifications, just like for packages. In addition to that, use the new make requirements
to pin and hash all recursive dependencies for the application, and install the application using that requirements file.
Note that this does not address pip
’s weak point to reliably detect and resolve dependency conflicts, but that’s not the point of this issue anyway.
Also note that npm
uses package-lock.json
as well as npm-shrinkwrap.json
which overrides the package lock; more specifically it says:
The recommended use-case for
npm-shrinkwrap.json
is applications deployed through the publishing process on the registry: for example, daemons and command-line tools intended as global installs or devDependencies. It's strongly discouraged for library authors to publish this file, since that would prevent end users from having control over transitive dependency updates.
Case in point 🤓
from python-package-template.
Closing this because we decided that the current approach of using pip
and setup.py
and requirements.txt
is sufficient. There are issues around deep package conflict detection and resolution but that’s not anything we can/should address with this template — users may choose to use Poetry or Pipenv.
from python-package-template.
Related Issues (20)
- Consider adding a .gitattributes file. HOT 3
- Merging to main without bump fails CI. HOT 2
- Slack notification should also msg the Changelog.
- Add support for cibuildwheel package.
- Revisit how we create sync PRs HOT 1
- Building local docker image needs to clean dist folder HOT 1
- Add the Release workflow logs as a release artifact? HOT 1
- Discuss accuracy of the epoch artifact. HOT 2
- Wiki workflow does not run. HOT 1
- Update arguments for the different Step Security workflows. HOT 3
- Should we consider `.pyi` files at all? HOT 2
- Tag a previous Upstream Sync PR as duplicate of a new Sync PR
- Add CODEOWNERS file to the repository. HOT 1
- Add pre-release support HOT 4
- Improve scripted building
- Consider tracking performance across package releases. HOT 1
- Review and fix pruning of installed packages.
- Synchronizing with an upstream template repository fails if the `.template_version` contains an incorrect tag.
- Switch to pre-build SLSA provenance generator HOT 1
- Add perflint
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from python-package-template.