Identify unused dependencies and avoid a bloated virtual environment.
Install creosote in separate virtual environment (using e.g. pipx
):
pipx install creosote
Scan virtual environment for unused dependencies (PEP-621 example below, but Poetry, Pipenv and requirements.txt
files are also supported, see this table):
$ creosote
Found dependencies in pyproject.toml: distlib, dotty-dict, loguru, pip-requirements-parser, requests, toml
Oh no, bloated venv! 🤢 🪣
Unused dependencies found: requests
And after having removed/uninstalled requests
:
$ creosote
Found dependencies in pyproject.toml: distlib, dotty-dict, loguru, pip-requirements-parser, toml
No unused dependencies found! ✨
Get help:
creosote --help
Some data is required as input:
Argument | Default value | Description |
---|---|---|
--venv |
.venv |
The path to your virtual environment. |
--paths |
src |
The path to your source code, one or more files/folders. |
--deps-file |
pyproject.toml |
The path to the file specifying your dependencies, like pyproject.toml , requirements_*.txt | .in . |
--sections |
project.dependencies |
One or more toml sections to parse, e.g. project.dependencies . |
The creosote tool will first scan the given python file(s) for all its imports. Then it fetches all dependency names (from the dependencies spec file). Finally, all imports are associated with their corresponding dependency name (requires the virtual environment for resolving). If a dependency does not have any imports associated, it is considered unused.
See the main
function in cli.py
for a terse overview of the logic.
importlib
imports are not detected by the AST parser (a great first contribution for anyone inclined 😄, reach out or start here).
The idea of a project like this was hatched from having security vulnerability reports about production dependencies (shipped into production) which turned out to not not even be in use.
The goal of this project is to run the creosote
tool in CI, which will catch cases where the developer
forgets to remove unused dependencies. An example of such a case could be when doing refactorings.
Note: Creosote supports identifying both unused production dependencies and developer dependencies. It all depends on what you would like to achieve.
Tool/standard | Supported | --deps-file value |
Example --sections values |
---|---|---|---|
PEP-621 | ✅ | pyproject.toml |
project.dependencies ,project.optional-dependencies.<GROUP> |
Poetry | ✅ | pyproject.toml |
tool.poetry.dependencies ,tool.poetry.dev-dependencies (legacy),tool.poetry.group.<GROUP>.dependencies |
Pipenv | ✅ | pyproject.toml |
packages ,dev-packages |
PEP-508 (requirements.txt , pip-tools) |
✅ | *.[txt|in] |
N/A |
Legacy Setuptools (setup.py ) |
❌ |
📔 Notes on PEP-508 (requirements.txt
)
When using requirements.txt
files to specify dependencies, there is no way to tell which part of requirements.txt
specifies production vs developer dependencies. Therefore, you have to break your requirements.txt
file into e.g. requirements-prod.txt
and requirements-dev.txt
and use any of them as input. When using pip-tools, you likely want to point Creosote to scan your *.in
file(s).
Yes, you can specify a list of sections after the --sections
argument. It all depends on what your setup looks like and what you set out to achieve.
$ creosote --sections project.dependencies project.optional-dependencies.lint project.optional-dependencies.test
Yes, you can use the --exclude-deps
argument to specify one or more dependencies you do not wish to get warnings for.
This feature is intended for dependencies you must specify in your dependencies spec file, but which you don't import in your source code. An example of such a dependency are database drivers, which are commonly only defined in connection strings and will signal to the ORM which driver to use.
$ creosote --exclude-deps pyodbc starlette
Yes, please see the action
job example in .github/workflows/test.yml
.
Can I run Creosote with pre-commit?
Yes, see example in .pre-commit-config.yaml
.
Here's another example setup, if already have Creosote installed onto $PATH (via e.g. pipx).
# .pre-commit-config.yaml
repos:
- repo: local
hooks:
- id: system
name: creosote
entry: creosote --venv .venv --paths src --deps-file pyproject.toml --sections project.dependencies
pass_filenames: false
files: \.(py|toml|txt|in|lock)$
language: system
This tool has borrowed its name from the Monty Python scene about Mr. Creosote.
Because it makes me happy to see this tool can help others! 🥰
You can run in-development versions of Creosote. Examples below:
# Creosote build from main branch
$ pipx install --suffix=@main --force git+https://github.com/fredrikaverpil/creosote.git@main
$ creosote@main --venv .venv ...
$ pipx uninstall creosote@main
# Creosote build from PR #123
$ pipx install --suffix=@123 --force git+https://github.com/fredrikaverpil/creosote.git@refs/pull/123/head
$ creosote@123 --venv .venv ...
$ pipx uninstall creosote@123
- Bump version in
src/creosote/__about__.py
and.pre-commit-config.yaml
. - GitHub Action will run automatically on creating a release and deploy the release onto PyPi.