Simple code formatter and pre-commit checker used internally by ESSS.
- Imports sorted using isort
- Trim right spaces
- Expand tabs
- Formats Python code using black
- Formats C++ code using clang-format if a
.clang-format
file is available
conda install esss_fix_format
Note:
If executed from the root environment (or another environment) isort will classify modules incorrectly, so you should install and run it from the same environment you're using for your project.
Use fix-format
(or ff
for short) to reorder imports and format source code automatically.
To format files and/or directories:
fix-format <file1> <dir1> ...
Format only modified files in Git:
fix-format --commit
Or more succinctly:
ff -c
Options for fix-format
are defined in the section [tool.esss_fix_format]]
of a pyproject.toml
file. The
TOML file should be placed in an ancestor directory of the filenames passed on the command-line.
A list of file name patterns to be excluded from the formatting. Patterns are matched using python fnmatch
:
[tool.esss_fix_format] exclude = [ "src/generated/*.py", "tmp/*", ]
Since version 4.0.0
black is used as the
code formatter for Python code.
For consistentcy, fix-format
requires a pyproject.toml
at the root of your repository (recommended) or project.
It is suggested to use minimal configuration, usually just line length:
[tool.black]
line-length = 100
A popular option is also skip-string-normalization = true
, which is recommended when migrating from
other formatter to black
.
See "Converting master to black" below for details.
Follow this steps to re format an entire project and start using the pre-commit hook:
You should have
ff
available in your environment already:$ ff --help Usage: ff-script.py [OPTIONS] [FILES_OR_DIRECTORIES]... Fixes and checks formatting according to ESSS standards. Options: -k, --check Check if files are correctly formatted. --stdin Read filenames from stdin (1 per line). -c, --commit Use modified files from git. --git-hooks Add git pre-commit hooks to the repo in the current dir. --help Show this message and exit.
For each file you don't want imports reordered add
isort:skipfile
to the docstring:""" isort:skip_file """
Commit using
-n
to skip the current hook.If there are any sensitive imports in your code which you wouldn't like to
ff
to touch, use a comment to preventisort
from touching it:ConfigurePyroSettings() # must be called before importing Pyro4 import Pyro4 # isort:skip
If you want to use
clang-format
to format C++ code, you should copy the.clang-format
file fromesss-fix-format
to the root of your project. This is optional for now in order to allow incremental changes (if this file is not present, the legacy C++ formatter will be used):$ cd /path/to/repo/root $ curl -O https://raw.githubusercontent.com/ESSS/esss_fix_format/master/.clang-format
If you want to use
black
to format Python code, add apyproject.toml
to the root of your repository; an example can be found in "Converting master to black" below.Activate your project environment:
$ conda activate myproject-py36
Execute:
$ cd /path/to/repo/root $ ff .
After it completes, make sure there are no problems with the files:
$ ff . --check
Note
if the check fails, try running it again; there's a rare bug in isort that might require to run
ff /path/to/repo/root
twice.Commit:
$ git commit -anm "Apply fix-format on all files" --author="fix-format <[email protected]>"
Push and run your branch on CI.
If all goes well, it's possible to install pre-commit hooks by using
ff --git-hooks
so that any commit will be checked locally before commiting.Profit! ๐ฐ
Migrating an existing code base from a formatter to another can be a bit of pain. This steps will help you diminish that pain as much as possible.
The first step is converting your master
branch to black.
Add a
pyproject.toml
project with this contents:[tool.black] line-length = 100 skip-string-normalization = true
If your project doesn't have a
.isort.cfg
file, create one at the project's repository root with the same contents as the one in the root of this repository.Run the
upsert-isort-config
task to update it (it should be run regularly, specially when adding new dependencies to internal projects, known as "first party" dependencies); or, if the project needs special configurations due to dual package and source modes, add these lines (and do not runupsert-isort-config
):[settings] profile=black no_sections=True force_alphabetical_sort=True
This will use black-like grouping, and clump imports together regardless if they are standard library, third party, or local. This avoids getting different results if you have a different environment activated, or commiting from an IDE.
Commit, and save the commit hash, possible in a task that you created for this conversion:
$ git commit -anm "Add configuration files for black"
Execute on the root of the repository:
$ fix-format .
Ensure everything is fine:
$ fix-format --check .
If you don't see any "reformatting" messages, it means everything is formatted correctly.
Commit and then open a PR:
$ git commit -anm "Convert source files to black" --author="fix-format <[email protected]>"
Here we are in the situation where the master
is already blacken, and you want
to update your branch. There are two ways, and which way generates less conflicts really
depends on the contents of the source branch.
Merge with the target branch, resolve any conflicts and then commit normally.
Execute
fix-format
in the root of your repository:$ fix-format .
This should only change the files you have touched in your branch.
Commit and push:
$ git commit -anm "Convert source files to black" --author="fix-format <[email protected]>"
Cherry-pick the commit you saved earlier on top of your branch.
Execute
fix-format
in the root of your repository:$ fix-format .
(In very large repositories, this will be a problem on Windows because of the command-line size, do it in chunks).
Fix any conflicts and then commit:
$ git commit -anm "Convert source files to black" --author="fix-format <[email protected]>"
Create a conda environment (using Python 3 here) and install it in development mode.
Make sure you have conda configured to use ``conda-forge`` and ``esss`` conda channels.
$ conda install -n base conda-devenv
$ conda devenv
$ source activate esss-fix-format-py310
$ pre-commit install
$ pytest
When implementing changes, please do it in a separate branch and open a PR.
The release is done internally at ESSS using our conda-recipes repository.
Licensed under the MIT license.