keith / buildifier-prebuilt Goto Github PK

A bazel toolchain for using prebuilt binaries for buildifier and buildozer

License: MIT License

Starlark 88.49% Shell 11.51%

buildifier-prebuilt's Introduction

buildifier-prebuilt

This repo contains bazel rules for buildifier and buildozer using prebuilt binaries with bazel toolchains instead of requiring you depend on rules_go. This also means you won't download every possible version of these binaries, you'll only download the ones for the platform you're running on.

Usage

You can create a rule for running buildifier:

load("@buildifier_prebuilt//:rules.bzl", "buildifier")

buildifier(
    name = "buildifier.check",
    exclude_patterns = [
        "./.git/*",
    ],
    lint_mode = "warn",
    mode = "diff",
)

That can be run with:

bazel run //:buildifier.check

Or you can run buildifier or buildozer directly:

bazel run -- @buildifier_prebuilt//:buildozer ARGS
bazel run -- @buildifier_prebuilt//:buildifier ARGS

Installation

Bzlmod: Add `bazel_dep` to `MODULE.bazel` file

bazel_dep(
    name = "buildifier_prebuilt",
    version = "6.4.0",
    dev_dependency = True,
)

Legacy: Add declarations to `WORKSPACE` file

Add the following to your WORKSPACE file.

load("@bazel_tools//tools/build_defs/repo:http.bzl", "http_archive")

http_archive(
    name = "buildifier_prebuilt",
    sha256 = "8ada9d88e51ebf5a1fdff37d75ed41d51f5e677cdbeafb0a22dda54747d6e07e",
    strip_prefix = "buildifier-prebuilt-6.4.0",
    urls = [
        "http://github.com/keith/buildifier-prebuilt/archive/6.4.0.tar.gz",
    ],
)

load("@buildifier_prebuilt//:deps.bzl", "buildifier_prebuilt_deps")

buildifier_prebuilt_deps()

load("@bazel_skylib//:workspace.bzl", "bazel_skylib_workspace")

bazel_skylib_workspace()

load("@buildifier_prebuilt//:defs.bzl", "buildifier_prebuilt_register_toolchains")

buildifier_prebuilt_register_toolchains()

Specify Version of Buildtools

By default releases of these rules hardcode the most up to date versions of the tools at release time. If you would like to specify a specific version of buildtools to use, you can do one of the following.

Option 1: Quick and Easy

Update the buildifier_prebuilt_register_toolchains declaration in your WORKSPACE file to specify the version.

# Use buildtools version 4.2.5.
buildifier_prebuilt_register_toolchains(
    assets = buildtools_assets(version = "4.2.5"),
)

The above example will download version 4.2.5 of the buildtools binaries. The only downside is that you will see warnings stating that a canonical version can be specified using SHA256 values.

Option 2: Manually Add SHA256 Values

To add SHA256 values to the declaration, add a sha256_values attribute and specify the values in a dict where the key is <tool>_<platform>_<arch> and the value is the SHA256 value.

# Use buildtools version 4.2.5.
buildifier_prebuilt_register_toolchains(
    assets = buildtools_assets(version = "4.2.5"),
    sha256_values = {
        "buildifier_darwin_amd64": "757f246040aceb2c9550d02ef5d1f22d3ef1ff53405fe76ef4c6239ef1ea2cc1",
        "buildifier_darwin_arm64": "4cf02e051f6cda18765935cb6e77cc938cf8b405064589a50fe9582f82c7edaf",
        "buildifier_linux_amd64": "f94e71b22925aff76ce01a49e1c6c6d31f521bbbccff047b81f2ea01fd01a945",
        "buildifier_linux_arm64": "2113d79e45efb51e2b3013c8737cb66cadae3fd89bd7e820438cb06201e50874",
        "buildozer_darwin_amd64": "3fe671620e6cb7d2386f9da09c1de8de88b02b9dd9275cdecd8b9e417f74df1b",
        "buildozer_darwin_arm64": "ff4d297023fe3e0fd14113c78f04cef55289ca5bfe5e45a916be738b948dc743",
        "buildozer_linux_amd64": "e8e39b71c52318a9030dd9fcb9bbfd968d0e03e59268c60b489e6e6fc1595d7b",
        "buildozer_linux_arm64": "96227142969540def1d23a9e8225524173390d23f3d7fd56ce9c4436953f02fc",
    },
)

The downside to this is that you will need to manually download each binary that you will use in your builds and calculate the SHA256 value.

Option 3: Quick, Easy and Canonical

We have included a utility which will generate a buildifier_prebuilt_register_toolchains declaration with the appropriate SHA256 values. If you execute it without any arguments, it will use the latest release of buildtools. Just copy and paste the declaration into your WORKSPACE file.

# Generate the declaration for the latest
$ bazel run //tools:generate_assets_declaration
load("@buildifier_prebuilt//:defs.bzl", "buildifier_prebuilt_register_toolchains", "buildtools_assets")

buildifier_prebuilt_register_toolchains(
    assets = buildtools_assets(
        version = "4.2.5",
        names = ["buildifier", "buildozer"],
        platforms = ["darwin", "linux"],
        arches = ["amd64", "arm64"],
        sha256_values = {
            "buildifier_darwin_amd64": "757f246040aceb2c9550d02ef5d1f22d3ef1ff53405fe76ef4c6239ef1ea2cc1",
            "buildifier_darwin_arm64": "4cf02e051f6cda18765935cb6e77cc938cf8b405064589a50fe9582f82c7edaf",
            "buildifier_linux_amd64": "f94e71b22925aff76ce01a49e1c6c6d31f521bbbccff047b81f2ea01fd01a945",
            "buildifier_linux_arm64": "2113d79e45efb51e2b3013c8737cb66cadae3fd89bd7e820438cb06201e50874",
            "buildozer_darwin_amd64": "3fe671620e6cb7d2386f9da09c1de8de88b02b9dd9275cdecd8b9e417f74df1b",
            "buildozer_darwin_arm64": "ff4d297023fe3e0fd14113c78f04cef55289ca5bfe5e45a916be738b948dc743",
            "buildozer_linux_amd64": "e8e39b71c52318a9030dd9fcb9bbfd968d0e03e59268c60b489e6e6fc1595d7b",
            "buildozer_linux_arm64": "96227142969540def1d23a9e8225524173390d23f3d7fd56ce9c4436953f02fc",
        },
    ),
)

You may also specify a specific version of buildtools by adding it to the end of the command.

# Generate the declaration for version 4.2.3
$ bazel run //tools:generate_assets_declaration -- 4.2.3
load("@buildifier_prebuilt//:defs.bzl", "buildifier_prebuilt_register_toolchains", "buildtools_assets")

buildifier_prebuilt_register_toolchains(
    assets = buildtools_assets(
        version = "4.2.3",
        names = ["buildifier", "buildozer"],
        platforms = ["darwin", "linux"],
        arches = ["amd64", "arm64"],
        sha256_values = {
            "buildifier_darwin_amd64": "954ec397089344b1564e45dc095e9331e121eb0f20e72032fcc8e94de78e5663",
            "buildifier_darwin_arm64": "9434043897a3c3821fda87046918e5a6c4320d8352df700f62046744c4d168a3",
            "buildifier_linux_amd64": "a19126536bae9a3917a7fc4bdbbf0378371a1d1683ab2415857cf53bce9dee49",
            "buildifier_linux_arm64": "39bd9d01d3638902a1e4cef353048ed160f0575f5df1bef175bd7637386d183c",
            "buildozer_darwin_amd64": "edcabae1d97bdc42559d7d1d65dfe7f8970db8d95d4bc9e7bf6656a9f2fb5592",
            "buildozer_darwin_arm64": "f8d0994620dec1247328f13db1d434b6489dd007f8e9b961dbd9363bc6fe7071",
            "buildozer_linux_amd64": "6b4177321b770fb788b618caa453d34561b8c05081ae8b27657e527c2a3b5d52",
            "buildozer_linux_arm64": "edfa964b283352ffd7503faca503de8f06dfcd1c7c96a6737e9452167e93c687",
        },
    ),
)

NOTE: The utility uses the GitHub CLI. If you haven't already done so, install it.

buildifier-prebuilt's People

Contributors

Stargazers

Watchers

Forkers

cgrindel minkuan88 jmelahman dhalperi erikkerber fmeum jbedard gzgavinzhao millerjhang thii noots-globally hunshcn peakschris

buildifier-prebuilt's Issues

Add option to specify the version for buildifier/buildozer and optional sha256 dict.

First, thanks for putting this repository together. I want to remove the requirement to build buildifier for repositories that use bzlformat. Your solution looks great. However, I wonder if you would be open to a PR that allows the client for this repository to specify the version and, optionally, the sha256 values? If so, I will put one together and send it over for review.

buildifier check and fix fails silently on linux with --noenable_runfiles

Without runfiles, buildifier check and fix fails silently. The command has no output or errors. It would be good at least to error in this case, if norunfiles mode will not be supported.

This issue can be covered by the test case in #90 if linux platform is enabled for the norunfiles cases.

To reproduce:

> git clone
> cd buildifier_prebuilt/examples/simple
> BUILD <-- make some edits to BUILD, breaking its formatting
> export BUILDIFIER_DIFF=diff
> bazel run //:buildifier.check --noenable_runfiles

outputs:

INFO: Analyzed target //:buildifier.check (66 packages loaded, 293 targets configured).
INFO: Found 1 target...
Target //:buildifier.check up-to-date:
  bazel-bin/buildifier.check.bash
INFO: Elapsed time: 0.406s, Critical Path: 0.00s
INFO: 5 processes: 5 internal.
INFO: Build completed successfully, 5 total actions
INFO: Running command line: bazel-bin/buildifier.check.bash
>

Add release-related rules and GitHub Actions workflow

I implemented some Bazel rules (bzlrelease) that work with a GitHub Actions workflow to create releases for my Bazel rule repositories. In short, it generates the release notes along with a workspace snippet, creates the GitHub release, and creates an auto-merge PR to update the README.md with the latest workspace snippet.

All of the pre-requisites are already being loaded in this repository. I would add a release Bazel package that would contain a BUILD file, README.md file, and a workspace snippet template (i.e., contains the load statements that need to be added to client workspaces). I would update the repository's README.md to include two markers as markdown comments to indicate where to insert/replace the generated workspace snippet. Releases can be created using a Bazel run command or via the GitHub Actions web interface.

If you would like to see an actual implementation, here is the release package, the GitHub Actions workflow and a release generated by this process for cgrindel/bazel-starlib. I could add this pretty quickly if you are interested.

//tests/tools_tests:generate_assets_declaration_test | which: no gh in (.:/bin:/usr/bin:/usr/local/bin)

@ b02049d

$ bazel test //tests/tools_tests:generate_assets_declaration_test --test_output=errors
INFO: Analyzed target //tests/tools_tests:generate_assets_declaration_test (2 packages loaded, 10 targets configured).
INFO: Found 1 test target...
FAIL: //tests/tools_tests:generate_assets_declaration_test (see /home/jamison/.cache/bazel/_bazel_jamison/04f084c324ba0be0b9506e8091939f14/execroot/buildifier_prebuilt/bazel-out/k8-fastbuild/testlogs/tests/tools_tests/generate_assets_declaration_test/test.log)
INFO: From Testing //tests/tools_tests:generate_assets_declaration_test:
==================== Test output for //tests/tools_tests:generate_assets_declaration_test:
which: no gh in (.:/bin:/usr/bin:/usr/local/bin)
Could not find Github CLI (gh).
================================================================================
Target //tests/tools_tests:generate_assets_declaration_test up-to-date:
  bazel-bin/tests/tools_tests/generate_assets_declaration_test
INFO: Elapsed time: 0.134s, Critical Path: 0.04s
INFO: 2 processes: 2 linux-sandbox.
INFO: Build completed, 1 test FAILED, 2 total actions
//tests/tools_tests:generate_assets_declaration_test                     FAILED in 0.0s
  /home/jamison/.cache/bazel/_bazel_jamison/04f084c324ba0be0b9506e8091939f14/execroot/buildifier_prebuilt/bazel-out/k8-fastbuild/testlogs/tests/tools_tests/generate_assets_declaration_test/test.log

INFO: Build completed, 1 test FAILED, 2 total actions

buildifier test passes on failing build files

Maybe I don't understand how to run this, but my naive experiements suggest some issue:

git clone
cd examples/simple
edit BUILD, mess up formatting
bazel run //:buildifier.check shows diffs with corrections
bazel test //:buildifier.test reports PASSED

The build file is:

load("@buildifier_prebuilt//:rules.bzl", "buildifier", "buildifier_test")

buildifier(
    name = "buildifier.check",
    exclude_patterns = [
        "./.git/*"],
    lint_mode = "warn",
    lint_warnings = [
        "-cc-native",
    ],
    mode = "diff",
)

buildifier_test(
    name = "buildifier.test",
    srcs = ["BUILD"],
    lint_mode = "warn",
)

sh_binary(
    name = "hello_world",
    srcs = ["hello_world.sh"],
)

sh_test(
    name = "hello_world_test",
    srcs = ["hello_world_test.sh"],
    data = [
        ":hello_world",
    ],
    deps = [
        "@bazel_tools//tools/bash/runfiles",
    ],
)

Thanks!

Add buildifier_test

The rule here has diverged a bit from buildtools, namely that buildtools wraps the "diff" mode in a test rule useful for CI checks

https://github.com/bazelbuild/buildtools/blob/master/buildifier/buildifier.bzl#L37-L41

I was going to adopt this prebuilt strategy and clean up our WORKSPACE a bit, but hit a snag in that small difference.

Checking here if matching the rules in the buildtools source is a goal, and if so I'll open a patch.

No repository visible as '@cgrindel_bazel_starlib' from repository '@@buildifier_prebuilt~6.4.0'

Steps to reproduce

$ echo 7.0.0 > .bazelversion
$ echo 'bazel_dep(name = "buildifier_prebuilt", version = "6.4.0", dev_dependency = True)' > MODULE.bazel
$ bazelisk query '@buildifier_prebuilt//...'

Expected behavior
List of targets

Actual behavior

ERROR:
error loading package under directory '':
error loading package '@@buildifier_prebuilt~6.4.0//release':
Unable to find package for @@[unknown repo 'cgrindel_bazel_starlib' requested from @@buildifier_prebuilt~6.4.0]//bzlrelease:defs.bzl:
The repository '@@[unknown repo 'cgrindel_bazel_starlib' requested from @@buildifier_prebuilt~6.4.0]' could not be resolved:
No repository visible as '@cgrindel_bazel_starlib' from repository '@@buildifier_prebuilt~6.4.0'.

(Line breaks added after colon to make it easier to read, but it's all on one line really.)

Specifying raw warning categories and those with modifiers results in an error.

Be able to do the following:

buildifier(
    name = "buildifier.check",
    exclude_patterns = _BUILDIFIER_EXCLUDE_PATTERNS,
    lint_mode = "warn",
    lint_warnings = [
        "all",
        "-cc-native",
    ],
    mode = "diff",
)

If you specify this today, you get the following error:

buildifier: warning categories with modifiers ("+" or "-") can't be mixed with raw warning categories

Look at using aliases instead of toolchains

The only reason this repo uses toolchains is to avoid downloading multiple versions of buildifier that you'll never use on your platform. It looks like if you use some indirection with a bazel alias in a select() you can avoid this as well. bazelbuild/bazel@903c272

If someone can confirm this is intentional behavior I can switch to this method instead to remove a lot of boilerplate.

Create release 0.1.4 or 0.2.0

I think that we are ready to create a release with the new code. Would you like to do the honors? I am happy to kick it off. I just need to know what version number you would like to use.

Add shell helper to make it easier to execute `buildozer` commands on a repository

The current README.md states that the following should work:

bazel run -- @buildifier_prebuilt//:buildozer ARGS

However, this will not work correctly. Bazel will execute the command in a sandbox, not the workspace. Hence, buildozer will never find any of the paths.

A possible fix would be to provide a shell script that performs a cd "${BUILD_WORKSPACE_DIRECTORY}" and then executes the buildozer commands.

NOTE: The buildifier would also benefit from this technique as today one must specify absolute paths for buildifier to find files and directories.

When enabling build --nolegacy_external_runfiles, buildifier-prebuilt still looks for external/buildifier_darwin_arm64/file/buildifier

Reproduction:

Have a simple Bazel WorkSpace, the following steps will take this one from rules_python as the example
Change WORKSPACE lines 64-72 as instructions in line 63
Integrate buildifier-prebuilt with the legacy-workspace-file method
Create a rule for running buildifier
Run "bazel run //:buildifier.check". it works.
Choose one of the below two steps to operate:
6-1. Add build --nolegacy_external_runfiles into the file .bazelrc; I added it in the first line. Then run bazel run //:buildifier.check agan.
or
6-2. Run "bazel run //:buildifier.check --nolegacy_external_runfiles"
It does not work (no fixes found) now.
(Optional) Add arguments --run_under="bash -x" and --sandbox_debug and run again for more details.

Expectation:

buildifier-prebuilt do compatible with build --nolegacy_external_runfiles