bazel: Migrate Buildifier to Bazel Lint Umbrella (#9858)#9865
bazel: Migrate Buildifier to Bazel Lint Umbrella (#9858)#9865Divinesoumyadip wants to merge 3 commits intoThe-OpenROAD-Project:masterfrom
Conversation
097a416 to
4935d3d
Compare
There was a problem hiding this comment.
Code Review
This pull request migrates Buildifier into the Bazel lint umbrella, which is a great improvement for the local development workflow. The changes follow the existing pattern for TCL linting by adding new test and tidy targets.
My main feedback is to make the new Buildifier test targets fully consistent with the TCL ones by using a non-sandboxed execution (tags = ["local"]) and git ls-files for discovering files. This approach is more robust than using glob() in a sandboxed test, as it correctly discovers all tracked files without being sensitive to the Bazel cache state. I've left specific suggestions on the sh_test targets in BUILD.bazel and on the new shell scripts to implement this.
Overall, this is a valuable change that will improve developer experience.
| sh_test( | ||
| name = "fmt_bzl_test", | ||
| size = "small", | ||
| srcs = ["//bazel:bzl_fmt_test.sh"], | ||
| args = ["$(rootpath @buildifier_prebuilt//:buildifier)"], | ||
| data = [ | ||
| "BUILD.bazel", | ||
| "MODULE.bazel", | ||
| "@buildifier_prebuilt//:buildifier", | ||
| ] + glob( | ||
| ["**/*.bzl"], | ||
| allow_empty = True, | ||
| ), | ||
| ) |
There was a problem hiding this comment.
To improve consistency with the existing TCL linting pattern and make file discovery more robust, I suggest a few changes. The current glob() can miss newly added files until a bazel clean is performed. The TCL tests avoid this by using tags = ["local"] and git ls-files in the test script.
I recommend applying the same pattern here:
- Add
tags = ["local"]. - Simplify
datato only include the buildifier binary. - Update
bzl_fmt_test.shto usegit ls-files.
This same feedback applies to lint_bzl_test. The duplication of the data attribute between the two tests could also be addressed by extracting it to a variable if you stick with the glob approach.
sh_test(
name = "fmt_bzl_test",
size = "small",
srcs = ["//bazel:bzl_fmt_test.sh"],
args = ["$(rootpath @buildifier_prebuilt//:buildifier)"],
data = ["@buildifier_prebuilt//:buildifier"],
tags = ["local"],
)
| sh_test( | ||
| name = "lint_bzl_test", | ||
| size = "small", | ||
| srcs = ["//bazel:bzl_lint_test.sh"], | ||
| args = ["$(rootpath @buildifier_prebuilt//:buildifier)"], | ||
| data = [ | ||
| "BUILD.bazel", | ||
| "MODULE.bazel", | ||
| "@buildifier_prebuilt//:buildifier", | ||
| ] + glob( | ||
| ["**/*.bzl"], | ||
| allow_empty = True, | ||
| ), | ||
| ) |
There was a problem hiding this comment.
Similar to my comment on fmt_bzl_test, I recommend using tags = ["local"] and updating the script to use git ls-files for consistency with the TCL linting setup and for more robust file discovery.
sh_test(
name = "lint_bzl_test",
size = "small",
srcs = ["//bazel:bzl_lint_test.sh"],
args = ["$(rootpath @buildifier_prebuilt//:buildifier)"],
data = ["@buildifier_prebuilt//:buildifier"],
tags = ["local"],
)
bazel/bzl_fmt_test.sh
Outdated
| #!/bin/bash | ||
| set -e | ||
| # Capture absolute path BEFORE changing directories | ||
| BUILDIFIER_BIN=$(realpath "$1") | ||
|
|
||
| if [ -n "$BUILD_WORKSPACE_DIRECTORY" ]; then | ||
| cd "$BUILD_WORKSPACE_DIRECTORY" | ||
| fi | ||
|
|
||
| "$BUILDIFIER_BIN" -r -mode=check -lint=off . |
There was a problem hiding this comment.
This script can be made more robust and consistent with the existing TCL linting scripts. Using git ls-files ensures that only tracked files are checked, and it's more reliable for discovering all relevant files than running on . in a sandboxed environment populated by glob. Also, set -euo pipefail is generally safer for shell scripts.
This change assumes the corresponding sh_test in BUILD.bazel is updated with tags = ["local"].
| #!/bin/bash | |
| set -e | |
| # Capture absolute path BEFORE changing directories | |
| BUILDIFIER_BIN=$(realpath "$1") | |
| if [ -n "$BUILD_WORKSPACE_DIRECTORY" ]; then | |
| cd "$BUILD_WORKSPACE_DIRECTORY" | |
| fi | |
| "$BUILDIFIER_BIN" -r -mode=check -lint=off . | |
| #!/bin/bash | |
| set -euo pipefail | |
| # Capture absolute path BEFORE changing directories | |
| BUILDIFIER_BIN=$(realpath "$1") | |
| if [ -n "$BUILD_WORKSPACE_DIRECTORY" ]; then | |
| cd "$BUILD_WORKSPACE_DIRECTORY" | |
| fi | |
| # Find all Starlark, BUILD, and WORKSPACE files tracked by git. | |
| git ls-files -- '*.bzl' 'BUILD' 'BUILD.bazel' 'MODULE.bazel' 'WORKSPACE' 'WORKSPACE.bazel' -z | xargs -0 "$BUILDIFIER_BIN" -mode=check -lint=off |
bazel/bzl_lint_test.sh
Outdated
| #!/bin/bash | ||
| set -e | ||
| BUILDIFIER_BIN=$(realpath "$1") | ||
|
|
||
| if [ -n "$BUILD_WORKSPACE_DIRECTORY" ]; then | ||
| cd "$BUILD_WORKSPACE_DIRECTORY" | ||
| fi | ||
|
|
||
| "$BUILDIFIER_BIN" -r -mode=check -lint=warn . |
There was a problem hiding this comment.
Similar to my comment on bzl_fmt_test.sh, I suggest using git ls-files for robust file discovery and set -euo pipefail for safer script execution. This aligns with the pattern used by the TCL linting scripts.
| #!/bin/bash | |
| set -e | |
| BUILDIFIER_BIN=$(realpath "$1") | |
| if [ -n "$BUILD_WORKSPACE_DIRECTORY" ]; then | |
| cd "$BUILD_WORKSPACE_DIRECTORY" | |
| fi | |
| "$BUILDIFIER_BIN" -r -mode=check -lint=warn . | |
| #!/bin/bash | |
| set -euo pipefail | |
| BUILDIFIER_BIN=$(realpath "$1") | |
| if [ -n "$BUILD_WORKSPACE_DIRECTORY" ]; then | |
| cd "$BUILD_WORKSPACE_DIRECTORY" | |
| fi | |
| git ls-files -- '*.bzl' 'BUILD' 'BUILD.bazel' 'MODULE.bazel' 'WORKSPACE' 'WORKSPACE.bazel' -z | xargs -0 "$BUILDIFIER_BIN" -mode=check -lint=warn |
bazel/bzl_tidy.sh
Outdated
| #!/bin/bash | ||
| set -e | ||
| BUILDIFIER_BIN=$(realpath "$1") | ||
|
|
||
| if [ -z "$BUILD_WORKSPACE_DIRECTORY" ]; then | ||
| echo "Error: This script must be run via 'bazelisk run'" | ||
| exit 1 | ||
| fi | ||
|
|
||
| cd "$BUILD_WORKSPACE_DIRECTORY" | ||
| "$BUILDIFIER_BIN" -r -mode=fix -lint=fix . | ||
| echo "Buildifier tidy complete!" |
There was a problem hiding this comment.
For consistency with the TCL tidy script (tcl_tidy.sh) and to ensure only tracked files are modified, I recommend using git ls-files to find the files to format. Also, set -euo pipefail is a good practice for safer script execution.
| #!/bin/bash | |
| set -e | |
| BUILDIFIER_BIN=$(realpath "$1") | |
| if [ -z "$BUILD_WORKSPACE_DIRECTORY" ]; then | |
| echo "Error: This script must be run via 'bazelisk run'" | |
| exit 1 | |
| fi | |
| cd "$BUILD_WORKSPACE_DIRECTORY" | |
| "$BUILDIFIER_BIN" -r -mode=fix -lint=fix . | |
| echo "Buildifier tidy complete!" | |
| #!/bin/bash | |
| set -euo pipefail | |
| BUILDIFIER_BIN=$(realpath "$1") | |
| if [ -z "$BUILD_WORKSPACE_DIRECTORY" ]; then | |
| echo "Error: This script must be run via 'bazelisk run'" | |
| exit 1 | |
| fi | |
| cd "$BUILD_WORKSPACE_DIRECTORY" | |
| git ls-files -- '*.bzl' 'BUILD' 'BUILD.bazel' 'MODULE.bazel' 'WORKSPACE' 'WORKSPACE.bazel' -z | xargs -0 "$BUILDIFIER_BIN" -mode=fix -lint=fix | |
| echo "Buildifier tidy complete!" |
There was a problem hiding this comment.
Thanks for the suggestion. The git ls-files approach fails in Bazel's sandboxed test environment with fatal: not a git repository (exit 128) since the sandbox doesn't have access to the .git directory. The tags = ["local"] has been added for consistency with the TCL pattern, and the -r recursive flag on buildifier correctly discovers all Starlark files without needing git. Both tests pass locally.
|
clang-tidy review says "All clean, LGTM! 👍" |
1 similar comment
|
clang-tidy review says "All clean, LGTM! 👍" |
793a68c to
46106d3
Compare
|
clang-tidy review says "All clean, LGTM! 👍" |
Signed-off-by: Divinesoumyadip <soumyacode7@gmail.com>
46106d3 to
b389365
Compare
|
clang-tidy review says "All clean, LGTM! 👍" |
2 similar comments
|
clang-tidy review says "All clean, LGTM! 👍" |
|
clang-tidy review says "All clean, LGTM! 👍" |
|
@maliberty all CI checks including Jenkins are passing. Would appreciate a review when you get a chance! |
|
Nice! I have not reviewed it, but just a note on buildifier versions and operating systems. I am not sure why the MODULE.bazel.lock is in git, but I think it is a security issue, users can tell bazel to check for supply chain attacks. I wonder if it is possible to make the lint check ignore non security concerns in .lock diffs. Would require some unittests in python also under CI tests as such a script cant be checked on any specific dev machine and would need to be curated to encode all these headaches. The upside to making this script is reduced cognitive load of contributors. |
|
I usually just point Claude at the PR and ask it to create new commits to address concerns in the PR, which it happily does. Except when Gemmini and Claude disagrees, in which case I ask Claude to give a snarky explanation why Gemmini is hallucinating with a record of tests or evidence that Gemmini is wrong. |
Thanks @oharboe! The |
bazel/bzl_fmt_test.sh
Outdated
| @@ -0,0 +1,7 @@ | |||
| #!/bin/bash | |||
| set -e | |||
There was a problem hiding this comment.
| set -e | |
| set -euo pipefail |
bazel/bzl_fmt_test.sh
Outdated
| @@ -0,0 +1,7 @@ | |||
| #!/bin/bash | |||
| set -e | |||
| BUILDIFIER_BIN=$(realpath "$1") | |||
There was a problem hiding this comment.
is realpath portable across all of our supported platforms?
bazel/bzl_fmt_test.sh
Outdated
| #!/bin/bash | ||
| set -e | ||
| BUILDIFIER_BIN=$(realpath "$1") | ||
| if [ -n "${BUILD_WORKSPACE_DIRECTORY:-}" ]; then |
There was a problem hiding this comment.
Refer to how other .sh locates WORKSPACE, let's keep things consistent.
Signed-off-by: Divinesoumyadip <soumyacode7@gmail.com>
|
clang-tidy review says "All clean, LGTM! 👍" |
Signed-off-by: Divinesoumyadip <soumyacode7@gmail.com>
|
@luarss I've addressed all three suggestions in the latest commits. Please let me know if anything else needs changing. |
|
clang-tidy review says "All clean, LGTM! 👍" |
Fixes Issue #9858
This migrates Buildifier into the Bazel lint umbrella, following the same pattern already established for TCL linting. Previously, Buildifier only ran as a standalone GitHub Actions workflow, meaning developers had to push to CI just to discover formatting violations in BUILD and .bzl files.
The change adds
buildifier_prebuilt(v8.2.1) as a Bzlmod dev dependency and introduces three new shell scriptsbzl_fmt_test.sh,bzl_lint_test.sh, andbzl_tidy.sh, mirroring the existing TCL lint pattern exactly. The two read-only check targets (fmt_bzl_testandlint_bzl_test) are wired into the//:lint_testumbrella, andfix_linthas been updated to run bothtidy_bzlandtidy_tclso developers get a single command that auto-fixes both Starlark and TCL files locally. The old.github/workflows/buildifier.yamlworkflow has been retired. Both new tests pass locally.