build(proto): make make proto fully hermetic#199
Closed
albertywu wants to merge 1 commit into
Closed
Conversation
There was a problem hiding this comment.
Pull request overview
This PR makes protobuf generation reproducible across hosts by pinning protoc and all Go codegen plugins, wiring the hermetic flow into make proto, and enforcing freshness in CI.
Changes:
- Add hermetic wrappers:
./tool/protoc(pinned via.protocversion+ SHA256) and./tool/protoc-gen-*(run pinned Go tools viago tool). - Update
make prototo use the wrappers and run a pinnedgoimportspass; addmake check-proto. - Wire
check-protointo CI and update module/docs to reflect the pinned toolchain.
Reviewed changes
Copilot reviewed 11 out of 12 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
tool/protoc-gen-yarpc-go |
Adds a go tool wrapper to run the pinned YARPC plugin hermetically. |
tool/protoc-gen-go-grpc |
Adds a go tool wrapper to run the pinned gRPC plugin hermetically. |
tool/protoc-gen-go |
Adds a go tool wrapper to run the pinned protobuf Go plugin hermetically. |
tool/protoc |
Adds a Bazelisk-style protoc downloader/verifier driven by .protocversion. |
tool/BUILD.bazel |
Exposes the new tool wrappers as exported files for Bazel usage. |
MODULE.bazel |
Adds the Bazel repo for protoc-gen-go-grpc. |
Makefile |
Switches proto generation to the hermetic wrappers; adds check-proto; runs pinned goimports. |
go.sum |
Records sums for the newly added protoc-gen-go-grpc module. |
go.mod |
Adds tool directives to pin plugin versions and pulls in protoc-gen-go-grpc. |
doc/howto/DEVELOPMENT.md |
Updates proto generation instructions to reflect hermetic behavior. |
.protocversion |
Pins the protoc version used by ./tool/protoc. |
.github/workflows/ci.yml |
Adds make check-proto to the tidy job to prevent stale generated protos. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
`make proto` previously used the host's protoc and plugins (Homebrew protoc,
`go install ...@latest` for protoc-gen-go / protoc-gen-go-grpc /
protoc-gen-yarpc-go), so generated stubs could drift between machines. Pin
the whole toolchain so one command produces identical output anywhere.
- Pin protoc via ./tool/protoc, a Bazelisk-style wrapper that downloads and
SHA-256-verifies the version in .protocversion (29.3) and caches it.
- Pin all three plugins via `tool` directives in go.mod at the versions that
reproduce the committed output (protoc-gen-go v1.36.10, protoc-gen-go-grpc
v1.5.1, protoc-gen-yarpc-go from yarpc v1.81.0); protoc-gen-go-grpc is a
separate module, now added to the graph.
- Add tool/protoc-gen-{go,go-grpc,yarpc-go} wrappers that run the pinned
plugins via `go tool`, and pass them to protoc with explicit --plugin=
flags so $PATH is never consulted.
- Run a pinned goimports pass so `make proto` emits the committed (formatted)
form in a single command.
- Add a `check-proto` target wired into the CI `tidy` job so stale generated
files fail the required-checks gate.
- Drop the host protoc/plugin install steps from DEVELOPMENT.md.
No change to the generated stubs: the committed yarpc output stays on the
existing gogo codec. Switching to the protobuf v2 codec will be a separate PR.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
albertywu
force-pushed
the
hermetic-proto-toolchain
branch
from
e632592 to
0fec623
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Pin the entire codegen toolchain so a single
make protoproduces byte-identical output on any host, with every version in version control.Previously,
make protowas dependent on host-installed versions ofprotoc+ plugins which led to different output when run on different machine.Validation
✅
make fmt && make proto && make build && make test && make e2e-test✅ Confirmed that after running
make protonone of the already committed proto source code was changed✅ Confirmed that I can make a .proto change and
make protoupdates the generated source code as expected🤖 Generated with Claude Code