Skip to content

verify-action-build: credit vendored files git normalised from CRLF - #1297

Open
potiuk wants to merge 1 commit into
mainfrom
verify-crlf-registry-normalisation
Open

potiuk wants to merge 1 commit into
mainfrom
verify-crlf-registry-normalisation

Conversation

@potiuk

@potiuk potiuk commented Sep 16, 2026

Copy link
Copy Markdown
Member

Vendored npm registry check byte-compares each committed file against the integrity-verified registry tarball. A package whose tarball ships CRLF can never pass that comparison: git normalises line endings on commit, so the committed blob holds the LF form while the tarball does not. Every such file was reported as modified, which reads as tampering.

  • Retries a failed byte comparison against the tarball content with \r\n folded to \n, and credits the file when that hashes to the committed blob.
  • Reports those files in a new crlf_normalized category rather than silently folding them into verified, so the output says why the file did not byte-match.
  • Adds a regression test for the shape, plus a precision guard asserting a real content edit still fails after the folding is in place.

This is a proof rather than a heuristic: if the folded bytes hash to the committed blob, the committed file is the published content, line endings aside.

Live case — #1283. benchmark-action/github-action-benchmark vendors node-fetch's nested whatwg-url@5.0.0, a 2016-era package whose tarball ships lib/url-state-machine.js with CRLF:

line endings CR count bytes git blob
registry tarball CRLF 1297 33573 c25dbc2c4862...
committed in the action LF 0 32276 27d977a25f90...

33573 - 32276 = 1297, exactly the CR count, and diff --strip-trailing-cr reports the two identical. Folding the tarball's CRLF hashes to 27d977a25f9011fc31ed28c17ba11f729b85edea - the committed blob GitHub reports. The other 26 vendored packages in that action matched byte-for-byte, so this was a single file failing for a reason unrelated to the action's security posture, on every bump.

Test plan: uv run pytest utils/tests/ - 353 passed. prek run --files ... passes. The two new tests fail without the fix (reproducing #1283's exact output: content differs from registry tarball) and pass with it.

🤖 Generated with Claude Code

A package whose npm tarball ships CRLF can never byte-match the committed
copy: git normalises line endings on commit, so the vendored blob is the
LF form while the tarball is not. The registry check reported every such
file as modified.

whatwg-url@5.0.0, vendored under node-fetch by benchmark-action/github-
action-benchmark (#1283), is the live case -- lib/url-state-machine.js is
33573 bytes with 1297 CRs as published and 32276 bytes with none as
committed. Folding the tarball's CRLF to LF hashes to
27d977a25f9011fc31ed28c17ba11f729b85edea, exactly the committed blob.

Hashing the folded bytes is a proof, not a heuristic: a match means the
committed file is the published content, line endings aside. A real edit
still fails, which the precision-guard test pins.

Reported separately from verified so the output says why the file did not
byte-match rather than hiding it.

Generated-by: Claude Opus 5
@potiuk

potiuk commented Sep 18, 2026

Copy link
Copy Markdown
Member Author

Nudge on my open PRs, @dfoulks1 @ppkarwasz - five are green and waiting, and they are easier to read as one set than separately.

Two are checker fixes, and these are the ones worth your time:

Two record how the checkers get used:

And one tool change:

No rush on any of them, and happy to split #1263 up if the rebase makes it awkward to review. #1297 is the one I would land first if you only have time for one.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant