Add CI to validate contributor runner archives#13
Merged
Conversation
Runner zips are committed by contributors and handed to RunCat Neo users as-is from the gallery download link, with nothing between the two but a maintainer reading a checklist. This checks every pull request instead. The core property is that a frame archive may contain nothing but PNG files, and that every PNG consists entirely of CRC-valid chunks ending at IEND with no trailing bytes. Together those leave no room in the archive for a payload to hide, which is a stronger guarantee than scanning for known signatures would give. On top of that the validator rejects symlinks, executables, encrypted entries, zip-slip paths, zip bombs, and data appended to the archive, then fully decodes every frame and checks it against the sizes CONTRIBUTING.md documents. Interlaced PNGs are rejected: Adam7 is pointless at 36px tall, and Pillow cannot decode the frames of an interlaced APNG at all, so accepting one would mean giving up the full-decode check for exactly the files that are unusual. Runs on pull_request rather than pull_request_target. The job feeds untrusted binaries to zlib, libpng, and Pillow's C decoders, so it is the last thing here that should hold a writable token or secrets; the read-only sandbox is the actual control. That costs the ability to post a comment, so results land as inline annotations on the offending file plus a job summary. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Every other runner names its frames <runner-name>-frame-<N>.png, but this archive used record-player-<N>.png, so it was the one runner the new validation would reject. The frame images themselves are byte-for-byte unchanged; only the zip entry names differ. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The frame file name was only ever shown as an example, so state it as a requirement now that CI enforces it, and note that PNGs must not be interlaced. Also point contributors at the validator itself. Someone who runs it before pushing never has to see a red check at all, which matters more than any wording in the failure messages. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The first run produced a completely empty step log: on success the validator emits no annotations, and the summary goes to a file. That leaves no way to tell from the log whether it validated a runner or silently validated nothing, which matters because the scope comes from a diff and an empty scope still exits 0. Print the result to stdout in both modes, and say so explicitly when nothing was in scope. Also move off Node 20, which the runner now warns about. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Type of Change
Summary
Runner zips are committed by contributors and handed to RunCat Neo users as-is from the gallery download link. Today the only thing between the two is a maintainer reading a checklist. This adds a check that runs automatically on every pull request and reports a pass/fail verdict.
Adds
.github/workflows/validate-runners.ymland.github/scripts/validate_runners.py. Only runners touched by the PR are validated; manifest/directory correspondence is always checked in full, since a manifest edit can drop an entry for a runner the PR never touched.What it actually guarantees
The core property is that a frame archive may contain nothing but PNG files, and that every PNG consists entirely of CRC-valid chunks ending at IEND with no trailing bytes. Together those leave nowhere in the archive for a payload to hide — a stronger guarantee than scanning for known signatures, which is why there is no ClamAV step here. On top of that:
.., absolute paths, drive letters), zip bombs, archive comments, and data appended before or after the archive.0..N-1.metadata.jsonshape, thatpreview.pngis a real APNG, and thatmanifest.jsonand the runner directories agree in both directions — a name listed with a broken directory is silently dropped from the gallery byscript.js, which is the failure this is most worth catching.Interlaced PNGs are rejected. Adam7 is pointless at 36px tall, and Pillow cannot decode the frames of an interlaced APNG at all (
load_prepareappends todecoderconfigon every seek, so frame 1 raisesTypeError), so accepting one would mean giving up the full-decode check for exactly the files that are unusual.Security posture
Runs on
pull_request, notpull_request_target. This job feeds untrusted binaries to zlib, libpng, and Pillow's C decoders, so it is the last thing in this repository that should hold a writable token or secrets — the read-only fork sandbox is the actual control. That costs the ability to post a PR comment, so results land as inline annotations on the offending file plus a job summary.There is deliberately no
paths:filter: once this is a required check, a filtered-out PR would never report it and could never merge. The job takes well under a second.Verification
Passes on all 18 existing runners in 0.13s. Verified against deliberately hostile inputs — payload appended to the zip, an
MZexecutable appended after a frame's IEND, a symlink entry, zip-slip, an ELF disguised as.png, a 9KB payload hidden in atEXtchunk, Finder junk, frame gaps — each is caught with an actionable message.Calibrated against the currently open runner PRs, which is where the real value shows:
preview.gifinstead ofpreview.png, 8 AppleDouble files in the zip,escapement/linkagelisted twice in the manifestspamton-frames/directory, interlaced preview, preview is 64x76Notes
record-player-frames.zipis rebuilt in its own commit. It was the one runner that named framesrecord-player-<N>.pnginstead ofrecord-player-frame-<N>.png, so it was the only existing violation. The frame images are byte-for-byte unchanged; only the zip entry names differ.mainis namedValidate runner assets.🤖 Generated with Claude Code