Skip to content

fix: allow all-evals sweep artifact reuse#1892

Merged
Oseltamivir merged 1 commit into
mainfrom
fix/all-evals-sweep-reuse
Jun 23, 2026
Merged

fix: allow all-evals sweep artifact reuse#1892
Oseltamivir merged 1 commit into
mainfrom
fix/all-evals-sweep-reuse

Conversation

@Oseltamivir

@Oseltamivir Oseltamivir commented Jun 23, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • allow full-sweep PRs with all-evals to use /reuse-sweep-run
  • keep evals-only reuse-incompatible
  • update workflow gating, merge helper, tests, and policy documentation

Testing

  • uv run --python 3.11 --with pytest --with pyyaml --with pydantic python -m pytest utils/test_find_reusable_sweep_run.py utils/changelog_gate_tests/test_run_sweep_gating.py utils/changelog_gate_tests/test_validate_perf_changelog.py -q
  • uv run --python 3.11 --with pytest --with pyyaml --with pydantic python -m pytest utils/changelog_gate_tests/ -q
  • actionlint .github/workflows/run-sweep.yml
  • bash -n utils/merge_with_reuse.sh
  • live selector check against PR [NV] Add MiniMax M3 B300 Dynamo vLLM recipes #1863 and run 27951287552

Note

Medium Risk
Changes CI merge-time sweep reuse gating and the supported merge helper path; wrong policy could skip needed benchmarks or accept stale artifacts, though evals-only remains blocked and tests encode the new rules.

Overview
Full-sweep PRs with the all-evals modifier can now use /reuse-sweep-run and merge_with_reuse.sh, instead of being treated like evals-only runs that must re-benchmark.

The reuse-sweep-gate job in run-sweep.yml no longer blocks on the all-evals label—only evals-only (including PRs with both modifiers) still skips the reuse gate and cannot reuse artifacts. find_reusable_sweep_run.py defaults --reuse-incompatible-label to evals-only only, and the merge helper drops its early exit for all-evals.

AGENTS.md and .github/workflows/README.md document the split: all-evals full sweeps are reusable; evals-only runs are not. Gating and reuse tests are updated to match (e.g. full-sweep-enabled + all-evals + authorized reuse now skips setup).

Reviewed by Cursor Bugbot for commit a821264. Bugbot is set up for automated code reviews on this repo. Configure here.

@Oseltamivir Oseltamivir requested a review from a team June 23, 2026 04:04
@Oseltamivir Oseltamivir merged commit 4ad21ea into main Jun 23, 2026
6 checks passed
@Oseltamivir Oseltamivir deleted the fix/all-evals-sweep-reuse branch June 23, 2026 04:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Development

Successfully merging this pull request may close these issues.

1 participant