fix(fleet): bind bootstrap reuse to lease identity - #72
Merged
Conversation
Integrate PR #58 onto current main while preserving active-harness hydration. Cover legacy manifests, mismatched and matching lease IDs, active-run preservation, and retry after a failed bootstrap. Co-authored-by: Vincent Koc <vincentkoc@ieee.org>
|
🦞👀 Pull request received. I will update this pull request when review starts. |
5 tasks
clawsweeper
Bot
added
merge-risk: 🚨 compatibility
|
Codex review: needs real behavior proof before merge. Reviewed August 28, 2026, 5:21 AM ET / 09:21 UTC. ClawSweeper reviewWhat this changesThe PR makes native fleet recovery rehydrate a replacement machine unless the recorded bootstrap lease ID matches the active lease, with regression coverage for legacy manifests and failed hydration retries. Merge readiness⛔ Blocked until real behavior proof is added - 2 items remain Keep open: the pinned base still trusts a timestamp alone, while this focused branch binds reuse to the active lease and covers legacy, replacement, and failed-hydration paths. No correctness defect was found, but external-contributor real-behavior proof is still required before merge. Priority: P2 Review scores
Verification
How this fits togetherThe native-eval fleet controller resumes benchmark runs on leased remote machines. It reads the local run manifest, decides whether the remote runner is usable, hydrates a lease when needed, and then dispatches the selected harness. flowchart LR
A[Run manifest] --> B[Fleet recovery]
C[Active remote lease] --> B
B --> D{Remote run missing?}
D -->|Yes| E{Bootstrap bound to lease?}
E -->|No| F[Hydrate remote runner]
E -->|Yes| G[Dispatch harness]
F --> G
D -->|No| H[Preserve or resume remote run]
Before merge
Agent review detailsSecurityNone. Review metrics
Root-cause clusterRelationship: Members:
Proposal only: this assessment does not dispatch repair, suppress jobs, mutate sibling items, close, or merge anything. Merge-risk optionsMaintainer options:
Technical reviewBest possible solution: Land the lease-identity check with a redacted real controller trace showing a legacy timestamp-only recovery hydrates the replacement lease and dispatches successfully. Do we have a high-confidence way to reproduce the issue? Yes, from source: a recovery entry with an old bootstrap timestamp, no valid active bootstrap lease ID, and a missing remote run follows the former skip-hydration path. The focused test models that state, although this review did not run the remote fleet. Is this the best way to solve the issue? Yes. Binding bootstrap reuse to the actual lease identity is the narrowest durable repair because timestamps cannot establish that a replacement machine contains the runner. AGENTS.md: not found in the target repository. Codex review notes: model internal, reasoning high; reviewed against 643c395f6793. LabelsLabel changes:
Label justifications:
EvidenceWhat I checked:
Likely related people:
Rank-up movesOptional improvements that raise the rating; they are not merge blockers.
Rating scale
Overall follows the weaker of proof and patch quality. Workflow
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Reviewed integration of #58: rebootstraps native evals when recovery replaces a lease, handling legacy timestamp-only manifests, replacement leases, and failed hydration retries without disturbing running jobs. Regression reproduced first; 48 focused + 460 full-suite tests pass, autoreview clean through P2. Credit to @vincentkoc for the original repair in #58. Fixes #55.