Skip to content

fix: make the plugin generation store actually reclaimable (#4689) - #4690

Merged
Trecek merged 7 commits into
developfrom
worktree-impl-retirement-engine-4689
Aug 18, 2026
Merged

fix: make the plugin generation store actually reclaimable (#4689)#4690
Trecek merged 7 commits into
developfrom
worktree-impl-retirement-engine-4689

Conversation

@Trecek

@Trecek Trecek commented Aug 18, 2026

Copy link
Copy Markdown
Collaborator

Fixes #4689.

What was broken

No directory under ~/.autoskillit/plugin-generations/ has been reclaimable since the generation store was introduced in 0.10.933. Observed on a live machine: 47 version directories accumulated over ~10 days, plus a retirement record sitting 10 days past its deadline with its target genuinely superseded.

Three defects compounded, and a fourth would have been triggered by fixing them.

1. Records were routed to an owner that could never contain them

default_plugin_retirement_coordinator() registered the INSTALLED_PLUGIN owner at current_installed_plugin_root().parent — the legacy Claude plugin cache — while publish_generation() writes records under the generation store. sweep_due() dispatches by kind, so every generation record reached an owner whose managed_root could not contain it.

try_reclaim() rejects an uncontained record without removing it, so each record was re-rejected on every sweep, forever.

Fixed by giving the generation store its own PLUGIN_GENERATION kind and an owner rooted at generation_store_root() — all versions, not one. The legacy tree keeps its own owner: reconcile_install_artifacts() still enqueues it, and the two roots are disjoint, so one owner cannot serve both.

2. Retirement was scoped to a single version

_enqueue_prior_generation() used managed_root=version_root, so it only queued superseded incarnations of the same version. A superseded version was never queued at all. prune_stale_generations() replaces it, walking every version × incarnation in one pass.

3. Every superseded version vouched for itself

Each version keeps its own per-version current symlink pointing at its own incarnation forever — nothing rewrites it when a newer version is published. Honoring that as protection meant no version was ever retirable.

A plugin-level current selector is now authoritative once established; only the live version's per-version selector is still honored (a consumer that resolved through it just before the flip may still be using it). Before any plugin-level selector exists, the old per-version behavior is retained — over-retaining rather than deleting something in use.

4. The Codex config would have broken the moment the above was fixed

~/.codex/config.toml bakes absolute generation paths for all its hooks and re-reads them at launch without re-resolving, and codex_prelaunch_transaction can write a per-session snapshot that outlives the version it names.

So retiring old generations breaks every safety guard baked into a stale config — branch_protection_guard, github_mutation_guard, write_guard, quota_guard, and the rest. This is not hypothetical: it happened during the investigation that produced this PR.

The bindingless resolver now prefers a version-independent selector whose absolute path stays valid across version bumps. trusted_hash is unaffected — it hashes the dispatcher through the symlink, and _dispatch.py is byte-identical across versions by design. Hook scripts need no change: .resolve() dereferences arbitrary symlink chains.

5. legacy_evidence was unreachable — and reaching it naively is dangerous

Migrated v1 evidence was never passed to try_reclaim, so it could not be reclaimed at all (12 entries sat unswept for three weeks). try_promote_legacy_evidence() re-derives an exact identity and hands it to the normal queue rather than deleting anything itself, so removal still flows through the existing lease- and identity-checked path.

Warning

The live legacy_evidence array records plugin-projections/.artifact-leases — the directory holding every running session's lease locks — as a "projection". An earlier prune_stale_projections enumerated that root with no dot-prefix filter. Anything that trusts the stored recognized_kind deletes the lease infrastructure out from under every live session.

is_reclaimable_artifact_path() admits only non-hidden direct children of the managed root, and promotion re-derives eligibility from scratch rather than trusting persisted classification — because that stored field is exactly what a past classification bug got wrong.

Why this shipped undetected

Nothing exercised the real coordinator against a real generation artifact. Every existing fixture (_installed_identity() and friends) builds the legacy cache shape, and tests/server/test_lifespan.py mocks sweep_due entirely.

tests/contracts/test_generation_retirement.py closes that gap: the routing regression, cross-version queueing, both selectors, version-bump survival of the pinned Codex path, and the .artifact-leases landmine. Every test fails against pre-fix code.

Design notes

  • Enqueue-only. prune_stale_generations() never deletes; removal stays in try_reclaim under its own lease and identity checks. This also keeps the change clear of the raw-mutation AST ratchet in tests/infra/test_plugin_source_ratchets.py.
  • Publish-time trigger only. Enqueueing recomputes a full content-tree digest per candidate, so wiring this into session launch would re-hash the whole backlog on every launch. Publication is the only event that creates staleness.
  • 24h grace for cross-version retirement: the sweep runs once per MCP server startup, not on a timer, so the window must outlast the gap between restarts.
  • Plugin-level selector flip is best-effort — the per-version flip is already durable and must not be rolled back. A persistent failure fails safe (over-retention, never deletion).
  • An arch guard asserts every PluginArtifactKind has a registered owner, so a future storage location can't silently repeat defect 1.

Known limitation

Codex sessions whose config snapshot predates this fix keep an old-style resolved path and stay as exposed as today until their next fresh prelaunch. Bounded by the grace window, not eliminated — an already-written config in a running process can't be retroactively fixed.

Still open (not in this PR)

Whether Codex fails open (guards silently stop running — a safety hole) or closed (tool calls denied — an outage) when a hook path is missing. tests/execution/backends/test_hook_deny_efficacy_probe.py documents this as uncharacterized; it needs a live CLI probe and determines defect 4's true severity.

Testing

task test-check: 39762 passed. The 16 remaining failures are pre-existing on develop — verified by stashing this branch and re-running — and are environmental (they need a real claude executable and /dev/shm paths unavailable in an isolated worktree venv). pre-commit run --all-files passes, including mypy.

🤖 Generated with Claude Code

No directory under ~/.autoskillit/plugin-generations/ has been reclaimable
since the generation store was introduced in 0.10.933. Three defects
compounded, and a fourth would have been triggered by fixing them.

Routing. default_plugin_retirement_coordinator() registered the
INSTALLED_PLUGIN owner at current_installed_plugin_root().parent — the
legacy Claude plugin cache — while publish_generation() writes records
under the generation store. sweep_due() dispatches by kind, so every
generation record reached an owner whose managed_root could not contain
it. try_reclaim() rejects an uncontained record WITHOUT removing it, so
each record was re-rejected on every sweep forever. Observed: a record
10 days past its deadline, target genuinely superseded, never collected.

Fixed by giving the generation store its own PLUGIN_GENERATION kind and
an owner rooted at generation_store_root() — all versions, not one. The
legacy tree keeps its own owner, since reconcile_install_artifacts()
still enqueues it and the two roots are disjoint.

Scope. _enqueue_prior_generation() was scoped managed_root=version_root,
so it only queued superseded incarnations of the same version; a
superseded VERSION was never queued at all. prune_stale_generations()
replaces it, walking every version x incarnation in one pass.

Self-protection. Each superseded version keeps its own per-version
`current` symlink pointing at its own incarnation forever — nothing
rewrites it on a newer publish. Honoring that meant every version
vouched for itself. A plugin-level `current` selector is now
authoritative once established, and only the live version's per-version
selector is still honored.

Codex coupling. ~/.codex/config.toml bakes absolute generation paths for
all its hooks and re-reads them at launch without re-resolving; a
per-session snapshot can outlive the version it names. Retiring old
generations therefore breaks every safety guard baked into a stale
config — this happened in practice. The bindingless resolver now prefers
the version-independent selector, whose absolute path stays valid across
version bumps. trusted_hash is unaffected: it hashes the dispatcher
through the symlink, and _dispatch.py is byte-identical across versions.

legacy_evidence. Migrated v1 evidence was never passed to try_reclaim,
so it could not be reclaimed at all. try_promote_legacy_evidence()
re-derives an exact identity and hands it to the normal queue rather
than deleting anything itself.

That path is only safe with a guard: the live legacy_evidence array
records plugin-projections/.artifact-leases — the directory holding
every running session's lease locks — as a "projection". An earlier
prune enumerated that root with no dot-prefix filter. Anything that
trusts the stored recognized_kind would delete the lease infrastructure
out from under every live session. is_reclaimable_artifact_path()
admits only non-hidden direct children of the managed root, and
promotion re-derives eligibility from scratch instead of trusting
persisted classification.

Coverage. Nothing exercised the real coordinator against a real
generation artifact: every existing fixture built the legacy cache
shape, and the lifespan test mocks sweep_due entirely. That is why this
shipped. tests/contracts/test_generation_retirement.py covers the
routing regression, cross-version queueing, both selectors, the
version-bump-survival of the pinned Codex path, and the
.artifact-leases landmine.

Refs #4689
Trecek added 6 commits August 18, 2026 12:47
… three retirement owners

InstalledPluginArtifactRetirementOwner.identity_for_path was the only one of
the three owner implementations (Installed/Projected/Generation) that skipped
the containment check before validating identity, so a caller holding an
out-of-root path got a different exception class here than from the other
two owners. Add the same self._contains() guard + PluginArtifactValidationError
already used by ProjectedPluginRetirementOwner and GenerationArtifactRetirementOwner.

Addresses review findings at cli/_plugin_artifact.py:439 and
workspace/_projected_artifact/_generation_publication.py:359 (same underlying
asymmetry, two vantage points).
…legacy_evidence

evidence.path being None or otherwise non-string-coercible raises TypeError
from Path(), which the existing except (OSError, ValueError) does not catch,
aborting the whole sweep pass. Add TypeError to the caught tuple so a
malformed legacy-evidence record degrades to LEGACY_EVIDENCE instead.

Addresses review finding at core/_plugin_cache.py (try_promote_legacy_evidence).
…'autoskillit' in _resolve_codex_hooks_dir

The bindingless resolution path hardcoded the string literal 'autoskillit' at
four call sites in the same function that already imports and uses
_AUTOSKILLIT_PLUGIN_KEY for the sibling semantic-key computation a few lines
below. A future plugin-key change would silently desync this resolver from
the constant it already depends on elsewhere in the same function.

Addresses review finding at execution/backends/_codex_hooks.py:118.
_is_selected_generation and prune_stale_generations docstrings each carried a
paragraph of historical/performance-rationale narrative duplicating the PR
commit message. Kept the load-bearing behavioral contract (dual-branch
fallback logic; the session-launch-vs-publish-time warning) and cut the
narrative exposition.

Addresses info findings at
workspace/_projected_artifact/_generation_publication.py:289 and :393.
is_reclaimable_artifact_path and try_promote_legacy_evidence docstrings each
carried a sentence of historical-bug narrative (a past prune_stale_projections
defect; a past .artifact-leases classification bug) better suited to a commit
message. Kept the load-bearing safety contract in each and cut the narrative.

Addresses info findings at core/_plugin_cache.py:297 and :726.
…ntract tests

test_superseded_version_is_enqueued_on_publish only asserted the superseded
generation was queued; add the complementary assertion that the currently
selected generation is NOT queued.

test_selected_generations_are_never_reclaimed discarded sweep_due()'s
outcome tuple, so it would pass vacuously if the sweep silently did nothing
(directory-existence and selector-resolution checks would still hold).
Assert RetirementOutcome.RECLAIMED is present to prove 1.0.0 was actually
processed while 2.0.0 (current) survives.

Addresses info findings at tests/contracts/test_generation_retirement.py:168
and :197.
@Trecek
Trecek added this pull request to the merge queue Aug 18, 2026
Merged via the queue into develop with commit 0949f8a Aug 18, 2026
4 checks passed
@Trecek
Trecek deleted the worktree-impl-retirement-engine-4689 branch August 18, 2026 20:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant