Repository navigation
simplify(automation): hold one lifetime file lease per task key - #4542
Conversation
Replace the token-file task-lock protocol (random ownership token, hard-link publication, staging files, PID/age reclamation, retrying destructor unlink) with a nonblocking exclusive FileLease on a stable `<key>.lease` inode held for the task and retained-settlement lifetime. The lease file is never unlinked, so exclusion cannot split across inodes. A live pre-lease holder (`<key>.lock` naming a live pid) still keeps the key; dead or pid-reused owners' artifacts are retired and unverifiable ownership is a typed error. The stale-lock timeout no longer applies. Refs #4426 Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
|
Fleet logs show five complete 48,000-function generations take 53–65 seconds each. The default medium deadline kills the final refresh near 300 seconds despite steady successful publication. Declare the measured aggregate workload through the existing Cargo-to-Bazel size metadata, preserving every refresh, memory bound, and assertion. The current #4542 run independently confirmed the same limit: 170/171 tests passed; complete generations published at 62, 123, 184, and 245 seconds before the final refresh was interrupted at 300.1 seconds with nominal memory usage. Failure evidence: https://github.com/ScriptedAlchemy/tracedecay/actions/runs/40248084891/job/121393230765
Resolve .github/workflows/ci.yml to master's ci-windows configuration, which already carries this branch's --jobs=2 Windows concurrency bound. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
PASS+NOTES. Independent runtime verification of exact head Fresh macOS checks used
Source review confirms that both stable coordination files remain held through retained settlement. Dead-owner retirement removes only the owner record. The cross-process test directly checks the pre-publication gap, old-versus-new exclusion, explicit release, process death, and stable legacy coordination identity. Evidence limits. The legacy child replays the shipped protocol, checked against master source, rather than launching a published older binary. PID-identity and symlink denial received source review only. This is not a full journey or Windows runtime verdict. No source edits, assertion changes, or merges were made. Net CI changes are empty against master. The separate CLI test-size adjustment preserves the workload and assertions; its archived failure log confirms the 300.1-second deadline. The known unused |
Refs #4426. Parent link: none (stack S3 link 1, base master).
Problem
Automation task exclusivity was a second locking protocol built on top of a file lock: a published
<key>.lockfile with random ownership token, staging file, hard-link publication, PID/start-time/age reclamation, and a destructor that retried durable unlinks with sleeps.Fix
AutomationTaskLockis now one nonblocking exclusiveFileLeaseon a stable<key>.leaseinode, held for the task and retained-settlement lifetime (the retained settlement guard already owns the lock value). The lease file is never unlinked, so exclusion cannot split across inodes. Deleted: token generation, hard-link publication, staging ownership, uncertain-publication states, age/PID reclamation, destructor unlink/retry/fsync,stale_lock_secs()and the job stale-lock constant, and the acquirenow_secs/stale_after_secsparameters.Transition from the old format
A live pre-lease holder (
<key>.locknaming a live pid whose start time matches) still keeps the key. Artifacts of a confirmed-dead or pid-reused owner are retired (.lockand.lock.lock). Unparseable or unverifiable ownership is a typed error, never treated as abandoned.Not done (needs a settings-contract decision)
The
stale_lock_secssetting (domainAutomationTaskSettingsV1,deny_unknown_fields, CLI flags, generated SDK) is still accepted and validated but no longer read. Removing it breaks persisted settings without a migration, so it is left for a follow-up.Verification (Bazel, nonzero counts)
//crates/tracedecay-automation-runtime:unit_test: 432+ passed, including new tests: one-winner under 16 concurrent acquirers, independent keys, release admits next, lease file/inode survives release and is observed by a foreign open file description, no-runtime release, final-symlink rejection, live/dead/reused/unverifiable pre-lease owners.//crates/tracedecay:automation_runner_test: PASSED.//crates/tracedecay-daemon-service:unit_test: PASSED.//crates/tracedecay:unit_test: the lock-touching tests (shutdown_waits_for_accepted_curator_settlement_and_lock_release,partial_replay_reuses_prior_scheduler_skip_without_current_publication) pass;an_in_flight_files_resource_observes_transport_cancellationandindependent_reads_complete_out_of_order_with_exact_idsfailed only in the full run under load average ~45 and pass alone.bazel build --config=clippyfor automation-runtime, daemon-service and tracedecay: clean.🤖 Generated with Claude Code