Skip to content

fix: prevent query-sensitive video collisions and unify ingestion identity - #3

Draft
ArietidsZ wants to merge 5 commits into
dots/validated-feed-redirects-20261028from
dots/source-identity-v2-20261028
Draft

ArietidsZ wants to merge 5 commits into
dots/validated-feed-redirects-20261028from
dots/source-identity-v2-20261028

Conversation

@ArietidsZ

@ArietidsZ ArietidsZ commented Oct 28, 2026 •

Copy link
Copy Markdown
Owner

Owned-client shutdown — 2026-10-30

Current head f938dd09510ecbd16fd62ca8428adb2401785b69 additionally closes the feed HTTP and transcription clients created by build_runtime. These were previously omitted from normal shutdown. An exit stack also closes these resources and SQLite if scheduler/summarizer cleanup raises.

Two new offline tests fail before the fix and pass afterwards (normal close and a failing summarizer close). Full local suite: 247 tests passed, one dependency deprecation warning; full Ruff and diff checks passed. No requests to model or feed endpoints. Local/published tree: 01a9a7b2b677f4d0b2a63f679bfdeb68fc3462b3. All six exact-head CI checks passed: run 40979177883, verified 2026-10-30 00:24 UTC. Construction-time failures and concurrent close remain outside this fix.


Normal-shutdown recovery — 2026-10-30

Previous head 67c186f1e17bb762b22c62fa23fc5015a8a325d8 records interrupted jobs before the runtime closes SQLite. Previously normal shutdown left freshly claimed jobs running; a restart within 30 minutes skipped them and the startup-only stale recovery never revisited them.

  • Preserve cancellation propagation and existing backoff/attempt limits. Interrupted in-progress videos become failed; already-done videos are not downgraded.
  • Five new offline regression cases failed before the fix and pass afterwards: INGEST/PROCESS × retry/terminal attempt, plus runtime shutdown and reopening the database to check durable state.
  • Local full tests/v2 suite passed (245 cases, one dependency deprecation warning); full ruff check vra tests and git diff --check passed. Tests use SQLite and fake services, not model APIs or user devices.
  • Local validated tree 3a04a9f55c385681ce2c704a81c1d714f7485163 matches the published tree. Remote CI for this exact head passed all six Linux/macOS/Windows × Python 3.11/3.12 checks: run 40978686135, verified 2026-10-30 00:18 UTC. Older-head CI remains historical.
  • Scope: cooperative normal shutdown only. Forced termination, repeated cancellation and database failures are not claimed fixed.

Stacked on #2. New source:v2 SHA-256 keys preserve query/order/encoding/path case; feed GUID remains metadata. Feed and API share this identity and default non-force queue payload, preventing separate jobs for one new source.

Legacy rows, UUIDs, artifacts and jobs are preserved without schema version changes. They are not automatically adopted: old lossy hashes could have overwritten source_url while retaining another video's completed summary. New ingestion may create a duplicate historical RSS entry; explicit legacy video_id processing remains available. This does not silently repair historical ambiguity.

Historical source-identity validation: 229 tests and Ruff passed locally. Coverage includes distinct watch?v=A/B, reused GUID, conservative URL distinctions, feed/API order permutations through real SQLite + ASGI transport, default-job dedup and legacy summary/job preservation. No database migration/reset, inference, downloads, deployment or merge. That earlier CI result does not validate the current head.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant