Own source capture and publish revision-safe search

[rohans0509]

The inherited ingestion pipeline transforms provider data directly into search entities. This fork retains native records and immutable blobs first, then projects them into search independently. Exact read/list/change, blob, email-thread, calendar and native Google Docs APIs preserve provider identities and enforce current visibility and revision checks.

The existing database and worker lifecycle own transactions, writer fencing, durable page progress and recovery. Native checkpoints advance only after committed terminal scans. Discovery-only sources explicitly report incomplete coverage; missing rows are not automatically classified as deleted. No second connector database or per-provider scheduling loop is introduced.

Owned capture implementations exist for Gmail, Google Calendar, Google Drive, Slack, Wispr, Linear, GitHub, Notion and Attio, with different qualification levels. Attio replaces the legacy entry-dropping extraction with separate CRM records, list memberships and notes; typed native models and an offline projection preserve that ownership. Its offset scans remain discovery-only: no live qualification, webhook delta or deletion guarantee. Wispr preserves per-meeting progress and rejects ambiguous continuation markers; observed rate-limit text stops capture without inventing a retry delay.

Managed Composio transport checks connected-account/toolkit identity and restricts provider hosts. Native sources verify their account/workspace boundaries. This does not establish a hosted Almanac end-user ownership binding by itself.

Validation and limits:

• The capture regression target passes288 tests. Attio's15 focused tests include real PostgreSQL: a failure after50 committed notes resumes at offset50 with a fresh writer, retaining51 notes without duplicate revisions. Eleven offline Attio projection tests cover identity separation, unavailable content and URL validation. Provider responses in these tests are synthetic.
• Prior pushed CI at8927e48 passed correctness and image build. Newer commits require fresh CI; local tests are not deployment proof.
• Live Gmail completed bounded frozen-query captures of242 records; filtered queries do not prove whole-mailbox history coverage. Calendar completed initial and saved-token cycles with181 active records and unchanged revisions. Drive captured55 records, then a new worker used its saved token with zero new changes. These were read-only trials, not induced edit/delete tests.
• A real Google Docs sample passed through captured native JSON, real local store HTTP, fixture-authenticated Almanac routing and the actual CLI: five provider GETs, one file, three blobs and four tabs. Exact JSON hashes and stale/wrong-account/revoked-binding rejection were checked. Comments and embedded media remain omitted; no Sheets or full-Drive completeness claim.
• Slack has partial restart evidence but no completed full-cycle proof. Wispr's bounded process-loss trial preserved three completed bodies and added a fourth; full traversal and exhaustive discovery remain unproven.
• Real-model search tests exercise semantic retrieval, exact titles, isolation and index lifecycle on a small corpus, not broad search-quality evaluation. Notion property projection, comments and file bodies remain unfinished.

Remaining: all17-provider coverage and action parity, full source lifecycle qualification, broader deletion/access withdrawal guarantees, hosted ownership attestation, backup/restore and cost-bounded Porter/AWS staging. No production cutover or release. Draft targets dev.

Almanac consumer integration: https://github.com/AlmanacCode/almanac/pull/430.