Reduce archived conversation disk usage with cold storage

[Quicksaver]

Summary

This adds compressed cold storage for archived conversations. Once a thread is archived, its conversation data and attachments are moved out of the hot state database into a dedicated archive.sqlite bundle, while unarchiving restores the complete thread tree before applying the domain command.

The lifecycle is durable and restart-safe: migrations 035/036 queue existing archived and deleted threads, background work resumes interrupted archive/delete jobs, and restoration keeps the cold bundle authoritative until the unarchive command commits successfully. Attachment ownership is preserved from exact persisted IDs and cold-bundle filenames throughout archive, restore, and durable deletion.

What Changed

• Added a dedicated cold-storage service that gzip-compresses thread-scoped SQL rows and attachments into chunked records in archive.sqlite.
• Removed successfully archived rows, attachments, and per-thread provider logs from hot storage, with incremental vacuuming to reclaim database pages.
• Restored archived thread trees transactionally before unarchive commands, with rollback on command failure and cleanup of the cold bundle only after a successful commit.
• Serialized archive-tree lifecycle operations with idle lock eviction, restored cleanup_pending bundles, and prevented queued archive work or post-commit failures from undoing a successful restore.
• Added migrations 035/036 with persistent manifests, maintenance records, and delete queues so interrupted work and legacy archived/deleted threads are recovered after restart without replacing fork-local migration metadata.
• Rechecked archived shell state at the destructive transaction boundary and required active provider, terminal, and log writers to quiesce before archiving hot rows.
• Changed thread deletion to permanently remove hot rows, cold bundles, attachments, provider logs, and related cleanup records.
• Kept non-missing attachment/provider-log directory failures retryable instead of treating them as empty directories.
• Matched attachment ownership by exact persisted IDs and cold-bundle filenames so colliding sanitized thread segments cannot remove another live thread's files.
• Resumed cleanup_pending manifests even when the archived shell row is missing, while preserving exact attachment metadata until durable delete cleanup succeeds.
• Added one-time compaction of legacy hot storage and startup processing for pending archive/delete work.
• Closed per-thread provider log writers before lifecycle cleanup so files can be removed safely.
• Applied optimistic archive visibility consistently to rendered rows, project sorting, and keyboard navigation on web, and kept in-progress unarchive states across the web and mobile archive interfaces.
• Added coverage for archive/restore round trips, binary SQL values, attachment handling, schema evolution, bounded restore reads, failure recovery, migrations, orchestration integration, and sidebar archive visibility.

Why

With fairly moderate usage, the user .t3 folder grew to more than 30 GB. That is far too much local data growth for normal use, and without lifecycle controls it gets out of hand very quickly. Archived and deleted conversations need to stop accumulating indefinitely in hot storage while remaining reliably recoverable when a user unarchives them.

Validation

• pnpm exec vp check passed with exit 0; it reported 9 pre-existing lint warnings outside the changed files.
• pnpm exec vp run typecheck passed with exit 0.
• pnpm exec vp run lint:mobile passed with exit 0; optional SwiftLint, ktlint, and detekt checks were skipped because those tools are not installed locally.
• pnpm exec vp test apps/server/src/orchestration/Layers/ThreadColdStorage.test.ts apps/server/src/orchestration/Layers/ThreadDeletionReactor.test.ts apps/server/src/orchestration/Layers/OrchestrationEngine.test.ts apps/server/src/persistence/Migrations/035_036_ThreadStorageLifecycle.test.ts passed: 4 files, 31 tests.
• pnpm exec vp test apps/web/src/components/Sidebar.logic.test.ts passed: 1 file, 58 tests.
• Playwright against isolated ports 5213/14213 created and archived a disposable thread; after the exit animation, the row was absent from the rendered and keyboard-addressable sidebar and the project showed No threads yet.

Proof

No attached visual proof is necessary; the data lifecycle and sidebar interaction are covered by the automated and isolated Playwright validation above.

---

Note

High Risk
Large new data-lifecycle path on the critical unarchive dispatch path; destructive archive/delete and filesystem I/O can corrupt or lose conversation data if rollback or queue recovery fails.

Overview
Adds gzip-compressed cold storage for archived conversations: thread-scoped SQL rows and attachments move into archive.sqlite, hot rows/attachments/provider logs are removed, and migrations queue legacy archived/deleted threads for background work.

Unarchive restores the cold bundle before the domain command commits; failed dispatches roll back the restore, and successful commits finalize by dropping the cold bundle. The lifecycle reactor now handles archive as well as delete, with startup replay of pending jobs and one-time legacy DB compaction.

Web and mobile get optimistic sidebar hide on archive and in-progress unarchive (spinners, disabled swipe/actions). unarchiveThread is now async on mobile so the route can track loading state.

^{Reviewed by Cursor Bugbot for commit a42fe41. Bugbot is set up for automated code reviews on this repo. Configure here.}

Note

Reduce archived conversation disk usage by introducing cold storage archival and restore

• Adds a ThreadColdStorage service (ThreadColdStorage.ts) that compresses and archives thread SQLite rows to a separate archive.sqlite database on thread archive, and restores them on unarchive.
• Extends ThreadDeletionReactor to handle archive, delete, and legacy-compaction lifecycle jobs, resuming pending work on startup via two new migrations (035 thread_archive_manifests, 036 thread_cleanup_queue).
• OrchestrationEngine now calls restoreTree before committing a thread.unarchive command, rolling back on dispatch failure and finalizing on success.
• Adds optimistic UI hiding for archived threads on web and mobile: threads disappear immediately on archive and reappear on failure, with per-row loading states during unarchive.
• Risk: two new migrations run on startup; existing archived/deleted threads are seeded into the new queue tables automatically.

^{Macroscope summarized 1ace10b.}

[coderabbitai]

Important

Review skipped

Auto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro

Run ID: dde76a98-8236-4972-9bf7-171701a7e78a

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[macroscopeapp]

Approvability

Verdict: Needs human review

This PR introduces a substantial new cold storage feature for archived conversations, including ~830 lines of new service code, database migrations, compression logic, and integration into the orchestration engine's dispatch flow. The scope and complexity of data lifecycle changes warrant human review.

^{You can customize Macroscope's approvability policy. Learn more.}

[macroscopeapp]

One Effect service convention issue found in the new ThreadColdStorage service. See the inline comment.

^{Posted via Macroscope — Effect Service Conventions}

[cursor]

Cursor Bugbot has reviewed your changes using high effort and found 1 potential issue.

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.}

^{Reviewed by Cursor Bugbot for commit a42fe41. Configure here.}