Skip to content

Move library data and imported files into a standalone service - #154

Merged
RobRoyce merged 2 commits into
mainfrom
feature/standalone-storage
Oct 8, 2026
Merged

RobRoyce merged 2 commits into
mainfrom
feature/standalone-storage

Conversation

@RobRoyce

@RobRoyce RobRoyce commented Oct 8, 2026 •

Copy link
Copy Markdown
Owner

Store projects, sources, and managed file copies in a standalone Node.js SQLite service. Use authenticated local requests from the desktop client instead of renderer storage. Add repeatable legacy import and portable library backup. Imported files remain available after the originals are moved or deleted.

Depends on #153. Merge the stack in order with merge commits, retargeting each next PR to main after its predecessor lands. Do not squash or delete intermediate branches during integration.

Validation on the combined stack at 6ee979b2615db242ca6525bb7d0ccc36d563479f, using Node.js 24.21.0 in an isolated checkout:

  • Clean immutable dependency install: passed.
  • Development build and storage type check: passed.
  • Unit tests: 8 passed.
  • Storage tests: 35 passed.
  • Hidden-window desktop tests: 4 passed.
  • Headless Chrome tests: 6 passed.
  • Unsigned production package and packaged workflow: passed (1 test).

Each historical intermediate branch was not rebuilt independently. Signing, live AI credentials, other operating systems, and broad dependency security auditing are outside this review.

Focused review found no confirmed blocker in the combined stack. The review covers basic workflows, data preservation, authentication, and the dependency changes. Broader modernization remains separate work.

RobRoyce and others added 2 commits October 6, 2026 16:02
Move projects, sources, and imported files out of renderer localStorage
into a standalone service that runs with Node.js 24, without Electron.

Storage service (src/kc_storage):
- SQLite through node:sqlite with versioned schema migrations.
- Loopback HTTP API v1 with bearer token, Host and Origin checks, input
  validation, and size limits. No route takes a filesystem path.
- Managed file storage: uploads are copied, hashed, and stored by asset ID.
- One process per data directory (lock file); clean shutdown on SIGTERM,
  SIGINT, or closed stdin; temporary and unreferenced files are swept at
  start.
- migrate: imports a knowledge-backup v1 file with dry run, validation
  before writes, a pre-migration snapshot, one transaction, idempotent
  reruns, and a report of every item and missing file.
- backup/restore: portable tar with manifest and SHA-256 per file.
- Contracts and record mapping in src/kc_contracts, with no app imports.

Desktop integration:
- Electron starts the service per instance (data/library, ephemeral port,
  random token) and passes the address through A2E:Backend:Info.
- StorageService loads the library at startup and writes changes in order.
  The renderer no longer stores projects or sources.
- File sources are copied into the service when added to a project.
  The Document tab, thumbnails, "open file", and chat text extraction use
  the managed copy.
- Settings > Backup exports the library; the renderer backup now covers
  chat history and preferences.

Tests: 16 service tests (yarn test-storage) and an end-to-end workflow
with restart, deleted originals, export, and restore into a new profile.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

@RobRoyce RobRoyce left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Focused integration review of PR #154 at 46ef2269bfd909e3ff0fe89876390872f9e78f65.

No confirmed blocking finding in the combined revival stack. The independent code review checked storage writes, schema migration, managed files, backup/restore, local authentication, session recovery, Electron startup, and dependency removal. Main's conflicts were resolved without downgrading the stack's dependencies. All original commits are preserved.

Validation on the combined stack at 6ee979b2615db242ca6525bb7d0ccc36d563479f, using Node.js 24.21.0 in an isolated checkout:

  • Clean immutable dependency install: passed.
  • Development build and storage type check: passed.
  • Unit tests: 8 passed.
  • Storage tests: 35 passed.
  • Hidden-window desktop tests: 4 passed.
  • Headless Chrome tests: 6 passed.
  • Unsigned production package and packaged workflow: passed (1 test).

Each historical intermediate branch was not rebuilt independently. Signing, live AI credentials, other operating systems, and broad dependency security auditing are outside this review.

This is a review record submitted through the PR author's account, not the independent approving review required by main's branch protection. CodeQL status is reported separately by GitHub.

@RobRoyce
RobRoyce marked this pull request as ready for review October 8, 2026 17:41
@RobRoyce
RobRoyce changed the base branch from recovery/local-dev to main October 8, 2026 18:14
@RobRoyce
RobRoyce merged commit 9acd06e into main Oct 8, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant