Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
87 changes: 79 additions & 8 deletions DEVELOPMENT.md
Original file line number Diff line number Diff line change
Expand Up @@ -38,15 +38,32 @@ yarn start

## Architecture

| Part | Location | Runs in |
| Part | Location | Runs in | Owns |
| --- | --- | --- | --- |
| Frontend | `src/kc_angular` | Electron renderer or a browser | Navigation, presentation, editing, temporary UI state, API calls |
| Backend (storage service) | `src/kc_storage` | Separate Node.js process | Projects, sources, managed files, validation, persistence, backup and restore, browser sessions, browser UI files |
| Desktop | `src/kc_electron` | Electron main process | Windows, native file access by path, watched folders, thumbnails and file icons, embedded browser, drag-out, backend start and stop. Also the chat server (for now). |
| Contracts | `src/kc_contracts`, `src/kc_shared` | Shared | Data types, record mapping, default settings. No Angular or Electron imports. |

The frontend reaches platform functions only through small capability
interfaces in `src/kc_angular/src/app/platform/`:

| Interface | Desktop | Browser |
| --- | --- | --- |
| UI | `src/kc_angular` | Electron renderer |
| Desktop functions, chat server | `src/kc_electron` | Electron main process |
| Storage service | `src/kc_storage` | Separate Node.js process |
| API contracts | `src/kc_contracts` | Shared. No application imports. |
| `Platform` | All features | Library features; desktop-only actions are hidden or disabled with a reason |
| `BackendService` | IPC addresses, bearer tokens | Same origin, session cookie, CSRF header |
| `SettingsStore` | Electron settings file | `localStorage` |
| `WindowControls` | Minimize, maximize, zoom | Not available |
| `NativeFiles` | Original file path, copy by path, open in default app | Not available |
| `ManagedFiles` | `blob:` view, default app | Same-origin URL with filename, new tab |

The design of the storage service is in
[docs/storage-service.md](docs/storage-service.md).
Both clients import files the same way: the user selects a `File`, the
frontend uploads its bytes (`POST /v1/assets`), and the source stores the
asset ID. IDs come from `crypto.randomUUID()`.

Designs: [docs/storage-service.md](docs/storage-service.md),
[docs/desktop-storage-completion.md](docs/desktop-storage-completion.md),
[docs/browser-library.md](docs/browser-library.md).

### Data ownership

Expand All @@ -70,14 +87,16 @@ addresses and tokens through IPC (`A2E:Backend:Info`).

### Access boundary

- Verified: a request without the correct token gets `401`. The window of
- Verified: a request without the correct token or browser session gets `401`. The window of
one instance does not know the address or token of another instance, so
it cannot use the other instance's servers by mistake.
- Verified: the servers do not accept connections from other computers.
- Not protected: other programs that run as the same macOS user. Such a
program can read the token (for example, from the storage service's
environment, `KC_STORAGE_TOKEN`), or read the profile files directly.
The real boundary is the operating-system user account.
- Verified (browser client): another local site in the same browser cannot
read the API or write to it, and its requests leave no records or files.
- The browser extension server (port 9000, off by default) has no token.

Only one instance can use a profile at a time. A second start with the same
Expand Down Expand Up @@ -130,6 +149,48 @@ out of the autoscan folder. Make a backup first.
Do not confuse `~/Library/Application Support/Knowledge`. macOS owns that
directory (`knowledgeC.db`). Knowledge does not use it.

## Browser client (no Electron)

```sh
yarn build-angular-dev
yarn browser
```

`yarn browser` starts the storage service with the built UI and prints a
one-time launch link, for example
`http://127.0.0.1:53124/#launch=...`. Open it in Chrome. Press Enter in the
terminal for a new link. Press Ctrl+C to stop.

Options: `--data-dir <dir>` (default `.dev-profiles/browser/library`),
`--web-root <dir>` (default `src/kc_angular/dist/main`), `--open` (macOS:
open the default browser).

What works in the browser: projects, file upload (PDF, text, and other
files), the Document view for PDF, text, and images, topics and metadata,
search, opening managed files in a new tab, library export and restore,
chat and preferences backup (browser local storage).

Desktop only (hidden or disabled in the browser, with a reason): saving
websites and the example websites, chat, the built-in browser, watched
folders and extension settings, window controls, opening files in other
apps, showing files in Finder, file thumbnails and file icons, dragging
files out.

### Browser session

- The launch code is 256 random bits, single use, and expires after 5
minutes. Only a bearer-token client (the launcher) can create one.
- The page exchanges it for an `HttpOnly`, `SameSite=Strict` cookie,
`kc_session_<port>`, and a CSRF token that stays in page memory.
- Writes need the CSRF token and the service's own `Origin`. Requests that
a browser marks as cross-site or same-site (another local port) are
refused.
- A session ends after 30 minutes without requests, after 12 hours, on
`DELETE /v1/session`, or when the service stops.
- The bearer token is never printed or sent to the browser. The launch
code appears once in the terminal and in the first URL fragment.
- This is a local, single-user setup. It is not a hosted deployment.

## Storage service without Electron

```sh
Expand Down Expand Up @@ -265,6 +326,7 @@ Stop the app first. The command uses the same checks as the app.
| `yarn workspace kc_storage typecheck` | Storage service types (TypeScript 5) | |
| `yarn e2e` | Desktop UI with real profiles (below) | macOS, `yarn build-dev` first |
| `yarn e2e-packaged` | The unsigned package (below) | macOS arm64, `yarn package-local` first |
| `yarn e2e-browser` | The browser client in Google Chrome, no Electron (below) | Google Chrome, `yarn build-angular-dev` first |

The end-to-end tests start the real app with new, empty profiles in
`e2e/.output/<run>/`. They never use your normal profile.
Expand All @@ -291,6 +353,15 @@ The end-to-end tests start the real app with new, empty profiles in
two instances at the same time, and that no service process remains.
Output stays in `$TMPDIR/kc-packaged-*`. The test removes the app copy.

- `e2e/browser/library.e2e.mjs`: starts `yarn browser` with scratch
libraries and opens the launch link in headless Google Chrome. It
creates a project, uploads a PDF and a text file, shows both, annotates,
searches, restarts the service and the browser, checks that the records
remain, opens the managed files in new tabs, exports the library, and
restores it through the UI into a second library. It also checks
requests without a session, reads and writes from another local site,
and that the launcher does not print the bearer token.

Test files are in `e2e/fixtures/`. To delete old test output, delete
`e2e/.output/`.

Expand Down
138 changes: 138 additions & 0 deletions docs/browser-library.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,138 @@
# Browser Library Workflow (Milestone 3)

## Purpose

Run one complete library workflow of the existing Angular interface in a
normal browser, without Electron. Keep the desktop app working.

Workflow: create a project, upload files, show sources, annotate, search,
restart, open managed files, export and restore the library.

Not in this milestone: website extraction in the browser, chat in the
browser, hosted deployment, accounts.

## Startup dependencies found

The Angular app could not start in a browser. These dependencies run at
startup, before any user action:

| Dependency | Where | Problem in a browser |
| --- | --- | --- |
| Settings from Electron (`A2E:Settings:*`) | `SettingsService`, used by most services | `window.api` is undefined. No settings, no theme, no search options. |
| Field initializers `window.api.send` | `SettingsService`, `AutoscanService`, `ExtensionService`, `ElectronIpcService`, display settings | Throw when the service is created. |
| UUIDs from Electron (`A2E:Uuid:Generate`) | `UuidService` (projects, sources) | No IDs. |
| Server addresses and tokens (`A2E:Backend:Info`) | `BackendService` | No API address. |
| App version (`A2E:Version:Get`) | `StartupService` | Throws. |
| Window controls | `AppComponent` | Throws on click. |
| `kc_shared` imports an Angular model | `kc_shared/models/project.model.ts` | Shared code depends on the frontend. |

Workflow dependencies:

| Step | Desktop dependency |
| --- | --- |
| Select and import a file | `File.path`; copy by path through Electron (`A2E:Storage:ImportFile`) |
| File icon and thumbnail | `A2E:FileSystem:FileIcon`, `A2E:FileSystem:FileThumbnail` |
| Show a managed file | Fetch with bearer token, then `blob:` URL (filename lost) |
| Open a managed file | `A2E:Storage:OpenAsset` (default app) |
| Save a website | Renderer fetches other sites (needs Electron) |
| Chat | Electron chat server |

## Responsibilities

| Part | Owns |
| --- | --- |
| Frontend (`kc_angular`) | Navigation, presentation, user input and editing, temporary UI state, calls to the application API. Uses only the capability interfaces below. |
| Backend (`kc_storage`) | Projects, sources, managed files, validation, persistence, backup and restore, browser sessions, serving the browser UI files. Same API for all clients. |
| Electron (`kc_electron`) | Windows and menus, native file selection and file access by path, watched folders (autoscan), desktop capture, native shortcuts and drag-out, embedded browsing, thumbnails and file icons, local backend start and stop. |
| Contracts (`kc_contracts`, `kc_shared`) | Data types, record mapping, default settings. No Angular or Electron imports. |
| Chat and extraction | Stay in the Electron chat server for now. See "Next extraction steps". |

## Capability interfaces (frontend)

Small interfaces. Each has a desktop and a browser implementation. The
platform is chosen once at startup: desktop if the preload bridge exists.

| Interface | Purpose | Desktop | Browser |
| --- | --- | --- | --- |
| `Platform` | Which optional features exist | All features | Library features only |
| `BackendConnection` | Address and request authorization for the library API (and chat) | IPC info, bearer token | Same origin, session cookie, CSRF header |
| `SettingsStore` | Load, save, and default settings | Electron settings file (IPC) | `localStorage`, shared defaults |
| `WindowControls` | Minimize, maximize, zoom | IPC | Not available (controls hidden) |
| `NativeFiles` | Original path of a selected file, copy a file by path, open in default app, show in folder | IPC | Not available |
| `ManagedFiles` | URL to show or open a managed file | `blob:` URL from an authorized fetch | Direct same-origin URL with the filename |

Other changes:

- IDs: `crypto.randomUUID()` on both platforms. No IPC.
- File import on both platforms: the user selects a `File`. The frontend
uploads its bytes with `POST /v1/assets` and stores the returned asset
ID in the source. The desktop also sends the original path as metadata.
Path-based copy stays in Electron for watched folders only.
- `ElectronIpcService` stays the desktop adapter for embedded browsing and
other desktop features. In a browser it is inert and not called.
- Desktop-only actions are hidden or disabled in the browser, with an
explanation: save website, chat, embedded browser, import and watched
folder settings, window controls, open in default app, show in folder,
drag-out, file thumbnails.

## Browser session

The service serves the built Angular files and the API from one origin,
`http://127.0.0.1:<port>`. The frontend uses its own origin as the API
address. So the page always talks to the service instance that served it.

| Question | Answer |
| --- | --- |
| How does the user open a session? | `yarn browser` starts the service with a random bearer token that it never prints. It asks the service for a launch code and prints `http://127.0.0.1:<port>/#launch=<code>`. The page reads the code from the URL fragment (not sent to servers, not in server logs), removes it from the address bar, and exchanges it with `POST /v1/session`. |
| What is the launch code? | 256 random bits. Single use. Expires 5 minutes after creation. Only a bearer-token client can create one (`POST /v1/session/launch`). |
| How are requests authenticated? | Cookie `kc_session` (256 random bits, `HttpOnly`, `SameSite=Strict`, `Path=/`). The service keeps sessions in memory. Desktop clients keep using the bearer token. |
| How are state-changing requests protected? | For cookie sessions, `POST`, `PUT`, and `DELETE` need all of: header `X-Knowledge-CSRF` equal to the session's CSRF token, and `Origin` equal to the service origin. Any cookie request with `Sec-Fetch-Site: cross-site` is refused. `SameSite=Strict` stops the browser from sending the cookie from other sites. The CSRF token lives only in page memory. After a reload, `GET /v1/session` returns it. |
| How does a session end? | 30 minutes without requests, 12 hours after creation, `DELETE /v1/session`, or a service restart (sessions are in memory). |
| What stays outside URLs, storage, and logs? | The bearer token. The session ID is only in an `HttpOnly` cookie. The launch code is in the terminal output and in the first URL only; it is single use and short-lived. |
| Other checks | `Host` must be `127.0.0.1:<port>` or `localhost:<port>` (DNS rebinding). The service listens on `127.0.0.1` only. CORS allows only the desktop origin `null`. CORS and loopback are not authentication. |

Trust boundary: the operating-system user. Programs that run as the same
user can read process environments, the profile files, and the terminal
output. This design does not protect against them, and it is not a design
for a hosted service.

## Managed file display

- The service serves `GET /v1/assets/<id>/content/<filename>`. The last part
sets the name that the browser PDF viewer and downloads show.
- Inline only for PDF, plain text, and raster images. Text is served as
`text/plain`. All other types are downloads (`Content-Disposition:
attachment`). Every response has `X-Content-Type-Options: nosniff` and a
sandbox `Content-Security-Policy`, except PDF. So imported files cannot
run scripts with the application's origin.
- The browser shows PDFs with its own viewer.

## Tests

- Service: sessions (launch code single use and expiry, cookie flags, CSRF
header, Origin, cross-site fetch metadata, idle and absolute expiry,
logout), bearer clients unchanged, UI file serving, content headers.
- Browser (Chrome through `playwright-core`, no Electron): the workflow
above, restart and reopen, unauthorized requests.
- Existing service, desktop, and packaged tests.

## Next extraction steps (not in this milestone)

- Chat: move the chat routes and OpenAI calls from the Electron chat server
into the backend. Keep the API key in the backend data directory.
- Extraction: move website fetching and PDF text extraction into the
backend, so the browser can save websites.

## Verified (2026-10-07, macOS 26.6 arm64, Google Chrome 154)

| Command | Result |
| --- | --- |
| `yarn build-dev` | Pass |
| `yarn test` | 8 of 8 pass |
| `yarn test-storage` | 28 of 28 pass (6 new session and UI-serving tests) |
| `yarn workspace kc_storage typecheck` | Pass |
| `yarn e2e` | 2 of 2 pass (desktop) |
| `yarn e2e-browser` | 1 of 1 pass (Chrome, no Electron) |
| `yarn package-local`, `yarn e2e-packaged` | Pass |

Not verified: Safari, Firefox, and Chrome on other systems.
14 changes: 11 additions & 3 deletions docs/follow-ups.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,12 +7,11 @@ milestones. Not started.

| Item | Reason |
| --- | --- |
| Orphan asset cleanup policy | Deleting a source keeps its managed file. Only files without records after a rollback, crash, or failed restore are removed. |
| Orphan asset cleanup policy | Deleting a source keeps its managed file. Files are uploaded when selected, so a file removed from the inbox also leaves a managed file. Only files without records after a rollback, crash, or failed restore are removed. |
| Restore into a library that has records (merge or replace) | Restore refuses a non-empty library now. |
| Move the extracted-text cache into the service | The chat server keeps it in `data/storage/sources/`, outside the library. |
| Thumbnails in the service | Electron makes thumbnails from a local copy of the managed file. |
| Move favicon cache, inbox, and chat history out of local storage | They stay in the renderer for this milestone. |
| Remove `kc_shared` dependency on Angular models | `kc_shared/models/project.model.ts` imports an Angular model. Electron needs the `@shared` alias because of it. |
| Stronger local access control | The token stops requests without it. Programs that run as the same OS user can read it. |

## Packaging and release
Expand All @@ -35,9 +34,18 @@ milestones. Not started.
| Replace the fixed API key passphrase | The key file is obfuscated, not encrypted. |
| Root `postinstall` key is outside `scripts` | It never runs. Remove it or move it. |
| Fresh-profile route error (`inbox/undefined`) | Cosmetic error in the log on first start. |
| PDF viewer title shows the blob ID | Managed PDFs open from a `blob:` URL. |
| Desktop PDF viewer title shows the blob ID | The desktop shows managed PDFs from a `blob:` URL. The browser client shows the filename. |
| Select a project after a library restore | After restore, no project is selected. The user selects one in the tree. |

## Browser client

| Item | Reason |
| --- | --- |
| Save websites in the browser | Needs website fetching and extraction in the backend. |
| Chat in the browser | Needs the chat server in the backend. |
| Browser settings and inbox | Kept in browser local storage, separate from the desktop. |
| Other browsers | Only Google Chrome is tested. |

## Features (separate tasks)

- Full-text search over extracted text.
Expand Down
Loading