diff --git a/documentor/FORK.md b/documentor/FORK.md index b71aedebed2b..3fac724f655e 100644 --- a/documentor/FORK.md +++ b/documentor/FORK.md @@ -13,25 +13,27 @@ what a rebase meets. ## Upstream files changed -| File | What differs | Why | -| -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| The 11 table files: `packages/core/src/{account,credential,event,permission,project,session,share}/sql.ts`, `control-plane/workspace.sql.ts`, `data-migration.sql.ts`, `database/schema.sql.ts`, `database/path.ts` | Postgres table builder; `count`, `jsonb` and `text` columns from `effect-drizzle-pg`; time columns are `double precision` | Written by `documentor/scripts/convert-schema.py`, never by hand | -| `packages/core/src/database/schema.gen.ts`, `packages/core/schema.json`, `packages/core/drizzle.config.ts` | The Postgres baseline, as one statement batch | Generated; see "Regenerating the baseline" | -| `packages/core/src/database/database.ts` | Connects with `@effect/sql-pg`; verifies collation and migrations and changes nothing; a throwaway schema per layer when `OPENCODE_DATABASE_EPHEMERAL=1` | The service itself | -| `packages/core/src/database/migration.ts` | `migrate`, `migrateAll`, `verify`, `pending`, `BASELINE_IDS`, `ported` | Schema work is a command, not a start-up side effect | -| `packages/core/src/project/directories.ts` | `Transaction` is `Database.Transaction` | It was typed against the SQLite client | -| `packages/core/src/flag/flag.ts` | No `OPENCODE_DB` | Nothing reads it | -| `packages/core/package.json` | Depends on `@effect/sql-pg`, `pg`, `@opencode-ai/effect-drizzle-pg`; tests run with `--timeout 30000` | Each test database is a real schema, about 110 ms to build | -| `packages/opencode/src/cli/cmd/db.ts` | `db `, `db migrate [--all]`, `db provision`, `db grant-read`, all outside the application runtime | The runtime's database service refuses the schemas these commands exist to create and migrate | -| `packages/opencode/src/session/processor.ts` | One lock per processor around each read-then-write of a tool part; `tool-input-delta` and `tool-input-end` do not read a part the processor already tracks | The processor and the running tool both update a tool part. SQLite made each update atomic; a network database lets them interleave, and the tool's metadata was lost. Each read is a round trip, and a large edit streams thousands of deltas | -| `packages/opencode/src/acp/service.ts` | `prompt` passes a `messageId` that starts with `msg` to v1 as `messageID`, for a normal prompt and a slash command; an empty prompt passes none | DocuMentor names the user message a prompt creates, so its optimistic, live and durable copies share one id. ACP clients that send UUIDs keep opencode-minted ids | -| `packages/opencode/src/session/prompt.ts` | `prompt` with no parts and no `noReply`, on a session that already has a user message, stores nothing and runs `loop`; on a session with no user message it stores a zero-part row as upstream does | DocuMentor's abort-then-answer: after a cancel, an empty ACP prompt answers the user messages that arrived during the cancelled step, which opencode already holds. A v1 client sending `parts: []` gets the same resume; upstream clients send none | -| `packages/opencode/src/acp/event.ts` | `agent_message_chunk`, `agent_thought_chunk`, `tool_call` and `tool_call_update`, live and replayed, carry `_meta.opencode = { messageId, parentId }`: the assistant message and the user message it answers. The subscription caches each assistant message's `parentID` from `message.updated` for the sessions it tracks. On a miss it fetches the message once, shared by concurrent updates; a failed fetch is remembered until the session goes idle, and meanwhile the updates carry `parentId: null`. A session's cache is forgotten when it goes idle, and the whole cache when the event stream disconnects | DocuMentor detects from the ACP stream alone, in order, the moment the agent starts answering a message sent while the turn ran | -| Fork cases in `packages/opencode/test/acp/event.test.ts`, `test/acp/service-session.test.ts` and `test/session/prompt.test.ts` | Tests named `documentor: …` | They pin the three rows above; a rebase keeps them | -| `packages/core/src/integration.ts` | An OAuth attempt's credential is stored before the attempt reads as complete, inside one update of the attempts map | The attempt was marked complete first and the credential written after. On SQLite nothing could look in between; on a network database a caller that saw "complete" could list no credential | -| `patches/effect@4.0.0-beta.83.patch` | Adds Effect's own fix for re-entrant interruption, first released in `4.0.0-beta.100` | A fiber that woke another fiber which interrupted it synchronously had its continuation stack corrupted once its cleanup awaited anything. With Postgres that leaked a pooled connection. Drop these hunks when upstream moves to `beta.100` or later | -| `turbo.json` | `OPENCODE_TEST_DATABASE_URL` passes through to every task; `@opencode-ai/effect-drizzle-pg#test` is a task | turbo strips variables it is not told about | -| Test helpers: `packages/core/test/preload.ts`, `packages/opencode/test/preload.ts`, `packages/opencode/test/fixture/db.ts`, `packages/opencode/test/server/httpapi-exercise/environment.ts`, `packages/sdk-next/test/embedded.test.ts` | Ask for a throwaway schema; no database file to delete | | +| File | What differs | Why | +| -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| The 11 table files: `packages/core/src/{account,credential,event,permission,project,session,share}/sql.ts`, `control-plane/workspace.sql.ts`, `data-migration.sql.ts`, `database/schema.sql.ts`, `database/path.ts` | Postgres table builder; `count`, `jsonb` and `text` columns from `effect-drizzle-pg`; time columns are `double precision` | Written by `documentor/scripts/convert-schema.py`, never by hand | +| `packages/core/src/database/schema.gen.ts`, `packages/core/schema.json`, `packages/core/drizzle.config.ts` | The Postgres baseline, as one statement batch | Generated; see "Regenerating the baseline" | +| `packages/core/src/database/database.ts` | Connects with `@effect/sql-pg`; verifies collation and migrations and changes nothing; a throwaway schema per layer when `OPENCODE_DATABASE_EPHEMERAL=1` | The service itself | +| `packages/core/src/database/migration.ts` | `migrate`, `migrateAll`, `verify`, `pending`, `BASELINE_IDS`, `ported` | Schema work is a command, not a start-up side effect | +| `packages/core/src/project/directories.ts` | `Transaction` is `Database.Transaction` | It was typed against the SQLite client | +| `packages/core/src/flag/flag.ts` | No `OPENCODE_DB` | Nothing reads it | +| `packages/core/package.json` | Depends on `@effect/sql-pg`, `pg`, `@opencode-ai/effect-drizzle-pg`; tests run with `--timeout 30000` | Each test database is a real schema, about 110 ms to build | +| `packages/opencode/src/cli/cmd/db.ts` | `db `, `db migrate [--all]`, `db provision`, `db grant-read`, all outside the application runtime | The runtime's database service refuses the schemas these commands exist to create and migrate | +| `packages/opencode/src/session/processor.ts` | One lock per processor around each read-then-write of a tool part; `tool-input-delta` and `tool-input-end` do not read a part the processor already tracks | The processor and the running tool both update a tool part. SQLite made each update atomic; a network database lets them interleave, and the tool's metadata was lost. Each read is a round trip, and a large edit streams thousands of deltas | +| `packages/opencode/src/acp/service.ts` | `prompt` passes a `messageId` that starts with `msg` to v1 as `messageID`, for a normal prompt and a slash command; an empty prompt passes none. `newSession` passes a `_meta.opencode.sessionId` that starts with `ses` to `session.create` as `id` | DocuMentor names the user message a prompt creates and the session a new chat creates, so their optimistic, live and durable copies share one id. ACP clients that send UUIDs, or no `_meta`, keep opencode-minted ids | +| `packages/opencode/src/session/prompt.ts` | `prompt` with no parts and no `noReply`, on a session that already has a user message, stores nothing and runs `loop`; on a session with no user message it stores a zero-part row as upstream does. `createUserMessage` stores the message and its parts with `Session.saveMessage` | DocuMentor's abort-then-answer: after a cancel, an empty ACP prompt answers the user messages that arrived during the cancelled step, which opencode already holds. A v1 client sending `parts: []` gets the same resume; upstream clients send none. A reader never sees a user message without all of its parts | +| `packages/opencode/src/session/session.ts` | `CreateInput` and `create` take an optional `id`; `create` refuses an id that names an existing session. `saveMessage` publishes a message and its parts in one transaction | DocuMentor shows a new chat under its final id before opencode answers `session/new`. Each part was its own commit: on a network database a reader between them saw the context part of a user message without its text | +| `packages/sdk/js/src/v2/gen/*` | `session.create` takes `id` | Generated by `packages/sdk/js/script/build.ts` from the route above | +| `packages/opencode/src/acp/event.ts` | `agent_message_chunk`, `agent_thought_chunk`, `tool_call` and `tool_call_update`, live and replayed, carry `_meta.opencode = { messageId, parentId }`: the assistant message and the user message it answers. The subscription caches each assistant message's `parentID` from `message.updated` for the sessions it tracks. On a miss it fetches the message once, shared by concurrent updates; a failed fetch is remembered until the session goes idle, and meanwhile the updates carry `parentId: null`. A session's cache is forgotten when it goes idle, and the whole cache when the event stream disconnects | DocuMentor detects from the ACP stream alone, in order, the moment the agent starts answering a message sent while the turn ran | +| Fork cases in `packages/opencode/test/acp/event.test.ts`, `test/acp/service-session.test.ts`, `test/session/prompt.test.ts` and `test/session/session.test.ts` | Tests named `documentor: …` | They pin the three rows above; a rebase keeps them | +| `packages/core/src/integration.ts` | An OAuth attempt's credential is stored before the attempt reads as complete, inside one update of the attempts map | The attempt was marked complete first and the credential written after. On SQLite nothing could look in between; on a network database a caller that saw "complete" could list no credential | +| `patches/effect@4.0.0-beta.83.patch` | Adds Effect's own fix for re-entrant interruption, first released in `4.0.0-beta.100` | A fiber that woke another fiber which interrupted it synchronously had its continuation stack corrupted once its cleanup awaited anything. With Postgres that leaked a pooled connection. Drop these hunks when upstream moves to `beta.100` or later | +| `turbo.json` | `OPENCODE_TEST_DATABASE_URL` passes through to every task; `@opencode-ai/effect-drizzle-pg#test` is a task | turbo strips variables it is not told about | +| Test helpers: `packages/core/test/preload.ts`, `packages/opencode/test/preload.ts`, `packages/opencode/test/fixture/db.ts`, `packages/opencode/test/server/httpapi-exercise/environment.ts`, `packages/sdk-next/test/embedded.test.ts` | Ask for a throwaway schema; no database file to delete | | Upstream tests that are changed or deleted are listed, with reasons, in [`upstream-test-exceptions.md`](upstream-test-exceptions.md). diff --git a/packages/opencode/src/acp/service.ts b/packages/opencode/src/acp/service.ts index e0eb17e95e6d..ce0673009dec 100644 --- a/packages/opencode/src/acp/service.ts +++ b/packages/opencode/src/acp/service.ts @@ -162,6 +162,7 @@ export function make(input: { const newSession = Effect.fn("ACP.newSession")(function* (params: NewSessionRequest) { const started = performance.now() + const requestedSessionID = sessionIdFromMeta(params._meta) const snapshot = yield* directorySnapshot(params.cwd) const selected = selectDefaultModel(snapshot) const variant = selectVariant(snapshot, selected) @@ -172,6 +173,7 @@ export function make(input: { input.sdk.session.create( { directory: params.cwd, + ...(requestedSessionID ? { id: requestedSessionID } : {}), ...(modeId ? { agent: modeId } : {}), model: { providerID: selected.providerID, @@ -839,6 +841,17 @@ function detectSlashCommand(parts: ReturnType) { return { name, args: rest.join(" ").trim() } } +// A client may name the session it creates in `_meta.opencode.sessionId`, so that the chat it shows before +// opencode answers and the session opencode stores share one id. Only an id with the "ses" prefix is used; anything +// else gets an opencode-minted id. +function sessionIdFromMeta(meta: unknown) { + if (typeof meta !== "object" || meta === null) return undefined + const opencode = (meta as { opencode?: unknown }).opencode + if (typeof opencode !== "object" || opencode === null) return undefined + const sessionId = (opencode as { sessionId?: unknown }).sessionId + return typeof sessionId === "string" && sessionId.startsWith("ses") ? sessionId : undefined +} + // A client may name the user message its prompt creates, so that its own copy and opencode's share one id. // Only an id that is already an opencode message id (prefix "msg") is used; ACP clients that send UUIDs get an // opencode-minted id. An empty prompt creates no message (SessionPrompt.prompt), so it names none. diff --git a/packages/opencode/src/session/prompt.ts b/packages/opencode/src/session/prompt.ts index 0d3898ac8b38..2095f585548f 100644 --- a/packages/opencode/src/session/prompt.ts +++ b/packages/opencode/src/session/prompt.ts @@ -1043,8 +1043,7 @@ const layer = Layer.effect( }) } - yield* sessions.updateMessage(info) - for (const part of parts) yield* sessions.updatePart(part) + yield* sessions.saveMessage(info, parts) return { info, parts } }, Effect.scoped) diff --git a/packages/opencode/src/session/session.ts b/packages/opencode/src/session/session.ts index 9a47017aef96..cb3544c8fab5 100644 --- a/packages/opencode/src/session/session.ts +++ b/packages/opencode/src/session/session.ts @@ -259,6 +259,7 @@ export type GlobalInfo = Types.DeepMutable export const CreateInput = Schema.optional( Schema.Struct({ + id: Schema.optional(SessionID), parentID: Schema.optional(SessionID), title: Schema.optional(Schema.String), agent: Schema.optional(Schema.String), @@ -414,6 +415,7 @@ export interface Interface { readonly list: (input?: ListInput) => Effect.Effect readonly listGlobal: (input?: GlobalListInput) => Effect.Effect readonly create: (input?: { + id?: SessionID parentID?: SessionID title?: string agent?: string @@ -457,6 +459,7 @@ export interface Interface { partID: PartID }) => Effect.Effect readonly updatePart: (part: T) => Effect.Effect + readonly saveMessage: (msg: T, parts: readonly SessionV1.Part[]) => Effect.Effect readonly updatePartDelta: (input: { sessionID: SessionID messageID: MessageID @@ -642,6 +645,19 @@ const layer: Layer.Layer< return part }).pipe(Effect.withSpan("Session.updatePart")) + // A message and its parts are published in one transaction, so their rows commit together. A reader between + // the message row and its last part would otherwise see a user message without its text. + const saveMessage = (msg: T, parts: readonly SessionV1.Part[]): Effect.Effect => + db + .transaction(() => + Effect.gen(function* () { + yield* updateMessage(msg) + for (const part of parts) yield* updatePart(part) + return msg + }), + ) + .pipe(Effect.orDie, Effect.withSpan("Session.saveMessage")) + const getPart: Interface["getPart"] = Effect.fn("Session.getPart")(function* (input) { const row = yield* db .select() @@ -665,6 +681,7 @@ const layer: Layer.Layer< }) const create = Effect.fn("Session.create")(function* (input?: { + id?: SessionID parentID?: SessionID title?: string agent?: string @@ -675,7 +692,14 @@ const layer: Layer.Layer< }) { const ctx = yield* InstanceState.context const workspace = yield* InstanceState.workspaceID + // A caller-chosen id names a new session, never an existing one. The projector refuses a duplicate row as + // well; checking first turns that into an error that names the id instead of a failed projection. + if (input?.id !== undefined) { + const existing = yield* get(input.id).pipe(Effect.option) + if (existing._tag === "Some") return yield* Effect.die(new Error(`Session already exists: ${input.id}`)) + } return yield* createNext({ + id: input?.id, parentID: input?.parentID, directory: ctx.directory, path: sessionPath(ctx.worktree, ctx.directory), @@ -933,6 +957,7 @@ const layer: Layer.Layer< removeMessage, removePart, updatePart, + saveMessage, getPart, updatePartDelta, findMessage, diff --git a/packages/opencode/test/acp/service-session.test.ts b/packages/opencode/test/acp/service-session.test.ts index 094f63cc6ed6..e2eca2d2c0ab 100644 --- a/packages/opencode/test/acp/service-session.test.ts +++ b/packages/opencode/test/acp/service-session.test.ts @@ -990,6 +990,64 @@ describe("ACP service sessions", () => { expect(result.configOptions?.find((option) => option.id === "model")?.currentValue).toBe("test/configured-model") }) + function sessionIdSdk(creates: Array<{ id?: string }>) { + return { + config: { + providers: () => Promise.resolve({ data: { providers: [provider], default: { test: modelID } } }), + get: () => Promise.resolve({ data: {} }), + }, + app: { + agents: () => Promise.resolve({ data: [{ name: "build", mode: "primary", permission: [], options: {} }] }), + skills: () => Promise.resolve({ data: [] }), + }, + command: { + list: () => Promise.resolve({ data: [] }), + }, + session: { + create: (input: { id?: string }) => { + creates.push(input) + return Promise.resolve({ data: { id: input.id ?? "ses_minted" } }) + }, + list: () => Promise.resolve({ data: [] }), + }, + mcp: { + add: () => Promise.resolve({ data: {} }), + }, + } as unknown as OpencodeClient + } + + it("documentor: creates the session under the id named in _meta.opencode.sessionId", async () => { + const creates: Array<{ id?: string }> = [] + const service = ACPService.make({ sdk: sessionIdSdk(creates) }) + + const result = await Effect.runPromise( + service.newSession({ + cwd: "/workspace", + mcpServers: [], + _meta: { opencode: { sessionId: "ses_fe0123456789ClientChosen1" } }, + }), + ) + + expect(creates[0]?.id).toBe("ses_fe0123456789ClientChosen1") + expect(result.sessionId).toBe("ses_fe0123456789ClientChosen1") + }) + + it("documentor: lets opencode mint the session id when _meta names none or one without the ses prefix", async () => { + const creates: Array<{ id?: string }> = [] + const service = ACPService.make({ sdk: sessionIdSdk(creates) }) + + await Effect.runPromise(service.newSession({ cwd: "/workspace", mcpServers: [] })) + await Effect.runPromise( + service.newSession({ + cwd: "/workspace", + mcpServers: [], + _meta: { opencode: { sessionId: "00000000-0000-4000-8000-000000000001" } }, + }), + ) + + expect(creates.map((input) => input.id)).toEqual([undefined, undefined]) + }) + it("does not scan last-used sessions when resolving the new session default", async () => { const historyCalls: string[] = [] const sdk = { diff --git a/packages/opencode/test/session/session.test.ts b/packages/opencode/test/session/session.test.ts index c4301d20401c..cae5fee46f4c 100644 --- a/packages/opencode/test/session/session.test.ts +++ b/packages/opencode/test/session/session.test.ts @@ -283,3 +283,88 @@ describe("Session", () => { }), ) }) + +describe("documentor: caller-chosen session id", () => { + it.instance("creates the session under the id the caller names", () => + Effect.gen(function* () { + const session = yield* SessionNs.Service + const id = "ses_fe0123456789CallerChosen1" as SessionID + + const info = yield* session.create({ id }) + + expect(info.id).toBe(id) + expect((yield* session.get(id)).id).toBe(id) + yield* session.remove(id) + }), + ) + + it.instance("refuses an id that names an existing session and leaves that session as it was", () => + Effect.gen(function* () { + const session = yield* SessionNs.Service + const existing = yield* session.create({ title: "kept" }) + + const exit = yield* session.create({ id: existing.id, title: "replacement" }).pipe(Effect.exit) + + expect(Exit.isFailure(exit)).toBe(true) + expect((yield* session.get(existing.id)).title).toBe("kept") + yield* session.remove(existing.id) + }), + ) +}) + +describe("documentor: a message and its parts are saved together", () => { + const userMessage = (sessionID: SessionID, id: MessageID) => + ({ + id, + sessionID, + role: "user", + time: { created: Date.now() }, + agent: "user", + model: { providerID: "test", modelID: "test" }, + tools: {}, + mode: "", + }) as unknown as SessionV1.Info + + const textPart = (sessionID: SessionID, messageID: MessageID, text: string) => + ({ id: PartID.ascending(), messageID, sessionID, type: "text", text }) as SessionV1.Part + + it.instance("stores the message with every part", () => + Effect.gen(function* () { + const session = yield* SessionNs.Service + const info = yield* session.create({}) + const messageID = MessageID.ascending() + + yield* session.saveMessage(userMessage(info.id, messageID), [ + textPart(info.id, messageID, "context"), + textPart(info.id, messageID, "Find me my passport"), + ]) + + const [stored] = yield* session.messages({ sessionID: info.id }) + expect(stored?.info.id).toBe(messageID) + expect(stored?.parts.map((part) => (part as SessionV1.TextPart).text)).toEqual([ + "context", + "Find me my passport", + ]) + yield* session.remove(info.id) + }), + ) + + it.instance("stores none of it when a part cannot be stored", () => + Effect.gen(function* () { + const session = yield* SessionNs.Service + const info = yield* session.create({}) + const messageID = MessageID.ascending() + + const exit = yield* session + .saveMessage(userMessage(info.id, messageID), [ + textPart(info.id, messageID, "context"), + textPart(info.id, MessageID.ascending(), "a part of a message that does not exist"), + ]) + .pipe(Effect.exit) + + expect(Exit.isFailure(exit)).toBe(true) + expect(yield* session.messages({ sessionID: info.id })).toEqual([]) + yield* session.remove(info.id) + }), + ) +}) diff --git a/packages/sdk/js/src/v2/gen/sdk.gen.ts b/packages/sdk/js/src/v2/gen/sdk.gen.ts index a2bcd4252c6d..01f4483284c0 100644 --- a/packages/sdk/js/src/v2/gen/sdk.gen.ts +++ b/packages/sdk/js/src/v2/gen/sdk.gen.ts @@ -3411,6 +3411,7 @@ export class Session2 extends HeyApiClient { parameters?: { directory?: string workspace?: string + id?: string parentID?: string title?: string agent?: string @@ -3434,6 +3435,7 @@ export class Session2 extends HeyApiClient { args: [ { in: "query", key: "directory" }, { in: "query", key: "workspace" }, + { in: "body", key: "id" }, { in: "body", key: "parentID" }, { in: "body", key: "title" }, { in: "body", key: "agent" }, diff --git a/packages/sdk/js/src/v2/gen/types.gen.ts b/packages/sdk/js/src/v2/gen/types.gen.ts index f06c20cc413e..a1f2bd854e31 100644 --- a/packages/sdk/js/src/v2/gen/types.gen.ts +++ b/packages/sdk/js/src/v2/gen/types.gen.ts @@ -9475,6 +9475,7 @@ export type SessionListResponse = SessionListResponses[keyof SessionListResponse export type SessionCreateData = { body?: { + id?: string parentID?: string title?: string agent?: string