Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
24 changes: 24 additions & 0 deletions docs/agent-dx-log.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,30 @@
How measurements from [honojs/agent-dx](https://github.com/honojs/agent-dx)
changed Hono CLI. Newest first.

## 2026-09-06: An executable spec wins — when the user hands it over

**Experiment**: the auto-mode task re-run with `expect` (`next.4`),
haiku, 3 runs per condition.

**Findings**:

- The agent writing `checks.jsonl` from the spec table itself: 2/3,
110k — transcription mixes interpretation in.
- A ready-made `checks.jsonl` shipped with the task: 3/3, median 65k
tokens, 43s — the top of every condition measured so far, beating
hand-written scripts (3/3, 92k, 92s). Every run opened with
`--batch checks.jsonl`, implemented, and finished at `failed: 0`.
- So `expect` pays off with a workflow: the user (or an approved
draft) hands the agent an executable acceptance spec.
- Also recorded: a real Claude Code run tried `npx hono dev &` — it
expects a dev server to exist.

**Changes**: `dev` / `serve` / `start` now answer with the point of
the whole design: "No server needed. Send requests directly:
hono request /path". Open ideas, not yet built: scaffolding
`checks.jsonl` from the routes, and a docs section on handing specs
to agents.

## 2026-09-05: `expect` returns — measurement beats our reasoning

**Experiment**: an auto-mode task (build a shop API against an
Expand Down
7 changes: 7 additions & 0 deletions src/utils/output.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -55,6 +55,13 @@ describe('formatArgumentsError flag fixes', () => {
])
})

it('should point an invented server command at request', () => {
const parsed = JSON.parse(formatArgumentsError("error: unknown command 'dev'"))
expect(parsed.error.suggestions).toEqual([
'No server needed. Send requests directly: hono request /path',
])
})

it('should map an invented flag to the real one', () => {
const body = JSON.parse(formatArgumentsError("error: unknown option '--body'"))
expect(body.error.suggestions[0]).toContain('-d')
Expand Down
11 changes: 10 additions & 1 deletion src/utils/output.ts
Original file line number Diff line number Diff line change
Expand Up @@ -47,10 +47,19 @@ const FLAG_FIXES: Record<string, string> = {
'-m': 'The method flag is -X: hono request /api/users -X POST',
}

// There is no server command on purpose. Agents that expect one get
// pointed at the serverless flow.
const COMMAND_FIXES: Record<string, string> = {
dev: 'No server needed. Send requests directly: hono request /path',
serve: 'No server needed. Send requests directly: hono request /path',
start: 'No server needed. Send requests directly: hono request /path',
}

export const formatArgumentsError = (message: string): string => {
const cleaned = message.replace(/^error: /, '').trim()
const flag = cleaned.match(/unknown option '([^']+)'/)?.[1]
const fix = flag ? FLAG_FIXES[flag] : undefined
const command = cleaned.match(/unknown command '([^']+)'/)?.[1]
const fix = (flag && FLAG_FIXES[flag]) || (command && COMMAND_FIXES[command]) || undefined
const suggestions = [fix ?? 'Check the usage: hono <command> --help']
return formatError(new CliError('INVALID_ARGUMENTS', cleaned, { suggestions }))
}
Expand Down
Loading