Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
61 changes: 49 additions & 12 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,12 @@ hono routes
# Send request to Hono app
hono request /

# Run multiple requests from JSONL
hono batch -

# Print the current behavior as batch JSONL lines
hono snapshot

# Measure the performance of your Hono app
hono benchmark

Expand All @@ -56,6 +62,8 @@ Inspect and test:

- `routes [file]` - Show routes of your Hono app
- `request <path> [file]` - Send request to Hono app using `app.request()`
- `batch <source> [file]` - Run multiple requests from JSONL using `app.request()`
- `snapshot [file]` - Print the current behavior as batch JSONL lines
- `benchmark [file]` - Measure the performance of your Hono app

Build:
Expand Down Expand Up @@ -132,7 +140,6 @@ hono request <path> [file] [options]
- `--runtime <runtime>` - runtime to execute the app: `node` (default), `bun`, `deno`, or `workerd`
- `-i, --include` - Include status and headers in the output (with `--plain`)
- `-I, --head` - Show only status and headers in the output (with `--plain`)
- `--batch <source>` - Run multiple requests from JSONL (`-` reads stdin)
- `-e, --external <package>` - Mark package as external (can be used multiple times)

**Examples:**
Expand Down Expand Up @@ -175,19 +182,8 @@ hono request / --runtime deno
# Run the app on workerd with your wrangler config: bindings (c.env) are the local ones
hono request /api --runtime workerd

# Run many requests in one call. One JSON object per line.
# `save` stores a value from the response body, later steps use it as {{name}}.
hono request --batch - <<'EOF'
{"path":"/users","expect":{"status":200}}
{"method":"POST","path":"/users","body":{"name":"Momo"},"expect":{"status":201,"body":{"name":"Momo"}},"save":{"id":".id"}}
{"path":"/users/{{id}}","expect":{"status":200}}
{"method":"DELETE","path":"/users/{{id}}","expect":{"status":204}}
{"path":"/users/{{id}}","expect":{"status":404}}
EOF
```

A batch runs in order against one app instance, so in-memory state carries between steps. Each step reports the actual `status` and `body`, and `expect` declares the acceptance criteria: `status` matches exactly, `body` is a deep partial match (declared fields must match, extra response fields are ignored). The output carries `pass` per step and a `summary` — rerun until `failed` is 0. A shared header from `-H` goes to every step.

`workerd` starts the app with the wrangler config of the project, so pass no file argument. It needs [wrangler](https://developers.cloudflare.com/workers/wrangler/) installed in the project. wrangler is not a dependency of Hono CLI.

With `--trace`, the output has `matchedRoutes`. `responded` marks the route that returned the response:
Expand Down Expand Up @@ -232,6 +228,47 @@ The result is JSON with the shared envelope. A JSON response body is embedded as

A binary response body becomes `"body": null` with `"binary": true` — save it with `-o`. Use `--plain` to print the raw body like curl. A 404 result includes a suggestion to run `--trace`.

### `batch`

Run multiple requests from JSONL in one call, in order, against one app instance — in-memory state carries between steps.

```bash
hono batch <source> [file]
```

**Arguments:**

- `source` - JSONL file, or `-` to read stdin
- `file` - Path to the Hono app file (optional)

**Options:**

- `-H, --header <header>` - Shared headers for every step
- `-e, --external <package>` - Mark package as external (can be used multiple times)

```bash
hono batch - <<'EOF'
{"path":"/users","expect":{"status":200}}
{"method":"POST","path":"/users","body":{"name":"Momo"},"expect":{"status":201,"body":{"name":"Momo"}},"save":{"id":".id"}}
{"path":"/users/{{id}}","expect":{"status":200}}
{"method":"DELETE","path":"/users/{{id}}","expect":{"status":204}}
EOF
```

One JSON object per line: `method`, `path`, `body`, `headers`, `expect`, `save`. `save` stores a value from the response body by dot path, and later steps use it as `{{id}}` (a whole-variable string keeps the saved type). `expect` declares the acceptance criteria: `status` matches exactly, `body` is a deep partial match (declared fields must match, extra response fields are ignored). The output carries the actual `status` and `body`, `pass` per step, and a `summary` — rerun until `failed` is 0.

### `snapshot`

Print the current behavior of the app as batch JSONL lines, to stdout — no file is written.

```bash
hono snapshot [file]
```

Paramless GET routes are executed and their actual response becomes the `expect`. Param and non-GET routes are printed without one, to fill in. One probe line records the current response for a path that matches no route. Capture before a refactor, then rerun the lines with `hono batch` until `failed` is 0.

Unlike `routes`, this command sends real requests to the app — middleware runs. `routes` never sends a request.

### `benchmark`

Measure the performance of your Hono app. It is a micro benchmark of routing and handlers: `app.request()` is called directly, with no HTTP stack and no network. Each run happens in a fresh process, so results are comparable.
Expand Down
22 changes: 22 additions & 0 deletions docs/agent-dx-log.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,28 @@
How measurements from [honojs/agent-dx](https://github.com/honojs/agent-dx)
changed Hono CLI. Newest first.

## 2026-09-06: The spec travels in the conversation, not in a file

**Experiment**: the `expect` re-run (`next.4`): a ready-made
`checks.jsonl` scored 3/3 at a 65k median — the top of every
condition — while agents transcribing the spec themselves lost runs
to interpretation (2/3, 110k).

**Findings**: what won is an executable spec that exists before the
implementation, is reviewable by a human, and reruns until green.
The file was the harness's delivery detail: ten JSONL lines travel
fine in the request itself and run as a heredoc — no artifact to
clean up.

**Changes**: two new commands, split out of `request` (both rejected
almost every single-request option — the sign of separate commands
under one flag). `hono batch <source> [file]` runs the JSONL lines;
`hono snapshot [file]` prints the current behavior as batch JSONL
lines, to stdout: paramless GET routes run and their actual response
becomes the `expect`; param and non-GET routes print without one; a
probe line records the current not-found behavior as a fact. The
taxonomy: `routes` never sends a request; `request`, `batch`, and
`snapshot` exist to send them.
## 2026-09-06: An executable spec wins — when the user hands it over

**Experiment**: the auto-mode task re-run with `expect` (`next.4`),
Expand Down
4 changes: 4 additions & 0 deletions src/cli.ts
Original file line number Diff line number Diff line change
Expand Up @@ -3,10 +3,12 @@ import { readFileSync } from 'node:fs'
import { dirname, join } from 'node:path'
import { fileURLToPath } from 'node:url'
import { agentContextCommand } from './commands/agent-context/index.js'
import { batchCommand } from './commands/batch/index.js'
import { benchmarkCommand } from './commands/benchmark/index.js'
import { optimizeCommand } from './commands/optimize/index.js'
import { requestCommand } from './commands/request/index.js'
import { routesCommand } from './commands/routes/index.js'
import { snapshotCommand } from './commands/snapshot/index.js'
import { ssgCommand } from './commands/ssg/index.js'
import { formatArgumentsError } from './utils/output.js'

Expand All @@ -30,6 +32,8 @@ program
agentContextCommand(program)
routesCommand(program)
requestCommand(program)
batchCommand(program)
snapshotCommand(program)
benchmarkCommand(program)
optimizeCommand(program)
ssgCommand(program)
Expand Down
4 changes: 4 additions & 0 deletions src/commands/agent-context/document.ts
Original file line number Diff line number Diff line change
@@ -1,10 +1,12 @@
import type { Command } from 'commander'
import type { CommandAgentContext } from '../../utils/agent-context.js'
import { bullets, codeBlock, section, steps } from '../../utils/markdown.js'
import { agentContext as batchContext } from '../batch/index.js'
import { agentContext as benchmarkContext } from '../benchmark/index.js'
import { agentContext as optimizeContext } from '../optimize/index.js'
import { agentContext as requestContext } from '../request/index.js'
import { agentContext as routesContext } from '../routes/index.js'
import { agentContext as snapshotContext } from '../snapshot/index.js'
import { agentContext as ssgContext } from '../ssg/index.js'

const contexts: Record<string, CommandAgentContext> = {
Expand All @@ -13,6 +15,8 @@ const contexts: Record<string, CommandAgentContext> = {
optimize: optimizeContext,
ssg: ssgContext,
benchmark: benchmarkContext,
batch: batchContext,
snapshot: snapshotContext,
}

const commandDoc = (command: Command, context?: CommandAgentContext): string => {
Expand Down
6 changes: 6 additions & 0 deletions src/commands/agent-context/index.test.ts
Original file line number Diff line number Diff line change
@@ -1,8 +1,10 @@
import { Command } from 'commander'
import { describe, it, expect, vi, beforeEach, afterEach } from 'vitest'
import { batchCommand } from '../batch/index.js'
import { optimizeCommand } from '../optimize/index.js'
import { requestCommand } from '../request/index.js'
import { routesCommand } from '../routes/index.js'
import { snapshotCommand } from '../snapshot/index.js'
import { agentContextCommand } from './index.js'

describe('agentContextCommand', () => {
Expand All @@ -15,6 +17,8 @@ describe('agentContextCommand', () => {
optimizeCommand(program)
requestCommand(program)
routesCommand(program)
batchCommand(program)
snapshotCommand(program)
agentContextCommand(program)
consoleLogSpy = spyOnLog()
})
Expand Down Expand Up @@ -48,6 +52,8 @@ describe('agentContextCommand', () => {
expect(output).toContain('### hono optimize [entry]')
expect(output).toContain('### hono request [path] [file]')
expect(output).toContain('### hono routes [file]')
expect(output).toContain('### hono batch <source> [file]')
expect(output).toContain('### hono snapshot [file]')
expect(output).not.toContain('### hono agent-context')
})

Expand Down
File renamed without changes.
File renamed without changes.
76 changes: 76 additions & 0 deletions src/commands/batch/index.test.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,76 @@
import { Command } from 'commander'
import { Hono } from 'hono'
import { describe, it, expect, vi, beforeEach, afterEach } from 'vitest'

vi.mock('node:fs', () => ({
existsSync: vi.fn(),
realpathSync: vi.fn(),
readFileSync: vi.fn(),
}))

vi.mock('node:path', () => ({
resolve: vi.fn(),
}))

vi.mock('../../utils/build.js', () => ({
buildAndImportApp: vi.fn(),
}))

import { batchCommand } from './index.js'

describe('batchCommand', () => {
let program: Command
let consoleLogSpy: ReturnType<typeof vi.spyOn>

async function* iteratorOf(app: Hono): AsyncGenerator<Hono> {
yield app
}

beforeEach(async () => {
program = new Command()
batchCommand(program)
consoleLogSpy = vi.spyOn(console, 'log').mockImplementation(() => {})
const fs = await import('node:fs')
const path = await import('node:path')
const build = await import('../../utils/build.js')
vi.mocked(fs.existsSync).mockReturnValue(true)
vi.mocked(fs.realpathSync).mockReturnValue('test-app.js')
vi.mocked(path.resolve).mockImplementation((cwd: string, p: string) => `${cwd}/${p}`)
const app = new Hono()
app.get('/data', (c) => c.json({ ok: 1 }))
vi.mocked(build.buildAndImportApp).mockReturnValue(iteratorOf(app))
vi.mocked(fs.readFileSync).mockReturnValue('{"path":"/data","expect":{"status":200}}')
})

afterEach(() => {
vi.restoreAllMocks()
vi.clearAllMocks()
})

it('should run the steps from a JSONL file and print the envelope', async () => {
await program.parseAsync(['node', 'test', 'batch', 'steps.jsonl', 'test-app.js'])
expect(JSON.parse(consoleLogSpy.mock.calls[0][0] as string)).toEqual({
ok: true,
data: {
steps: [
{
method: 'GET',
path: '/data',
status: 200,
body: { ok: 1 },
pass: true,
expect: { status: 200 },
},
],
summary: { total: 1, passed: 1, failed: 0 },
},
})
})

it('should reject the app and the batch both from stdin', async () => {
await program.parseAsync(['node', 'test', 'batch', '-', '-'])
const output = JSON.parse(consoleLogSpy.mock.calls[0][0] as string)
expect(output.ok).toBe(false)
expect(output.error.code).toBe('INVALID_OPTION')
})
})
85 changes: 85 additions & 0 deletions src/commands/batch/index.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,85 @@
import type { Command } from 'commander'
import { existsSync, readFileSync } from 'node:fs'
import { resolve } from 'node:path'
import type { CommandAgentContext } from '../../utils/agent-context.js'
import { parseHeaders } from '../../utils/headers.js'
import { getBuildIterator, readStdin } from '../../utils/load-app.js'
import { CliError, handleErrors, printResult } from '../../utils/output.js'
import { parseBatch, runBatch } from './batch.js'

export const agentContext: CommandAgentContext = {
output:
'{ "steps": [{ "method": "GET", "path": "/users", "status": 200, "body": [], "pass": true, "expect": { "status": 200 } }], "summary": { "total": 1, "passed": 1, "failed": 0 } }',
errors: ['BATCH_INVALID', 'BATCH_NOT_FOUND', 'ENTRY_NOT_FOUND', 'BUILD_FAILED', 'INVALID_APP'],
examples: [
`hono batch - <<'EOF'
{"path":"/users","expect":{"status":200}}
{"method":"POST","path":"/users","body":{"name":"Momo"},"expect":{"status":201,"body":{"name":"Momo"}},"save":{"id":".id"}}
{"path":"/users/{{id}}","expect":{"status":200}}
EOF`,
],
notes: [
'Runs many requests in one call, in order, against one app instance — in-memory state carries between steps. One JSON object per line: {"method","path","body","headers","expect","save"}.',
'"save" stores a value from the response body by dot path (e.g. {"id":".id"}), and later steps use it as {{id}}. A whole-variable string like "{{id}}" keeps the saved type.',
'Declare the acceptance criteria in "expect": {"status":201} and/or {"body":{...}} (a deep partial match — declared fields must match, extra response fields are ignored). Turn the spec into batch lines and rerun until "failed" is 0 — comparing a spec table by eye misses lines.',
'A shared header from -H goes to every step. Prefer a heredoc over writing a file: the lines live in your context.',
'hono snapshot prints the current behavior of an app in this format — capture before a refactor, rerun after.',
],
}

interface BatchOptions {
header?: string[]
external?: string[]
}

export function batchCommand(program: Command) {
program
.command('batch')
.description('Run multiple requests from JSONL using app.request()')
.argument('<source>', 'JSONL file (- reads stdin)')
.argument('[file]', 'Path to the Hono app file')
.option(
'-H, --header <header>',
'Shared headers for every step',
(value: string, previous: string[]) => {
return previous ? [...previous, value] : [value]
},
[] as string[]
)
.option(
'-e, --external <package>',
'Mark package as external (can be used multiple times)',
(value: string, previous: string[]) => {
return previous ? [...previous, value] : [value]
},
[] as string[]
)
.action(
handleErrors(async (source: string, file: string | undefined, options: BatchOptions) => {
if (source === '-' && file === '-') {
throw new CliError(
'INVALID_OPTION',
'Cannot read both the app and the batch from stdin',
{
suggestions: ['Pass the app as a file, or the batch as a file'],
}
)
}
const input = source === '-' ? await readStdin() : readBatchFile(source)
const steps = parseBatch(input)
for await (const app of getBuildIterator(file, false, options.external || [])) {
printResult(await runBatch(app, steps, parseHeaders(options.header)))
}
})
)
}

const readBatchFile = (source: string): string => {
const filepath = resolve(process.cwd(), source)
if (!existsSync(filepath)) {
throw new CliError('BATCH_NOT_FOUND', `Batch file ${source} does not exist`, {
suggestions: ['Pass a JSONL file, or - to read stdin'],
})
}
return readFileSync(filepath, 'utf-8')
}
6 changes: 3 additions & 3 deletions src/commands/benchmark/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -109,7 +109,7 @@ export function benchmarkCommand(program: Command) {
}

const external = options.external || []
const entry = resolveEntry(file)
const entry = await resolveEntry(file)
const targets = await collectTargets(file, entry, options, method, external)

const sources: HonoSource[] = []
Expand Down Expand Up @@ -145,7 +145,7 @@ export function benchmarkCommand(program: Command) {

const collectTargets = async (
file: string | undefined,
entry: ReturnType<typeof resolveEntry>,
entry: Awaited<ReturnType<typeof resolveEntry>>,
options: BenchmarkOptions,
method: string,
external: string[]
Expand All @@ -158,7 +158,7 @@ const collectTargets = async (
headers[key.trim()] = value.trim()
}
}
const body = resolveData(options.data)
const body = await resolveData(options.data)
return options.path.map((path) => ({
method,
path,
Expand Down
Loading
Loading