Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,8 +4,8 @@ Notable FlexDoc product releases are recorded here. Detailed release notes remai

## Unreleased

- An implemented route that OpenAPI does not document fails `flexdoc validate` when runtime discovery is complete, and the failure names the method and path. Partial discovery keeps that finding a warning. `runtimeIntelligence.acknowledgedUndocumented` omits operations that are intentionally absent from the contract. Those routes stay in the runtime route list, and `summary.runtime` counts only `matched + runtimeOnly`.
- Opening that finding shows the registered route. It is not treated as an OpenAPI operation.
- An implemented route that OpenAPI does not document fails `flexdoc validate` when runtime discovery is complete, and the failure names the method and path. Partial discovery keeps that finding a warning. `runtimeIntelligence.acknowledgedUndocumented` records that route as acknowledged info. The route stays registered, `summary.runtime` still counts it, and the acknowledgement does not fail validation.
- Opening that finding shows the missing contract beside the registered method and path, and can send it from the API Client. The URL uses the snapshot `serverOrigin` when present, and is same-origin otherwise. The request asks to prefer the API host only when host execution is available and the server has not opted out.

## 3.4.0 — 3.3.x hardening package release

Expand Down
130 changes: 65 additions & 65 deletions adapters/elixir/assets/flexdoc.standalone.js

Large diffs are not rendered by default.

130 changes: 65 additions & 65 deletions adapters/go/assets/flexdoc.standalone.js

Large diffs are not rendered by default.

130 changes: 65 additions & 65 deletions adapters/php/assets/flexdoc.standalone.js

Large diffs are not rendered by default.

130 changes: 65 additions & 65 deletions adapters/python/src/prauga_flexdoc/_assets/flexdoc.standalone.js

Large diffs are not rendered by default.

130 changes: 65 additions & 65 deletions adapters/ruby/assets/flexdoc.standalone.js

Large diffs are not rendered by default.

130 changes: 65 additions & 65 deletions adapters/rust-actix/assets/flexdoc.standalone.js

Large diffs are not rendered by default.

130 changes: 65 additions & 65 deletions adapters/rust/assets/flexdoc.standalone.js

Large diffs are not rendered by default.

2 changes: 1 addition & 1 deletion docs/runtime-intelligence.md
Original file line number Diff line number Diff line change
Expand Up @@ -148,7 +148,7 @@ The Node Runtime Intelligence snapshot can include one backend-produced `validat

The four finding codes in this cut are:

- `runtime.operation-undocumented` — the running backend positively exposes an operation that OpenAPI does not document. Severity: `error` when discovery is complete, `warning` when discovery is partial, matching the other findings that must not treat an incomplete inventory as authoritative. The finding names the method and path and opens that registered route. It is not an OpenAPI operation. When discovery is complete the derived status is `fail`, and `flexdoc validate` exits `1` under the default policy. Set `runtimeIntelligence.acknowledgedUndocumented` to method/path pairs that are intentionally absent from OpenAPI; those operations are omitted from this finding.
- `runtime.operation-undocumented` — the running backend positively exposes an operation that OpenAPI does not document. Severity: `error` when discovery is complete, `warning` when discovery is partial, matching the other findings that must not treat an incomplete inventory as authoritative. The finding names the method and path and opens that registered route. It is not an OpenAPI operation. The page shows the contract claim — OpenAPI does not declare the operation — beside the registered method and path. It does not invent a handler or a schema. **Open in API Client** sends that method and path on `serverOrigin` when the snapshot has one, and as a same-origin path otherwise. It does not use the OpenAPI server URL. The request asks to prefer the API host only when host execution is available and the server has not set `preferHostExecution` to false. `runtimeIntelligence.acknowledgedUndocumented` keeps the route registered and records the finding as acknowledged info, so that finding does not fail validation. An unacknowledged route still fails when discovery is complete, and `flexdoc validate` exits `1`.
- `runtime.operation-unobserved` — OpenAPI documents an operation that runtime discovery did not observe. Severity: `error` when discovery is complete, `info` when discovery is partial.
- `runtime.method-mismatch` — the same wire-equivalent path exists in both sources but under different HTTP methods. Severity: `error` when discovery is complete, `warning` when partial. The finding carries an expected documented method so the renderer can navigate to the spec operation.
- `runtime.duplicate-operation` — the host exposes multiple registrations for one wire-equivalent method/path. Severity: `warning`. In 3.1 this is an explicitly supported/verified claim for Express (including Nest-on-Express) and Hono. Fastify participates in the other validation checks, but exact duplicate-registration reporting is not claimed for Fastify in this cut.
Expand Down
40 changes: 40 additions & 0 deletions e2e/runtime-intelligence.spec.cjs
Original file line number Diff line number Diff line change
Expand Up @@ -82,3 +82,43 @@ test('runtime drift rows navigate to documented operations', async ({ page }, te
await expect(page.getByRole('heading', { name: 'Get a pet' })).toBeVisible();
await expect(page).toHaveURL(/#get-~2Fpets~2F~7Bid~7D$/);
});

test('an undocumented runtime finding opens, shows the contract gap, and hands off to the API Client', async ({ page }, testInfo) => {
test.skip(testInfo.project.name !== 'chromium-desktop', 'desktop runtime disagreement loop');
const disagreement = {
...runtimeSnapshot,
discoveryComplete: true,
serverOrigin: 'https://staging.internal',
routes: [{ method: 'POST', path: '/internal/reindex' }],
runtimeOnly: [{ method: 'POST', path: '/internal/reindex' }],
documentedOnly: [],
summary: { documented: 1, runtime: 1, matched: 0, runtimeOnly: 1, documentedOnly: 0 },
validation: {
status: 'fail',
complete: true,
findings: [{
id: 'runtime.operation-undocumented:POST:/internal/reindex',
code: 'runtime.operation-undocumented',
severity: 'error',
location: { kind: 'operation', method: 'POST', path: '/internal/reindex' },
message: 'Runtime implements POST /internal/reindex, but OpenAPI does not document that operation.',
expected: 'Operation is represented in OpenAPI',
observed: 'Operation exists only in the running backend',
}],
summary: { total: 1, errors: 1, warnings: 0, info: 0 },
},
};
await page.route('**/e2e/__flexdoc/runtime', async (route) => {
await route.fulfill({ status: 200, contentType: 'application/json', body: JSON.stringify(disagreement) });
});
await page.goto('/e2e/index.html?runtime=1');
await page.getByRole('button', { name: 'Open runtime intelligence' }).click();
const dialog = page.getByRole('dialog', { name: 'Runtime intelligence' });
await dialog.getByRole('button', { name: 'Open runtime route POST /internal/reindex' }).click();
await expect(dialog).toBeHidden();
await expect(page.getByText('OpenAPI does not declare this operation.')).toBeVisible();
await expect(page.getByText('POST /internal/reindex is registered by the running service.')).toBeVisible();
await page.getByRole('button', { name: 'Open in API Client' }).click();
await expect(page.getByLabel('Request URL')).toHaveValue('https://staging.internal/internal/reindex');
await expect(page.getByLabel('HTTP method')).toHaveValue('POST');
});
2 changes: 1 addition & 1 deletion examples/javascript-express/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

This is the primary backend-native FlexDoc 3.1 showcase. Express serves the shared OpenAPI 3.1 contract, while FlexDoc runs inside the same backend and exposes the completed documentation/API Client surface plus live Runtime Intelligence.

The example registers `GET /internal/health` without documenting it, and lists that operation in `runtimeIntelligence.acknowledgedUndocumented`. Contract validation stays green. The route stays on the snapshot's `routes` array. The Runtime panel shows findings rather than that array, so an acknowledged route does not appear there.
`GET /internal/health` is registered and listed in `runtimeIntelligence.acknowledgedUndocumented`. It stays a runtime-only route, the finding is informational, and it does not fail validation. `POST /internal/reindex` is also registered and is not acknowledged, so contract validation fails. Open **Runtime**, then that finding, to see the registered route beside the missing contract and send it from the API Client.

The renderer also demonstrates the completed documentation/API Client surface: persisted viewer preferences, deep links, keyboard command palette, mobile navigation, JSON/YAML download, Basic/Advanced Try It, handoff to the sibling API Client page, environments, scripts, the CodeMirror-backed editor, response inspection, collection/history workflows, and code samples.

Expand Down
3 changes: 2 additions & 1 deletion examples/javascript-express/index.js
Original file line number Diff line number Diff line change
Expand Up @@ -57,6 +57,7 @@ function buildApp() {

// Intentionally omitted from the OpenAPI document so Runtime Intelligence has real drift to report.
app.get('/internal/health', (_req, res) => res.json({ status: 'internal-ok' }));
app.post('/internal/reindex', (_req, res) => res.json({ accepted: true }));

// Security ordering matters for host execution:
// 1. authenticate the whole documentation subtree;
Expand Down Expand Up @@ -106,7 +107,7 @@ function buildApp() {
}

if (require.main === module) {
buildApp().listen(3000, () => console.log('Login: http://localhost:3000/example-login\nAPI: http://localhost:3000/pets\nDocs: http://localhost:3000/docs\nGET /internal/health is acknowledged as intentionally undocumented'));
buildApp().listen(3000, () => console.log('Login: http://localhost:3000/example-login\nAPI: http://localhost:3000/pets\nDocs: http://localhost:3000/docs\nGET /internal/health is acknowledged. POST /internal/reindex is undocumented and fails validation.'));
}

module.exports = { buildApp };
2 changes: 1 addition & 1 deletion examples/javascript-fastify/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

This example serves the shared OpenAPI 3.1 showcase through Fastify and enables FlexDoc 3.1 Runtime Intelligence. FlexDoc reads Fastify's registered route tree at runtime, normalizes framework route syntax, and compares it with the exact OpenAPI document passed to the integration.

`GET /internal/health` is absent from the spec and listed in `runtimeIntelligence.acknowledgedUndocumented`, so contract validation stays green. The route stays on the snapshot's `routes` array. The Runtime panel shows findings rather than that array, so the acknowledged route does not appear there. The renderer still exposes Basic/Advanced Try It, handoff to the sibling API Client page, persisted workspace state, scripts, response inspection, keyboard workflows and code samples.
`GET /internal/health` is absent from the spec and listed in `runtimeIntelligence.acknowledgedUndocumented`, so that finding is informational. `POST /internal/reindex` is also absent from the spec and is not acknowledged, so contract validation fails. The renderer still exposes Basic/Advanced Try It, handoff to the sibling API Client page, persisted workspace state, scripts, response inspection, keyboard workflows and code samples.

```bash
npm install
Expand Down
3 changes: 2 additions & 1 deletion examples/javascript-fastify/index.js
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@ async function buildApp() {
app.patch('/pets/:petId', async (request) => ({ id: request.params.petId, name: 'Miso', age: 3, tags: ['friendly'], status: 'available', ...(request.body || {}) }));
app.get('/search', async (request) => ({ terms: request.query.terms || [], count: 1 }));
app.get('/internal/health', async () => ({ status: 'internal-ok' }));
app.post('/internal/reindex', async () => ({ accepted: true }));

setupFastifyFlexDoc(app, '/docs', {
spec,
Expand Down Expand Up @@ -48,7 +49,7 @@ async function main() {
await app.listen({ port: 3000, host: '0.0.0.0' });
console.log('API: http://localhost:3000/pets');
console.log('Docs: http://localhost:3000/docs');
console.log('GET /internal/health is acknowledged as intentionally undocumented');
console.log('GET /internal/health is acknowledged. POST /internal/reindex is undocumented and fails validation.');
}

if (require.main === module) {
Expand Down
2 changes: 1 addition & 1 deletion examples/javascript-hono/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,6 @@

This example uses Hono `4.13.x` with the optional `setupHonoFlexDoc` integration. Its registry dependency is pinned to published `@prauga/flexdoc-backend` `3.3.0` for reproducible standalone installs, while repository CI validates against the backend package built from the current commit. Runtime Intelligence reads Hono's actual registered routes instead of inferring them from the OpenAPI document.

The contract documents `/health` and `/pets/{petId}`. `GET /internal/health` is registered only in Hono and listed in `runtimeIntelligence.acknowledgedUndocumented`, so validation stays green. The route stays on the snapshot's `routes` array. The Runtime panel shows findings rather than that array, so the acknowledged route does not appear there. The docs surface also enables Basic/Advanced Try It, handoff to the sibling API Client page, persisted workspace state and code samples.
The contract documents `/health` and `/pets/{petId}`. `GET /internal/health` is listed in `runtimeIntelligence.acknowledgedUndocumented`, so that finding is informational. `POST /internal/reindex` is registered only in Hono and is not acknowledged, so contract validation fails. The docs surface also enables Basic/Advanced Try It, handoff to the sibling API Client page, persisted workspace state and code samples.

The integration remains framework-native without making Hono a hard dependency of the core backend package.
3 changes: 2 additions & 1 deletion examples/javascript-hono/index.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -37,8 +37,9 @@ export const app = new Hono();

app.get('/health', (c) => c.json({ status: 'ok' }));
app.get('/pets/:petId', (c) => c.json({ id: c.req.param('petId'), name: 'Miso' }));
// Absent from the contract and acknowledged so validation stays green.
// Acknowledged, so the finding is informational. /internal/reindex is the drift that fails validation.
app.get('/internal/health', (c) => c.json({ status: 'internal-ok' }));
app.post('/internal/reindex', (c) => c.json({ accepted: true }));

setupHonoFlexDoc(app, '/docs', {
spec,
Expand Down
10 changes: 8 additions & 2 deletions packages/backend/src/contract-validation.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -89,7 +89,7 @@ describe('runtime contract validation', () => {
}));
});

it('omits runtime operations the operator has acknowledged as undocumented', () => {
it('keeps an acknowledged undocumented route as info and does not fail validation', () => {
const result = validateRuntimeContract({
documentedRoutes: [{ method: 'GET', path: '/pets' }],
runtimeRoutes: [
Expand All @@ -100,7 +100,13 @@ describe('runtime contract validation', () => {
discoveryComplete: true,
});
expect(result.status).toBe('pass');
expect(result.findings).toEqual([]);
expect(result.summary).toEqual({ total: 1, errors: 0, warnings: 0, info: 1 });
expect(result.findings[0]).toEqual(expect.objectContaining({
code: 'runtime.operation-undocumented',
severity: 'info',
disposition: 'acknowledged',
location: { kind: 'operation', method: 'GET', path: '/internal/health' },
}));
});

it('reports duplicate host registrations that standalone OpenAPI tooling cannot observe', () => {
Expand Down
11 changes: 7 additions & 4 deletions packages/backend/src/contract-validation.ts
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,7 @@ export interface FlexDocContractValidationFinding {
/** Human-readable explanation suitable for UI and CI output. */ message: string;
/** Concise expected contract/runtime state. */ expected?: string | string[];
/** Concise state actually observed from the running backend. */ observed?: string | string[];
/** Operator policy applied to this finding. Absence means the finding is still actionable. */ disposition?: 'acknowledged';
}

/** Aggregate finding counts for one contract-validation pass. */
Expand Down Expand Up @@ -63,7 +64,7 @@ export interface ValidateRuntimeContractOptions {
/** Normalized operations discovered from the running backend. */ runtimeRoutes: FlexDocContractRoute[];
/** Runtime operations that were registered more than once by the host framework. */ duplicateRuntimeRoutes?: FlexDocContractDuplicateRuntimeRoute[];
/** Whether route discovery is believed to cover the complete running application. */ discoveryComplete: boolean;
/** Runtime operations intentionally absent from OpenAPI. They are not reported as undocumented. */ acknowledgedUndocumented?: FlexDocContractRoute[];
/** Runtime operations the operator has accepted as intentionally absent from OpenAPI. They remain registered and are reported as acknowledged info. */ acknowledgedUndocumented?: FlexDocContractRoute[];
}

function routeShape(path: string): string {
Expand Down Expand Up @@ -147,15 +148,17 @@ export function validateRuntimeContract(options: ValidateRuntimeContractOptions)
for (const route of options.runtimeRoutes) {
const shape = routeShape(route.path);
if (methodMismatchShapes.has(shape)) continue;
if (documentedKeys.has(exactShapeKey(route)) || acknowledgedKeys.has(exactShapeKey(route))) continue;
if (documentedKeys.has(exactShapeKey(route))) continue;
const acknowledged = acknowledgedKeys.has(exactShapeKey(route));
findings.push({
id: findingId('runtime.operation-undocumented', route.path, route.method),
code: 'runtime.operation-undocumented',
severity: options.discoveryComplete ? 'error' : 'warning',
severity: acknowledged ? 'info' : (options.discoveryComplete ? 'error' : 'warning'),
...(acknowledged ? { disposition: 'acknowledged' as const } : {}),
location: { kind: 'operation', method: route.method.toUpperCase(), path: route.path },
message: `Runtime implements ${route.method.toUpperCase()} ${route.path}, but OpenAPI does not document that operation.`,
expected: 'Operation is represented in OpenAPI',
observed: 'Operation exists only in the running backend',
observed: acknowledged ? 'Operation exists only in the running backend and is acknowledged' : 'Operation exists only in the running backend',
});
}

Expand Down
10 changes: 6 additions & 4 deletions packages/backend/src/runtime-intelligence.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -156,11 +156,13 @@ describe('runtime intelligence', () => {
{ method: 'GET', path: '/internal/health' },
{ method: 'GET', path: '/pets' },
]);
expect(snapshot.runtimeOnly).toEqual([]);
expect(snapshot.summary).toEqual({ documented: 1, runtime: 1, matched: 1, runtimeOnly: 0, documentedOnly: 0 });
expect(snapshot.summary.matched + snapshot.summary.runtimeOnly).toBe(snapshot.summary.runtime);
expect(snapshot.runtimeOnly).toEqual([{ method: 'GET', path: '/internal/health' }]);
expect(snapshot.summary).toEqual({ documented: 1, runtime: 2, matched: 1, runtimeOnly: 1, documentedOnly: 0 });
expect(snapshot.validation.status).toBe('pass');
expect(snapshot.validation.findings).toEqual([]);
expect(snapshot.validation.findings[0]).toEqual(expect.objectContaining({
disposition: 'acknowledged',
severity: 'info',
}));
});

it('feeds duplicate host registrations into contract validation', () => {
Expand Down
7 changes: 3 additions & 4 deletions packages/backend/src/runtime-intelligence.ts
Original file line number Diff line number Diff line change
Expand Up @@ -43,7 +43,7 @@ export interface FlexDocRuntimeDiscovery {
/** Aggregate route counts in a Runtime Intelligence snapshot. */
export interface FlexDocRuntimeIntelligenceSummary {
/** Number of documented OpenAPI operations. */ documented: number;
/** Runtime routes in the drift comparison. Equals `matched + runtimeOnly`. Acknowledged undocumented routes stay in `routes` and are excluded here. */ runtime: number;
/** Number of routes the framework registered, including acknowledged undocumented routes. */ runtime: number;
/** Number of wire-equivalent method/path matches between runtime and OpenAPI. */ matched: number;
/** Number of runtime-only routes. */ runtimeOnly: number;
/** Number of documented-only routes. */ documentedOnly: number;
Expand Down Expand Up @@ -351,9 +351,8 @@ export function buildRuntimeIntelligenceSnapshot(input: FlexDocRuntimeIntelligen
const runtimeRoutes = uniqueSorted(input.discovery.routes);
const documentedKeys = new Set(documented.map(contractRouteKey));
const runtimeKeys = new Set(runtimeRoutes.map(contractRouteKey));
const acknowledgedKeys = new Set((input.acknowledgedUndocumented || []).map(contractRouteKey));
const matched = runtimeRoutes.filter((route) => documentedKeys.has(contractRouteKey(route))).length;
const runtimeOnly = runtimeRoutes.filter((route) => !documentedKeys.has(contractRouteKey(route)) && !acknowledgedKeys.has(contractRouteKey(route)));
const runtimeOnly = runtimeRoutes.filter((route) => !documentedKeys.has(contractRouteKey(route)));
const documentedOnly = documented.filter((route) => !runtimeKeys.has(contractRouteKey(route)));
const environment = input.environment || nodeEnvironmentMetadata();
const validation = validateRuntimeContract({
Expand All @@ -377,7 +376,7 @@ export function buildRuntimeIntelligenceSnapshot(input: FlexDocRuntimeIntelligen
documentedOnly,
summary: {
documented: documented.length,
runtime: matched + runtimeOnly.length,
runtime: runtimeRoutes.length,
matched,
runtimeOnly: runtimeOnly.length,
documentedOnly: documentedOnly.length,
Expand Down
Loading
Loading