Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,11 @@

Notable FlexDoc product releases are recorded here. Detailed release notes remain under `docs/releases/` and are linked from each entry.

## Unreleased

- An implemented route that OpenAPI does not document fails `flexdoc validate` when runtime discovery is complete, and the failure names the method and path. Partial discovery keeps that finding a warning. `runtimeIntelligence.acknowledgedUndocumented` omits operations that are intentionally absent from the contract. Those routes stay in the runtime route list, and `summary.runtime` counts only `matched + runtimeOnly`.
- Opening that finding shows the registered route. It is not treated as an OpenAPI operation.

## 3.4.0 — 3.3.x hardening package release

Package semver is independent from internal product-milestone labels: **3.4.0** publishes the completed 3.3.x hardening stack.
Expand Down
116 changes: 58 additions & 58 deletions adapters/elixir/assets/flexdoc.standalone.js

Large diffs are not rendered by default.

116 changes: 58 additions & 58 deletions adapters/go/assets/flexdoc.standalone.js

Large diffs are not rendered by default.

116 changes: 58 additions & 58 deletions adapters/php/assets/flexdoc.standalone.js

Large diffs are not rendered by default.

116 changes: 58 additions & 58 deletions adapters/python/src/prauga_flexdoc/_assets/flexdoc.standalone.js

Large diffs are not rendered by default.

116 changes: 58 additions & 58 deletions adapters/ruby/assets/flexdoc.standalone.js

Large diffs are not rendered by default.

116 changes: 58 additions & 58 deletions adapters/rust-actix/assets/flexdoc.standalone.js

Large diffs are not rendered by default.

116 changes: 58 additions & 58 deletions adapters/rust/assets/flexdoc.standalone.js

Large diffs are not rendered by default.

2 changes: 1 addition & 1 deletion docs/runtime-intelligence.md
Original file line number Diff line number Diff line change
Expand Up @@ -148,7 +148,7 @@ The Node Runtime Intelligence snapshot can include one backend-produced `validat

The four finding codes in this cut are:

- `runtime.operation-undocumented` — the running backend positively exposes an operation that OpenAPI does not document. Severity: `error`. The finding names the method and path. This is runtime evidence only and is not clickable as a spec operation. Because it is an error, the derived status is `fail`, and `flexdoc validate` exits `1` under the default policy.
- `runtime.operation-undocumented` — the running backend positively exposes an operation that OpenAPI does not document. Severity: `error` when discovery is complete, `warning` when discovery is partial, matching the other findings that must not treat an incomplete inventory as authoritative. The finding names the method and path and opens that registered route. It is not an OpenAPI operation. When discovery is complete the derived status is `fail`, and `flexdoc validate` exits `1` under the default policy. Set `runtimeIntelligence.acknowledgedUndocumented` to method/path pairs that are intentionally absent from OpenAPI; those operations are omitted from this finding.
- `runtime.operation-unobserved` — OpenAPI documents an operation that runtime discovery did not observe. Severity: `error` when discovery is complete, `info` when discovery is partial.
- `runtime.method-mismatch` — the same wire-equivalent path exists in both sources but under different HTTP methods. Severity: `error` when discovery is complete, `warning` when partial. The finding carries an expected documented method so the renderer can navigate to the spec operation.
- `runtime.duplicate-operation` — the host exposes multiple registrations for one wire-equivalent method/path. Severity: `warning`. In 3.1 this is an explicitly supported/verified claim for Express (including Nest-on-Express) and Hono. Fastify participates in the other validation checks, but exact duplicate-registration reporting is not claimed for Fastify in this cut.
Expand Down
2 changes: 2 additions & 0 deletions e2e/runtime-intelligence.spec.cjs
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,8 @@ test('runtime panel remains accessible with a hidden topbar and behaves as a mod
await expect(dialog.getByLabel('Close runtime intelligence panel')).toBeFocused();
await expect.poll(() => page.evaluate(() => document.body.style.overflow)).toBe('hidden');

await page.keyboard.press('Tab');
await expect(dialog.getByRole('button', { name: 'Open runtime route POST /internal/reindex' })).toBeFocused();
await page.keyboard.press('Tab');
await expect(dialog.getByLabel('Close runtime intelligence panel')).toBeFocused();
await page.keyboard.press('Escape');
Expand Down
2 changes: 1 addition & 1 deletion examples/javascript-express/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

This is the primary backend-native FlexDoc 3.1 showcase. Express serves the shared OpenAPI 3.1 contract, while FlexDoc runs inside the same backend and exposes the completed documentation/API Client surface plus live Runtime Intelligence.

The example intentionally registers `GET /internal/health` without documenting it. Open **Runtime** in FlexDoc to see a genuine implemented-but-undocumented route alongside the matched OpenAPI operations. That is the kind of backend knowledge a spec-only renderer cannot provide.
The example registers `GET /internal/health` without documenting it, and lists that operation in `runtimeIntelligence.acknowledgedUndocumented`. Contract validation stays green. The route stays on the snapshot's `routes` array. The Runtime panel shows findings rather than that array, so an acknowledged route does not appear there.

The renderer also demonstrates the completed documentation/API Client surface: persisted viewer preferences, deep links, keyboard command palette, mobile navigation, JSON/YAML download, Basic/Advanced Try It, handoff to the sibling API Client page, environments, scripts, the CodeMirror-backed editor, response inspection, collection/history workflows, and code samples.

Expand Down
7 changes: 5 additions & 2 deletions examples/javascript-express/index.js
Original file line number Diff line number Diff line change
Expand Up @@ -85,7 +85,10 @@ function buildApp() {
requiredPropsFirst: true,
sortPropsAlphabetically: true,
showRequestHeaders: true,
runtimeIntelligence: true,
runtimeIntelligence: {
enabled: true,
acknowledgedUndocumented: [{ method: 'GET', path: '/internal/health' }],
},
expand: 'interactive',
tryIt: {
enabled: true,
Expand All @@ -103,7 +106,7 @@ function buildApp() {
}

if (require.main === module) {
buildApp().listen(3000, () => console.log('Login: http://localhost:3000/example-login\nAPI: http://localhost:3000/pets\nDocs: http://localhost:3000/docs\nRuntime drift: GET /internal/health is intentionally undocumented'));
buildApp().listen(3000, () => console.log('Login: http://localhost:3000/example-login\nAPI: http://localhost:3000/pets\nDocs: http://localhost:3000/docs\nGET /internal/health is acknowledged as intentionally undocumented'));
}

module.exports = { buildApp };
2 changes: 1 addition & 1 deletion examples/javascript-fastify/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

This example serves the shared OpenAPI 3.1 showcase through Fastify and enables FlexDoc 3.1 Runtime Intelligence. FlexDoc reads Fastify's registered route tree at runtime, normalizes framework route syntax, and compares it with the exact OpenAPI document passed to the integration.

`GET /internal/health` is intentionally absent from the spec so the Runtime panel shows real drift. The normal renderer still exposes Basic/Advanced Try It, handoff to the sibling API Client page, persisted workspace state, scripts, response inspection, keyboard workflows and code samples.
`GET /internal/health` is absent from the spec and listed in `runtimeIntelligence.acknowledgedUndocumented`, so contract validation stays green. The route stays on the snapshot's `routes` array. The Runtime panel shows findings rather than that array, so the acknowledged route does not appear there. The renderer still exposes Basic/Advanced Try It, handoff to the sibling API Client page, persisted workspace state, scripts, response inspection, keyboard workflows and code samples.

```bash
npm install
Expand Down
7 changes: 5 additions & 2 deletions examples/javascript-fastify/index.js
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,10 @@ async function buildApp() {
requiredPropsFirst: true,
sortPropsAlphabetically: true,
showRequestHeaders: true,
runtimeIntelligence: true,
runtimeIntelligence: {
enabled: true,
acknowledgedUndocumented: [{ method: 'GET', path: '/internal/health' }],
},
expand: 'interactive',
tryIt: { enabled: true, defaultServer: 'http://localhost:3000', credentials: 'same-origin', apiClientPersistenceKey: 'flexdoc-fastify-3-showcase' },
codeSamples: { enabled: true, languages: ['curl', 'javascript', 'python', 'go', 'java'] },
Expand All @@ -45,7 +48,7 @@ async function main() {
await app.listen({ port: 3000, host: '0.0.0.0' });
console.log('API: http://localhost:3000/pets');
console.log('Docs: http://localhost:3000/docs');
console.log('Runtime drift: GET /internal/health is intentionally undocumented');
console.log('GET /internal/health is acknowledged as intentionally undocumented');
}

if (require.main === module) {
Expand Down
2 changes: 1 addition & 1 deletion examples/javascript-hono/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,6 @@

This example uses Hono `4.13.x` with the optional `setupHonoFlexDoc` integration. Its registry dependency is pinned to published `@prauga/flexdoc-backend` `3.3.0` for reproducible standalone installs, while repository CI validates against the backend package built from the current commit. Runtime Intelligence reads Hono's actual registered routes instead of inferring them from the OpenAPI document.

The contract documents `/health` and `/pets/{petId}`. `GET /internal/health` is deliberately registered only in Hono so the Runtime panel demonstrates implemented-but-undocumented drift. The docs surface also enables Basic/Advanced Try It, handoff to the sibling API Client page, persisted workspace state and code samples.
The contract documents `/health` and `/pets/{petId}`. `GET /internal/health` is registered only in Hono and listed in `runtimeIntelligence.acknowledgedUndocumented`, so validation stays green. The route stays on the snapshot's `routes` array. The Runtime panel shows findings rather than that array, so the acknowledged route does not appear there. The docs surface also enables Basic/Advanced Try It, handoff to the sibling API Client page, persisted workspace state and code samples.

The integration remains framework-native without making Hono a hard dependency of the core backend package.
7 changes: 5 additions & 2 deletions examples/javascript-hono/index.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,7 @@ export const app = new Hono();

app.get('/health', (c) => c.json({ status: 'ok' }));
app.get('/pets/:petId', (c) => c.json({ id: c.req.param('petId'), name: 'Miso' }));
// Intentionally omitted from the contract so Runtime Intelligence reports it as runtime-only.
// Absent from the contract and acknowledged so validation stays green.
app.get('/internal/health', (c) => c.json({ status: 'internal-ok' }));

setupHonoFlexDoc(app, '/docs', {
Expand All @@ -46,7 +46,10 @@ setupHonoFlexDoc(app, '/docs', {
title: 'FlexDoc Hono 3.0 showcase',
description: 'Live Hono route discovery plus the canonical FlexDoc 3.0 renderer and API Client surface.',
version: '3.0.0',
runtimeIntelligence: true,
runtimeIntelligence: {
enabled: true,
acknowledgedUndocumented: [{ method: 'GET', path: '/internal/health' }],
},
expand: 'interactive',
tryIt: {
enabled: true,
Expand Down
27 changes: 27 additions & 0 deletions packages/backend/src/contract-validation.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -76,6 +76,33 @@ describe('runtime contract validation', () => {
})]);
});

it('downgrades an undocumented route to a warning when discovery is partial', () => {
const result = validateRuntimeContract({
documentedRoutes: [{ method: 'GET', path: '/pets' }],
runtimeRoutes: [{ method: 'POST', path: '/internal/reindex' }],
discoveryComplete: false,
});
expect(result.status).toBe('warn');
expect(result.findings[0]).toEqual(expect.objectContaining({
code: 'runtime.operation-undocumented',
severity: 'warning',
}));
});

it('omits runtime operations the operator has acknowledged as undocumented', () => {
const result = validateRuntimeContract({
documentedRoutes: [{ method: 'GET', path: '/pets' }],
runtimeRoutes: [
{ method: 'GET', path: '/pets' },
{ method: 'GET', path: '/internal/health' },
],
acknowledgedUndocumented: [{ method: 'GET', path: '/internal/health' }],
discoveryComplete: true,
});
expect(result.status).toBe('pass');
expect(result.findings).toEqual([]);
});

it('reports duplicate host registrations that standalone OpenAPI tooling cannot observe', () => {
const result = validateRuntimeContract({
documentedRoutes: [{ method: 'GET', path: '/pets/{petId}' }],
Expand Down
6 changes: 4 additions & 2 deletions packages/backend/src/contract-validation.ts
Original file line number Diff line number Diff line change
Expand Up @@ -63,6 +63,7 @@ export interface ValidateRuntimeContractOptions {
/** Normalized operations discovered from the running backend. */ runtimeRoutes: FlexDocContractRoute[];
/** Runtime operations that were registered more than once by the host framework. */ duplicateRuntimeRoutes?: FlexDocContractDuplicateRuntimeRoute[];
/** Whether route discovery is believed to cover the complete running application. */ discoveryComplete: boolean;
/** Runtime operations intentionally absent from OpenAPI. They are not reported as undocumented. */ acknowledgedUndocumented?: FlexDocContractRoute[];
}

function routeShape(path: string): string {
Expand Down Expand Up @@ -142,14 +143,15 @@ export function validateRuntimeContract(options: ValidateRuntimeContractOptions)
});
}

const acknowledgedKeys = new Set((options.acknowledgedUndocumented || []).map(exactShapeKey));
for (const route of options.runtimeRoutes) {
const shape = routeShape(route.path);
if (methodMismatchShapes.has(shape)) continue;
if (documentedKeys.has(exactShapeKey(route))) continue;
if (documentedKeys.has(exactShapeKey(route)) || acknowledgedKeys.has(exactShapeKey(route))) continue;
findings.push({
id: findingId('runtime.operation-undocumented', route.path, route.method),
code: 'runtime.operation-undocumented',
severity: 'error',
severity: options.discoveryComplete ? 'error' : 'warning',
location: { kind: 'operation', method: route.method.toUpperCase(), path: route.path },
message: `Runtime implements ${route.method.toUpperCase()} ${route.path}, but OpenAPI does not document that operation.`,
expected: 'Operation is represented in OpenAPI',
Expand Down
3 changes: 2 additions & 1 deletion packages/backend/src/framework-adapters.ts
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ import { generateFlexDocHTML } from './template';
import { createHostExecutionState, publicHostExecutionOptions } from './host-execution';
import { hostExecutionRequestOrigin, runHostCookiesRoute, runHostExecutionRoute } from './host-execution-route';
import { createCachedFlexDocPage, matchesFlexDocEtag } from './page-cache';
import { buildRuntimeIntelligenceSnapshot, discoverFastifyRoutes, runtimeIntelligenceEnabled } from './runtime-intelligence';
import { acknowledgedUndocumentedRoutes, buildRuntimeIntelligenceSnapshot, discoverFastifyRoutes, runtimeIntelligenceEnabled } from './runtime-intelligence';

/** Minimal Express application surface required by `setupExpressFlexDoc`. */
export interface ExpressLikeApplication {
Expand Down Expand Up @@ -177,6 +177,7 @@ function setupFastifyFlexDocInternal(
const snapshot = buildRuntimeIntelligenceSnapshot({
spec: await resolvedSpec(),
discovery: await discoverFastifyRoutes(app, normalizedPath),
acknowledgedUndocumented: acknowledgedUndocumentedRoutes(options.options?.runtimeIntelligence),
serverOrigin,
});
return reply
Expand Down
3 changes: 2 additions & 1 deletion packages/backend/src/hono-adapter.ts
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ import { generateFlexDocHTML } from './template';
import { createHostExecutionState, publicHostExecutionOptions } from './host-execution';
import { hostExecutionRequestOrigin, runHostCookiesRoute, runHostExecutionRoute } from './host-execution-route';
import { createCachedFlexDocPage, matchesFlexDocEtag } from './page-cache';
import { buildRuntimeIntelligenceSnapshot, discoverHonoRoutes, runtimeIntelligenceEnabled } from './runtime-intelligence';
import { acknowledgedUndocumentedRoutes, buildRuntimeIntelligenceSnapshot, discoverHonoRoutes, runtimeIntelligenceEnabled } from './runtime-intelligence';

/** Minimal Hono request surface required by FlexDoc's adapter. */
export interface HonoLikeRequest {
Expand Down Expand Up @@ -117,6 +117,7 @@ export function setupHonoFlexDoc(
const snapshot = buildRuntimeIntelligenceSnapshot({
spec: await resolvedSpec(),
discovery: discoverHonoRoutes(app, base),
acknowledgedUndocumented: acknowledgedUndocumentedRoutes(options.options?.runtimeIntelligence),
serverOrigin,
});
return context.body(JSON.stringify(snapshot), 200, {
Expand Down
2 changes: 1 addition & 1 deletion packages/backend/src/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -124,6 +124,6 @@ export type {
FlexDocContractValidationSummary,
ValidateRuntimeContractOptions,
} from './contract-validation';
export { buildRuntimeIntelligenceSnapshot, discoverExpressRoutes, discoverFastifyRoutes, discoverHonoRoutes, documentedOpenApiRoutes, nodeRuntimeMetadata, normalizeRuntimePath } from './runtime-intelligence';
export { acknowledgedUndocumentedRoutes, buildRuntimeIntelligenceSnapshot, discoverExpressRoutes, discoverFastifyRoutes, discoverHonoRoutes, documentedOpenApiRoutes, nodeRuntimeMetadata, normalizeRuntimePath } from './runtime-intelligence';
export type { FlexDocRuntimeDiscovery, FlexDocRuntimeDuplicateRoute, FlexDocRuntimeIntelligenceSnapshot, FlexDocRuntimeMetadata, FlexDocRuntimeRoute } from './runtime-intelligence';
export type { HonoLikeApplication, HonoLikeContext } from './hono-adapter';
1 change: 1 addition & 0 deletions packages/backend/src/interfaces.ts
Original file line number Diff line number Diff line change
Expand Up @@ -227,6 +227,7 @@ export interface FlexDocHostExecutionPublicOptions {
/** Explicit Runtime Intelligence opt-in configuration. */
export interface FlexDocRuntimeIntelligenceOptions {
/** Enable runtime route discovery and drift metadata for this documentation mount. */ enabled: true;
/** Runtime operations that are intentionally absent from OpenAPI and must not be reported as undocumented. */ acknowledgedUndocumented?: Array<{ method: string; path: string }>;
}

/** Runtime Intelligence endpoint metadata safe to serialize to the renderer. */
Expand Down
24 changes: 24 additions & 0 deletions packages/backend/src/runtime-intelligence.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -139,6 +139,30 @@ describe('runtime intelligence', () => {
expect(snapshot.runtime).toEqual({ name: 'node', version: 'v22.22.3', platform: 'linux', arch: 'x64' });
});

it('omits acknowledged undocumented routes from drift and validation', () => {
const snapshot = buildRuntimeIntelligenceSnapshot({
spec: { openapi: '3.1.0', paths: { '/pets': { get: {} } } },
discovery: {
framework: 'express',
complete: true,
routes: [
{ method: 'GET', path: '/pets' },
{ method: 'GET', path: '/internal/health' },
],
},
acknowledgedUndocumented: [{ method: 'GET', path: '/internal/health' }],
});
expect(snapshot.routes).toEqual([
{ method: 'GET', path: '/internal/health' },
{ method: 'GET', path: '/pets' },
]);
expect(snapshot.runtimeOnly).toEqual([]);
expect(snapshot.summary).toEqual({ documented: 1, runtime: 1, matched: 1, runtimeOnly: 0, documentedOnly: 0 });
expect(snapshot.summary.matched + snapshot.summary.runtimeOnly).toBe(snapshot.summary.runtime);
expect(snapshot.validation.status).toBe('pass');
expect(snapshot.validation.findings).toEqual([]);
});

it('feeds duplicate host registrations into contract validation', () => {
const snapshot = buildRuntimeIntelligenceSnapshot({
spec: { openapi: '3.1.0', paths: { '/pets/{petId}': { get: {} } } },
Expand Down
Loading
Loading