Skip to content

feat: add vendor extensions metric with per-extension counts - #3021

Open
n0rahh wants to merge 29 commits into
mainfrom
feat/stats-for-vendor-extensions
Open

feat: add vendor extensions metric with per-extension counts#3021
n0rahh wants to merge 29 commits into
mainfrom
feat/stats-for-vendor-extensions

Conversation

@n0rahh

@n0rahh n0rahh commented Aug 5, 2026

Copy link
Copy Markdown
Contributor

What/Why/How?

Adds a Vendor Extensions metric to the stats command. It reports how many distinct x- extensions a document uses and how many times each one occurs, shown in the stylish, json, and markdown output. Works across OpenAPI and AsyncAPI.

The stats visitors collect extensions through a single SpecExtension entrypoint. Two walker fixes make that possible:

  • SpecExtension nodes now dedupe by location, so every occurrence is visited — previously occurrences with equal scalar values (e.g. x-internal: true on many operations) were visited only once. This also means visitors and configurable rules targeting SpecExtension now fire per occurrence.
  • Fixed a pre-existing bug where x- properties with a declared type (e.g. x-codeSamples) were walked twice, which inflated other metrics.

For the stats walk only, ensureSpecExtensionDispatch adjusts the command's normalized types so every x- key dispatches as SpecExtension — including natively-typed extensions and AsyncAPI types that don't declare extensionsPrefix. The structural extensions x-webhooks and x-query keep their declared types so the webhooks/operations/tags metrics still traverse their subtrees; the visitors count those two explicitly. Lint and bundle behavior is unchanged.

The collector also gathers per-extension prop names and value samples for the portal's stats collector (telemetry) via the accumulator — the CLI prints only totals and counts. Samples are bounded (20 props / 20 values per extension), long strings become a <string:N> marker, and credential-like keys and values are masked.

Fixed the stats command always reporting Parameters: 0 for AsyncAPI 2.x and 3.x descriptions. Channel parameters are keyed by name rather than carrying a name property, so none of them were counted.

Reference

Testing

Covered with unit and e2e tests.
Published snapshot and tested cli commands in terminal.

Screenshots (optional)

image image image

Check yourself

  • This PR follows the contributing guide
  • All new/updated code is covered by tests
  • Core code changed? - Tested with other Redocly products (internal contributions only)
  • New package installed? - Tested in different environments (browser/node)
  • Documentation update has been considered

Security

  • The security impact of the change has been considered
  • Code follows company security practices and guidelines

Note

Medium Risk
Walker and resolve dispatch changes affect all lint/stats walks and SpecExtension rule behavior (more visits, fewer duplicates); scope is large but covered by new unit and e2e tests.

Overview
Adds a Vendor Extensions (xExtensions) metric to redocly stats: total distinct x- names plus per-extension occurrence counts, surfaced in stylish, JSON, and Markdown output (including optional counts on each metric row in JSON).

The stats visitors count extensions via SpecExtension enter hooks; AsyncAPI parameter totals now use the channel parameter map key instead of parameter.name, fixing Parameters: 0 on AsyncAPI 2.x/3.x.

Core walker/resolver changes route x- properties to SpecExtension (including when additionalProperties would otherwise match), attach SpecExtension visitors for declared extensions while keeping typed walks where defined, dedupe seen nodes by location for scalars so repeated identical extension values are counted separately, and avoid double-visiting declared extensions like x-codeSamples. AsyncAPI/OAS type trees gain broader extensionsPrefix: 'x-' coverage; Swagger 2 scopes use a dedicated Scopes map so x- keys inside scopes dispatch correctly.

Reviewed by Cursor Bugbot for commit 33e26b1. Bugbot is set up for automated code reviews on this repo. Configure here.

@changeset-bot

changeset-bot Bot commented Aug 5, 2026

Copy link
Copy Markdown

🦋 Changeset detected

Latest commit: 33e26b1

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 4 packages
Name Type
@redocly/openapi-core Minor
@redocly/cli Minor
@redocly/client-generator Patch
@redocly/respect-core Minor

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@github-actions

github-actions Bot commented Aug 5, 2026

Copy link
Copy Markdown
Contributor

Coverage Report

Status Category Percentage Covered / Total
🔵 Lines 77.31% (🎯 77%) 11680 / 15107
🔵 Statements 77.4% (🎯 77%) 12502 / 16152
🔵 Functions 81.2% (🎯 81%) 2355 / 2900
🔵 Branches 71% (🎯 71%) 8607 / 12121
File Coverage
File Stmts Branches Functions Lines Uncovered Lines
Changed Files
packages/cli/src/commands/stats/visitor-and-accumulator-resolver.ts 0% 0% 0% 0% 13-57
packages/cli/src/commands/stats/print-stats/json.ts 0% 100% 0% 0% 8-15
packages/cli/src/commands/stats/print-stats/markdown.ts 0% 0% 0% 0% 10-24
packages/cli/src/commands/stats/print-stats/stylish.ts 0% 0% 0% 0% 11-15
packages/core/src/resolve.ts 97.17% 96.5% 100% 97.1% 96, 143, 301, 492-493
packages/core/src/walk.ts 98.3% 96.64% 85% 98.8% 222, 237, 484
packages/core/src/rules/other/stats.ts 5.08% 0% 0% 5.17% 19-34, 38-107, 114-166, 173-227
packages/core/src/types/asyncapi-bindings.ts 95.37% 0% 0% 95.37% 245-246, 257, 268, 283
packages/core/src/types/asyncapi2.ts 85.71% 57.14% 100% 85.71% 580-582, 588, 596-598, 604
packages/core/src/types/asyncapi3.ts 57.14% 15% 25% 57.14% 188-194, 262-270, 500-528
packages/core/src/types/index.ts 97.29% 88.46% 100% 97.29% 140
packages/core/src/types/json-schema-draft7.shared.ts 71.42% 50% 71.42% 71.42% 68, 91-94
packages/core/src/types/oas2.ts 52.23% 21.95% 33.33% 52.23% 173, 176, 179, 209-212, 266-269, 320-332, 390-428
packages/core/src/types/oas3.ts 75.36% 37.5% 66.66% 75.36% 618, 625, 760-763, 823-847
packages/core/src/types/oas3_2.ts 62.5% 14.28% 50% 62.5% 68-155
Generated in workflow #11305 for commit 33e26b1 by the Vitest Coverage Report Action

@github-actions

github-actions Bot commented Aug 5, 2026

Copy link
Copy Markdown
Contributor

Performance Benchmark (Lower is Faster)

CLI Version Bundle Lint Check Config
cli-latest ▓ 1.00x (Fastest) ▓ 1.00x (Fastest) ▓ 1.00x ± 0.01
cli-next ▓ 1.01x ± 0.01 ▓ 1.00x ± 0.01 ▓ 1.00x (Fastest)

@n0rahh n0rahh added the snapshot Create experimental release PR label Aug 5, 2026
@github-actions

github-actions Bot commented Aug 5, 2026

Copy link
Copy Markdown
Contributor

📦 A new experimental 🧪 version v0.0.0-snapshot.1785934483 of Redocly CLI has been published for testing.

Install with NPM:

npm install @redocly/cli@0.0.0-snapshot.1785934483

⚠️ Note: This is a development build and may contain unstable features.

@n0rahh
n0rahh marked this pull request as ready for review August 5, 2026 13:45
@n0rahh
n0rahh requested review from a team as code owners August 5, 2026 13:45
Comment thread packages/core/src/rules/other/spec-extensions.ts Outdated
Comment thread packages/core/src/rules/other/spec-extensions.ts Outdated
… to $ref and ignore map keys starting with x-
Comment thread packages/core/src/rules/other/spec-extensions.ts Outdated
Comment thread .changeset/seven-waves-create.md Outdated
Comment thread packages/core/src/rules/other/spec-extensions.ts Outdated
Comment thread packages/core/src/rules/other/spec-extensions.ts Outdated
@n0rahh n0rahh added snapshot Create experimental release PR and removed snapshot Create experimental release PR labels Aug 6, 2026
@n0rahh n0rahh added snapshot Create experimental release PR and removed snapshot Create experimental release PR labels Aug 7, 2026
@n0rahh
n0rahh requested a review from tatomyr August 7, 2026 10:49
@github-actions

github-actions Bot commented Aug 7, 2026

Copy link
Copy Markdown
Contributor

📦 A new experimental 🧪 version v0.0.0-snapshot.1786099785 of Redocly CLI has been published for testing.

Install with NPM:

npm install @redocly/cli@0.0.0-snapshot.1786099785

⚠️ Note: This is a development build and may contain unstable features.

@adamaltman

Copy link
Copy Markdown
Member

I found things I didn't know existed in there. Thank you.

Document: ./website/api-definitions/openapi/openapi.yaml stats:

🚗 References: 972
📦 External Documents: 0
📈 Schemas: 838
👉 Parameters: 61
🔗 Links: 0
🔀 Path Items: 382
🎣 Webhooks: 148
👷 Operations: 613
🔖 Tags: 88
🧩 Vendor Extensions: 16
  - x-badges: 151
  - x-basic: 92
  - x-codeSamples: 261
  - x-currency-field: 16
  - x-enumDescriptions: 79
  - x-is-free-form: 1
  - x-label: 26
  - x-logo: 1
  - x-multiline: 31
  - x-products: 761
  - x-sdk-operation-name: 613
  - x-sdk-resource: 43
  - x-sdk-skip: 5
  - x-sortable: 155
  - x-tagGroups: 1
  - x-type: 138

./website/api-definitions/openapi/openapi.yaml: stats processed in 143ms

Comment thread docs/@v2/commands/stats.md Outdated
@n0rahh
n0rahh requested a review from RomanHotsiy August 10, 2026 07:17
Comment thread packages/cli/src/commands/stats/visitor-and-accumulator-resolver.ts Outdated
Comment thread packages/core/src/rules/other/stats.ts Outdated
Comment thread tests/e2e/stats/stats-extensions/openapi.yaml
Comment thread packages/core/src/walk.ts Outdated

currentLocation = resolvedLocation;
const isNodeSeen = seenNodesPerType[type.name]?.has?.(resolvedNode);
const seenKey = type === SpecExtension ? location.absolutePointer : resolvedNode;

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is this change for?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SpecExtension nodes are often equal scalars (true, "internal"), and deduping them by node value made the walker visit only the first occurrence document-wide — deduping by location visits every occurrence exactly once (while a $ref-shared extension still counts once), which the stats need to count extensions correctly.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There are plenty of other scalar nodes. It looks like you've found a but that has to be fixed instead of masking.

export function ensureSpecExtensionDispatch(types: Record<string, NormalizedNodeType>) {
for (const type of Object.values(types)) {
if (type === SpecExtension) continue;
type.extensionsPrefix ??= EXTENSION_PREFIX;

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure this is still needed after using the SpecExtension visitor.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is needed: the walker only dispatches an x- key to SpecExtension when the type declares extensionsPrefix, and most AsyncAPI types (everything except SecurityScheme) plus Paths in OAS never declare it. Removing this line drops the AsyncAPI fixture's extension count — only the additionalProperties: {} path survives.

Adding extensionsPrefix to the AsyncAPI type definitions themselves would make extension values walked (and their $refs bundled/linted) for every consumer, which is a behavior change beyond stats — could be a follow-up if we want it.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So we can modify types to declare the extensions where needed. The purpose of having right visitors tree is exactly to avoid manipulations like this.

Comment thread packages/core/src/rules/other/stats.ts Outdated
Comment thread packages/core/src/rules/other/stats.ts Outdated
}
}
const extensionNames = Object.keys(extensions).sort();
statsAccumulator.xExtensions.total = extensionNames.length;

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why not use the same approach with Set like in other rows?

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Moreover, maybe we can simply calculate the items' size in place instead of assigning the total?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

On Set question already answered above.

Four consumers (three printers and the portal collector) read .total uniformly for every row, so deriving the size at read time would push an items ? items.size : total check into each of them. This one-time assignment at Root.leave also predates the PR — the loop just replaces the four per-row copies that main already had.

const entryType = type.additionalProperties;
// An untyped catch-all (`additionalProperties: {}`) swallows x- keys before the extensions fallback.
if (isPlainObject(entryType) && !isNamedType(entryType) && entryType.type === undefined) {
type.additionalProperties = (_value, key: string) =>

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think it's a good idea to modify types. What are you're trying to achieve?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This modifies only the stats command's own copy of the types — lint keeps the original ones where the declarations matter for validation. Without it, typed extensions (x-codeSamples, x-logo) and most AsyncAPI nodes never reach the SpecExtension visitor, since the walker dispatches them to their declared types instead. The only alternative is going back to key-scanning in the any hook — the approach you wanted to replace with SpecExtension

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Then we have wrong types or wrong walker logic. Making different types for different commands makes no sense to me.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agree, most of this can be fixed at the source instead of remapping types per command

  1. Declare extensionsPrefix: 'x-' in the type definitions where the specs allow extensions
  2. In the walker, check extensionsPrefix before falling back to additionalProperties

@tatomyr wdyt?

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agree. I believe it makes sense to do that in this PR as it directly touches the spec extensions functionality and it'd be easier to spot uncovered places.

Comment thread .changeset/olive-donkeys-shave.md Outdated
Comment thread packages/core/src/walk.ts Outdated

currentLocation = resolvedLocation;
const isNodeSeen = seenNodesPerType[type.name]?.has?.(resolvedNode);
const seenKey = type === SpecExtension ? location.absolutePointer : resolvedNode;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@n0rahh
why scope the fix to SpecExtension only? can other primitive nodes hit the same problem?
If they can, using resolvedLocation.absolutePointer as a key instead of checking type === SpecExtension would cover them all in one place

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Only SpecExtension holds arbitrary user data, so equal scalar values like true are its normal case — other node types hold objects, which never collide as keys. Using the location as the key for all types isn't safe either: a YAML anchor puts the same object in many places, and every rule would then visit and report it once per place instead of once

@n0rahh
n0rahh requested review from tatomyr and vadyvas August 11, 2026 13:21
}
}

export function collectSpecExtension(

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nothing in this repo reads props, the printers only use count, and CLI telemetry never sends it

So could the sampling (props, describe, these regexes) move to the portal collector instead?

@n0rahh
n0rahh marked this pull request as draft August 13, 2026 08:05
Comment thread packages/core/src/walk.ts
Comment thread packages/core/src/walk.ts
Comment thread packages/core/src/walk.ts

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

Reviewed by Cursor Bugbot for commit 33e26b1. Configure here.

@n0rahh n0rahh added snapshot Create experimental release PR and removed snapshot Create experimental release PR labels Aug 13, 2026
@n0rahh
n0rahh requested a review from vadyvas August 13, 2026 12:59
@n0rahh
n0rahh marked this pull request as ready for review August 13, 2026 12:59
@github-actions

Copy link
Copy Markdown
Contributor

📦 A new experimental 🧪 version v0.0.0-snapshot.1786626004 of Redocly CLI has been published for testing.

Install with NPM:

npm install @redocly/cli@0.0.0-snapshot.1786626004

⚠️ Note: This is a development build and may contain unstable features.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

snapshot Create experimental release PR

Projects

None yet

Development

Successfully merging this pull request may close these issues.

7 participants