feat(observability): generate internal metrics docs from configurable metadata#25580
Draft
gwenaskell wants to merge 7 commits into
Draft
feat(observability): generate internal metrics docs from configurable metadata#25580gwenaskell wants to merge 7 commits into
gwenaskell wants to merge 7 commits into
Conversation
github-actions
Bot
added
domain: vdev
gwenaskell
force-pushed
the
yoenn.burban/generate-website-metrics
branch
from
ff19032 to
0667048
Compare
github-actions
Bot
added
the
domain: external docs
gwenaskell
force-pushed
the
yoenn.burban/generate-website-metrics
branch
from
d6c971f to
a01198f
Compare
… metadata - Add `metric_tags!` macro and base tag-set statics to compose metric tag metadata ergonomically (inspired by CUE spread syntax) - Rewrite `CounterName`, `GaugeName`, and `HistogramName` enums with full doc comments and `#[configurable(metadata(docs::tags = ...))]` annotations referencing the new tag-set constants - Extend the `Configurable` derive macro AST parser to accept macro, reference, call, unary, and path expressions inside metadata values - Support optional deprecation messages on enum variants via `#[configurable(deprecated = "message")]` - Inject metric enum schemas into the root config schema in `generate_schema.rs` so downstream tooling can consume them - Add `generate_internal_metric_descriptions` to the vdev `component_docs` runner, producing a `internal_metric_descriptions.cue` file for the Vector website docs pipeline Co-authored-by: Cursor <cursoragent@cursor.com>
…gs metadata Introduces Option B from the tags annotation discussion: a first-class `tags` field on the `Configurable` derive's variant attributes that eliminates the verbose `metadata(docs::tags = ...)` wrapper. - Add `AnyExpr` to `ast/mod.rs`: a `FromMeta` impl that captures any expression (macro invocations, paths, literals) as a raw token stream, bypassing darling's built-in literal-only expression handling. - Add `tags: Option<AnyExpr>` to the variant `Attributes` struct in `ast/variant.rs`; the field is wired into `metadata()` and emits a `docs::tags` custom attribute automatically. - Add `DOCS_META_TAGS` constant to `vector-config-common/src/constants.rs`. - Replace all 161 `#[configurable(metadata(docs::tags = ...))]` annotations in `metric_name.rs` with the compact `#[configurable(tags = ...)]` form, including multi-line block forms and variants that combine `deprecated` and `tags` in the same attribute. Before: #[configurable(metadata(docs::tags = metric_tags!(..COMPONENT_TAGS)))] After: #[configurable(tags = metric_tags!(..COMPONENT_TAGS))] Co-authored-by: Cursor <cursoragent@cursor.com>
…s = metric_tags!(...)
Replaces the `#[configurable(tags = metric_tags!(...))]` shorthand from the
previous commit with the cleaner list syntax originally intended:
#[configurable(tags(..COMPONENT_TAGS))]
#[configurable(tags { ..COMPONENT_TAGS, "protocol": { ... } })]
Implementation:
- Replace `AnyExpr` (which hooked `FromMeta::from_expr` for `name = value`
pairs) with `TagsTokens` that overrides `FromMeta::from_meta` directly.
This intercepts the `Meta::List` node before darling tries to parse its
contents as `NestedMeta` items — which would fail for spread syntax (`..X`)
since `..` is not valid attribute meta syntax.
- The captured `TokenStream` is wrapped in `crate::metric_tags! { ... }` by
the code generator so the annotation body is forwarded verbatim to the macro.
- All 161 annotations in `metric_name.rs` migrated to the new list form;
`use crate::metric_tags;` removed (no longer referenced in source).
Co-authored-by: Cursor <cursoragent@cursor.com>
…able code gen
The previous implementation generated `crate::metric_tags! { ... }` which required
the callsite crate to define `metric_tags!` at its root — an implicit coupling to
`vector-common`.
This commit inlines the entire spread/merge logic into the proc macro itself and
roots all generated paths in `::vector_config`:
- Add `::vector_config::merge_tags(base, extra)` function that merges JSON objects
(replaces `merge_lazy` from `vector-common`).
- Add `pub use serde_json::json` to `vector-config` so generated code can use
`::vector_config::json!({ ... })` without requiring a direct `serde_json` dep.
- Add `TagsBody` syn parser in `ast/mod.rs` that understands the three forms of the
tags body (empty / spread-only / spread + entries).
- `TagsTokens::into_value_tokens()` converts each form to fully qualified inline code:
tags() → ::vector_config::json!({})
tags(..BASE) → (*BASE).clone()
tags { ..BASE, k:v } → ::vector_config::merge_tags((*BASE).clone(), ::vector_config::json!({k:v}))
- `variant.rs` now calls `t.into_value_tokens()` instead of emitting the macro call.
The only external name the generated code now references is the user-supplied base
constant (e.g. `COMPONENT_TAGS`) — which is the user's responsibility to have in
scope, just as with the `metadata(key = expr)` form.
Co-authored-by: Cursor <cursoragent@cursor.com>
…ng_self_convention) Co-authored-by: Cursor <cursoragent@cursor.com>
gwenaskell
force-pushed
the
yoenn.burban/generate-website-metrics
branch
from
a01198f to
8b6ddc1
Compare
Co-authored-by: Cursor <cursoragent@cursor.com>
gwenaskell
force-pushed
the
yoenn.burban/generate-website-metrics
branch
from
8b6ddc1 to
9ff4327
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Makes internal metric metadata (tags, unit) machine-readable and automatically generates the
internal_metric_descriptions.cuefile for the Vector docs website from source annotations.#[configurable(tags(..BASE))]and#[configurable(tags { ..BASE, \"key\": { ... } })]shorthand attributes on metric enum variants. The spread body is parsed and inlined by the proc macro — no coupling to any external macro.#[configurable(unit = \"event\")]shorthand for the canonical Datadog metric unit, validated at compile time against the full unit enum from the OPW metrics schema.CounterName,GaugeName, andHistogramNamevariants inmetric_name.rswith doc comments,tags(...), andunit = \"...\"annotations covering every metric.vdev build component-docsto producewebsite/cue/reference/generated/internal_metric_descriptions.cuefrom the injected metric enum schemas, replacing the previously hand-maintained CUE entries.unit?: stringto#MetricOutputincomponents.cueso the generatedunit:field passes CUE validation.metric_tags!macro fromvector-common.How did you test this PR?
cargo check -p vector-config-macros -p vector-common -p vector-config— all proc macro and annotation changes compile cleanly.cargo run --bin vector -- generate-schema— metric enums appear in the root schema under_metric_schemas.vdev build component-docs—internal_metric_descriptions.cueis generated withdescription,type,tags, andunitfor every metric variant;cue vetpasses.Change Type
Is this a breaking change?
Does this PR include user facing changes?
References