Forward-merge release/0.6 into main#515
Conversation
> [!WARNING] > **BREAKING CHANGE — Rust built-in PII configuration API and serialized configuration:** `BuiltinBackendConfig` now includes the public `preset` and `custom_mark_payload_policy` fields, so downstream exhaustive struct literals must initialize them or use `..Default::default()`. The default `action = "remove"` value is now omitted during serialization, so consumers that depend on the previous exact serialized shape must update. Existing TOML and JSON configurations remain accepted, and existing runtime behavior is unchanged when `preset` is omitted; structure-preserving trajectory redaction is opt-in. #### Overview Adds a structure-preserving `trajectory_context` policy to the composable PII-redaction contract. Exported trajectories can retain conversation topology, tool-call relationships, agent hierarchy, routing, usage, cost, and optimization telemetry while removing request/response content. This draft is intentionally stacked on [#512](#512). Review this PR relative to `bbednarski/pii-profile-composition`; its base can move to `release/0.6` after #512 merges. - [x] I confirm this contribution is my own work, or I have the right to submit it under this project's license. - [x] I searched existing issues and open pull requests, and this does not duplicate existing work. #### Details - Adds the built-in `trajectory_context` preset for OpenAI Chat Completions, OpenAI Responses, and Anthropic Messages event shapes, including buffered and streaming flows. - Redacts chat text, reasoning, multimodal references, tool inputs/outputs, nested-agent content, and stringified tool arguments while preserving structural identifiers and analytical fields. - Sanitizes typed Relay marks semantically so lifecycle, model-routing, token, usage, cost, and optimization data remain useful. - Adds `custom_mark_payload_policy`: - `preserve` (default) leaves opaque third-party mark payloads unchanged. - `redact_all_leaves` recursively replaces scalar payload leaves while retaining keys, arrays, objects, and typed category identity. - Keeps email redaction composable as a later profile so addresses are also removed from retained metadata and custom marks. - Sanitizes canonical event fields before subscriber fan-out. Exporter-owned resource attributes remain outside this plugin boundary. - Updates the CLI editor, schema coverage, and crate documentation. The policy changes only observability data; it does not rewrite the provider request or client-visible provider response. #### Where should the reviewer start? Start in `crates/pii-redaction/src/trajectory.rs` for the semantic redaction boundary, then review preset validation and wiring in `crates/pii-redaction/src/component.rs`. Provider-shape, streaming, mark-policy, and exporter-facing regression coverage is in `crates/pii-redaction/tests/unit/component_tests.rs`. Validation performed on the complete stack: ```text cargo fmt --all -- --check cargo test -p nemo-relay-pii-redaction --features schema cargo clippy -p nemo-relay-pii-redaction --all-targets --features schema -- -D warnings cargo test -p nemo-relay-cli pii_redaction ``` The PII crate suite passes 83 tests, and the focused CLI suite passes 7 tests. #### Related Issues: (use one of the action keywords Closes / Fixes / Resolves / Relates to) - Depends on #512. ## Summary by CodeRabbit * **New Features** * Added multiple ordered PII redaction profiles with profile-scoped backend registration and rollback-aware setup. * Added `trajectory_context` built-in preset for structure-preserving conversational sanitization. * Added `custom_mark_payload_policy` handling for unknown custom marks (preserve vs redact-all-leaves), including preset-specific constraints. * **Documentation** * Updated plugin scope and configuration docs for profile-array mode, ordering/compatibility rules, and trajectory sanitization workflows. * **Tests** * Expanded coverage for profile validation/registration, `trajectory_context` request/response/event sanitization, custom mark policies, and streaming finalization/export redaction. Authors: - Bryan Bednarski (https://github.com/bbednarski9) Approvers: - Will Killian (https://github.com/willkill07) URL: #513
|
FAILURE - Unable to forward-merge due to an error, manual merge is necessary. Do not use the IMPORTANT: When merging this PR, do not use the auto-merger (i.e. the |
#### Overview > [!WARNING] > **BREAKING CHANGE:** **[annotated request schema and plugin compatibility]** `AnnotatedLlmRequest` adds public fields and message, content, tool, and tool-choice variants. Rust exhaustive matches and direct struct literals must be updated. Request annotations and request-intercept outcomes now use the `@2` schema tags. Rust native plugins and Rust `grpc-v1` workers that consume annotations must rebuild against NeMo Relay 0.6; Python workers must update to the 0.6 worker SDK. Dynamic plugins that register LLM request intercepts must declare `compat.relay = ">=0.6,<1.0"`; the host rejects ranges that still admit Relay 0.5. Node.js, Go, and raw C FFI callback boundaries remain JSON-based and do not require new binding DTOs. Fix Issue 501 by making request-codec annotations lossless across Anthropic Messages, OpenAI Responses, and OpenAI Chat Completions, and by enabling those request codecs in the `nemo-relay` gateway. - [X] I confirm this contribution is my own work, or I have the right to submit it under this project's license. - [X] I searched existing issues and open pull requests, and this does not duplicate existing work. #### Details * Extend `AnnotatedLlmRequest` with shared provider instructions, a tagged mutable `api_specific` surface, portable message/content/tool unions, component extras, and explicit provider-native fallbacks. * Complete request modeling against the pinned Anthropic and OpenAI SDK schemas, including provider-only input items, built-in tools, tool choices, legacy Chat fields, and provider-specific top-level controls. * Guarantee JSON-value identity for no-op `encode(decode(original), original)` calls by comparing edits with a decoded baseline and patching only changed fields. Unknown fields, nulls, alternate representations, item order, and untouched native values remain unchanged. * Patch edited nested messages, content blocks, tools, and tool choices against their original provider values so sibling unknown fields and explicit nulls survive portable edits. * Reject malformed recognized structures, provider-mismatched native edits, unencodable normalized edits, and raw-body mutation before the provider callback. Remove the Responses private sentinel and Chat's implicit `stream_options.include_usage` injection. * Enable request codecs for buffered and streaming `/v1/messages`, `/v1/chat/completions`, and `/v1/responses` gateway calls. Count-token, model, probe, and non-LLM passthrough routes remain unannotated. * Add Python constructor properties and serialization coverage. Version request annotations and request-intercept outcomes as `@2`, require the 0.6 SDK for Python workers, and enforce a Relay 0.6 compatibility floor for dynamic plugins that register request intercepts. Node, FFI, and Go callback boundaries continue carrying annotations as JSON. * Add provider schema fixtures, exact round-trip and surgical-edit tests, an Issue 501 managed-runtime reproduction, a six-way gateway route/streaming matrix, migration documentation, and updated typed-wrapper guidance. * Migration note: gateway interceptors for the three generation routes must mutate `annotated_request`; changing raw `request.content` while a request codec is active remains an explicit runtime error. Validation: * `just test-rust` * `just test-python` (536 passed) * `just test-node` (277 passed) * `just test-go` * `just test-python-plugin` * `just test-claude-plugin-e2e` * `just test-codex-plugin-e2e` * `just docs` * `uv run pre-commit run --all-files` #### Where should the reviewer start? Start with `crates/types/src/codec/request.rs` for the public annotation contract, then review the baseline-diff helper and provider codecs under `crates/core/src/codec/`. The gateway wiring and buffered/streaming matrix are in `crates/cli/src/gateway/mod.rs` and `crates/cli/tests/coverage/shared/server_tests.rs`. #### Related Issues: (use one of the action keywords Closes / Fixes / Resolves / Relates to) * Closes #501 ## Summary by CodeRabbit * **New Features** * Added lossless request annotations for Anthropic Messages, OpenAI Chat Completions, and OpenAI Responses routes, including streaming. * Added support for shared instructions, developer messages, tool calls/results, multimodal content, and provider-native fields. * Exposed `instructions` and `api_specific` through Python request objects. * **Bug Fixes** * Preserves unchanged fields, ordering, explicit nulls, metadata, and unknown provider data during edits. * Rejects unsupported raw-body mutations when request codecs are active. * **Documentation** * Expanded codec behavior, routing, migration, and interception guidance. ## Summary by CodeRabbit - **New Features** - Added lossless, baseline-aware request codecs for Anthropic Messages, OpenAI Chat Completions, and OpenAI Responses to enable safe annotation edits while preserving unchanged JSON. - Expanded annotated requests to support `instructions`, `api_specific`, developer/function/tool/provider-native variants, and richer tool schemas. - Added Python accessors for `instructions` and `api_specific`; enhanced OpenInference message-role projection. - **Bug Fixes** - Stricter validation/encoding behavior, improved refusal/text extraction, and enforced rejection of unsupported raw-body and stream-mode edits when codecs are active. - Enforced annotated-request consumer compatibility. - **Documentation** - Updated codec and migration guidance for the new annotation surfaces. - **Tests** - Updated and expanded codec, gateway, plugin, and observability test coverage. Authors: - Will Killian (https://github.com/willkill07) Approvers: - Bryan Bednarski (https://github.com/bbednarski9) URL: #509
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## main #515 +/- ##
==========================================
+ Coverage 92.44% 93.36% +0.92%
==========================================
Files 268 315 +47
Lines 75150 84874 +9724
Branches 0 100 +100
==========================================
+ Hits 69465 79237 +9772
+ Misses 5685 5637 -48
... and 53 files with indirect coverage changes Continue to review full report in Codecov by Harness.
🚀 New features to boost your workflow:
|
#### Overview Update CODEOWNERS to use the NeMo Relay developer, dependency approver, documentation reviewer, and administrator teams. - [x] I confirm this contribution is my own work, or I have the right to submit it under this project's license. - [x] I searched existing issues and open pull requests, and this does not duplicate existing work. #### Details - Replace NAT ownership teams with NeMo Relay ownership teams. - Assign `/.github/CODEOWNERS` to `@NVIDIA/nemo-relay-admins`. - Assign documentation paths to the NeMo Relay documentation reviewers and developers. #### Where should the reviewer start? Review `.github/CODEOWNERS`. #### Related Issues: (use one of the action keywords Closes / Fixes / Resolves / Relates to) - Relates to: none ## Summary by CodeRabbit * **Chores** * Updated repository ownership and review assignments. * Added dedicated administrative review for ownership configuration. * Updated documentation and attribution review assignments to the appropriate teams. Authors: - Will Killian (https://github.com/willkill07) Approvers: - Bryan Bednarski (https://github.com/bbednarski9) URL: #516
#### Overview Clarify when NeMo Relay subscribers and observability exporters finish processing events, including the behavior during graceful and abrupt shutdown. - [x] I confirm this contribution is my own work, or I have the right to submit it under this project's license. - [x] I searched existing issues and open pull requests, and this does not duplicate existing work. #### Details - Define emitted, delivered, and exported events in the subscriber concept documentation. - Document the shared subscriber flush barrier and the downstream barriers for ATOF, ATIF, OpenTelemetry, and OpenInference. - Explain the file and stream differences for ATOF, including the limits of process-writer flushing and abrupt termination. - Clarify ATIF export timing and remove the redundant subscriber flush from the Rust example. - Update the Node.js custom subscriber example to allow an event-loop turn after native dispatch. - Update public ATOF method comments in the Rust, Python, and Node.js bindings without changing behavior or signatures. - Preserve the existing fail-open telemetry behavior. This change has no runtime, configuration, compatibility, or breaking change. Validation: - `uv run pre-commit run --all-files` - `just docs-linkcheck` - `just docs` - `git diff --check` - Runtime test suites were not run because the change is limited to documentation, examples, and public doc comments. #### Where should the reviewer start? Start with `docs/about-nemo-relay/concepts/subscribers.mdx`, which defines the canonical delivery contract and the boundary between subscriber flush and exporter-specific completion. #### Related Issues: (use one of the action keywords Closes / Fixes / Resolves / Relates to) - Relates to: RELAY-541 ## Summary by CodeRabbit - **Documentation** - Clarified telemetry lifecycle milestones (emitted vs delivered vs exported) and the correct barrier semantics for queued delivery, file sink flushing, and stream sink draining/closing (including stream timeouts being logged without failing). - Updated `force_flush()`/`shutdown()` and `flush_subscribers()` guidance across Rust, Python, native typings, and FFI, including re-entrant behavior and ordering. - Expanded observability plugin teardown and configuration guidance (including plugin teardown steps and warnings about telemetry loss/truncated outputs on abrupt termination). - Refreshed ATIF/ATOF, OpenTelemetry/OpenInference, and Node “Getting Started” (extra event-loop turn after scope creation). Authors: - Will Killian (https://github.com/willkill07) Approvers: - Bryan Bednarski (https://github.com/bbednarski9) URL: #518
#### Overview Update the NeMo Relay 0.6 release notes for the PII redaction changes in PRs #512 and #513. The release notes now explain the new composable profiles and structure-preserving trajectory redaction in plain language, with migration warnings for the breaking Rust configuration and serialization changes. - [x] I confirm this contribution is my own work, or I have the right to submit it under this project's license. - [x] I searched existing issues and open pull requests, and this does not duplicate existing work. #### Details - Update the 0.6 release summary and highlights with the new PII redaction capabilities. - Add migration warnings for `PiiRedactionConfig.profiles`, `BuiltinBackendConfig.preset`, `custom_mark_payload_policy`, and the changed serialized defaults. - Document that `trajectory_context` is opt-in and changes observability data without changing provider requests or client-visible responses. - Keep the change limited to `docs/about-nemo-relay/release-notes/`. Validation: - `git diff --check` - `just docs` - `just docs-linkcheck` - `uv run pre-commit run --files docs/about-nemo-relay/release-notes/index.mdx docs/about-nemo-relay/release-notes/highlights.mdx docs/about-nemo-relay/release-notes/known-issues.mdx` #### Where should the reviewer start? Start with `docs/about-nemo-relay/release-notes/known-issues.mdx` to review the migration warnings, then review the user-facing summaries in `index.mdx` and `highlights.mdx`. #### Related Issues: (use one of the action keywords Closes / Fixes / Resolves / Relates to) - Relates to #512 - Relates to #513 ## Summary by CodeRabbit * **Documentation** * Documented composable, priority-based PII redaction profiles. * Added details about the opt-in `trajectory_context` preset, which removes sensitive request and response content while preserving conversation structure and telemetry. * Added migration guidance for configuration changes, serialization behavior, and custom mark redaction policies. * Updated release scope and known-issues documentation for Relay 0.6 compatibility. Authors: - Will Killian (https://github.com/willkill07) Approvers: - Bryan Bednarski (https://github.com/bbednarski9) URL: #522
#### Overview Resolve the NeMo Relay 0.6 documentation gaps identified in the VDR preparation report. - [x] I confirm this contribution is my own work, or I have the right to submit it under this project's license. - [x] I searched existing issues and open pull requests, and this does not duplicate existing work. #### Details - Pin published Node.js installation examples to `nemo-relay-node@0.6.0`. - Replace the stale versioned Build Plugins README link. - Add factual positioning for OpenTelemetry GenAI conventions and observability or evaluation products, with links from the Overview and README. #### Where should the reviewer start? Review `docs/about-nemo-relay/ecosystem.mdx` for the new comparison, then the Node.js installation examples and README entry points. #### Related Issues: (use one of the action keywords Closes / Fixes / Resolves / Relates to) - Relates to: none ## Summary by CodeRabbit - **Documentation** - Added an ecosystem guide explaining how NeMo Relay complements telemetry, observability, and evaluation tooling. - Linked the new ecosystem guidance from relevant introductory documentation. - Updated Build Plugins documentation links. - Standardized Node.js installation instructions to use `nemo-relay-node@0.6.0` across the README and setup guides. Authors: - Will Killian (https://github.com/willkill07) Approvers: - Eric Evans II (https://github.com/ericevans-nv) URL: #521
#### Overview Extend Relay's best-effort `skill.load` detection to cover additional reviewed structured readers, path fields, and single-layer shell wrappers while preserving the existing event contract. - [x] I confirm this contribution is my own work, or I have the right to submit it under this project's license. - [x] I searched existing issues and open pull requests, and this does not duplicate existing work. #### Details - Recognize the structured-reader suffixes `read_resource`, `get_file_contents`, and `read_file_content`. - Recognize `uri` and `absolute_path` fields while retaining whole-file and exact-`SKILL.md` requirements. - Support one exact wrapper layer for `sh`, `bash`, `zsh`, `fish`, PowerShell, and `pwsh` around the corresponding complete-file readers. - Continue rejecting partial reads, range controls, pipelines, redirections, substitutions, compound commands, nested wrappers, Windows `type`, and PowerShell aliases. - Add unit and lifecycle integration coverage for the new positive and negative cases. - Document the complete registry and the request-based, best-effort detection contract in the event reference and Codex guide. - Preserve the existing `skill.load` data and metadata fields. This change has no breaking API changes. Validation completed: - `cargo fmt --all` - `cargo test -p nemo-relay skill_load` - `just test-rust` - `cargo clippy --workspace --all-targets -- -D warnings` - `just test-python` - `just test-go` - `just test-node` - `just docs` - `uv run pre-commit run --all-files` The authenticated SQA characterization fixture was not updated or run because this workspace does not have access to the external SQA repository. That follow-up remains required before RELAY-540 is complete. #### Where should the reviewer start? Start with `crates/core/src/api/skill_load.rs`, especially the reviewed registries and the single-layer wrapper parsing. Then review `crates/core/tests/unit/skill_load_tests.rs` for the supported and deliberately unsupported cases. #### Related Issues: (use one of the action keywords Closes / Fixes / Resolves / Relates to) - Closes: RELAY-540 ## Summary by CodeRabbit * **New Features** * Expanded automatic `skill.load` detection for additional shell tools/wrappers (zsh/fish, `pwsh`) and more StructuredRead/MCP resource/file read shapes. * **Bug Fixes** * Improved shell-read detection and completion parsing, including stricter PowerShell option/flag handling and tightened `SKILL.md` path validation. * **Documentation** * Updated “Automatic Skill-Load Marks” and Codex “Captured Events” to clarify best-effort criteria, timing, and supported wrapper/request forms. * **Tests** * Added an integration test for minimal tool-parented `skill.load` emission; expanded unit detection acceptance/rejection coverage. Authors: - Will Killian (https://github.com/willkill07) Approvers: - Bryan Bednarski (https://github.com/bbednarski9) URL: #517
#### Overview Reorganizes the getting-started docs so the runtime primer lives with the About NeMo Relay conceptual content, while Getting Started gets a short About entry point and AI-coding-assistant guidance. This replaces draft PR #443 with a clean branch and a single signed-off docs commit. - [x] I confirm this contribution is my own work, or I have the right to submit it under this project's license. - [x] I searched existing issues and open pull requests; this supersedes #443 and does not duplicate remaining open work. #### Details - Moves Agent Runtime Primer from Getting Started to About NeMo Relay and adds a redirect from the old URL. - Adds a new Getting Started About page with starting-point guidance ordered from try-now paths to advanced development workflows. - Adds a "Using an AI coding assistant?" card to Getting Started About and Quick Start with the future no-clone `npx skills add nvidia/skills` path for installing the full Relay user-skill set. - Updates related docs and README links to the new primer location. #### Where should the reviewer start? Start with `docs/getting-started/about.mdx`, then review `docs/about-nemo-relay/agent-runtime-primer.mdx` and `docs/getting-started/quick-start/index.mdx` for the IA changes. #### Related Issues: (use one of the action keywords Closes / Fixes / Resolves / Relates to) - Relates to NVIDIA agent skills publication follow-up. - Supersedes #443. #### Testing - `just docs` passed. Fern reported one warning: missing redirects check skipped because this local environment is not authenticated with `fern login` / `FERN_TOKEN`. ## Summary by CodeRabbit * **Documentation** * Added/expanded NeMo Relay “About” getting-started content, including guidance on selecting the smallest matching path, prerequisites, installation, configuration, and quick-start steps. * Updated Quick Start with a new “Using an AI coding assistant?” card, including skill-installation commands and catalog discovery. * Refreshed documentation navigation ordering and updated Agent Runtime Primer links across relevant pages. * Improved “starting point” routing guidance and adjusted internal doc links. * **Chores** * Added a redirect to keep existing Agent Runtime Primer links working. Authors: - Allan (https://github.com/exactlyallan) Approvers: - Will Killian (https://github.com/willkill07) URL: #479
#### Overview Make Node tool and LLM callback failures reject their returned promises instead of terminating the Node process. - [x] I confirm this contribution is my own work, or I have the right to submit it under this project's license. - [x] I searched existing issues and open pull requests, and this does not duplicate existing work. #### Details - Wrap synchronous tool and LLM execution callbacks on the JavaScript thread so thrown exceptions cross N-API as error envelopes. - Normalize top-level undefined to null and copy callback results through recursive JSON validation before NAPI-RS conversion. - Reject BigInt, sparse arrays, circular values, non-finite numbers, and other unsupported JSON values without aborting Node. - Apply the same validation to promise-aware tool and LLM callbacks while preserving original rejection messages. - Add regressions for sync throws, sync and async BigInt results, stateful getters, custom array iterators, process survival, and supported undefined behavior. - Keep the public Node callback declarations and sync-only execution semantics unchanged. - Breaking changes: none. Validation: - Focused Node callback, tool, and LLM tests: 94 passed. - Full Node test suite: 283 passed. - `just build-node` - `just test-node` - `cargo fmt --all` - `just test-rust` - `cargo clippy --workspace --all-targets -- -D warnings` - `uv run pre-commit run --all-files` - Targeted pre-commit checks for the final staged files. #### Where should the reviewer start? Start with `safe_execution_callback` in `crates/node/src/callable.rs`, then compare the promise-aware validation in `crates/node/src/promise_call.rs` and the process-survival regressions in `crates/node/tests/tools_tests.mjs`. #### Related Issues: (use one of the action keywords Closes / Fixes / Resolves / Relates to) - Closes RELAY-544 ## Summary by CodeRabbit * **New Features** * Tool and LLM execution now accept regular JavaScript callback functions directly, with safer cross-boundary execution. * **Bug Fixes** * More consistent rejection for invalid or unsupported callback results (e.g., circular references, non-finite numbers, BigInt/sparse arrays), avoiding runtime instability. * Completion argument validation is stricter: invalid inputs now fail explicitly instead of defaulting. * **Tests** * Added/expanded tool and LLM execution cases covering thrown callbacks, invalid JSON-like outputs, getter/iterator edge cases, and continued operation after failures. Signed-off-by: Will Killian <wkillian@nvidia.com>
#### Overview Prevent cyclic Python tool callback results from overflowing native JSON conversion and terminating the process. Cyclic results now produce a catchable exception while shared non-cyclic values remain supported. - [x] I confirm this contribution is my own work, or I have the right to submit it under this project's license. - [x] I searched existing issues and open pull requests, and this does not duplicate existing work. #### Details - Validate Python dictionaries, lists, and tuples for active-path cycles before converting callback results to JSON. - Preserve ordinary shared references by removing container identities after each recursive branch completes. - Add Python regressions for cycle rejection, process recovery, and shared non-cyclic values. - Add Node regression coverage confirming synchronous and asynchronous circular results reject without terminating the process. - Breaking changes: none. Validation: - Focused Python cyclic/shared-reference tests: 2 passed. - Focused Node circular-result test: 1 passed. - Python and Node native binding builds passed. - Repository-wide pre-commit validation passed. #### Where should the reviewer start? Start with `validate_acyclic` in `crates/python/src/convert.rs`, then review the Python behavioral tests in `python/tests/test_tools.py`. #### Related Issues: (use one of the action keywords Closes / Fixes / Resolves / Relates to) - Closes RELAY-543 ## Summary by CodeRabbit - **Bug Fixes** - Tool executions now detect and reject circular result data with a clear “circular reference detected” error, instead of risking a process crash. - After a circular-result rejection, the tool system remains usable for subsequent executions. - Shared references in non-cyclic result graphs are preserved correctly. - **Tests** - Added regression coverage for rejecting cyclic tool results across sync/async and multiple data shapes, plus verification of continued operation. - Updated several codec/sanitizer tests to use synchronous callbacks consistently. Authors: - Eric Evans II (https://github.com/ericevans-nv) Approvers: - Bryan Bednarski (https://github.com/bbednarski9) URL: #524
Forward-merge triggered by push to release/0.6 that creates a PR to keep main up-to-date. If this PR is unable to be immediately merged due to conflicts, it will remain open for the team to manually merge. See forward-merger docs for more info.