Summary
NeMo Relay ships provider codecs for OpenAI Chat, OpenAI Responses, and Anthropic Messages, so middleware (PII redaction, guardrails, policy) and observability subscribers can reason over those providers' payloads through the normalized annotated shapes. There is currently no coverage for Oracle Cloud Infrastructure (OCI) Generative AI, whose chat API uses its own request envelope (ChatDetails with servingMode and chatRequest) and two wire formats (GENERIC and COHERE).
OCI Generative AI serves managed on-demand models (Meta Llama, Cohere Command, xAI Grok, Google Gemini, OpenAI) and dedicated AI cluster endpoints, including imported open-weights models such as NVIDIA Nemotron 3 — so agents calling those models today are invisible to Relay's request intercepts and get no normalized response annotations.
Proposal
- A supported-integrations guide for observing LangChain agents backed by
langchain-oci (plus a langchain-oci optional-dependency extra) with callback tests.
LlmCodec / LlmResponseCodec implementations for the OCI Generative AI chat API covering both the GENERIC and COHERE formats, meeting the Provider Codecs guide's guarantees (encode(decode(original), original) == original, unmodeled-field preservation via baseline-compare-and-patch), with unit tests and an opt-in live integration test against a real OCI endpoint.
Follow-ups under discussion with the maintainers: OCI model pricing entries for cost accounting, and Switchyard routing support.
I have a branch implementing 1 and 2, validated end-to-end against a dedicated AI cluster endpoint hosting an imported NVIDIA Nemotron 3 model; PR to follow referencing this issue.
Summary
NeMo Relay ships provider codecs for OpenAI Chat, OpenAI Responses, and Anthropic Messages, so middleware (PII redaction, guardrails, policy) and observability subscribers can reason over those providers' payloads through the normalized annotated shapes. There is currently no coverage for Oracle Cloud Infrastructure (OCI) Generative AI, whose chat API uses its own request envelope (
ChatDetailswithservingModeandchatRequest) and two wire formats (GENERICandCOHERE).OCI Generative AI serves managed on-demand models (Meta Llama, Cohere Command, xAI Grok, Google Gemini, OpenAI) and dedicated AI cluster endpoints, including imported open-weights models such as NVIDIA Nemotron 3 — so agents calling those models today are invisible to Relay's request intercepts and get no normalized response annotations.
Proposal
langchain-oci(plus alangchain-ocioptional-dependency extra) with callback tests.LlmCodec/LlmResponseCodecimplementations for the OCI Generative AI chat API covering both theGENERICandCOHEREformats, meeting the Provider Codecs guide's guarantees (encode(decode(original), original) == original, unmodeled-field preservation via baseline-compare-and-patch), with unit tests and an opt-in live integration test against a real OCI endpoint.Follow-ups under discussion with the maintainers: OCI model pricing entries for cost accounting, and Switchyard routing support.
I have a branch implementing 1 and 2, validated end-to-end against a dedicated AI cluster endpoint hosting an imported NVIDIA Nemotron 3 model; PR to follow referencing this issue.