From 5b6a08a777eafb3b71d8265ccff86ca312fbb4a1 Mon Sep 17 00:00:00 2001 From: rkoster Date: Thu, 17 Sep 2026 09:22:19 +0200 Subject: [PATCH 1/3] docs: add Agent Router research note --- .../2026-09-17-agent-router-research-note.md | 140 ++++++++++++++++++ ...09-17-agent-router-research-note-design.md | 49 ++++++ research/agent-router.md | 109 ++++++++++++++ 3 files changed, 298 insertions(+) create mode 100644 docs/superpowers/plans/2026-09-17-agent-router-research-note.md create mode 100644 docs/superpowers/specs/2026-09-17-agent-router-research-note-design.md create mode 100644 research/agent-router.md diff --git a/docs/superpowers/plans/2026-09-17-agent-router-research-note.md b/docs/superpowers/plans/2026-09-17-agent-router-research-note.md new file mode 100644 index 0000000..298eb63 --- /dev/null +++ b/docs/superpowers/plans/2026-09-17-agent-router-research-note.md @@ -0,0 +1,140 @@ +# Agent Router Research Note Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Add and publish a sourced architecture-first research note on Agent Router and its relevance to Cloud Foundry. + +**Architecture:** Create one flat Markdown note under `research/`, following the repository template. The note will separate upstream facts from Cloud Foundry analysis and will use Agent Router's control-plane/data-plane architecture as its organizing model. + +**Tech Stack:** Markdown, YAML frontmatter, repository note validator, Python tests, GitHub CLI. + +--- + +### Task 1: Create the research note + +**Files:** +- Create: `research/agent-router.md` + +- [ ] **Step 1: Add valid frontmatter** + +Use `title: Agent Router: Envoy-Based Control Plane for AI Traffic`, author `Ruben Koster (@rkoster)`, date `2026-09-17`, tags covering `routing`, `observability-governance`, `ecosystem-survey`, and `agent-runtime`, `cf_areas: []`, and `status: draft`. Add ratings from 0-100 with one-sentence explanations and link the upstream repository, Apache license, README, concepts, system architecture, control plane, data plane, resources, getting started, and CLI documentation. + +- [ ] **Step 2: Write the Summary section** + +Explain that Agent Router, formerly Envoy AI Gateway, is an Apache-2.0 open-source control plane for AI and agent traffic. State that it exposes an OpenAI-compatible interface while centralizing provider routing, credentials, quotas, failover, and usage attribution, and that its production architecture is Kubernetes- and Envoy-based with a standalone CLI option. + +- [ ] **Step 3: Write the Key findings section** + +Cover these concrete findings as sourced bullets: + +1. The control plane watches `AIGatewayRoute`, `AIServiceBackend`, and `BackendSecurityPolicy` resources, creates or manages standard Envoy Gateway resources, and fine-tunes xDS through the Envoy Gateway extension server. +2. The data plane is Envoy Proxy plus the AI Gateway External Processor and Rate Limit Service; the processor selects providers, transforms request and response formats, manages upstream authentication, tracks tokens, and supports streaming and non-streaming responses. +3. `AIGatewayRoute` defines the unified client-facing API and routing, `AIServiceBackend` represents a provider or service endpoint, and `BackendSecurityPolicy` supplies API-key or AWS credential behavior. +4. The gateway supports hosted providers, self-hosted inference, and MCP servers behind a consistent interface, including a two-tier pattern for centralized entry routing and self-hosted model clusters. +5. `aigw run` can run locally without Kubernetes or Docker, while production deployment uses Kubernetes and Envoy Gateway; the configuration model is intended to carry between these modes. +6. The architecture is an AI-aware gateway and policy layer, not an agent workflow engine or durable execution runtime; it does not replace application-level orchestration, memory, or task recovery. + +- [ ] **Step 4: Write the CF relevance section** + +Assess the architecture against Cloud Foundry without claiming existing integration. Explain that CF could expose Agent Router as a shared platform service or a dedicated gateway app, with service bindings carrying endpoint and client credentials while platform operators retain provider credentials. Map Envoy's edge routing and policy role to CF routing and platform-managed ingress, but note that token-aware rate limiting and model-aware transformations are beyond ordinary HTTP routing. Discuss operational trade-offs: the standalone binary is easier to run as a CF app, while the controller, CRDs, admission webhooks, xDS, and sidecar injection assume Kubernetes and would need a replacement control plane or an external Kubernetes installation. Mention that shared gateway observability could provide cost and usage attribution, but the platform would need explicit log, metric, trace, and tenant isolation choices. + +- [ ] **Step 5: Write the Open questions section** + +Ask whether CF should host one shared AI gateway, offer a brokered/bound gateway service, or leave routing to application teams; whether provider credentials and quotas should be platform-managed; whether token-aware limits and cost attribution belong in the platform; whether Kubernetes should remain a dependency for the full Agent Router architecture; and which interoperability boundary CF should standardize on for OpenAI-compatible APIs, MCP, and future agent protocols. + +### Task 2: Validate the note + +**Files:** +- Test: `tests/test_workflows.py` +- Test: `.github/scripts/validate_notes.py` + +- [ ] **Step 1: Run the note validator** + +Run: + +```bash +python .github/scripts/validate_notes.py +``` + +Expected: successful validation with no template placeholders, valid YAML frontmatter, a lowercase kebab-case filename, and all four required sections. + +- [ ] **Step 2: Run the test suite** + +Run: + +```bash +pytest -q +``` + +Expected: all tests pass, including tests that parse and validate research-note metadata. + +- [ ] **Step 3: Inspect the diff** + +Run: + +```bash +git diff --check +git status --short +git diff -- research/agent-router.md +``` + +Expected: no whitespace errors; only the intended research note and approved design/plan documents are reviewed for staging, while unrelated untracked environment artifacts remain untouched. + +### Task 3: Commit the note and create the PR + +**Files:** +- Modify: `research/agent-router.md` +- Include: `docs/superpowers/specs/2026-09-17-agent-router-research-note-design.md` +- Include: `docs/superpowers/plans/2026-09-17-agent-router-research-note.md` + +- [ ] **Step 1: Create a feature branch** + +Run: + +```bash +git switch -c research/agent-router +``` + +Expected: the new branch is based on the current `main` branch and contains no unrelated staged files. + +- [ ] **Step 2: Stage only intended files** + +Run: + +```bash +git add research/agent-router.md docs/superpowers/specs/2026-09-17-agent-router-research-note-design.md docs/superpowers/plans/2026-09-17-agent-router-research-note.md +git diff --cached --check +git status --short +``` + +Expected: only the three listed files are staged. + +- [ ] **Step 3: Commit the research note** + +Run: + +```bash +git commit -m "docs: add Agent Router research note" +``` + +Expected: one commit containing the research note and its approved design artifacts. + +- [ ] **Step 4: Push and open the PR** + +Run: + +```bash +git push -u origin research/agent-router +``` + +Before submitting, replace the PR template comments with a concise description and check every checklist item. The PR should target `main`, contain only the note and approved planning artifacts, and link the upstream Agent Router sources through the note itself. + +- [ ] **Step 5: Verify the created PR** + +Run: + +```bash +gh pr view --json number,url,title,baseRefName,headRefName,state,statusCheckRollup +``` + +Expected: an open PR from `research/agent-router` into `main`, with CI checks reported and the final URL recorded for the user. diff --git a/docs/superpowers/specs/2026-09-17-agent-router-research-note-design.md b/docs/superpowers/specs/2026-09-17-agent-router-research-note-design.md new file mode 100644 index 0000000..76d42c9 --- /dev/null +++ b/docs/superpowers/specs/2026-09-17-agent-router-research-note-design.md @@ -0,0 +1,49 @@ +# Agent Router Research Note Design + +## Goal + +Add a sourced research note on Agent Router, formerly Envoy AI Gateway, to help the +Cloud Foundry agent-runtime research group understand its architecture and relevance. + +## Scope + +The note will describe documented behavior from the Agent Router repository and +documentation, then clearly separate Cloud Foundry analysis from upstream facts. It will +cover: + +- The OpenAI-compatible API and Agent Router's role as an AI traffic control plane. +- The Kubernetes control plane: Agent Router controller, Envoy Gateway controller, + Kubernetes API, CRDs, and xDS configuration. +- The data plane: Envoy Proxy, AI Gateway External Processor, provider adapters, response + normalization, credentials, and token-based rate limiting. +- The relationship between `AIGatewayRoute`, `AIServiceBackend`, and + `BackendSecurityPolicy`. +- Standalone `aigw run` and Kubernetes deployment modes. +- Cloud Foundry relevance, including a shared AI gateway service, service bindings and + secret management, routing and policy enforcement, observability, and the implications + of Agent Router's Kubernetes-specific control plane. + +## Structure + +Create `research/agent-router.md` using the repository template and required sections: + +1. Summary +2. Key findings +3. CF relevance +4. Open questions + +The frontmatter will include the author, current date, relevant tags, `status: draft`, +primary upstream sources, and provisional ratings with concise justifications. + +## Sources and evidence + +Use the Agent Router GitHub repository README, Apache 2.0 license, concepts documentation, +system architecture, control-plane, data-plane, resources, getting-started, and CLI pages. +Claims about Cloud Foundry will be framed as analysis or open questions rather than +presented as Agent Router capabilities. + +## Validation + +Run the repository's note validation and test suite. Confirm that the new file has valid +frontmatter, a kebab-case filename, all required sections, linked sources, no template +placeholders, and only the intended tracked changes are committed. diff --git a/research/agent-router.md b/research/agent-router.md new file mode 100644 index 0000000..3d3e11d --- /dev/null +++ b/research/agent-router.md @@ -0,0 +1,109 @@ +--- +title: "Agent Router: Envoy-Based Control Plane for AI Traffic" +author: Ruben Koster (@rkoster) +date: 2026-09-17 +tags: [routing, observability-governance, agent-runtime, ecosystem-survey] +cf_areas: [capi, diego, loggregator] +status: draft +ratings: + platform-impact: + value: 78 + note: "A shared AI gateway could centralize provider access, quotas, credentials, and usage attribution for many CF applications." + maturity: + value: 76 + note: "The project is an active Apache-2.0 Envoy-based system with a Kubernetes deployment model and a standalone CLI, but the CLI is documented as experimental." + novelty: + value: 58 + note: "The distinctive contribution is applying Envoy Gateway and an external processor to model-aware routing and token accounting rather than introducing a new agent workflow model." + actionability: + value: 72 + note: "The standalone router can be evaluated quickly, while adapting the Kubernetes control plane to CF would require a deliberate platform integration design." +sources: + - https://github.com/theagentrouter/agent-router + - https://github.com/theagentrouter/agent-router/blob/main/LICENSE + - https://theagentrouter.ai/docs/concepts/ + - https://theagentrouter.ai/docs/concepts/architecture/system-architecture + - https://theagentrouter.ai/docs/concepts/architecture/control-plane + - https://theagentrouter.ai/docs/concepts/architecture/data-plane + - https://theagentrouter.ai/docs/concepts/resources + - https://theagentrouter.ai/docs/getting-started/ + - https://theagentrouter.ai/docs/cli/ +--- + +## Summary + +Agent Router, formerly Envoy AI Gateway, is an Apache-2.0 open-source control plane for AI +and agent traffic. It gives applications one OpenAI-compatible interface while centralizing +provider routing, credentials, quotas, failover, and usage attribution. Its production +architecture is Kubernetes- and Envoy-based, with a standalone `aigw run` mode for local or +dependency-light evaluation. + +## Key findings + +- **The architecture separates control and data planes.** The Agent Router controller watches + AI Gateway custom resources through the Kubernetes API, creates or manages Envoy Gateway + resources, and fine-tunes xDS through the Envoy Gateway extension-server protocol. Envoy + Gateway then applies the resulting configuration to the Envoy Proxy data plane. +- **The data plane is AI-aware rather than only an HTTP reverse proxy.** Envoy Proxy works + with the AI Gateway External Processor and a Rate Limit Service. The external processor + selects a provider from request paths, headers, and model names; transforms request and + response formats; applies upstream authentication; and tracks token usage for streaming and + non-streaming responses. The Rate Limit Service enforces budgets based on token consumption. +- **Three custom resources define the policy model.** `AIGatewayRoute` describes the unified + client-facing API, routing rules, transformations, and possible cost tracking. + `AIServiceBackend` represents a provider or other AI service endpoint. A + `BackendSecurityPolicy` supplies backend authentication behavior, including API-key and AWS + credential authentication. +- **The gateway fronts more than hosted model APIs.** The project presents one consistent + interface for hosted providers, self-hosted inference, and MCP servers. Its documented + two-tier pattern uses a centralized Tier One Gateway for authentication, top-level routing, + and global rate limits, with a Tier Two Gateway providing finer-grained access to a + self-hosted model-serving cluster. +- **There are two operational paths.** `aigw run` starts an OpenAI-compatible router locally + without Docker or Kubernetes and can front providers, self-hosted models, and MCP servers. + The production path installs Agent Router with Envoy Gateway in Kubernetes. This makes the + local mode useful for experimentation, but the full controller, CRD, admission-webhook, + xDS, and sidecar architecture remains Kubernetes-specific. +- **Agent Router is a gateway and policy layer, not an agent runtime.** It normalizes access + to model and tool endpoints and can enforce traffic policy, but it does not replace an + application's agent orchestration, memory, durable execution, or task-recovery logic. + +## CF relevance + +Agent Router suggests a useful shared-service boundary for Cloud Foundry: applications could +bind to one platform-operated AI gateway rather than each carrying provider-specific SDKs, +credentials, retry policy, and quota logic. A CF service binding could provide the gateway +endpoint and client credentials to an application, while provider credentials remain owned by +the platform operator. This would make provider rotation and centralized usage attribution +possible without distributing vendor keys across application spaces. + +The standalone binary appears easier to operate as a CF application than the complete +Kubernetes installation. CF routing could expose the gateway endpoint, and Diego could +supervise the router process. However, ordinary HTTP routing is not equivalent to Agent +Router's model-aware processing: token accounting, provider-specific transformations, +failover, and token-based budgets require the external processor or equivalent gateway logic. +Those policies would also need clear tenant and space boundaries. + +The production architecture exposes a more substantial integration question. Agent Router's +controller depends on Kubernetes APIs, CRDs, admission webhooks, Envoy Gateway, xDS, and +sidecar insertion. Running that unchanged would require a separate Kubernetes control plane, +not just a CF deployment. CF could instead provide a smaller adapter that translates a +platform service definition into Agent Router configuration, or adopt the architectural +pattern independently using CF routing, service bindings, and platform-managed policy +services. In either case, Loggregator-compatible metrics and logs, request-level cost +attribution, trace correlation, and credential isolation would need explicit design rather +than being assumed from the Envoy deployment. + +## Open questions + +- Should Cloud Foundry offer one shared AI gateway, a brokered service that teams provision, + or only primitives that let application teams run their own gateway? +- Which provider credentials, model allowlists, quotas, and failover policies should be + platform-managed, and which should remain application-owned? +- Should token-aware rate limiting and cost attribution be a CF platform capability, a + gateway service capability, or an external service bound to applications? +- Is Kubernetes an acceptable dependency for the full Agent Router architecture, or would a CF + integration need a native control plane that reconciles gateway policy without CRDs and + admission webhooks? +- What interoperability boundary should CF standardize on: an OpenAI-compatible API, MCP + gatewaying, A2A or another agent protocol, or a combination of these interfaces? From b330c959fbcd3a895a20ec45f15d62973feedb85 Mon Sep 17 00:00:00 2001 From: rkoster Date: Sat, 19 Sep 2026 16:29:36 +0200 Subject: [PATCH 2/3] docs: update generated research map --- generated/research-map.html | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/generated/research-map.html b/generated/research-map.html index bbe5f21..8a40d3b 100644 --- a/generated/research-map.html +++ b/generated/research-map.html @@ -23,9 +23,9 @@

Focus use cases

Attested Workload Authority and Mediated Tool AccessExchange platform-attested workload identity for scoped authority while credentials and outbound tool access remain mediated by the platform.Strategic decision: Decide whether CF should become the portable trust and policy layer between agent workloads and the tools they invoke.
Gap, experiments, and evidence
Current CF gap
CF issues workload identity certificates but does not exchange them for scoped tool authority, keep third-party credentials out of workloads, mediate off-platform access, or record delegation-aware audit events.
Candidate POC
Exchange a Diego instance identity certificate for a short-lived scoped token, invoke one allowed tool through a credential proxy and egress mediator, deny another, and emit attributable audit events.
Candidate RFC scope
Define workload token exchange, authority and delegation claims, credential brokering, outbound mediation and policy enforcement, audit events, revocation, and integration boundaries for UAA, routing, and service brokers.

Core

Supporting

-
Gap, experiments, and evidence
Current CF gap
CF can stage apps and run ephemeral tasks but cannot cheaply compose a reusable environment with per-session workspace state, select stronger isolation, constrain session networking, or resume the session lifecycle.
Candidate POC
Start two isolated sessions from one content-addressed staged environment, attach separate mutable workspaces, apply per-session egress policy, stop one session, and resume it on fresh compute.
Candidate RFC scope
Define environment and workspace references, session identity and lifecycle, isolation classes, network policy, workspace persistence and cleanup, scheduling, quotas, and compatibility with existing CF staging and task APIs.

Core

Supporting

ResearchIdea

Platform Impact x Maturity

Emerging < Maturity > EstablishedLocal concern < Platform Impact > Platform-wide concern
Unplaced notes (0)
  • All notes are placed.
+
Gap, experiments, and evidence
Current CF gap
CF can stage apps and run ephemeral tasks but cannot cheaply compose a reusable environment with per-session workspace state, select stronger isolation, constrain session networking, or resume the session lifecycle.
Candidate POC
Start two isolated sessions from one content-addressed staged environment, attach separate mutable workspaces, apply per-session egress policy, stop one session, and resume it on fresh compute.
Candidate RFC scope
Define environment and workspace references, session identity and lifecycle, isolation classes, network policy, workspace persistence and cleanup, scheduling, quotas, and compatibility with existing CF staging and task APIs.

Core

Supporting

ResearchIdea

Platform Impact x Maturity

Emerging < Maturity > EstablishedLocal concern < Platform Impact > Platform-wide concern
Unplaced notes (0)
  • All notes are placed.
 -