diff --git a/AGENTS.md b/AGENTS.md index 556717cd..8516748c 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -60,7 +60,7 @@ fail-closed and only pauses when the agent actually proposed a reviewed output. │ │ ├── mod.rs # Module entry point and Compiler trait │ │ ├── common.rs # Shared helpers across targets │ │ ├── ado_bundle.rs # Registry of ado-script bundles and their compile-time env contracts: Bundle enum (path + auth), apply_bundle_auth() (single chokepoint projecting SYSTEM_ACCESSTOKEN into every REST-calling bundle step), token_source_for() (System.AccessToken vs SC_WRITE_TOKEN selection), is_redundant_ado_mirror() (identifies auto-injected ADO predefined var re-projections) -│ │ ├── agentic_pipeline.rs # Canonical Setup → Agent → Detection → (ManualReview?) → SafeOutputs(+SafeOutputs_Reviewed?) → Teardown → Conclusion shape (Conclusion emitted when configured; shared by every target); BuiltPipelineContext, build_pipeline_context, build_canonical_jobs, per-job builders incl. build_manual_review_job + SafeOutputsVariant split, fold_agent_conditions, agent_job_variables_hoist +│ │ ├── agentic_pipeline.rs # Canonical Setup → Agent → Detection → (ManualReview?) → Custom_* → SafeOutputs(+SafeOutputs_Reviewed?) → Teardown → Conclusion shape (Conclusion emitted when configured; shared by every target); BuiltPipelineContext, build_pipeline_context, build_canonical_jobs, per-job builders incl. build_manual_review_job + custom safe-output jobs + SafeOutputsVariant split, fold_agent_conditions, agent_job_variables_hoist │ │ ├── standalone.rs # Standalone pipeline compiler │ │ ├── standalone_ir.rs # Standalone target typed-IR builder │ │ ├── onees.rs # 1ES Pipeline Template compiler @@ -74,6 +74,8 @@ fail-closed and only pauses when the agent actually proposed a reviewed output. │ │ ├── filter_ir.rs # Filter expression IR: Fact/Predicate types, lowering, validation, codegen │ │ ├── pr_filters.rs # PR trigger filter generation (native ADO + gate steps) │ │ ├── path_layout_check.rs # Warning-only checkout-aware path validation: $(Build.SourcesDirectory)/ refs in steps, runtime-import targets, deprecated directory markers in the body +│ │ ├── custom_tools.rs # Compile-time custom safe-output MCP schema generation for safe-outputs.scripts/jobs (closed scalar input schemas, custom-tools JSON) +│ │ ├── imports/ # Reusable component imports: mod.rs resolver + SHA-keyed .ado-aw/imports cache, schema.rs import-schema/with validation + substitution, alias.rs generated repo aliases + template diagnostics, merge.rs consumer-wins front-matter/body merge │ │ ├── extensions/ # CompilerExtension trait and infrastructure extensions │ │ │ ├── mod.rs # Trait, Extension enum, collect_extensions(), re-exports │ │ │ ├── ado_aw_marker.rs # Always-on metadata marker extension (emits # ado-aw-metadata JSON) @@ -118,10 +120,11 @@ fail-closed and only pauses when the agent actually proposed a reviewed output. │ │ ├── emit.rs # Thin `lower() + serde_yaml::to_string()` wrapper │ │ └── summary.rs # Public, serializable PipelineSummary / GraphSummary for agent-facing tooling (see docs/ir.md Public JSON summary) │ ├── init.rs # Repository initialization for AI-first authoring: scaffolds a dispatcher agent (.github/agents/ado-aw.agent.md) AND skill (.github/skills/ado-aw/SKILL.md); `--agency` plugin scaffold embeds agency/plugins/ado-aw/ via include_str! -│ ├── execute.rs # Stage 3 safe output execution +│ ├── execute.rs # Stage 3 safe output execution, including custom safe-output modes: scripts-style `--custom-config` native dispatch and jobs-style `--custom-phase pre|post` wrapper/result validation │ ├── fuzzy_schedule.rs # Fuzzy schedule parsing │ ├── logging.rs # File-based logging infrastructure │ ├── mcp.rs # SafeOutputs stdio MCP server +│ ├── mcp_custom_tools.rs # Dynamic SafeOutputs MCP tool registration from compiler-generated custom-tools JSON │ ├── mcp_author/ # Author-facing read-only MCP server for local IDE/Copilot Chat integrations │ │ ├── mod.rs # Tool router + handlers for inspect/graph/deps/outputs/whatif/lint/catalog/trace/audit │ │ └── tests.rs # MCP-author integration / contract tests @@ -306,6 +309,10 @@ index to jump to the right page. - [`docs/front-matter.md`](docs/front-matter.md) — full agent file format (markdown body + YAML front matter grammar) with every supported field. +- [`docs/imports.md`](docs/imports.md) — reusable local and SHA-pinned + cross-repository markdown components: `imports:`, `import-schema:`, committed + `.ado-aw/imports/` cache, merge semantics, and custom safe-output component + examples. - [`docs/runtime-imports.md`](docs/runtime-imports.md) — runtime prompt import markers, path resolution, and `inlined-imports:` behavior. - [`docs/schedule-syntax.md`](docs/schedule-syntax.md) — fuzzy schedule time @@ -330,7 +337,8 @@ index to jump to the right page. configured via the `execution-context:` front-matter block. - [`docs/safe-outputs.md`](docs/safe-outputs.md) — full reference for every safe-output tool agents can use to propose actions (PRs, work items, wiki - pages, comments, etc.) plus their per-agent configuration. + pages, comments, etc.), custom `safe-outputs.scripts` / `safe-outputs.jobs` + components, and per-agent configuration. - [`docs/safe-output-permissions.md`](docs/safe-output-permissions.md) — diagnosis and fix reference for Stage 3 401/403 failures: the default build identity (PCBS vs project-scoped Build Service), @@ -371,8 +379,9 @@ index to jump to the right page. - [`docs/mcpg.md`](docs/mcpg.md) — MCP Gateway architecture and pipeline integration. - [`docs/network.md`](docs/network.md) — AWF network isolation, default - allowed domains, ecosystem identifiers, blocking, and ADO `permissions:` - service-connection model. + allowed domains, ecosystem identifiers, blocking, repository-resource + `endpoint:` service connections, and ADO `permissions:` service-connection + model. - [`docs/extending.md`](docs/extending.md) — adding new CLI commands, compile targets, front-matter fields, typed IR extensions, safe-output tools, first-class tools, and runtimes; the `CompilerExtension` trait. diff --git a/Cargo.lock b/Cargo.lock index fd31a8b0..860a5ffd 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -23,6 +23,7 @@ dependencies = [ "glob-match", "indexmap", "inquire", + "libc", "log", "percent-encoding", "rand", @@ -38,6 +39,7 @@ dependencies = [ "tempfile", "tokio", "url", + "windows-sys 0.61.2", "wiremock", "zip", ] diff --git a/Cargo.toml b/Cargo.toml index f35b91ee..331246d7 100644 --- a/Cargo.toml +++ b/Cargo.toml @@ -38,3 +38,15 @@ zip = { version = "8.6.0", default-features = false, features = ["deflate"] } [dev-dependencies] reqwest = { version = "0.12", features = ["blocking"] } wiremock = "0.6" + +[target."cfg(unix)".dependencies] +libc = "0.2" + +[target."cfg(windows)".dependencies] +windows-sys = { version = "0.61", features = [ + "Win32_Foundation", + "Win32_Security", + "Win32_System_Diagnostics_ToolHelp", + "Win32_System_JobObjects", + "Win32_System_Threading", +] } diff --git a/docs/front-matter.md b/docs/front-matter.md index ee7bb27c..73f2d936 100644 --- a/docs/front-matter.md +++ b/docs/front-matter.md @@ -39,8 +39,16 @@ repos: # compact repository declarations (replaces rep - my-org/my-repo # shorthand: alias="my-repo", type=git, ref=refs/heads/main, checkout=true - reponame=my-org/another-repo # shorthand with explicit alias - name: my-org/templates # object form for full control + type: github # external repo resource type; default is git ref: refs/heads/release/2.x checkout: false # declared as resource only, not checked out by the agent + endpoint: github-templates # required for type: github/githubenterprise/bitbucket +imports: # reusable markdown components; see docs/imports.md + - ./components/local-guidance.md + - uses: octo/shared/components/notify.md@0123456789abcdef0123456789abcdef01234567 + endpoint: github-shared-components + with: # non-secret import-schema inputs + channel: service-alerts tools: # optional tool configuration bash: ["cat", "ls", "grep"] # explicit bash allow-list; when omitted, all bash tools are allowed (unrestricted) edit: true # enable file editing tool (default: true) @@ -247,6 +255,61 @@ runtime — write it as clear, structured natural-language instructions. > report on pipeline failures and surfaces diagnostic signals. See > [`docs/conclusion.md`](conclusion.md). +## Reusable Imports (`imports:` / `import-schema:`) + +`imports:` lets a workflow reuse local or SHA-pinned cross-repository markdown +components. Each imported file is parsed as regular ado-aw markdown with YAML +front matter; the compiler validates optional `import-schema:` inputs, applies +`{{ inputs. }}` substitutions (a compile-time `{{ ... }}` +replacement — not the ADO `${{ ... }}` template delimiter), then merges the +imported front matter and body into the consumer workflow. Imported **body** +content is inlined into the agent prompt at compile time (ahead of the +consumer's own body); see [`imports.md`](imports.md) for the full reference. + +```yaml +imports: + - ./components/local-policy.md + - octo/shared-agents/components/notify.md@0123456789abcdef0123456789abcdef01234567 + - uses: octo/shared-agents/components/deploy.md@89abcdef0123456789abcdef0123456789abcdef + endpoint: github-shared-components + with: + environment: prod + region: westus3 +``` + +Object-form fields: + +| Field | Description | +|-------|-------------| +| `uses` | Import spec. Local paths are relative to the importing `.md` file. Cross-repo specs use `owner/repo/path@<40-character-sha>`; branches/tags are rejected. | +| `with` | Non-secret values validated against the imported file's `import-schema:`. | +| `endpoint` | Azure DevOps service connection for GitHub/GitHub Enterprise runtime repository resources created for imported component sources. | + +Import specs may also include `#Section` to import only a markdown heading +section, and a trailing `?` to make the import optional. + +Reusable components declare compile-time inputs with `import-schema:`: + +```yaml +import-schema: + channel: + type: string + required: true + severity: + type: choice + options: [info, warning, critical] + default: info + labels: + type: array + items: + type: string +``` + +Supported types are `string`, `number`, `boolean`, `choice`, `array`, and +`object` (object properties are currently one level deep). See +[`imports.md`](imports.md) for the full syntax, cache layout, merge semantics, +limitations, and custom safe-output component examples. + ## Inline step validation (`setup` / `steps` / `post-steps` / `teardown`) Inline steps are authored as raw Azure DevOps YAML and are emitted into the @@ -407,6 +470,7 @@ Object fields: | `alias` | last segment of `name` | Repository alias (maps to ADO `repository:`) | | `type` | `git` | ADO repository resource type | | `ref` | `refs/heads/main` | Branch or tag reference | +| `endpoint` | *(none)* | Azure DevOps service connection. Required for `type: github`, `githubenterprise`, or `bitbucket`; not needed for same-org Azure Repos `git`. | | `checkout` | `true` | Whether the agent job clones this repo | | `fetch-depth` | *(ADO default)* | Shallow-clone depth for this repo's checkout (ADO `fetchDepth`). `0` = full history | | `fetch-tags` | *(ADO default)* | Whether to fetch git tags during checkout (ADO `fetchTags`) | diff --git a/docs/imports.md b/docs/imports.md new file mode 100644 index 00000000..9fad7e17 --- /dev/null +++ b/docs/imports.md @@ -0,0 +1,333 @@ +# Reusable Imports + +_Part of the [ado-aw documentation](../AGENTS.md)._ + +`imports:` lets one agent file reuse another markdown component, including +cross-repository components pinned to an immutable commit SHA. The imported file +is the same markdown + YAML-front-matter format as a normal workflow: its front +matter is merged into the consumer, and its markdown body is prepended to the +consumer's prompt. + +This is separate from [`{{#runtime-import}}`](runtime-imports.md). Runtime +imports expand prompt snippets on the pipeline runner; `imports:` is resolved by +`ado-aw compile`, validates optional `import-schema:` inputs, and can contribute +front-matter configuration such as tools, runtimes, MCP servers, and custom safe +outputs. + +## Syntax + +`imports:` is a flat list. Each entry is either a bare spec string or an object +with `uses`, optional `with`, and optional `endpoint`: + +```yaml +imports: + # Same-org Azure Repos (the primary, default source — no endpoint): + - myproject/shared-agents/components/notify.md@0123456789abcdef0123456789abcdef01234567 + # Local import: + - ./components/local-guidance.md + # GitHub.com, via an ADO service connection (bare-string endpoint shorthand): + - uses: octo/shared-agents/components/deploy.md@89abcdef0123456789abcdef0123456789abcdef + endpoint: github-shared-components + with: + environment: prod + region: westus3 +``` + +### Import specs + +| Form | Meaning | +|------|---------| +| `path/to/component.md` or `./path/to/component.md` | Local import, resolved relative to the importing `.md` file. Exactly one leading `./` is accepted. | +| `owner/repo/path/to/component.md@` | Cross-repository import. `` must be a full 40-character commit SHA; branches and tags are rejected. | +| `...#Section` | Import only a `# Section` or `## Section` from the markdown body. | +| `...?` | Optional import. If the target is missing, it is skipped. | + +The optional marker is trailing, so a sectioned optional import looks like +`owner/repo/component.md@0123...cdef#Usage?`. + +For a cross-repository spec `owner/repo/path@`, `owner` maps to the Azure +DevOps **project** (or GitHub owner) and `repo` to the repository name. + +### `endpoint:` — source type and service connection + +`endpoint:` selects the **source type** of a cross-repository import and names +the Azure DevOps service connection the generated runtime repository resource +authenticates with. It drives **both** the compile-time manifest fetch **and** +the runtime checkout, so the two can never disagree. + +- **Absent** → **same-organization Azure Repos** (the primary, default case for + this ADO-native compiler). Fetched at compile time via the ADO Git Items API + and checked out at runtime with `System.AccessToken` (`type: git`, no + endpoint). +- **Bare string** (`endpoint: my-connection`) → shorthand for a **GitHub.com** + service connection. +- **Object form** with an explicit `type`: + + | `type` | Extra fields | Source | Runtime `type` | + |--------|--------------|--------|----------------| + | `github` (default) | — | GitHub.com | `github` | + | `ghe` | `host:` (GHES server host, e.g. `ghe.acme.com`) | GitHub Enterprise | `githubenterprise` | + | `azure-repos` | `org:` (target collection URL, e.g. `https://dev.azure.com/otherorg`) | **Cross-organization** Azure Repos | `git` | + +```yaml +imports: + # Cross-org Azure Repos: + - uses: otherproject/otherrepo/component.md@0123456789abcdef0123456789abcdef01234567 + endpoint: + name: other-org-repos-connection + type: azure-repos + org: https://dev.azure.com/otherorg + # GitHub Enterprise: + - uses: octo/components/deploy.md@89abcdef0123456789abcdef0123456789abcdef + endpoint: + name: ghe-connection + type: ghe + host: ghe.acme.com +``` + +See [Repository resource endpoints](network.md#repository-resource-endpoints). + +Cross-organization `org:` values are credential-bearing compile-time +destinations and are therefore restricted to HTTPS Azure DevOps collection +URLs: `https://dev.azure.com/` or the legacy +`https://.visualstudio.com[/DefaultCollection]` form. Arbitrary hosts, +non-default ports, embedded credentials, query strings, and project-level paths are +rejected before authentication is resolved. + +## Cross-repository resolution and cache + +Cross-repository imports are immutable: the spec must include a full commit SHA. +At compile time, ado-aw fetches the imported **markdown manifest** and stores a +SHA-keyed copy under: + +```text +.ado-aw/imports/////.md +``` + +The component's directory structure under `/` mirrors its path in the +source repository (e.g. `components/deploy.md`). + +The cache is intended to be committed. ado-aw also creates +`.ado-aw/imports/.gitattributes` marking cached imports as generated and using +`merge=ours`, mirroring gh-aw's committed import-cache model. + +Only the markdown manifest is cached. Script files and other executor source are +not vendored into `.ado-aw/imports/`; script-bearing custom safe-output +components are checked out in their dedicated executor job and verified at the +pinned SHA before their code runs. + +### Compile-time manifest fetch + +The manifest fetcher is selected from the import's `endpoint` type so that the +compile-time fetch always matches the runtime checkout source: + +- **Azure Repos** (endpoint-less same-org, or `type: azure-repos` cross-org) — + fetched via the ADO Git Items API. Credentials are resolved + **non-interactively** in this precedence: `SYSTEM_ACCESSTOKEN` → + `AZURE_DEVOPS_EXT_PAT` → `az account get-access-token`. The consumer + organization is taken from `AZURE_DEVOPS_ORG_URL` / `SYSTEM_COLLECTIONURI` or + the repo's Azure Repos git remote; cross-org imports use the endpoint's + `org:`. +- **GitHub / GitHub Enterprise** (`type: github` / `type: ghe`) — fetched via + `gh api` using the compiler host's GitHub auth. For GHES, `host:` is the + server hostname consumed by `GH_HOST` (not the `api.` API hostname). + +Routing is **fail-closed**: an endpoint-less (Azure-Repos-intended) import never +silently falls back to GitHub, and a GitHub-typed import is never served by the +Azure Repos fetcher. + +Current MVP notes: + +- Nested/transitive import resolution is not expanded yet; the current resolver + processes the workflow's top-level `imports:` list. +- A workflow may declare at most 20 imports, and each resolved manifest is + capped at 256 KiB. + +## `import-schema:` and `with:` + +A reusable component can declare non-secret inputs with `import-schema:`. +Consumers pass values through `with:`. Values are validated at compile time, +defaults are applied, and placeholders of the form +`{{ inputs. }}` are substituted throughout the imported front +matter and body before merge. + +> **Delimiter.** Import inputs use the compile-time `{{ ... }}` delimiter (the +> same family as `{{ workspace }}`), **not** the Azure DevOps template-expression +> delimiter `${{ ... }}`. The substituted output is embedded directly into the +> pipeline YAML and agent prompt, where ADO template-processes any `${{ ... }}` +> it finds — so reusing that delimiter would be a footgun. A `{{` immediately +> preceded by `$` is treated as an ADO `${{ ... }}` expression and left +> untouched. Any `{{ inputs. }}` still present after +> substitution (an input the consumer did not supply and the schema did not +> default) is a **compile-time error**. An unclosed `{{ inputs.` marker is +> also rejected with an error identifying the missing `}}`, and an empty +> `{{ inputs. }}` marker is rejected as malformed. + +Supported schema types are `string`, `number`, `boolean`, `choice`, `array`, and +`object`. `choice` uses an `options:` list. `array` uses an `items:` schema. +`object` uses `properties:`; object properties are currently one level deep. +Unknown `with:` keys, missing required inputs, and values of the wrong type are +compile-time errors. + +```markdown +--- +import-schema: + channel: + type: string + required: true + description: Notification channel name. + severity: + type: choice + options: [info, warning, critical] + default: info + labels: + type: array + items: + type: string + delivery: + type: object + properties: + retries: + type: number + default: 2 +safe-outputs: + scripts: + notify-team: + description: Send a team notification. + max: 3 + run: node tools/notify.js + inputs: + title: + type: string + required: true + max-length: 120 + body: + type: string + required: true + env: + NOTIFY_TOKEN: TEAM_NOTIFY_TOKEN +--- +When notifying the team, use channel `{{ inputs.channel }}` and +severity `{{ inputs.severity }}`. +``` + +Consumer: + +```yaml +imports: + - uses: octo/shared-agents/components/notify.md@0123456789abcdef0123456789abcdef01234567 + endpoint: github-shared-components + with: + channel: service-alerts + severity: warning + labels: [agentic, automated] + delivery: + retries: 3 +safe-outputs: + notify-team: + require-approval: true +``` + +`with:` values are not secrets. Pass secrets through custom safe-output `env:` +bindings, which name Azure DevOps variables and are scoped to the privileged +custom executor job. + +## Merge semantics + +Imports are merged in declaration order, then the consumer workflow is overlaid +on top. Precedence is: + +```text +consumer workflow > later import > earlier import +``` + +- Scalar/singleton fields use the highest-precedence explicit value. +- Mapping collections (`tools`, `mcp-servers`, `safe-outputs`, `runtimes`, + `env`) merge additively by key. Duplicate keys from two different imports are + hard errors. +- Sequence fields (`parameters`, `repos`, `variable-groups`) concatenate in + import order, then consumer entries. +- The consumer may configure an imported safe-output tool, for example by adding + `require-approval`, but may not replace executor-defining fields such as + `steps`, `env`, `inputs`, `run`, or `entrypoint`. +- Runtime component provenance (`component-source`, `component-sha`, + `manifest-digest`, `component-repo-type`, and `component-endpoint`) is + compiler-owned. Component-authored values are stripped and consumers cannot + override the compiler-resolved source, commit, repository type, or service + connection. +- Imported markdown bodies are concatenated in declaration order, followed by + the consumer body. Imported bodies are **inlined into the agent prompt at + compile time** (their `{{ inputs.* }}` placeholders are already + substituted); in the default `inlined-imports: false` mode the consumer's own + body is delivered ahead-of-time as a `{{#runtime-import}}` marker so it can + still be edited without recompiling, while imported bodies — which can only be + substituted at compile time — precede it inline. +- `import-schema:` and `imports:` are consumed by the merge and do not appear in + the merged workflow. + +## Example: shared custom safe-output job + +Shared component manifest: + +```markdown +--- +import-schema: + service: + type: string + required: true +safe-outputs: + jobs: + create-service-ticket: + description: Create an incident ticket in the service desk. + max: 2 + inputs: + title: + type: string + required: true + max-length: 160 + priority: + type: choice + options: [low, normal, high] + required: true + env: + SERVICE_DESK_TOKEN: SERVICE_DESK_TOKEN + SERVICE_NAME: SERVICE_NAME + steps: + - bash: | + set -euo pipefail + : > "$ADO_AW_SAFE_OUTPUT_RESULTS" + while IFS= read -r proposal; do + proposal_id=$(echo "$proposal" | jq -r '.proposal_id') + title=$(echo "$proposal" | jq -r '.title') + # Call your service-desk client here, honoring staged mode. + jq -cn \ + --arg proposal_id "$proposal_id" \ + --arg title "$title" \ + '{schema_version:1, proposal_id:$proposal_id, status:"success", message:("created ticket for " + $title)}' \ + >> "$ADO_AW_SAFE_OUTPUT_RESULTS" + done < "$ADO_AW_SAFE_OUTPUT_PROPOSALS" + displayName: Create service ticket +--- +Use `create-service-ticket` only when a durable service-desk record is needed +for `{{ inputs.service }}`. +``` + +Consumer workflow: + +```yaml +imports: + - uses: contoso/ado-aw-components/service-ticket.md@89abcdef0123456789abcdef0123456789abcdef + endpoint: github-components + with: + service: payments-api +safe-outputs: + create-service-ticket: + require-approval: + approvers: ["[Contoso]\\SRE Leads"] + instructions: Confirm the ticket title and priority before approving. +``` + +The imported tool appears to the agent as a typed SafeOutputs MCP tool. Agent +proposals still flow through Detection and optional manual review before the +isolated `Custom_create_service_ticket` executor job receives the secret env +bindings and performs the side effect. diff --git a/docs/network.md b/docs/network.md index 05af2261..826e1d69 100644 --- a/docs/network.md +++ b/docs/network.md @@ -190,6 +190,54 @@ network: - "*.github.com" # Remove wildcard variant too ``` +## Repository resource endpoints + +Azure DevOps repository resources backed by an external service connection must +declare `endpoint:`. ado-aw validates this at compile time so the generated YAML +does not fail later in Azure Pipelines. + +| Repository `type` | `endpoint:` required? | Notes | +|-------------------|-----------------------|-------| +| `git` | No | Same-organization Azure Repos checkout using the build's OAuth token. | +| `github` | Yes | Azure DevOps GitHub service connection. | +| `githubenterprise` | Yes | Azure DevOps GitHub Enterprise service connection. | +| `bitbucket` | Yes | Azure DevOps Bitbucket service connection. | + +```yaml +repos: + - name: octo/shared-components + alias: shared-components + type: github + endpoint: github-shared-components + ref: refs/heads/main + checkout: false +``` + +The same rule applies to repository resources generated for reusable +[`imports:`](imports.md): object-form imports can specify the ADO service +connection with `endpoint:`: + +```yaml +imports: + - uses: octo/shared-components/components/notify.md@0123456789abcdef0123456789abcdef01234567 + endpoint: github-shared-components +``` + +`endpoint:` is an Azure DevOps runtime authorization setting. It is not passed +to the agent, Detection, or the compile-time manifest fetcher. + +### Template targets (`target: job` / `target: stage`) + +`target: job` and `target: stage` emit Azure DevOps templates, and templates +cannot declare top-level `resources.repositories`. For imports that require a +generated repository resource, ado-aw emits a diagnostic naming the generated +alias (for example `import_octo_shared_components_`). The parent pipeline +that includes the template must declare and authorize that repository resource +with the same alias and endpoint. + +Standalone and 1ES targets own their top-level resources, so ado-aw emits the +repository resource directly. + ## Permissions (ADO Access Tokens) ADO does not support fine-grained permissions — there are two access levels: @@ -246,8 +294,9 @@ agents. Set `permissions.write` only when you need: ### Security Model -- **`permissions.read`**: Mints a read-only ADO-scoped token given to the - agent inside the AWF sandbox (Stage 1). The agent can query ADO APIs but +- **`permissions.read`**: Mints a read-only ADO-scoped token and maps it as + `AZURE_DEVOPS_EXT_PAT: $(SC_READ_TOKEN)` on the Agent step. AWF carries that + secret into the Stage 1 sandbox, where the agent can query ADO APIs but cannot write. - **`permissions.write` (optional)**: Mints a write-capable ADO-scoped token used **only** by the executor in Stage 3 (`SafeOutputs` job). Overrides diff --git a/docs/safe-outputs.md b/docs/safe-outputs.md index 1f2589e1..0a0d0ca3 100644 --- a/docs/safe-outputs.md +++ b/docs/safe-outputs.md @@ -175,6 +175,169 @@ ARM-minted token, e.g. for cross-org writes or named-identity attribution. See [`docs/network.md`](network.md) and [`docs/ir.md`](ir.md) for the typed SafeOutputs job wiring. +## Custom safe outputs (scripts & jobs) + +Reusable components imported with [`imports:`](imports.md) can add custom +agent-callable safe-output tools under `safe-outputs.scripts.` or +`safe-outputs.jobs.`. Both mechanisms preserve the standard trust +boundary: + +```text +Agent MCP proposal → Detection → optional ManualReview → Custom_ executor job +``` + +The Agent and Detection jobs see only the generated closed MCP schema and the +proposal artifact. Secret variables and service credentials named by the custom +tool are bound only in the dedicated `Custom_` executor job. + +### Shared fields + +Both custom forms accept: + +| Field | Description | +|-------|-------------| +| `description` | Human-readable MCP tool description shown to the agent. | +| `inputs` | Agent-facing typed input schema. MVP types are scalar only: `string`, `number`, `boolean`, and `choice`. | +| `max` | Maximum proposals of this custom tool to attempt in one run. Omitted tools default to 3. | +| `env` | Mapping of environment variable names to Azure DevOps variable names. Values are treated as secret bindings and scoped to the custom executor job. | + +The generated MCP schema is closed (`additionalProperties: false`), so extra +agent-supplied keys are rejected. String inputs default to `maxLength: 4000`; +set `max-length` per field to choose a smaller bound (up to the compiler's hard +limit of 8000). `choice` inputs require an `options:` list. + +Custom tool names must be safe tool identifiers, cannot collide with built-in +safe-output names, and cannot use the reserved structural names `scripts` or +`jobs`. `require-approval` works for custom tools exactly like it does for +built-ins: configure it under the tool's top-level name in the consumer +workflow, or set a section-level default. + +```yaml +safe-outputs: + notify-team: + require-approval: true +``` + +### `safe-outputs.scripts.` + +Scripts-style tools declare an entrypoint command with `run:` (or +`entrypoint:`). The compiled job writes a compiler-generated custom config and +runs one `ado-aw execute --custom-config ` step. The executor owns the +proposal loop: for each selected proposal it invokes the entrypoint, passes the +proposal JSON on stdin and in `AW_PROPOSAL`, enforces the budget/staged mode, +sanitizes the result, and appends the final execution record. + +`timeout-minutes` bounds each entrypoint invocation. It defaults to 10 minutes +and must be between 1 and 60. On expiry the executor terminates the child +process tree, waits for the direct child to exit, and records that proposal as +failed rather than waiting for the outer Azure Pipelines job timeout. This field +is scripts-only; jobs-style tools should set timeouts on their authored ADO +steps. Stdout and stderr capture are each capped at 1 MiB; exceeding either +limit terminates the process tree and fails the proposal. + +The script must print exactly one JSON line to stdout: + +```json +{"status":"success","message":"sent notification","data":{"id":"123"}} +``` + +`status` may be `success`/`succeeded`, `failure`/`failed`, or `staged`. + +```yaml +safe-outputs: + scripts: + notify-team: + description: Send a structured team notification. + max: 3 + timeout-minutes: 10 + run: node tools/notify.js + inputs: + title: + type: string + required: true + max-length: 120 + severity: + type: choice + options: [info, warning, critical] + env: + NOTIFY_TOKEN: TEAM_NOTIFY_TOKEN +``` + +### `safe-outputs.jobs.` + +Jobs-style tools declare arbitrary Azure DevOps `steps:`. The compiler wraps +those steps: + +1. `ado-aw execute --custom-phase pre --tool --max ` filters + proposals for this tool, applies the compiler-resolved `max`, assigns stable + `proposal_id` values, writes + `ADO_AW_SAFE_OUTPUT_PROPOSALS`, and sets `ADO_AW_SAFE_OUTPUTS_STAGED=true` + during dry runs. +2. The component's authored ADO steps run. They read + `$ADO_AW_SAFE_OUTPUT_PROPOSALS` and append one result record per attempted + proposal to `$ADO_AW_SAFE_OUTPUT_RESULTS`. +3. `ado-aw execute --custom-phase post --tool --max ` validates + and enriches the component results with compiler-owned provenance, sanitizes + output, and emits the standard executed-safe-output artifact. Missing, + malformed, or duplicate result records fail closed. + +The resolved budget is embedded in both wrapper commands at compile time, +including values contributed by imported components; Stage 3 does not reparse +the unmerged consumer source to infer it. + +Every `Custom_` job runs its finalization wrapper with `always()` and +publishes a unique `custom_safe_output__` artifact with the +execution records. This preserves failure/provenance evidence even when an +authored component step or scripts entrypoint fails. + +Each jobs-style result line must be JSON with `schema_version: 1`, +`proposal_id`, `status`, `message`, and optional `data`: + +```json +{"schema_version":1,"proposal_id":"deploy-thing-0","status":"success","message":"deployment queued"} +``` + +```yaml +safe-outputs: + jobs: + create-service-ticket: + description: Create an incident ticket in the service desk. + max: 2 + inputs: + title: + type: string + required: true + max-length: 160 + priority: + type: choice + options: [low, normal, high] + required: true + env: + SERVICE_DESK_TOKEN: SERVICE_DESK_TOKEN + steps: + - bash: | + set -euo pipefail + : > "$ADO_AW_SAFE_OUTPUT_RESULTS" + while IFS= read -r proposal; do + proposal_id=$(echo "$proposal" | jq -r '.proposal_id') + title=$(echo "$proposal" | jq -r '.title') + jq -cn --arg proposal_id "$proposal_id" --arg title "$title" \ + '{schema_version:1, proposal_id:$proposal_id, status:"success", message:("created ticket for " + $title)}' \ + >> "$ADO_AW_SAFE_OUTPUT_RESULTS" + done < "$ADO_AW_SAFE_OUTPUT_PROPOSALS" + displayName: Create service ticket +``` + +For cross-repository components that ship executor files, the custom executor +job checks out the component repository resource, detaches to the pinned commit +SHA, verifies `HEAD`, then runs the script or wrapped steps. Prompt-only or +configuration-only imports are fully inlined at compile time and do not need a +runtime checkout. Same-organization Azure Repos fetches the pin with +`$(System.AccessToken)`; GitHub, GitHub Enterprise, and cross-organization +Azure Repos reuse their repository-resource service-connection credentials. +Persisted Git auth headers are removed immediately after the pin is fetched and +verified, before component code runs. + ## Available Safe Output Tools ### comment-on-work-item diff --git a/docs/tools.md b/docs/tools.md index 56bfadae..176fabe1 100644 --- a/docs/tools.md +++ b/docs/tools.md @@ -146,7 +146,9 @@ general use. Two paths are supported: 1. **`az devops *` subcommands** (work items, repos, pipelines, etc.) are automatically authenticated via `AZURE_DEVOPS_EXT_PAT`, which the compiler populates inside AWF whenever `permissions.read` is - configured. No extra steps needed. + configured. The Agent step maps only + `AZURE_DEVOPS_EXT_PAT: $(SC_READ_TOKEN)`; neither `SC_WRITE_TOKEN` nor + `$(System.AccessToken)` is exposed to Stage 1. No extra steps needed. 2. **General `az` / ARM / Graph commands** (`az account get-access-token`, `az resource ...`, `az ad ...`, etc.) require their own authentication. The agent has no inherited cloud identity; you diff --git a/scripts/ado-script/.gitignore b/scripts/ado-script/.gitignore index f50cd05e..905b1521 100644 --- a/scripts/ado-script/.gitignore +++ b/scripts/ado-script/.gitignore @@ -15,6 +15,7 @@ approval-summary.js conclusion.js github-app-token.js prepare-pr-base.js +checkout-component.js schema *.tsbuildinfo test-bin diff --git a/scripts/ado-script/package.json b/scripts/ado-script/package.json index e95f33e9..5309b345 100644 --- a/scripts/ado-script/package.json +++ b/scripts/ado-script/package.json @@ -7,8 +7,8 @@ "node": ">=20.0.0" }, "scripts": { - "build": "npm run codegen && npm run clean && npm run build:gate && npm run build:import && npm run build:exec-context-pr && npm run build:exec-context-pr-synth && npm run build:exec-context-manual && npm run build:exec-context-pipeline && npm run build:exec-context-ci-push && npm run build:exec-context-workitem && npm run build:exec-context-schedule && npm run build:exec-context-pr-checks && npm run build:exec-context-repo && npm run build:conclusion && npm run build:approval-summary && npm run build:github-app-token && npm run build:prepare-pr-base", - "clean": "node -e \"const fs=require('node:fs'); fs.rmSync('.ado-build',{recursive:true,force:true}); for (const n of ['gate','import','exec-context-pr','exec-context-pr-synth','exec-context-manual','exec-context-pipeline','exec-context-ci-push','exec-context-workitem','exec-context-schedule','exec-context-pr-checks','exec-context-repo','conclusion','approval-summary','github-app-token','prepare-pr-base']) fs.rmSync(n+'.js',{force:true});\"", + "build": "npm run codegen && npm run clean && npm run build:gate && npm run build:import && npm run build:exec-context-pr && npm run build:exec-context-pr-synth && npm run build:exec-context-manual && npm run build:exec-context-pipeline && npm run build:exec-context-ci-push && npm run build:exec-context-workitem && npm run build:exec-context-schedule && npm run build:exec-context-pr-checks && npm run build:exec-context-repo && npm run build:conclusion && npm run build:approval-summary && npm run build:github-app-token && npm run build:prepare-pr-base && npm run build:checkout-component", + "clean": "node -e \"const fs=require('node:fs'); fs.rmSync('.ado-build',{recursive:true,force:true}); for (const n of ['gate','import','exec-context-pr','exec-context-pr-synth','exec-context-manual','exec-context-pipeline','exec-context-ci-push','exec-context-workitem','exec-context-schedule','exec-context-pr-checks','exec-context-repo','conclusion','approval-summary','github-app-token','prepare-pr-base','checkout-component']) fs.rmSync(n+'.js',{force:true});\"", "build:gate": "ncc build src/gate/index.ts -o .ado-build/gate -m -t && node -e \"const fs=require('node:fs'); fs.copyFileSync('.ado-build/gate/index.js','gate.js'); fs.rmSync('.ado-build/gate',{recursive:true,force:true});\"", "build:import": "ncc build src/import/index.ts -o .ado-build/import -m -t && node -e \"const fs=require('node:fs'); fs.copyFileSync('.ado-build/import/index.js','import.js'); fs.rmSync('.ado-build/import',{recursive:true,force:true});\"", "build:exec-context-pr": "ncc build src/exec-context-pr/index.ts -o .ado-build/exec-context-pr -m -t && node -e \"const fs=require('node:fs'); fs.copyFileSync('.ado-build/exec-context-pr/index.js','exec-context-pr.js'); fs.rmSync('.ado-build/exec-context-pr',{recursive:true,force:true});\"", @@ -24,13 +24,14 @@ "build:approval-summary": "ncc build src/approval-summary/index.ts -o .ado-build/approval-summary -m -t && node -e \"const fs=require('node:fs'); fs.copyFileSync('.ado-build/approval-summary/index.js','approval-summary.js'); fs.rmSync('.ado-build/approval-summary',{recursive:true,force:true});\"", "build:github-app-token": "ncc build src/github-app-token/index.ts -o .ado-build/github-app-token -m -t && node -e \"const fs=require('node:fs'); fs.copyFileSync('.ado-build/github-app-token/index.js','github-app-token.js'); fs.rmSync('.ado-build/github-app-token',{recursive:true,force:true});\"", "build:prepare-pr-base": "ncc build src/prepare-pr-base/index.ts -o .ado-build/prepare-pr-base -m -t && node -e \"const fs=require('node:fs'); fs.copyFileSync('.ado-build/prepare-pr-base/index.js','prepare-pr-base.js'); fs.rmSync('.ado-build/prepare-pr-base',{recursive:true,force:true});\"", + "build:checkout-component": "ncc build src/checkout-component/index.ts -o .ado-build/checkout-component -m -t && node -e \"const fs=require('node:fs'); fs.copyFileSync('.ado-build/checkout-component/index.js','checkout-component.js'); fs.rmSync('.ado-build/checkout-component',{recursive:true,force:true});\"", "build:executor-e2e": "ncc build src/executor-e2e/index.ts -o .ado-build/executor-e2e -m -t && node -e \"const fs=require('node:fs'); fs.mkdirSync('test-bin',{recursive:true}); fs.copyFileSync('.ado-build/executor-e2e/index.js','test-bin/executor-e2e.js'); fs.rmSync('.ado-build/executor-e2e',{recursive:true,force:true});\"", "build:trigger-e2e": "ncc build src/trigger-e2e/index.ts -o .ado-build/trigger-e2e -m -t && node -e \"const fs=require('node:fs'); fs.mkdirSync('test-bin',{recursive:true}); fs.copyFileSync('.ado-build/trigger-e2e/index.js','test-bin/trigger-e2e.js'); fs.rmSync('.ado-build/trigger-e2e',{recursive:true,force:true});\"", "build:compiler-smoke-e2e": "ncc build src/compiler-smoke-e2e/index.ts -o .ado-build/compiler-smoke-e2e -m -t && node -e \"const fs=require('node:fs'); fs.mkdirSync('test-bin',{recursive:true}); fs.copyFileSync('.ado-build/compiler-smoke-e2e/index.js','test-bin/compiler-smoke-e2e.js'); fs.rmSync('.ado-build/compiler-smoke-e2e',{recursive:true,force:true});\"", "build:check": "ls -lh gate.js && wc -c gate.js", "codegen": "node -e \"require('node:fs').mkdirSync('schema', { recursive: true })\" && cargo run --quiet --manifest-path ../../Cargo.toml -- export-gate-schema --output schema/gate-spec.schema.json && npx json2ts schema/gate-spec.schema.json -o src/shared/types.gen.ts --bannerComment \"// AUTO-GENERATED from Rust IR via cargo run -- export-gate-schema. Do not edit; run npm run codegen.\" && cargo run --quiet --manifest-path ../../Cargo.toml -- export-fact-catalog --output src/trigger-e2e/fact-catalog.gen.json", "test": "vitest run", - "test:smoke": "npm run build:gate && npm run build:import && npm run build:exec-context-pr && npm run build:exec-context-pr-synth && npm run build:exec-context-manual && npm run build:exec-context-pipeline && npm run build:exec-context-ci-push && npm run build:exec-context-workitem && npm run build:exec-context-schedule && npm run build:exec-context-pr-checks && npm run build:exec-context-repo && npm run build:conclusion && npm run build:approval-summary && npm run build:github-app-token && npm run build:prepare-pr-base && vitest run -c vitest.config.smoke.ts", + "test:smoke": "npm run build:gate && npm run build:import && npm run build:exec-context-pr && npm run build:exec-context-pr-synth && npm run build:exec-context-manual && npm run build:exec-context-pipeline && npm run build:exec-context-ci-push && npm run build:exec-context-workitem && npm run build:exec-context-schedule && npm run build:exec-context-pr-checks && npm run build:exec-context-repo && npm run build:conclusion && npm run build:approval-summary && npm run build:github-app-token && npm run build:prepare-pr-base && npm run build:checkout-component && vitest run -c vitest.config.smoke.ts", "lint": "echo TODO", "typecheck": "tsc --noEmit" }, diff --git a/scripts/ado-script/src/__tests__/bundle-coverage.test.ts b/scripts/ado-script/src/__tests__/bundle-coverage.test.ts index f0e31ebd..05320794 100644 --- a/scripts/ado-script/src/__tests__/bundle-coverage.test.ts +++ b/scripts/ado-script/src/__tests__/bundle-coverage.test.ts @@ -100,6 +100,30 @@ describe("ado-script bundle coverage", () => { ).toEqual([]); }); + it("clean removes every generated bundle", () => { + const { scripts } = readPackageJson(); + const clean = scripts.clean ?? ""; + const missing = listBundleDirs().filter( + (name) => !clean.includes(`'${name}'`) && !clean.includes(`"${name}"`), + ); + expect( + missing, + `src/ bundle dirs missing from the clean script: ${missing.join(", ")}`, + ).toEqual([]); + }); + + it("the smoke build chain rebuilds every bundle", () => { + const { scripts } = readPackageJson(); + const smoke = scripts["test:smoke"] ?? ""; + const missing = listBundleDirs().filter( + (name) => !smoke.includes(`build:${name}`), + ); + expect( + missing, + `src/ bundle dirs not referenced in the test:smoke build chain: ${missing.join(", ")}`, + ).toEqual([]); + }); + it("every bundle dir has an index.ts entry point", () => { const missing = listBundleDirs().filter((name) => { try { diff --git a/scripts/ado-script/src/checkout-component/__tests__/index.test.ts b/scripts/ado-script/src/checkout-component/__tests__/index.test.ts new file mode 100644 index 00000000..bcc656ad --- /dev/null +++ b/scripts/ado-script/src/checkout-component/__tests__/index.test.ts @@ -0,0 +1,224 @@ +import { describe, expect, it } from "vitest"; + +import type { GitResult } from "../../shared/git.js"; +import { main, parseArgs, type GitRunners } from "../index.js"; + +const SHA = "a".repeat(40); +const OTHER = "b".repeat(40); + +/** + * Build a scriptable `GitRunners` pair. `present` decides, per invocation + * count, whether `git cat-file -e ^{commit}` reports the object present + * (status 0) — so a test can model "absent until the Nth fetch". `checkout` + * and `head` control the detach + rev-parse outcomes. + */ +function makeRunners(opts: { + /** Return codes for successive `cat-file -e` probes (default: always 1/absent). */ + catFile?: number[]; + fetchStatus?: number; + checkoutStatus?: number; + head?: string; +}): { runners: GitRunners; calls: string[][] } { + const calls: string[][] = []; + let catIdx = 0; + const runGit: GitRunners["runGit"] = (args) => { + calls.push(args); + let result: GitResult = { stdout: "", stderr: "", status: 0 }; + if (args[0] === "cat-file") { + const seq = opts.catFile ?? [1]; + const status = (catIdx < seq.length ? seq[catIdx] : seq[seq.length - 1]) ?? 1; + catIdx++; + result = { stdout: "", stderr: "", status }; + } else if (args[0] === "fetch") { + result = { stdout: "", stderr: "", status: opts.fetchStatus ?? 0 }; + } else if (args[0] === "checkout") { + result = { stdout: "", stderr: "", status: opts.checkoutStatus ?? 0 }; + } + return result; + }; + const gitOk: GitRunners["gitOk"] = (args) => { + if (args[0] === "rev-parse") return opts.head ?? SHA; + return null; + }; + return { runners: { runGit, gitOk }, calls }; +} + +describe("parseArgs", () => { + it("parses --dir and --sha", () => { + expect(parseArgs(["--dir", "/src/comp", "--sha", SHA])).toEqual({ + dir: "/src/comp", + sha: SHA, + }); + }); + + it("defaults to empty strings when flags are absent", () => { + expect(parseArgs([])).toEqual({ dir: "", sha: "" }); + }); +}); + +describe("main", () => { + const okEnv = { SYSTEM_ACCESSTOKEN: "tok" } as NodeJS.ProcessEnv; + const noopChdir = () => {}; + + it("rejects a non-40-char sha (fail closed)", () => { + const { runners, calls } = makeRunners({}); + expect(main({ dir: "/c", sha: "main" }, okEnv, runners, noopChdir)).toBe(1); + // Must not touch git for an invalid pin. + expect(calls).toEqual([]); + }); + + it("requires --dir", () => { + const { runners } = makeRunners({}); + expect(main({ dir: "", sha: SHA }, okEnv, runners, noopChdir)).toBe(1); + }); + + it("fails closed when the component dir cannot be entered", () => { + const { runners } = makeRunners({}); + const throwingChdir = () => { + throw new Error("no such dir"); + }; + expect(main({ dir: "/missing", sha: SHA }, okEnv, runners, throwingChdir)).toBe(1); + }); + + it("checks out and verifies when the sha is already present (no fetch)", () => { + const { runners, calls } = makeRunners({ catFile: [0], head: SHA }); + expect(main({ dir: "/c", sha: SHA }, okEnv, runners, noopChdir)).toBe(0); + expect(calls.some((c) => c[0] === "fetch")).toBe(false); + expect(calls).toContainEqual(["checkout", "--detach", SHA]); + }); + + it("does a direct by-sha fetch when the sha is initially absent", () => { + // absent, then present after the direct fetch. + const { runners, calls } = makeRunners({ catFile: [1, 0], head: SHA }); + expect(main({ dir: "/c", sha: SHA }, okEnv, runners, noopChdir)).toBe(0); + expect(calls).toContainEqual(["fetch", "--no-tags", "--depth", "1", "origin", SHA]); + }); + + it("falls back to progressive deepening when by-sha fetch does not yield the object", () => { + // absent, absent after direct fetch, present after first deepen. + const { runners, calls } = makeRunners({ catFile: [1, 1, 0], head: SHA }); + expect(main({ dir: "/c", sha: SHA }, okEnv, runners, noopChdir)).toBe(0); + expect(calls).toContainEqual(["fetch", "--no-tags", "--depth=200", "origin"]); + }); + + it("fails closed when the sha can never be obtained", () => { + const { runners, calls } = makeRunners({ catFile: [1] }); // always absent + expect(main({ dir: "/c", sha: SHA }, okEnv, runners, noopChdir)).toBe(1); + // Never attempts the checkout of an unavailable object. + expect(calls.some((c) => c[0] === "checkout")).toBe(false); + }); + + it("fails closed when checkout fails", () => { + const { runners } = makeRunners({ catFile: [0], checkoutStatus: 1 }); + expect(main({ dir: "/c", sha: SHA }, okEnv, runners, noopChdir)).toBe(1); + }); + + it("fails closed when HEAD does not equal the pin after checkout", () => { + const { runners } = makeRunners({ catFile: [0], head: OTHER }); + expect(main({ dir: "/c", sha: SHA }, okEnv, runners, noopChdir)).toBe(1); + }); + + it("passes the bearer env to git fetch (never on argv)", () => { + const seen: Array | undefined> = []; + const runGit: GitRunners["runGit"] = (args, env) => { + if (args[0] === "fetch") seen.push(env); + // absent, then present after direct fetch. + const status = args[0] === "cat-file" ? (seen.length === 0 ? 1 : 0) : 0; + return { stdout: "", stderr: "", status }; + }; + const gitOk: GitRunners["gitOk"] = () => SHA; + main({ dir: "/c", sha: SHA }, okEnv, { runGit, gitOk }, noopChdir); + expect(seen.length).toBeGreaterThan(0); + // Bearer is delivered via GIT_CONFIG_* env, and the token is never in argv. + expect(seen[0]?.GIT_CONFIG_VALUE_0).toContain("bearer tok"); + }); + + it("uses persisted git credentials when no bearer is supplied", () => { + const seen: Array | undefined> = []; + const calls: string[][] = []; + const runGit: GitRunners["runGit"] = (args, env) => { + calls.push(args); + if (args[0] === "fetch") seen.push(env); + const status = args[0] === "cat-file" ? (seen.length === 0 ? 1 : 0) : 0; + return { stdout: "", stderr: "", status }; + }; + const gitOk: GitRunners["gitOk"] = () => SHA; + + expect(main({ dir: "/c", sha: SHA }, {}, { runGit, gitOk }, noopChdir)).toBe(0); + expect(seen).toEqual([{}]); + expect(calls).toContainEqual([ + "config", + "--local", + "--name-only", + "--get-regexp", + "^(http\\..*\\.extraheader|http(\\..*)?\\.proxy|credential\\..*)$", + ]); + }); + + it("fails closed when persisted credentials cannot be removed", () => { + let fetched = false; + const runGit: GitRunners["runGit"] = (args) => { + if (args[0] === "cat-file") { + return { stdout: "", stderr: "", status: fetched ? 0 : 1 }; + } + if (args[0] === "fetch") { + fetched = true; + return { stdout: "", stderr: "", status: 0 }; + } + if (args[0] === "config" && args.includes("--get-regexp")) { + return { + stdout: + "http.https://example.test.extraheader\nhttp.proxy\ncredential.helper\n", + stderr: "", + status: 0, + }; + } + if (args[0] === "config" && args.includes("--unset-all")) { + return { stdout: "", stderr: "denied", status: 1 }; + } + return { stdout: "", stderr: "", status: 0 }; + }; + const gitOk: GitRunners["gitOk"] = () => SHA; + + expect(main({ dir: "/c", sha: SHA }, {}, { runGit, gitOk }, noopChdir)).toBe(1); + }); + + it("removes persisted headers, proxies, and credential helpers", () => { + let fetched = false; + const removed: string[] = []; + const runGit: GitRunners["runGit"] = (args) => { + if (args[0] === "cat-file") { + return { stdout: "", stderr: "", status: fetched ? 0 : 1 }; + } + if (args[0] === "fetch") { + fetched = true; + return { stdout: "", stderr: "", status: 0 }; + } + if (args[0] === "config" && args.includes("--get-regexp")) { + return { + stdout: [ + "http.https://example.test.extraheader", + "http.proxy", + "http.https://example.test.proxy", + "credential.helper", + ].join("\n"), + stderr: "", + status: 0, + }; + } + if (args[0] === "config" && args.includes("--unset-all")) { + removed.push(args.at(-1) ?? ""); + } + return { stdout: "", stderr: "", status: 0 }; + }; + const gitOk: GitRunners["gitOk"] = () => SHA; + + expect(main({ dir: "/c", sha: SHA }, {}, { runGit, gitOk }, noopChdir)).toBe(0); + expect(removed).toEqual([ + "http.https://example.test.extraheader", + "http.proxy", + "http.https://example.test.proxy", + "credential.helper", + ]); + }); +}); diff --git a/scripts/ado-script/src/checkout-component/index.ts b/scripts/ado-script/src/checkout-component/index.ts new file mode 100644 index 00000000..3c5e2a28 --- /dev/null +++ b/scripts/ado-script/src/checkout-component/index.ts @@ -0,0 +1,230 @@ +/** + * checkout-component — make a SHA-pinned custom safe-output component available + * at the exact pinned commit on a shallow-default Azure DevOps agent pool. + * + * ## Why this exists + * + * A cross-repository custom safe-output component is checked out via an ADO + * repository resource, whose `ref` can only be a branch/tag — never a commit + * SHA (an ADO limitation). On a shallow-default pool the resource checkout + * pulls only the tip of `refs/heads/main` (`fetchDepth: 1`), so the pinned + * commit object is usually absent and a plain `git checkout --detach ` + * fails with `fatal: reference is not a tree`. That defeats the whole point of + * SHA-pinning: the pinned revision must actually run, reproducibly, regardless + * of where `main` has since moved. + * + * This bundle runs in the isolated custom safe-output job. Same-org Azure + * Repos receives `$(System.AccessToken)`; GitHub/GHE and cross-org Azure Repos + * reuse credentials persisted by the repository-resource checkout. It obtains + * the pinned commit object — first via a direct `git fetch origin `, then, + * if the server refuses a by-SHA fetch, by progressively deepening the checked + * out branch until the object is present — then checks it out detached and + * **verifies HEAD equals the pin, failing closed** on any mismatch or + * unrecoverable fetch. + * + * ## Trust boundary + * + * When present, the bearer (`SYSTEM_ACCESSTOKEN`) is passed to the spawned + * `git` child via `GIT_CONFIG_*` env vars (see `shared/git.ts::bearerEnv`) — + * never in argv and never written to `.git/config`. When absent, Git uses the + * checkout's persisted provider credentials, then removes every persisted + * `http.*.extraheader` before component code runs. The compiler-owned, + * non-secret `--dir` / `--sha` are argv flags (immune to ADO variable + * shadowing). + * + * ## Posture — FAIL CLOSED + * + * Unlike `prepare-pr-base` (which is a best-effort optimization and exits 0 on + * failure), this bundle is a **security control**: if the exact pinned commit + * cannot be obtained and verified, it exits non-zero so the custom job — and + * the pipeline — fails rather than running an unverified revision. + * + * Invocation: node checkout-component.js --dir --sha <40-hex> + * env: SYSTEM_ACCESSTOKEN (same-org Azure Repos only) + */ +import { + bearerEnv, + gitOk as defaultGitOk, + runGit as defaultRunGit, + type GitResult, +} from "../shared/git.js"; + +const SHA40_RE = /^[0-9a-f]{40}$/i; + +/** Injectable git runners (production uses the real ones; tests stub them). */ +export type GitRunners = { + runGit: (args: string[], env?: Record) => GitResult; + gitOk: (args: string[], env?: Record) => string | null; +}; + +const defaultRunners: GitRunners = { + runGit: defaultRunGit, + gitOk: defaultGitOk, +}; + +export interface CheckoutArgs { + /** The component checkout directory (as the compiler resolved it). */ + dir: string; + /** The full 40-char commit SHA the component is pinned to. */ + sha: string; +} + +/** Parse `--dir ` / `--sha <40-hex>` flags. */ +export function parseArgs(argv: string[]): CheckoutArgs { + let dir = ""; + let sha = ""; + for (let i = 0; i < argv.length; i++) { + if (argv[i] === "--dir") { + dir = argv[i + 1] ?? ""; + i++; + } else if (argv[i] === "--sha") { + sha = argv[i + 1] ?? ""; + i++; + } + } + return { dir, sha }; +} + +/** True when the pinned commit object is already present locally. */ +function shaPresent(sha: string, runners: GitRunners): boolean { + return runners.runGit(["cat-file", "-e", `${sha}^{commit}`]).status === 0; +} + +/** + * Obtain the pinned commit object in the current working directory. Tries a + * direct by-SHA fetch first, then progressively deepens the existing shallow + * history until the object appears. Returns `true` once the object is present. + */ +function ensureShaFetched( + sha: string, + fetchEnv: Record, + runners: GitRunners, +): boolean { + if (shaPresent(sha, runners)) { + return true; + } + + // 1. Direct by-SHA fetch (GitHub / GHE / Azure Repos support this). + runners.runGit(["fetch", "--no-tags", "--depth", "1", "origin", sha], fetchEnv); + if (shaPresent(sha, runners)) { + return true; + } + + // 2. Fall back to progressively deepening the checked-out history until the + // pinned object is reachable (servers that refuse by-SHA fetch). + const depths = ["--depth=200", "--depth=500", "--depth=2000", "--unshallow"]; + for (const depthArg of depths) { + runners.runGit(["fetch", "--no-tags", depthArg, "origin"], fetchEnv); + if (shaPresent(sha, runners)) { + return true; + } + } + + return false; +} + +function clearPersistedAuth(runners: GitRunners): boolean { + const listed = runners.runGit([ + "config", + "--local", + "--name-only", + "--get-regexp", + "^(http\\..*\\.extraheader|http(\\..*)?\\.proxy|credential\\..*)$", + ]); + // `git config --get-regexp` exits 1 when there are no matching keys (for + // example a public repository), which is already a clean state. + if (listed.status !== 0 && listed.status !== 1) { + return false; + } + const keys = [ + ...new Set( + listed.stdout + .split(/\r?\n/) + .map((key) => key.trim()) + .filter((key) => key.length > 0), + ), + ]; + return keys.every( + (key) => + runners.runGit(["config", "--local", "--unset-all", key]).status === 0, + ); +} + +export function main( + args: CheckoutArgs, + env: NodeJS.ProcessEnv = process.env, + runners: GitRunners = defaultRunners, + chdir: (dir: string) => void = process.chdir.bind(process), +): number { + const { dir, sha } = args; + + if (!SHA40_RE.test(sha)) { + process.stderr.write( + `[checkout-component] error: '--sha' must be a full 40-character commit SHA, got '${sha}'.\n`, + ); + return 1; + } + if (dir.length === 0) { + process.stderr.write("[checkout-component] error: '--dir' is required.\n"); + return 1; + } + + try { + chdir(dir); + } catch (err) { + // The pipeline just checked this dir out; a missing/unusable dir is a real + // infra error, not a benign skip — fail closed. + process.stderr.write( + `[checkout-component] error: could not enter component dir '${dir}': ${(err as Error).message}.\n`, + ); + return 1; + } + + const fetchEnv = bearerEnv(env.SYSTEM_ACCESSTOKEN); + const usingPersistedAuth = fetchEnv.GIT_CONFIG_COUNT === undefined; + + if (!ensureShaFetched(sha, fetchEnv, runners)) { + process.stderr.write( + `[checkout-component] error: could not obtain pinned commit ${sha} in '${dir}' ` + + "after a direct fetch and progressive deepening.\n", + ); + return 1; + } + + const checkout = runners.runGit(["checkout", "--detach", sha]); + if (checkout.status !== 0) { + process.stderr.write( + `[checkout-component] error: 'git checkout --detach ${sha}' failed in '${dir}': ${checkout.stderr.trim()}.\n`, + ); + return 1; + } + + const actual = runners.gitOk(["rev-parse", "HEAD"]) ?? ""; + if (actual.toLowerCase() !== sha.toLowerCase()) { + process.stderr.write( + `[checkout-component] error: checkout resolved '${actual}', expected pinned '${sha}' in '${dir}'.\n`, + ); + return 1; + } + if (usingPersistedAuth && !clearPersistedAuth(runners)) { + process.stderr.write( + `[checkout-component] error: could not remove persisted repository credentials in '${dir}'.\n`, + ); + return 1; + } + + process.stdout.write( + `[checkout-component] verified component checkout at ${actual} in '${dir}'.\n`, + ); + return 0; +} + +// CLI entry guard: only run when invoked directly (not when imported by tests). +if ( + typeof process !== "undefined" && + process.argv[1] && + /checkout-component(\/index)?\.js$/.test(process.argv[1]) +) { + const args = parseArgs(process.argv.slice(2)); + process.exit(main(args)); +} diff --git a/scripts/ado-script/src/compiler-smoke-e2e/__tests__/ado-rest.test.ts b/scripts/ado-script/src/compiler-smoke-e2e/__tests__/ado-rest.test.ts index 565e1acb..67237885 100644 --- a/scripts/ado-script/src/compiler-smoke-e2e/__tests__/ado-rest.test.ts +++ b/scripts/ado-script/src/compiler-smoke-e2e/__tests__/ado-rest.test.ts @@ -104,6 +104,64 @@ describe("AdoRest.getBuild / cancelBuild", () => { expect(build.result).toBe("succeeded"); }); + describe("AdoRest.getBuildTags", () => { + it("accepts the documented string-array response", async () => { + const fetchImpl = vi.fn(async () => + jsonResponse(200, ["ado-aw-custom-script-10", "ado-aw-custom-job-10"]), + ); + const rest = makeRest(fetchImpl as unknown as typeof fetch); + await expect(rest.getBuildTags(10)).resolves.toEqual([ + "ado-aw-custom-script-10", + "ado-aw-custom-job-10", + ]); + }); + + it("accepts an ADO collection wrapper defensively", async () => { + const fetchImpl = vi.fn(async () => + jsonResponse(200, { count: 1, value: ["tag-one"] }), + ); + const rest = makeRest(fetchImpl as unknown as typeof fetch); + await expect(rest.getBuildTags(10)).resolves.toEqual(["tag-one"]); + }); + + it("retries malformed responses and reports the final error", async () => { + const fetchImpl = vi.fn(async () => jsonResponse(200, { value: [42] })); + const rest = makeRest(fetchImpl as unknown as typeof fetch); + await expect( + rest.getBuildTags(10, { retries: 2, retryDelayMs: 1 }), + ).rejects.toThrow(/not a string array/); + expect(fetchImpl).toHaveBeenCalledTimes(2); + }); + + it("retries a successful response until every required tag is visible", async () => { + let calls = 0; + const fetchImpl = vi.fn(async () => { + calls++; + return jsonResponse( + 200, + calls === 1 + ? ["ado-aw-custom-script-10"] + : ["ado-aw-custom-script-10", "ado-aw-custom-job-10"], + ); + }); + const rest = makeRest(fetchImpl as unknown as typeof fetch); + await expect( + rest.getBuildTags(10, { + retries: 2, + retryDelayMs: 1, + required: [ + "ado-aw-custom-script-10", + "ado-aw-custom-job-10", + ], + }), + ).resolves.toEqual([ + "ado-aw-custom-script-10", + "ado-aw-custom-job-10", + ]); + expect(fetchImpl).toHaveBeenCalledTimes(2); + }); + }); + it("cancelBuild PATCHes the build to status=cancelling", async () => { let method: string | undefined; let body: unknown; diff --git a/scripts/ado-script/src/compiler-smoke-e2e/__tests__/assertions.test.ts b/scripts/ado-script/src/compiler-smoke-e2e/__tests__/assertions.test.ts index 65ad1d3b..d026647c 100644 --- a/scripts/ado-script/src/compiler-smoke-e2e/__tests__/assertions.test.ts +++ b/scripts/ado-script/src/compiler-smoke-e2e/__tests__/assertions.test.ts @@ -1,6 +1,11 @@ import { describe, expect, it } from "vitest"; -import { assertNoForbiddenReleaseUrls, assertPipelineArtifactValues } from "../assertions.js"; +import { + assertAgentCommandPolicy, + assertAdoTokenIsolation, + assertNoForbiddenReleaseUrls, + assertPipelineArtifactValues, +} from "../assertions.js"; const EXPECTED = { project: "AgentPlayground", @@ -26,6 +31,108 @@ steps: `; } +function agentTokenYaml(opts: { + agentReadToken?: string; + agentExtraEnv?: string; + detectionExtraEnv?: string; +} = {}): string { + const agentReadToken = + opts.agentReadToken === undefined + ? "" + : `\n AZURE_DEVOPS_EXT_PAT: ${opts.agentReadToken}`; + return ` +jobs: + - job: Agent + steps: + - bash: echo agent + displayName: Run copilot (AWF network isolated) + env: + GITHUB_TOKEN: $(GITHUB_TOKEN)${agentReadToken}${opts.agentExtraEnv ?? ""} + - job: Detection + steps: + - bash: echo detection + displayName: Run threat analysis (AWF network isolated) + env: + GITHUB_TOKEN: $(GITHUB_TOKEN)${opts.detectionExtraEnv ?? ""} +`; +} + +describe("assertAdoTokenIsolation", () => { + it("accepts the read-scoped Agent mapping while keeping Detection isolated", () => { + expect(() => + assertAdoTokenIsolation(agentTokenYaml({ agentReadToken: "$(SC_READ_TOKEN)" }), "canary", true), + ).not.toThrow(); + }); + + it("rejects a missing Agent read-token mapping", () => { + expect(() => assertAdoTokenIsolation(agentTokenYaml(), "canary", true)).toThrow( + /read-token contract mismatch/, + ); + }); + + it("accepts a workflow that does not request read permissions", () => { + expect(() => assertAdoTokenIsolation(agentTokenYaml(), "custom-safe-output", false)).not.toThrow(); + }); + + it("rejects a write-scoped token on the Agent", () => { + expect(() => + assertAdoTokenIsolation( + agentTokenYaml({ + agentReadToken: "$(SC_READ_TOKEN)", + agentExtraEnv: "\n SC_WRITE_TOKEN: $(SC_WRITE_TOKEN)", + }), + "canary", + true, + ), + ).toThrow(/Agent must not receive SC_WRITE_TOKEN/); + }); + + it("rejects any ADO token on Detection", () => { + expect(() => + assertAdoTokenIsolation( + agentTokenYaml({ + agentReadToken: "$(SC_READ_TOKEN)", + detectionExtraEnv: "\n AZURE_DEVOPS_EXT_PAT: $(SC_READ_TOKEN)", + }), + "canary", + true, + ), + ).toThrow(/Detection must not receive AZURE_DEVOPS_EXT_PAT/); + }); +}); + +describe("assertAgentCommandPolicy", () => { + it("accepts a restricted Agent command", () => { + const yaml = agentTokenYaml().replace( + "echo agent", + 'copilot --allow-tool "shell(az:*)" --allow-tool "shell(head)"', + ); + expect(() => + assertAgentCommandPolicy( + yaml, + "azure-cli", + ["shell(az", "shell(head"], + ["--allow-all-tools", "--allow-all-paths"], + ), + ).not.toThrow(); + }); + + it("rejects unrestricted Agent tools", () => { + const yaml = agentTokenYaml().replace( + "echo agent", + "copilot --allow-all-tools --allow-all-paths", + ); + expect(() => + assertAgentCommandPolicy( + yaml, + "azure-cli", + ["shell(az"], + ["--allow-all-tools", "--allow-all-paths"], + ), + ).toThrow(/missing required snippet|forbidden snippet/); + }); +}); + describe("assertNoForbiddenReleaseUrls", () => { it("passes for YAML with no forbidden release URL", () => { expect(() => assertNoForbiddenReleaseUrls(specificRunYaml(), "canary")).not.toThrow(); diff --git a/scripts/ado-script/src/compiler-smoke-e2e/__tests__/component-fixture.test.ts b/scripts/ado-script/src/compiler-smoke-e2e/__tests__/component-fixture.test.ts new file mode 100644 index 00000000..e330a266 --- /dev/null +++ b/scripts/ado-script/src/compiler-smoke-e2e/__tests__/component-fixture.test.ts @@ -0,0 +1,98 @@ +import { spawn } from "node:child_process"; +import { createServer } from "node:http"; +import { dirname, resolve } from "node:path"; +import { fileURLToPath } from "node:url"; + +import { describe, expect, it } from "vitest"; + +const fixtureScript = resolve( + dirname(fileURLToPath(import.meta.url)), + "../../../../../tests/compiler-smoke-e2e/component-fixture/components/custom-build-tags/tag-build-script.js", +); + +async function runFixture( + orgUrl: string, + proposal: unknown, +): Promise<{ status: number | null; stdout: string; stderr: string }> { + const child = spawn(process.execPath, [fixtureScript], { + env: { + ...process.env, + SYSTEM_COLLECTIONURI: orgUrl, + SYSTEM_TEAMPROJECT: "AgentPlayground", + BUILD_BUILDID: "42", + SYSTEM_ACCESSTOKEN: "token", + }, + stdio: ["pipe", "pipe", "pipe"], + }); + child.stdin.end(JSON.stringify(proposal)); + + let stdout = ""; + let stderr = ""; + child.stdout.setEncoding("utf8").on("data", (chunk) => { + stdout += chunk; + }); + child.stderr.setEncoding("utf8").on("data", (chunk) => { + stderr += chunk; + }); + const status = await new Promise((resolveExit) => { + child.on("close", resolveExit); + }); + return { status, stdout, stderr }; +} + +describe("scripts-style build-tag component fixture", () => { + it("adds the deterministic current-build tag and emits one JSON result", async () => { + let method = ""; + let url = ""; + let authorization = ""; + const server = createServer((request, response) => { + method = request.method ?? ""; + url = request.url ?? ""; + authorization = request.headers.authorization ?? ""; + response.writeHead(200, { "content-type": "application/json" }); + response.end("[]"); + }); + await new Promise((resolveListen) => + server.listen(0, "127.0.0.1", resolveListen), + ); + + try { + const address = server.address(); + if (!address || typeof address === "string") { + throw new Error("test server did not expose a TCP address"); + } + const outcome = await runFixture( + `http://127.0.0.1:${address.port}/`, + { proof: "candidate-smoke" }, + ); + expect(outcome.status).toBe(0); + expect(outcome.stderr).toBe(""); + expect(outcome.stdout.trim().split("\n")).toHaveLength(1); + expect(JSON.parse(outcome.stdout)).toEqual({ + status: "success", + message: "added scripts-style build tag ado-aw-custom-script-42", + data: { tag: "ado-aw-custom-script-42" }, + }); + expect(method).toBe("PUT"); + expect(url).toBe( + "/AgentPlayground/_apis/build/builds/42/tags/ado-aw-custom-script-42?api-version=7.1", + ); + expect(authorization).toBe( + `Basic ${Buffer.from(":token", "utf8").toString("base64")}`, + ); + } finally { + await new Promise((resolveClose, rejectClose) => + server.close((error) => (error ? rejectClose(error) : resolveClose())), + ); + } + }); + + it("rejects an unexpected proof before making a request", async () => { + const outcome = await runFixture("http://127.0.0.1:1/", { + proof: "wrong", + }); + expect(outcome.status).toBe(1); + expect(outcome.stdout).toBe(""); + expect(outcome.stderr).toMatch(/proof must equal candidate-smoke/); + }); +}); diff --git a/scripts/ado-script/src/compiler-smoke-e2e/__tests__/config.test.ts b/scripts/ado-script/src/compiler-smoke-e2e/__tests__/config.test.ts index e701d631..777841ff 100644 --- a/scripts/ado-script/src/compiler-smoke-e2e/__tests__/config.test.ts +++ b/scripts/ado-script/src/compiler-smoke-e2e/__tests__/config.test.ts @@ -18,8 +18,8 @@ function baseEnv(overrides: Record = {}): NodeJS.Pro COMPILER_SMOKE_CANARY_DEFINITION_ID: "2601", COMPILER_SMOKE_AZURE_CLI_DEFINITION_ID: "2602", COMPILER_SMOKE_NOOP_TARGET_DEFINITION_ID: "2603", - COMPILER_SMOKE_JANITOR_DEFINITION_ID: "2604", - COMPILER_SMOKE_REPORTER_DEFINITION_ID: "2605", + COMPILER_SMOKE_REPORTER_DEFINITION_ID: "2604", + COMPILER_SMOKE_CUSTOM_SAFE_OUTPUT_DEFINITION_ID: "2605", ...overrides, }; } @@ -35,8 +35,8 @@ describe("loadConfig", () => { canary: 2601, "azure-cli": 2602, "noop-target": 2603, - janitor: 2604, - "smoke-failure-reporter": 2605, + "smoke-failure-reporter": 2604, + "custom-safe-output": 2605, }); expect(config.concurrency).toBe(5); expect(config.childTimeoutMs).toBe(7_200_000); @@ -59,8 +59,8 @@ describe("loadConfig", () => { "COMPILER_SMOKE_CANARY_DEFINITION_ID", "COMPILER_SMOKE_AZURE_CLI_DEFINITION_ID", "COMPILER_SMOKE_NOOP_TARGET_DEFINITION_ID", - "COMPILER_SMOKE_JANITOR_DEFINITION_ID", "COMPILER_SMOKE_REPORTER_DEFINITION_ID", + "COMPILER_SMOKE_CUSTOM_SAFE_OUTPUT_DEFINITION_ID", ]) { it(`rejects a missing ${name}`, () => { expect(() => loadConfig(baseEnv({ [name]: undefined }))).toThrow(); @@ -84,7 +84,7 @@ describe("loadConfig", () => { }); it("rejects a non-integer fixture definition id", () => { - expect(() => loadConfig(baseEnv({ COMPILER_SMOKE_JANITOR_DEFINITION_ID: "12.5" }))).toThrow( + expect(() => loadConfig(baseEnv({ COMPILER_SMOKE_REPORTER_DEFINITION_ID: "12.5" }))).toThrow( /positive integer/, ); }); @@ -105,7 +105,7 @@ describe("loadConfig", () => { baseEnv({ COMPILER_SMOKE_AZURE_CLI_DEFINITION_ID: "2601", COMPILER_SMOKE_NOOP_TARGET_DEFINITION_ID: "2604", - COMPILER_SMOKE_JANITOR_DEFINITION_ID: "2604", + COMPILER_SMOKE_REPORTER_DEFINITION_ID: "2604", }), ), ).toThrow(/canary/); diff --git a/scripts/ado-script/src/compiler-smoke-e2e/__tests__/fixtures.test.ts b/scripts/ado-script/src/compiler-smoke-e2e/__tests__/fixtures.test.ts index 1e1f84f9..bcd986c8 100644 --- a/scripts/ado-script/src/compiler-smoke-e2e/__tests__/fixtures.test.ts +++ b/scripts/ado-script/src/compiler-smoke-e2e/__tests__/fixtures.test.ts @@ -1,6 +1,15 @@ import { describe, expect, it } from "vitest"; -import { ALL_FIXTURES, allowedChangedPaths, fixturePaths, FIXTURE_DIR } from "../fixtures.js"; +import { + ALL_FIXTURES, + CANDIDATE_FIXTURE_DIR, + CUSTOM_COMPONENT_CACHE_PATH, + CUSTOM_COMPONENT_DIGEST_PATH, + IMPORT_CACHE_ATTRIBUTES_PATH, + RELEASE_FIXTURE_DIR, + allowedChangedPaths, + fixturePaths, +} from "../fixtures.js"; describe("fixturePaths", () => { it("builds repo-relative md/lock paths under tests/safe-outputs", () => { @@ -8,34 +17,58 @@ describe("fixturePaths", () => { name: "canary", relMd: "tests/safe-outputs/canary.md", relLock: "tests/safe-outputs/canary.lock.yml", + requiresAgentReadToken: true, }); }); + + it("uses the candidate-only directory for the custom fixture", () => { + const fixture = fixturePaths("custom-safe-output"); + expect(fixture.relMd).toBe( + "tests/compiler-smoke-e2e/custom-safe-output.md", + ); + expect(fixture.relLock).toBe( + "tests/compiler-smoke-e2e/custom-safe-output.lock.yml", + ); + expect(fixture.requiresAgentReadToken).toBe(false); + expect(fixture.requiredBuildTags?.(42)).toEqual([ + "ado-aw-custom-script-42", + "ado-aw-custom-job-42", + ]); + }); }); describe("ALL_FIXTURES", () => { - it("has exactly the five fixtures in the required stable order", () => { + it("has exactly the five candidate fixtures in the required stable order", () => { expect(ALL_FIXTURES.map((f) => f.name)).toEqual([ "canary", "azure-cli", "noop-target", - "janitor", "smoke-failure-reporter", + "custom-safe-output", ]); + expect(ALL_FIXTURES.map((f) => f.name)).not.toContain("janitor"); }); - it("every fixture path lives under the shared FIXTURE_DIR", () => { - for (const f of ALL_FIXTURES) { - expect(f.relMd.startsWith(`${FIXTURE_DIR}/`)).toBe(true); - expect(f.relLock.startsWith(`${FIXTURE_DIR}/`)).toBe(true); + it("keeps release and candidate-only fixture paths separate", () => { + for (const fixture of ALL_FIXTURES) { + const directory = + fixture.name === "custom-safe-output" + ? CANDIDATE_FIXTURE_DIR + : RELEASE_FIXTURE_DIR; + expect(fixture.relMd.startsWith(`${directory}/`)).toBe(true); + expect(fixture.relLock.startsWith(`${directory}/`)).toBe(true); } }); }); describe("allowedChangedPaths", () => { - it("contains exactly the five md files, five lock files, and .gitattributes", () => { + it("contains the five source/lock pairs and exact compiler-managed attribute/cache paths", () => { const allowed = allowedChangedPaths(); - expect(allowed.size).toBe(11); + expect(allowed.size).toBe(14); expect(allowed.has(".gitattributes")).toBe(true); + expect(allowed.has(IMPORT_CACHE_ATTRIBUTES_PATH)).toBe(true); + expect(allowed.has(CUSTOM_COMPONENT_CACHE_PATH)).toBe(true); + expect(allowed.has(CUSTOM_COMPONENT_DIGEST_PATH)).toBe(true); for (const f of ALL_FIXTURES) { expect(allowed.has(f.relMd)).toBe(true); expect(allowed.has(f.relLock)).toBe(true); diff --git a/scripts/ado-script/src/compiler-smoke-e2e/__tests__/git.test.ts b/scripts/ado-script/src/compiler-smoke-e2e/__tests__/git.test.ts index b38bbd9b..7dabe06e 100644 --- a/scripts/ado-script/src/compiler-smoke-e2e/__tests__/git.test.ts +++ b/scripts/ado-script/src/compiler-smoke-e2e/__tests__/git.test.ts @@ -105,7 +105,7 @@ describe("parseCandidateBuildId", () => { describe("worktreeChangedFiles", () => { it("parses git status --porcelain=v1 output, one path per line", async () => { - const { runner } = fakeRunner(() => ({ + const { runner, calls } = fakeRunner(() => ({ status: 0, stdout: " M tests/safe-outputs/canary.md\n?? tests/safe-outputs/canary.lock.yml\n", })); @@ -114,6 +114,29 @@ describe("worktreeChangedFiles", () => { "tests/safe-outputs/canary.md", "tests/safe-outputs/canary.lock.yml", ]); + expect(calls[0]?.args).toEqual([ + "status", + "--porcelain=v1", + "--untracked-files=all", + ]); + }); + + it("receives untracked import-cache files individually instead of a collapsed directory", async () => { + const { runner } = fakeRunner(() => ({ + status: 0, + stdout: [ + "?? .ado-aw/imports/.gitattributes", + "?? .ado-aw/imports/owner/repo/sha/component.md", + "?? .ado-aw/imports/owner/repo/sha/component.md.sha256", + ].join("\n"), + })); + await expect( + worktreeChangedFiles({ worktreeDir: "/wt", timeoutMs: 1000 }, runner), + ).resolves.toEqual([ + ".ado-aw/imports/.gitattributes", + ".ado-aw/imports/owner/repo/sha/component.md", + ".ado-aw/imports/owner/repo/sha/component.md.sha256", + ]); }); it("expands a rename line into both the old and new path", async () => { diff --git a/scripts/ado-script/src/compiler-smoke-e2e/__tests__/index.test.ts b/scripts/ado-script/src/compiler-smoke-e2e/__tests__/index.test.ts index 557281f1..9bb4aceb 100644 --- a/scripts/ado-script/src/compiler-smoke-e2e/__tests__/index.test.ts +++ b/scripts/ado-script/src/compiler-smoke-e2e/__tests__/index.test.ts @@ -1,6 +1,8 @@ import { describe, expect, it, vi, beforeEach } from "vitest"; const mockCalls: string[] = []; +const compiledFixturePaths: string[] = []; +let queuedFixtureNames: string[] = []; const baseEnv = { SYSTEM_COLLECTIONURI: "https://dev.azure.com/org/", @@ -17,24 +19,39 @@ const baseEnv = { COMPILER_SMOKE_CANARY_DEFINITION_ID: "3001", COMPILER_SMOKE_AZURE_CLI_DEFINITION_ID: "3002", COMPILER_SMOKE_NOOP_TARGET_DEFINITION_ID: "3003", - COMPILER_SMOKE_JANITOR_DEFINITION_ID: "3004", - COMPILER_SMOKE_REPORTER_DEFINITION_ID: "3005", + COMPILER_SMOKE_REPORTER_DEFINITION_ID: "3004", + COMPILER_SMOKE_CUSTOM_SAFE_OUTPUT_DEFINITION_ID: "3005", COMPILER_SMOKE_CHILD_TIMEOUT_MS: "5000", COMPILER_SMOKE_POLL_MS: "1", }; -function specificRunYaml(): string { +function specificRunYaml(agentReadToken: boolean): string { + const readTokenEnv = agentReadToken + ? "\n AZURE_DEVOPS_EXT_PAT: $(SC_READ_TOKEN)" + : ""; return ` -steps: - - task: DownloadPipelineArtifact@2 - inputs: - targetPath: in - source: specific - project: AgentPlayground - pipeline: '2560' - runVersion: specific - runId: '630001' - artifact: ado-aw-candidate +jobs: + - job: Agent + steps: + - bash: copilot --allow-tool "shell(az)" --allow-tool "shell(head)" + displayName: Run copilot (AWF network isolated) + env: + GITHUB_TOKEN: $(GITHUB_TOKEN)${readTokenEnv} + - task: DownloadPipelineArtifact@2 + inputs: + targetPath: in + source: specific + project: AgentPlayground + pipeline: '2560' + runVersion: specific + runId: '630001' + artifact: ado-aw-candidate + - job: Detection + steps: + - bash: echo detection + displayName: Run threat analysis (AWF network isolated) + env: + GITHUB_TOKEN: $(GITHUB_TOKEN) `; } @@ -47,6 +64,10 @@ vi.mock("../ado-rest.js", () => { return { name: "ado-aw-candidate" }; }), getBuild: vi.fn(async () => ({ status: "completed", result: "succeeded" })), + getBuildTags: vi.fn(async (buildId: number) => [ + `ado-aw-custom-script-${buildId}`, + `ado-aw-custom-job-${buildId}`, + ]), queueBuild: vi.fn(async () => ({ id: 1 })), cancelBuild: vi.fn(async () => {}), buildUrl: (id: number) => `https://example/${id}`, @@ -79,10 +100,13 @@ vi.mock("../git.js", async (importOriginal) => { "tests/safe-outputs/azure-cli.lock.yml", "tests/safe-outputs/noop-target.md", "tests/safe-outputs/noop-target.lock.yml", - "tests/safe-outputs/janitor.md", - "tests/safe-outputs/janitor.lock.yml", "tests/safe-outputs/smoke-failure-reporter.md", "tests/safe-outputs/smoke-failure-reporter.lock.yml", + "tests/compiler-smoke-e2e/custom-safe-output.md", + "tests/compiler-smoke-e2e/custom-safe-output.lock.yml", + ".ado-aw/imports/.gitattributes", + ".ado-aw/imports/AgentPlayground/ado-aw-e2e-fixture/aa711dd17c4dfcde492b2bfad62e5fb1baad71f6/components/custom-build-tags/component.md", + ".ado-aw/imports/AgentPlayground/ado-aw-e2e-fixture/aa711dd17c4dfcde492b2bfad62e5fb1baad71f6/components/custom-build-tags/component.md.sha256", ]; }), commitAll: vi.fn(async () => { @@ -106,8 +130,9 @@ vi.mock("../git.js", async (importOriginal) => { }); vi.mock("../compile-cli.js", () => ({ - compileAndCheck: vi.fn(async () => { + compileAndCheck: vi.fn(async (opts: { relMd: string }) => { mockCalls.push("compileAndCheck"); + compiledFixturePaths.push(opts.relMd); return { ok: true, stdout: "", stderr: "" }; }), })); @@ -118,6 +143,7 @@ vi.mock("../runner.js", async (importOriginal) => { ...actual, runFixtures: vi.fn(async (_client: unknown, requests: { name: string }[]) => { mockCalls.push("runFixtures"); + queuedFixtureNames = requests.map((request) => request.name); return { ok: true, allTerminal: true, @@ -142,7 +168,9 @@ vi.mock("node:fs/promises", async (importOriginal) => { ...actual, mkdtemp: vi.fn(async () => "C:\\tmp\\compiler-smoke-xyz"), readFile: vi.fn(async (path: string) => { - if (String(path).endsWith(".lock.yml")) return specificRunYaml(); + if (String(path).endsWith(".lock.yml")) { + return specificRunYaml(!String(path).includes("custom-safe-output.lock.yml")); + } return "---\nname: fixture\n---\nBody.\n"; }), writeFile: vi.fn(async () => {}), @@ -152,32 +180,54 @@ vi.mock("node:fs/promises", async (importOriginal) => { beforeEach(() => { mockCalls.length = 0; + compiledFixturePaths.length = 0; + queuedFixtureNames = []; vi.clearAllMocks(); }); describe("compiler-smoke-e2e index.main (happy path)", () => { - it("checks artifact visibility before any git work, and deletes the ref before removing the worktree", async () => { - process.env = { ...process.env, ...baseEnv, VITEST: "true" }; - const { main } = await import("../index.js"); - const code = await main(); - expect(code).toBe(0); + it( + "checks artifact visibility before any git work, and deletes the ref before removing the worktree", + async () => { + process.env = { ...process.env, ...baseEnv, VITEST: "true" }; + const { main } = await import("../index.js"); + const code = await main(); + expect(code).toBe(0); - expect(mockCalls.indexOf("getArtifact")).toBeGreaterThanOrEqual(0); - expect(mockCalls.indexOf("getArtifact")).toBeLessThan(mockCalls.indexOf("verifyLocalCommit")); - expect(mockCalls.indexOf("verifyLocalCommit")).toBeLessThan(mockCalls.indexOf("createDetachedWorktree")); - expect(mockCalls.indexOf("createDetachedWorktree")).toBeLessThan(mockCalls.indexOf("compileAndCheck")); - expect(mockCalls.indexOf("compileAndCheck")).toBeLessThan(mockCalls.indexOf("worktreeChangedFiles")); - expect(mockCalls.indexOf("worktreeChangedFiles")).toBeLessThan(mockCalls.indexOf("commitAll")); - expect(mockCalls.indexOf("commitAll")).toBeLessThan(mockCalls.indexOf("pushCandidate")); - expect(mockCalls.indexOf("pushCandidate")).toBeLessThan(mockCalls.indexOf("verifyRemoteRef")); - expect(mockCalls.indexOf("verifyRemoteRef")).toBeLessThan(mockCalls.indexOf("runFixtures")); + expect(mockCalls.indexOf("getArtifact")).toBeGreaterThanOrEqual(0); + expect(mockCalls.indexOf("getArtifact")).toBeLessThan(mockCalls.indexOf("verifyLocalCommit")); + expect(mockCalls.indexOf("verifyLocalCommit")).toBeLessThan(mockCalls.indexOf("createDetachedWorktree")); + expect(mockCalls.indexOf("createDetachedWorktree")).toBeLessThan(mockCalls.indexOf("compileAndCheck")); + expect(mockCalls.indexOf("compileAndCheck")).toBeLessThan(mockCalls.indexOf("worktreeChangedFiles")); + expect(mockCalls.indexOf("worktreeChangedFiles")).toBeLessThan(mockCalls.indexOf("commitAll")); + expect(mockCalls.indexOf("commitAll")).toBeLessThan(mockCalls.indexOf("pushCandidate")); + expect(mockCalls.indexOf("pushCandidate")).toBeLessThan(mockCalls.indexOf("verifyRemoteRef")); + expect(mockCalls.indexOf("verifyRemoteRef")).toBeLessThan(mockCalls.indexOf("runFixtures")); + expect(compiledFixturePaths).toEqual([ + "tests/safe-outputs/canary.md", + "tests/safe-outputs/azure-cli.md", + "tests/safe-outputs/noop-target.md", + "tests/safe-outputs/smoke-failure-reporter.md", + "tests/compiler-smoke-e2e/custom-safe-output.md", + ]); + expect(compiledFixturePaths).not.toContain("tests/safe-outputs/janitor.md"); + expect(queuedFixtureNames).toEqual([ + "canary", + "azure-cli", + "noop-target", + "smoke-failure-reporter", + "custom-safe-output", + ]); + expect(queuedFixtureNames).not.toContain("janitor"); - // Cleanup ordering: the remote candidate ref must be deleted BEFORE the - // local worktree is removed (never leave the ref hanging around). - expect(mockCalls.indexOf("deleteRemoteRef")).toBeGreaterThanOrEqual(0); - expect(mockCalls.indexOf("removeWorktree")).toBeGreaterThanOrEqual(0); - expect(mockCalls.indexOf("deleteRemoteRef")).toBeLessThan(mockCalls.indexOf("removeWorktree")); - }); + // Cleanup ordering: the remote candidate ref must be deleted BEFORE the + // local worktree is removed (never leave the ref hanging around). + expect(mockCalls.indexOf("deleteRemoteRef")).toBeGreaterThanOrEqual(0); + expect(mockCalls.indexOf("removeWorktree")).toBeGreaterThanOrEqual(0); + expect(mockCalls.indexOf("deleteRemoteRef")).toBeLessThan(mockCalls.indexOf("removeWorktree")); + }, + 15_000, + ); }); describe("compiler-smoke-e2e index.main (unexpected path guard)", () => { diff --git a/scripts/ado-script/src/compiler-smoke-e2e/__tests__/pipeline-policy.test.ts b/scripts/ado-script/src/compiler-smoke-e2e/__tests__/pipeline-policy.test.ts new file mode 100644 index 00000000..1e49ae76 --- /dev/null +++ b/scripts/ado-script/src/compiler-smoke-e2e/__tests__/pipeline-policy.test.ts @@ -0,0 +1,116 @@ +import { readFileSync } from "node:fs"; +import { dirname, resolve } from "node:path"; +import { fileURLToPath } from "node:url"; + +import { describe, expect, it } from "vitest"; +import { parse } from "yaml"; + +const pipelinePath = resolve( + dirname(fileURLToPath(import.meta.url)), + "../../../../../tests/compiler-smoke-e2e/azure-pipelines.yml", +); + +describe("candidate compiler trigger policy", () => { + const text = readFileSync(pipelinePath, "utf8"); + const pipeline = parse(text) as { + trigger?: string; + pr?: { branches?: { include?: string[] } }; + schedules?: Array<{ + cron?: string; + branches?: { include?: string[] }; + always?: boolean; + }>; + jobs?: Array<{ + steps?: Array<{ + condition?: string; + displayName?: string; + inputs?: { + artifact?: string; + targetPath?: string; + }; + script?: string; + task?: string; + }>; + }>; + }; + const steps = pipeline.jobs?.flatMap((job) => job.steps ?? []) ?? []; + + it("keeps PRs eligible for the Azure Pipelines comment trigger", () => { + expect(pipeline.trigger).toBe("none"); + expect(pipeline.pr?.branches?.include).toEqual(["main"]); + }); + + it("runs the latest main candidate every day", () => { + expect(pipeline.schedules).toEqual([ + expect.objectContaining({ + cron: "0 1 * * *", + branches: { include: ["main"] }, + always: true, + }), + ]); + }); + + it("fails when the live orchestrator loses its all-PR comment gate", () => { + for (const contract of [ + ".isCommentRequiredForPullRequest == true", + ".requireCommentsForNonTeamMembersOnly == false", + ".requireCommentsForNonTeamMemberAndNonContributors == false", + ".isCommentRequiredForInternalRepoPRs == true", + '.commentOptionInternalRepos == "all"', + ]) { + expect(text).toContain(contract); + } + }); + + it("preserves bounded ADO response diagnostics when the policy audit fails", () => { + const initialize = steps.find( + (step) => + step.displayName === "Initialize candidate smoke diagnostics", + ); + expect(initialize?.script).toContain('mkdir -p "$DIAGNOSTICS"'); + expect(initialize?.script).toContain( + '"ado-aw/candidate-smoke-diagnostics/1"', + ); + + const audit = steps.find( + (step) => step.displayName === "Audit AgentPlayground trigger policy", + ); + expect(audit?.script).toContain("for attempt in 1 2 3"); + expect(audit?.script).toContain("--fail-with-body"); + expect(audit?.script).toContain('--dump-header "$raw_headers"'); + expect(audit?.script).toContain( + 'RAW_DIAGNOSTICS="$(Agent.TempDirectory)/compiler-smoke-policy"', + ); + expect(audit?.script).toContain( + 'body="$RAW_DIAGNOSTICS/definition-${id}-attempt-${attempt}.body"', + ); + expect(audit?.script).toContain( + 'raw_headers="$RAW_DIAGNOSTICS/definition-${id}-attempt-${attempt}.headers.raw"', + ); + expect(audit?.script).not.toContain( + 'body="$DIAGNOSTICS/definition-${id}-attempt-${attempt}.body"', + ); + expect(audit?.script).toContain( + '-H "Authorization: Bearer $SYSTEM_ACCESSTOKEN"', + ); + expect(audit?.script).not.toContain("Authorization: ******"); + expect(audit?.script).not.toContain('SELF_JSON="$(curl'); + expect(audit?.script).toContain("http_code=%{http_code}"); + expect(audit?.script).toContain("jq_error_begin"); + expect(audit?.script).toContain('head -c 16384 "$body"'); + expect(audit?.script).toContain("response_sample_begin"); + + const publish = steps.find( + (step) => step.displayName === "Publish candidate smoke diagnostics", + ); + expect(publish).toMatchObject({ + condition: "always()", + inputs: { + artifact: "compiler-smoke-diagnostics", + targetPath: + "$(Build.ArtifactStagingDirectory)/compiler-smoke-diagnostics", + }, + task: "PublishPipelineArtifact@1", + }); + }); +}); diff --git a/scripts/ado-script/src/compiler-smoke-e2e/__tests__/report.test.ts b/scripts/ado-script/src/compiler-smoke-e2e/__tests__/report.test.ts index 9f61058c..97f8a301 100644 --- a/scripts/ado-script/src/compiler-smoke-e2e/__tests__/report.test.ts +++ b/scripts/ado-script/src/compiler-smoke-e2e/__tests__/report.test.ts @@ -32,13 +32,13 @@ describe("renderResultsTable", () => { it("preserves the caller's declaration order", () => { const table = renderResultsTable([ - result({ name: "janitor", definitionId: 2604 }), + result({ name: "smoke-failure-reporter", definitionId: 2604 }), result({ name: "canary", definitionId: 2601 }), ]); - const janitorIdx = table.indexOf("janitor"); + const reporterIdx = table.indexOf("smoke-failure-reporter"); const canaryIdx = table.indexOf("canary"); - expect(janitorIdx).toBeGreaterThan(-1); - expect(canaryIdx).toBeGreaterThan(janitorIdx); + expect(reporterIdx).toBeGreaterThan(-1); + expect(canaryIdx).toBeGreaterThan(reporterIdx); }); it("renders a '-' placeholder for missing buildId/url", () => { diff --git a/scripts/ado-script/src/compiler-smoke-e2e/__tests__/runner.test.ts b/scripts/ado-script/src/compiler-smoke-e2e/__tests__/runner.test.ts index 6b0601cc..c9164fd7 100644 --- a/scripts/ado-script/src/compiler-smoke-e2e/__tests__/runner.test.ts +++ b/scripts/ado-script/src/compiler-smoke-e2e/__tests__/runner.test.ts @@ -230,7 +230,12 @@ describe("runFixtures", () => { it("respects the configured concurrency (never more than N builds polled in parallel)", async () => { let inFlight = 0; let maxInFlight = 0; - const fixtureNames = ["canary", "azure-cli", "noop-target", "janitor"] as const; + const fixtureNames = [ + "canary", + "azure-cli", + "noop-target", + "smoke-failure-reporter", + ] as const; const buildIds = [701, 702, 703, 704]; const client: FixtureBuildClient = { diff --git a/scripts/ado-script/src/compiler-smoke-e2e/__tests__/signals.test.ts b/scripts/ado-script/src/compiler-smoke-e2e/__tests__/signals.test.ts new file mode 100644 index 00000000..add580c1 --- /dev/null +++ b/scripts/ado-script/src/compiler-smoke-e2e/__tests__/signals.test.ts @@ -0,0 +1,98 @@ +import { describe, expect, it } from "vitest"; + +import type { FixtureBuildResult } from "../runner.js"; +import { verifyFixtureSignals } from "../signals.js"; + +function result( + overrides: Partial = {}, +): FixtureBuildResult { + return { + name: "custom-safe-output", + definitionId: 3006, + buildId: 42, + url: "https://example/42", + status: "succeeded", + result: "succeeded", + durationMs: 1, + terminalProven: true, + ...overrides, + }; +} + +describe("verifyFixtureSignals", () => { + it("passes when both custom executor tags exist", async () => { + const outcome = await verifyFixtureSignals( + { + getBuildTags: async () => [ + "unrelated", + "ado-aw-custom-script-42", + "ado-aw-custom-job-42", + ], + }, + [result()], + ); + expect(outcome.ok).toBe(true); + expect(outcome.results[0]?.status).toBe("succeeded"); + }); + + it("fails a successful child when either route tag is missing", async () => { + const outcome = await verifyFixtureSignals( + { getBuildTags: async () => ["ado-aw-custom-script-42"] }, + [result()], + ); + expect(outcome.ok).toBe(false); + expect(outcome.results[0]).toMatchObject({ + status: "failed", + terminalProven: true, + result: "succeeded", + }); + expect(outcome.results[0]?.message).toMatch( + /ado-aw-custom-job-42/, + ); + }); + + it("reports tag API failures without losing terminal proof", async () => { + const outcome = await verifyFixtureSignals( + { + getBuildTags: async () => { + throw new Error("tag API unavailable"); + }, + }, + [result()], + ); + expect(outcome.ok).toBe(false); + expect(outcome.results[0]?.terminalProven).toBe(true); + expect(outcome.results[0]?.message).toMatch(/tag API unavailable/); + }); + + it("does not query tags for a child that already failed", async () => { + let calls = 0; + const outcome = await verifyFixtureSignals( + { + getBuildTags: async () => { + calls++; + return []; + }, + }, + [result({ status: "failed", result: "failed" })], + ); + expect(calls).toBe(0); + expect(outcome.ok).toBe(false); + }); + + it("leaves fixtures without signal requirements unchanged", async () => { + let calls = 0; + const canary = result({ name: "canary" }); + const outcome = await verifyFixtureSignals( + { + getBuildTags: async () => { + calls++; + return []; + }, + }, + [canary], + ); + expect(calls).toBe(0); + expect(outcome).toEqual({ ok: true, results: [canary] }); + }); +}); diff --git a/scripts/ado-script/src/compiler-smoke-e2e/ado-rest.ts b/scripts/ado-script/src/compiler-smoke-e2e/ado-rest.ts index e0dad3da..1812eba1 100644 --- a/scripts/ado-script/src/compiler-smoke-e2e/ado-rest.ts +++ b/scripts/ado-script/src/compiler-smoke-e2e/ado-rest.ts @@ -41,6 +41,8 @@ export interface ArtifactInfo { const DEFAULT_ARTIFACT_RETRIES = 5; const DEFAULT_ARTIFACT_RETRY_DELAY_MS = 5_000; +const DEFAULT_TAG_RETRIES = 5; +const DEFAULT_TAG_RETRY_DELAY_MS = 2_000; export class AdoRest { private readonly base: string; @@ -153,6 +155,65 @@ export class AdoRest { return res; } + /** Read the observable tags on a completed child build. */ + async getBuildTags( + buildId: number, + opts: { + retries?: number; + retryDelayMs?: number; + required?: readonly string[]; + } = {}, + ): Promise { + const retries = opts.retries ?? DEFAULT_TAG_RETRIES; + const retryDelayMs = opts.retryDelayMs ?? DEFAULT_TAG_RETRY_DELAY_MS; + const path = this.projPath(`_apis/build/builds/${buildId}/tags?api-version=7.1`); + let lastErr: unknown; + + for (let attempt = 1; attempt <= retries; attempt++) { + try { + const response = await this.request(path); + const tags = Array.isArray(response) + ? response + : response && + typeof response === "object" && + Array.isArray((response as { value?: unknown }).value) + ? (response as { value: unknown[] }).value + : undefined; + if (!tags || !tags.every((tag) => typeof tag === "string")) { + throw new Error("ADO build-tags response was not a string array"); + } + const stringTags = tags as string[]; + const missing = (opts.required ?? []).filter( + (tag) => !stringTags.includes(tag), + ); + if (missing.length > 0) { + throw new Error( + `required build tag(s) not visible yet: ${missing.join(", ")}; ` + + `observed: ${stringTags.length > 0 ? stringTags.join(", ") : ""}`, + ); + } + return stringTags; + } catch (err) { + lastErr = err; + } + + if (attempt < retries) { + this.log( + `[build-tags] build #${buildId} attempt ${attempt}/${retries} failed, retrying in ${retryDelayMs}ms: ${ + (lastErr as Error).message + }`, + ); + await this.sleepImpl(retryDelayMs); + } + } + + throw new Error( + `could not read tags for build #${buildId} after ${retries} attempts: ${ + (lastErr as Error)?.message ?? "unknown error" + }`, + ); + } + async cancelBuild(buildId: number): Promise { const path = this.projPath(`_apis/build/builds/${buildId}?api-version=7.1`); await this.request(path, { method: "PATCH", body: { status: "cancelling" } }); diff --git a/scripts/ado-script/src/compiler-smoke-e2e/assertions.ts b/scripts/ado-script/src/compiler-smoke-e2e/assertions.ts index 0a107d47..0c64a99c 100644 --- a/scripts/ado-script/src/compiler-smoke-e2e/assertions.ts +++ b/scripts/ado-script/src/compiler-smoke-e2e/assertions.ts @@ -52,6 +52,114 @@ function collectDownloadPipelineArtifactSteps(node: unknown, out: Record[], +): void { + if (Array.isArray(node)) { + for (const item of node) collectStepsByDisplayName(item, displayName, out); + return; + } + if (node && typeof node === "object") { + const obj = node as Record; + if (obj.displayName === displayName) { + out.push(obj); + } + for (const value of Object.values(obj)) { + collectStepsByDisplayName(value, displayName, out); + } + } +} + +function singleStep( + docs: unknown[], + label: string, + displayName: string, +): Record { + const steps: Record[] = []; + for (const doc of docs) collectStepsByDisplayName(doc, displayName, steps); + if (steps.length !== 1) { + throw new Error( + `${label}: expected exactly one '${displayName}' step, found ${steps.length}`, + ); + } + return steps[0]!; +} + +/** + * Assert the Stage 1 ADO credential boundary in freshly compiled YAML. + * + * Workflows with `permissions.read` must project the read-scoped token to the + * Agent as `AZURE_DEVOPS_EXT_PAT`. Workflows without it must not gain the + * mapping. Detection must never receive any ADO token, and neither execution + * step may receive the write-scoped token. + */ +export function assertAdoTokenIsolation( + yamlText: string, + label: string, + expectsAgentReadToken: boolean, +): void { + const docs = parseAllDocuments(yamlText, { merge: false }).map((d) => d.toJS()); + const agent = singleStep(docs, label, "Run copilot (AWF network isolated)"); + const detection = singleStep(docs, label, "Run threat analysis (AWF network isolated)"); + const agentEnv = (agent.env ?? {}) as Record; + const detectionEnv = (detection.env ?? {}) as Record; + const actualReadToken = agentEnv.AZURE_DEVOPS_EXT_PAT; + + if (expectsAgentReadToken && actualReadToken !== "$(SC_READ_TOKEN)") { + throw new Error( + `${label}: Agent read-token contract mismatch — expected AZURE_DEVOPS_EXT_PAT='$(SC_READ_TOKEN)', got ${JSON.stringify(actualReadToken)}`, + ); + } + if (!expectsAgentReadToken && actualReadToken !== undefined) { + throw new Error( + `${label}: Agent unexpectedly received AZURE_DEVOPS_EXT_PAT=${JSON.stringify(actualReadToken)}`, + ); + } + + for (const forbidden of ["SC_READ_TOKEN", "SC_WRITE_TOKEN", "SYSTEM_ACCESSTOKEN"]) { + if (agentEnv[forbidden] !== undefined) { + throw new Error(`${label}: Agent must not receive ${forbidden}`); + } + } + for (const forbidden of [ + "AZURE_DEVOPS_EXT_PAT", + "SC_READ_TOKEN", + "SC_WRITE_TOKEN", + "SYSTEM_ACCESSTOKEN", + ]) { + if (detectionEnv[forbidden] !== undefined) { + throw new Error(`${label}: Detection must not receive ${forbidden}`); + } + } +} + +/** Assert the command/tool policy on the Agent execution step only. */ +export function assertAgentCommandPolicy( + yamlText: string, + label: string, + requiredSnippets: readonly string[], + forbiddenSnippets: readonly string[], +): void { + const docs = parseAllDocuments(yamlText, { merge: false }).map((d) => d.toJS()); + const agent = singleStep(docs, label, "Run copilot (AWF network isolated)"); + const script = agent.bash; + if (typeof script !== "string") { + throw new Error(`${label}: Agent execution step has no bash body`); + } + for (const snippet of requiredSnippets) { + if (!script.includes(snippet)) { + throw new Error(`${label}: Agent command is missing required snippet '${snippet}'`); + } + } + for (const snippet of forbiddenSnippets) { + if (script.includes(snippet)) { + throw new Error(`${label}: Agent command contains forbidden snippet '${snippet}'`); + } + } +} + /** * Throws unless every `DownloadPipelineArtifact` "specific run" step in the * compiled YAML carries exactly the expected project/pipeline/runId/artifact diff --git a/scripts/ado-script/src/compiler-smoke-e2e/config.ts b/scripts/ado-script/src/compiler-smoke-e2e/config.ts index ba94e545..030679c8 100644 --- a/scripts/ado-script/src/compiler-smoke-e2e/config.ts +++ b/scripts/ado-script/src/compiler-smoke-e2e/config.ts @@ -2,10 +2,9 @@ * Environment configuration for the deterministic compiler-smoke E2E harness. * * This harness stages the compiler candidate produced by the current build as - * a pinned `supply-chain.pipeline-artifact` source across five real, - * registered ADO pipeline fixtures (canary, azure-cli, noop-target, janitor, - * smoke-failure-reporter), pushes the staged candidate to a per-run branch on - * a mirror repo, queues all five, and asserts they all go green. + * a pinned `supply-chain.pipeline-artifact` source across five registered ADO + * pipeline fixtures, pushes the staged candidate to a per-run branch on a + * mirror repo, queues all five, and asserts they all go green. * * Strict, fail-closed parsing lives here so every other module can trust a * fully validated {@link CompilerSmokeConfig} rather than re-checking env @@ -27,16 +26,16 @@ export const DEFAULT_POLL_MS = 10_000; export const DEFAULT_STALE_REF_HOURS = 24; export const MIN_STALE_REF_HOURS = 6; -/** Stable declaration order for every fixture-keyed collection in the harness. */ -export const FIXTURE_NAMES = [ +/** Stable declaration order for the five workflows in the live candidate lane. */ +export const CANDIDATE_FIXTURE_NAMES = [ "canary", "azure-cli", "noop-target", - "janitor", "smoke-failure-reporter", + "custom-safe-output", ] as const; -export type FixtureName = (typeof FIXTURE_NAMES)[number]; +export type FixtureName = (typeof CANDIDATE_FIXTURE_NAMES)[number]; export interface CompilerSmokeConfig { /** ADO collection URI, e.g. https://dev.azure.com/org/. */ @@ -59,7 +58,7 @@ export interface CompilerSmokeConfig { readonly adoAwBin: string; /** Pipeline artifact name pinned into each fixture's supply-chain config. */ readonly artifactName: string; - /** ADO Git repo hosting the five registered fixture pipeline definitions. */ + /** ADO Git repo hosting the five registered candidate definitions. */ readonly mirrorRepo: string; /** Registered ADO pipeline definition id, keyed by fixture name. */ readonly definitionIds: Readonly>; @@ -89,8 +88,8 @@ const DEFINITION_ID_ENV_BY_FIXTURE: Readonly> = { canary: "COMPILER_SMOKE_CANARY_DEFINITION_ID", "azure-cli": "COMPILER_SMOKE_AZURE_CLI_DEFINITION_ID", "noop-target": "COMPILER_SMOKE_NOOP_TARGET_DEFINITION_ID", - janitor: "COMPILER_SMOKE_JANITOR_DEFINITION_ID", "smoke-failure-reporter": "COMPILER_SMOKE_REPORTER_DEFINITION_ID", + "custom-safe-output": "COMPILER_SMOKE_CUSTOM_SAFE_OUTPUT_DEFINITION_ID", }; /** ADO macros that failed to expand look like `$(NAME)`; treat them as unset. */ @@ -162,12 +161,12 @@ export function loadConfig(env: NodeJS.ProcessEnv = process.env): CompilerSmokeC const definitionId = requirePositiveInt(env, "SYSTEM_DEFINITIONID"); const definitionIds = {} as Record; - for (const fixture of FIXTURE_NAMES) { + for (const fixture of CANDIDATE_FIXTURE_NAMES) { definitionIds[fixture] = requirePositiveInt(env, DEFINITION_ID_ENV_BY_FIXTURE[fixture]); } const seen = new Map(); - for (const fixture of FIXTURE_NAMES) { + for (const fixture of CANDIDATE_FIXTURE_NAMES) { const id = definitionIds[fixture]; const existing = seen.get(id); if (existing) { diff --git a/scripts/ado-script/src/compiler-smoke-e2e/fixtures.ts b/scripts/ado-script/src/compiler-smoke-e2e/fixtures.ts index 137de4ed..451044ef 100644 --- a/scripts/ado-script/src/compiler-smoke-e2e/fixtures.ts +++ b/scripts/ado-script/src/compiler-smoke-e2e/fixtures.ts @@ -1,25 +1,27 @@ /** * Manifest of the five fixed compiler-smoke fixtures. * - * These are the same five agentic-pipeline sources documented in - * `tests/safe-outputs/README.md` (canary, azure-cli, noop-target, janitor, - * smoke-failure-reporter) — this harness does not invent its own fixture - * content. It reads the exact files from the detached candidate worktree - * (an exact checkout of `BUILD_SOURCEVERSION`, at - * `/tests/safe-outputs/.md` — never from - * `BUILD_SOURCESDIRECTORY`, which may sit at a different commit), stages a - * pinned `supply-chain.pipeline-artifact` transform of each onto the mirror - * repo, recompiles, and queues the five FIXED "candidate lane" pipeline - * definitions tracked in `tests/compiler-smoke-e2e/REGISTERED.md` (distinct - * from the release-backed definitions those same sources also feed). + * Four reuse release-backed sources under `tests/safe-outputs/`; the fifth is + * candidate-only and lives beside this harness. The weekly janitor is + * deliberately excluded from candidate checks. Every selected source is read from + * the detached candidate worktree (an exact checkout of + * `BUILD_SOURCEVERSION`, never the possibly-divergent + * `BUILD_SOURCESDIRECTORY`), transformed, compiled, and queued through its + * fixed definition tracked in `tests/compiler-smoke-e2e/REGISTERED.md`. * * Test-harness module; not shipped in `ado-script.zip`. */ import type { FixtureName } from "./config.js"; -import { FIXTURE_NAMES } from "./config.js"; +import { CANDIDATE_FIXTURE_NAMES } from "./config.js"; -/** Repo-relative directory containing every fixture source + compiled lock. */ -export const FIXTURE_DIR = "tests/safe-outputs"; +/** Repo-relative directory containing release-backed fixture sources. */ +export const RELEASE_FIXTURE_DIR = "tests/safe-outputs"; +/** Repo-relative directory containing the candidate-only custom fixture. */ +export const CANDIDATE_FIXTURE_DIR = "tests/compiler-smoke-e2e"; +export const CUSTOM_COMPONENT_CACHE_PATH = + ".ado-aw/imports/AgentPlayground/ado-aw-e2e-fixture/aa711dd17c4dfcde492b2bfad62e5fb1baad71f6/components/custom-build-tags/component.md"; +export const CUSTOM_COMPONENT_DIGEST_PATH = `${CUSTOM_COMPONENT_CACHE_PATH}.sha256`; +export const IMPORT_CACHE_ATTRIBUTES_PATH = ".ado-aw/imports/.gitattributes"; export interface FixturePaths { readonly name: FixtureName; @@ -27,28 +29,56 @@ export interface FixturePaths { readonly relMd: string; /** Repo-relative path to the compiled lock file, e.g. tests/safe-outputs/canary.lock.yml. */ readonly relLock: string; + /** Whether the Agent step must receive the read-scoped ADO token. */ + readonly requiresAgentReadToken: boolean; + /** Observable ADO build tags that must exist after this child succeeds. */ + readonly requiredBuildTags?: (buildId: number) => readonly string[]; } -/** Repo-relative path for a fixture's markdown source. Always POSIX-separated (a git path, not a filesystem path). */ +/** Repo-relative paths and signal contract for one fixture. */ export function fixturePaths(name: FixtureName): FixturePaths { + const directory = + name === "custom-safe-output" ? CANDIDATE_FIXTURE_DIR : RELEASE_FIXTURE_DIR; + const requiredBuildTags = + name === "custom-safe-output" + ? (buildId: number): readonly string[] => [ + `ado-aw-custom-script-${buildId}`, + `ado-aw-custom-job-${buildId}`, + ] + : undefined; return { name, - relMd: `${FIXTURE_DIR}/${name}.md`, - relLock: `${FIXTURE_DIR}/${name}.lock.yml`, + relMd: `${directory}/${name}.md`, + relLock: `${directory}/${name}.lock.yml`, + requiresAgentReadToken: name !== "custom-safe-output", + requiredBuildTags, }; } -/** All five fixtures in the stable declaration order used throughout the harness. */ -export const ALL_FIXTURES: readonly FixturePaths[] = FIXTURE_NAMES.map(fixturePaths); +/** All five candidate fixtures in the stable order used throughout the harness. */ +export const ALL_FIXTURES: readonly FixturePaths[] = + CANDIDATE_FIXTURE_NAMES.map(fixturePaths); + +export function fixtureByName(name: FixtureName): FixturePaths { + const fixture = ALL_FIXTURES.find((candidate) => candidate.name === name); + if (!fixture) { + throw new Error(`unknown compiler-smoke fixture '${name}'`); + } + return fixture; +} /** - * The exact set of repo-relative paths the candidate-staging commit is - * allowed to touch: the five markdown sources, their five compiled locks, - * and the compiler-managed `.gitattributes` block. Any other changed path - * fails the run before it pushes anything. + * The exact set of repo-relative paths the candidate-staging commit may touch: + * five markdown sources, five compiled locks, and the compiler-managed + * `.gitattributes` block. Any other changed path fails before push. */ export function allowedChangedPaths(): Set { - const paths = new Set([".gitattributes"]); + const paths = new Set([ + ".gitattributes", + IMPORT_CACHE_ATTRIBUTES_PATH, + CUSTOM_COMPONENT_CACHE_PATH, + CUSTOM_COMPONENT_DIGEST_PATH, + ]); for (const fixture of ALL_FIXTURES) { paths.add(fixture.relMd); paths.add(fixture.relLock); diff --git a/scripts/ado-script/src/compiler-smoke-e2e/git.ts b/scripts/ado-script/src/compiler-smoke-e2e/git.ts index fa1c4316..300f50d7 100644 --- a/scripts/ado-script/src/compiler-smoke-e2e/git.ts +++ b/scripts/ado-script/src/compiler-smoke-e2e/git.ts @@ -149,7 +149,11 @@ export async function worktreeChangedFiles( runner: GitRunner = defaultGitRunner, ): Promise { const stdout = await run( - ["status", "--porcelain=v1"], + // `--untracked-files=all` is security-significant: the default porcelain + // output collapses a wholly untracked directory to e.g. `.ado-aw/`, which + // prevents the exact-path allowlist below from validating each generated + // import-cache file independently. + ["status", "--porcelain=v1", "--untracked-files=all"], { cwd: opts.worktreeDir, timeoutMs: opts.timeoutMs }, runner, ); diff --git a/scripts/ado-script/src/compiler-smoke-e2e/index.ts b/scripts/ado-script/src/compiler-smoke-e2e/index.ts index fe3c4547..c320e888 100644 --- a/scripts/ado-script/src/compiler-smoke-e2e/index.ts +++ b/scripts/ado-script/src/compiler-smoke-e2e/index.ts @@ -3,10 +3,9 @@ * * Stages the compiler candidate produced by the current build (PR or * nightly `main`) as a pinned `supply-chain.pipeline-artifact` source across - * the five real fixtures documented in `tests/safe-outputs/README.md` - * (canary, azure-cli, noop-target, janitor, smoke-failure-reporter), pushes - * the staged candidate to a short-lived branch on the mirror repo, queues - * the five FIXED "candidate lane" pipeline definitions (tracked in + * five fixed fixtures, pushes the staged candidate to a short-lived branch on + * the mirror repo, queues the FIXED "candidate lane" pipeline definitions + * (tracked in * `tests/compiler-smoke-e2e/REGISTERED.md`), and asserts they all go green. * * See `config.ts` for the full required/optional env var contract. @@ -18,8 +17,18 @@ import { tmpdir } from "node:os"; import { join } from "node:path"; import { AdoRest } from "./ado-rest.js"; -import { assertNoForbiddenReleaseUrls, assertPipelineArtifactValues } from "./assertions.js"; -import { candidateRef, FIXTURE_NAMES, loadConfig, type CompilerSmokeConfig } from "./config.js"; +import { + assertAgentCommandPolicy, + assertAdoTokenIsolation, + assertNoForbiddenReleaseUrls, + assertPipelineArtifactValues, +} from "./assertions.js"; +import { + candidateRef, + CANDIDATE_FIXTURE_NAMES, + loadConfig, + type CompilerSmokeConfig, +} from "./config.js"; import { compileAndCheck } from "./compile-cli.js"; import { ALL_FIXTURES, allowedChangedPaths } from "./fixtures.js"; import { @@ -38,6 +47,7 @@ import { import { injectPipelineArtifact } from "./source.js"; import { renderResultsTable } from "./report.js"; import { runFixtures, type FixtureBuildRequest, type FixtureBuildResult } from "./runner.js"; +import { verifyFixtureSignals } from "./signals.js"; import { scanStaleRefs } from "./stale.js"; function log(msg: string): void { @@ -86,6 +96,15 @@ async function compileFixtures( const yamlText = await readFile(join(worktreeDir, fixture.relLock), "utf8"); assertNoForbiddenReleaseUrls(yamlText, fixture.name); + assertAdoTokenIsolation(yamlText, fixture.name, fixture.requiresAgentReadToken); + if (fixture.name === "azure-cli") { + assertAgentCommandPolicy( + yamlText, + fixture.name, + ["shell(az", "shell(head"], + ["--allow-all-tools", "--allow-all-paths"], + ); + } assertPipelineArtifactValues(yamlText, fixture.name, { project: config.project, pipeline: String(config.definitionId), @@ -108,7 +127,9 @@ async function cleanupStaleRefs(config: CompilerSmokeConfig, rest: AdoRest, mirr baseRef: config.sourceBranch, ownRef, definitionId: config.definitionId, - childDefinitionIds: FIXTURE_NAMES.map((name) => config.definitionIds[name]), + childDefinitionIds: CANDIDATE_FIXTURE_NAMES.map( + (name) => config.definitionIds[name], + ), staleRefHours: config.staleRefHours, client: rest, }); @@ -216,7 +237,7 @@ export async function main(): Promise { }); log(`[git] candidate ${candidateSha} pushed to ${ownRef}`); - const requests: FixtureBuildRequest[] = FIXTURE_NAMES.map((name) => ({ + const requests: FixtureBuildRequest[] = CANDIDATE_FIXTURE_NAMES.map((name) => ({ name, definitionId: config.definitionIds[name], sourceBranch: ownRef, @@ -235,8 +256,9 @@ export async function main(): Promise { pollMs: config.pollMs, log, }); - results = outcome.results; - overallOk = outcome.ok; + const signalOutcome = await verifyFixtureSignals(rest, outcome.results); + results = signalOutcome.results; + overallOk = outcome.ok && signalOutcome.ok; allChildrenTerminal = outcome.allTerminal; if (!overallOk) failureMessage = "one or more fixture builds did not succeed"; if (!allChildrenTerminal) { diff --git a/scripts/ado-script/src/compiler-smoke-e2e/runner.ts b/scripts/ado-script/src/compiler-smoke-e2e/runner.ts index 1fb6c87c..daa6b1c4 100644 --- a/scripts/ado-script/src/compiler-smoke-e2e/runner.ts +++ b/scripts/ado-script/src/compiler-smoke-e2e/runner.ts @@ -1,5 +1,5 @@ /** - * Bounded-concurrency queue/poll/cancel state machine for the five staged + * Bounded-concurrency queue/poll/cancel state machine for the staged * fixture builds. * * Contract: diff --git a/scripts/ado-script/src/compiler-smoke-e2e/signals.ts b/scripts/ado-script/src/compiler-smoke-e2e/signals.ts new file mode 100644 index 00000000..06a47652 --- /dev/null +++ b/scripts/ado-script/src/compiler-smoke-e2e/signals.ts @@ -0,0 +1,71 @@ +/** + * Post-build verification for fixture-specific observable signals. + * + * A successful child build is not sufficient for custom safe outputs: both + * executor modes must leave their distinct build tags on the actual child run. + */ +import { fixtureByName } from "./fixtures.js"; +import type { FixtureBuildResult } from "./runner.js"; + +export interface BuildTagClient { + getBuildTags( + buildId: number, + opts?: { required?: readonly string[] }, + ): Promise; +} + +export interface SignalVerificationOutcome { + readonly ok: boolean; + readonly results: FixtureBuildResult[]; +} + +export async function verifyFixtureSignals( + client: BuildTagClient, + results: readonly FixtureBuildResult[], +): Promise { + const verified: FixtureBuildResult[] = []; + + for (const result of results) { + const fixture = fixtureByName(result.name); + if ( + result.status !== "succeeded" || + result.buildId === undefined || + fixture.requiredBuildTags === undefined + ) { + verified.push({ ...result }); + continue; + } + + try { + const expected = fixture.requiredBuildTags(result.buildId); + const actual = await client.getBuildTags(result.buildId, { + required: expected, + }); + const missing = expected.filter((tag) => !actual.includes(tag)); + if (missing.length === 0) { + verified.push({ ...result }); + continue; + } + verified.push({ + ...result, + status: "failed", + message: + `build #${result.buildId} is missing required tag(s): ${missing.join(", ")}; ` + + `observed: ${actual.length > 0 ? actual.join(", ") : ""}`, + }); + } catch (error) { + verified.push({ + ...result, + status: "failed", + message: `build #${result.buildId} tag verification failed: ${ + error instanceof Error ? error.message : String(error) + }`, + }); + } + } + + return { + ok: verified.every((result) => result.status === "succeeded"), + results: verified, + }; +} diff --git a/scripts/ado-script/src/compiler-smoke-e2e/stale.ts b/scripts/ado-script/src/compiler-smoke-e2e/stale.ts index 2a31b950..eae85514 100644 --- a/scripts/ado-script/src/compiler-smoke-e2e/stale.ts +++ b/scripts/ado-script/src/compiler-smoke-e2e/stale.ts @@ -10,8 +10,8 @@ * (`COMPILER_SMOKE_STALE_REF_HOURS`), and (c) that parent build is * terminal. Note that (c) is NOT by itself proof the orchestration it * started is done — an abruptly canceled/killed parent process can reach a - * terminal ADO build status while the five fixture builds it queued are - * still running. The scanner therefore also queries each of the five FIXED + * terminal ADO build status while its fixture builds are still running. The + * scanner therefore also queries each fixed * child definitions on the ref's exact branch (see * `listBuildsForDefinitionBranch`) and inspects their statuses directly; * only when every child build found there is ALSO terminal (or none exist) @@ -157,7 +157,7 @@ export async function scanStaleRefs(opts: ScanStaleRefsOptions): Promise { const target = writeFixture( dir, "prompt.md", - "root=$(Build.SourcesDirectory) repo=$(Build.Repository.Name)\n", + "root=$(Build.SourcesDirectory) repo=$(Build.Repository.Name) build=$(Build.BuildId) org=$(System.CollectionUri)\n", ); const result = runImportSource(target, { vars: [ "Build.SourcesDirectory=/agent/_work/1/s", "Build.Repository.Name=my-repo", + "Build.BuildId=626006", + "System.CollectionUri=https://dev.azure.com/example/", ], }); expect(result.status).toBe(0); - expect(readText(target)).toBe("root=/agent/_work/1/s repo=my-repo\n"); + expect(readText(target)).toBe( + "root=/agent/_work/1/s repo=my-repo build=626006 org=https://dev.azure.com/example/\n", + ); }); }); diff --git a/scripts/ado-script/src/import/index.ts b/scripts/ado-script/src/import/index.ts index 5313f806..b5f28fde 100644 --- a/scripts/ado-script/src/import/index.ts +++ b/scripts/ado-script/src/import/index.ts @@ -72,8 +72,8 @@ function fail(messages: string[]): never { // src/compile/extensions/ado_script.rs. // // The repeatable `--var name=value` flag carries a small, fixed set of -// ADO path-anchor variables (e.g. `Build.SourcesDirectory`, -// `Build.Repository.Name`). ADO expands the `$(...)` macros into +// non-secret ADO variables (e.g. `Build.SourcesDirectory`, +// `Build.BuildId`). ADO expands the `$(...)` macros into // concrete values in the resolver step's bash args before node runs, so // the values arrive here already resolved. The allowlist is owned by the // compiler (which emits the flags); import.js is a dumb substitutor and @@ -188,7 +188,7 @@ function main(): void { fail(errors); } - // Substitute the compiler-provided ADO path-anchor variables (e.g. + // Substitute the compiler-provided ADO variables (e.g. // `$(Build.SourcesDirectory)`). Runs on the fully expanded prompt, so it // covers both the author body and any inlined snippets, giving the same // result whether imports are inlined at compile time (where ADO expands diff --git a/scripts/ado-script/test/smoke.test.ts b/scripts/ado-script/test/smoke.test.ts index b231ffc1..72769e93 100644 --- a/scripts/ado-script/test/smoke.test.ts +++ b/scripts/ado-script/test/smoke.test.ts @@ -3,7 +3,9 @@ * * The gate smoke test validates the existing gate.js bundle. * The import smoke test builds import.js and verifies it expands - * a prompt fixture in place. + * a prompt fixture in place. Git-oriented bundles run against disposable + * local repositories so their bundled ncc output is exercised without network + * or Azure DevOps dependencies. */ import { spawnSync } from "node:child_process"; import { randomUUID } from "node:crypto"; @@ -18,6 +20,7 @@ const gateBundlePath = resolve(__dirname, "../gate.js"); const importBundlePath = resolve(__dirname, "../import.js"); const execContextPrBundlePath = resolve(__dirname, "../exec-context-pr.js"); const preparePrBaseBundlePath = resolve(__dirname, "../prepare-pr-base.js"); +const checkoutComponentBundlePath = resolve(__dirname, "../checkout-component.js"); const gateFixturePath = resolve( __dirname, "fixtures/gate-spec-pr-title-match.json", @@ -430,3 +433,67 @@ describe("prepare-pr-base.js smoke", () => { }); }, 60000); }); + +describe("checkout-component.js smoke", () => { + it("fetches and verifies a pinned commit missing from a shallow checkout", () => { + expect(existsSync(checkoutComponentBundlePath)).toBe(true); + + withSmokeScratchDir("checkout-component", (dir) => { + const originDir = resolve(dir, "origin"); + mkdirSync(originDir, { recursive: true }); + runGitInRepo(originDir, ["init", "-q", "-b", "main"]); + + writeFileSync(resolve(originDir, "component.txt"), "pinned\n", "utf8"); + runGitInRepo(originDir, ["add", "-A"]); + runGitInRepo(originDir, ["commit", "-q", "-m", "pinned component"]); + const pinnedSha = spawnSync("git", ["rev-parse", "HEAD"], { + cwd: originDir, + encoding: "utf8", + }).stdout.trim(); + + writeFileSync(resolve(originDir, "component.txt"), "tip\n", "utf8"); + runGitInRepo(originDir, ["add", "-A"]); + runGitInRepo(originDir, ["commit", "-q", "-m", "advance default branch"]); + + const checkout = resolve(dir, "checkout"); + runGitInRepo(dir, [ + "clone", + "--depth", + "1", + "--single-branch", + "--branch", + "main", + pathToFileURL(originDir).href, + checkout, + ]); + expect( + gitStatusInRepo(checkout, ["cat-file", "-e", `${pinnedSha}^{commit}`]), + ).not.toBe(0); + + const result = spawnSync( + process.execPath, + [ + checkoutComponentBundlePath, + "--dir", + checkout, + "--sha", + pinnedSha, + ], + { + env: { ...process.env, SYSTEM_ACCESSTOKEN: "" }, + encoding: "utf8", + }, + ); + + expect(result.status).toBe(0); + expect(result.stdout).toContain("verified component checkout"); + expect(result.stderr).toBe(""); + const actual = spawnSync("git", ["rev-parse", "HEAD"], { + cwd: checkout, + encoding: "utf8", + }).stdout.trim(); + expect(actual).toBe(pinnedSha); + expect(gitStatusInRepo(checkout, ["symbolic-ref", "-q", "HEAD"])).not.toBe(0); + }); + }, 30000); +}); diff --git a/scripts/rotate-agentplayground-secrets.ps1 b/scripts/rotate-agentplayground-secrets.ps1 index fe4888ba..5b2cffd2 100644 --- a/scripts/rotate-agentplayground-secrets.ps1 +++ b/scripts/rotate-agentplayground-secrets.ps1 @@ -7,7 +7,7 @@ param( Set-StrictMode -Version Latest $ErrorActionPreference = "Stop" -$copilotDefinitionIds = @(2545, 2546, 2547, 2548, 2549, 2554, 2555, 2556, 2557, 2558) +$copilotDefinitionIds = @(2545, 2546, 2547, 2548, 2549, 2554, 2555, 2556, 2558, 2564) $reporterDefinitionIds = @(2549, 2558) $executorDefinitionIds = @(2550) $triggerDefinitionIds = @(2551) diff --git a/src/ado/mod.rs b/src/ado/mod.rs index 0d118768..e7ddfdbd 100644 --- a/src/ado/mod.rs +++ b/src/ado/mod.rs @@ -995,6 +995,41 @@ pub async fn resolve_auth(pat: Option<&str>) -> Result { } } +/// Resolve ADO authentication for **non-interactive** (compile-time / CI) +/// contexts, without ever prompting. +/// +/// Precedence: +/// 1. `SYSTEM_ACCESSTOKEN` — the ADO pipeline job token (bearer). +/// 2. `AZURE_DEVOPS_EXT_PAT` — a personal access token (HTTP Basic). +/// 3. `az account get-access-token` — an Azure CLI AAD token (bearer). +/// +/// Unlike [`resolve_auth`], this never falls back to an interactive prompt, so +/// it is safe to call from `ado-aw compile` and other unattended entry points. +pub async fn resolve_auth_non_interactive() -> Result { + if let Ok(token) = std::env::var("SYSTEM_ACCESSTOKEN") + && !token.trim().is_empty() + { + debug!("Using SYSTEM_ACCESSTOKEN for Azure DevOps auth"); + return Ok(AdoAuth::Bearer(token)); + } + if let Ok(pat) = std::env::var("AZURE_DEVOPS_EXT_PAT") + && !pat.trim().is_empty() + { + debug!("Using AZURE_DEVOPS_EXT_PAT for Azure DevOps auth"); + return Ok(AdoAuth::Pat(pat)); + } + match try_azure_cli_token().await { + Ok(token) => { + debug!("Using Azure CLI token for Azure DevOps auth"); + Ok(AdoAuth::Bearer(token)) + } + Err(e) => Err(anyhow::anyhow!( + "no non-interactive Azure DevOps credentials available: set \ + SYSTEM_ACCESSTOKEN or AZURE_DEVOPS_EXT_PAT, or run `az login` ({e:#})" + )), + } +} + /// Normalize a `--org` value to a full ADO organization URL. /// /// Users commonly pass just the org name (e.g. `myorg`) instead of the full @@ -1282,6 +1317,89 @@ pub async fn get_repository_id( .with_context(|| format!("Repository '{}' response has no 'id' field", repo_name)) } +/// Fetch the raw bytes of a single SHA-pinned file from an Azure Repos git +/// repository via the ADO Git Items API. +/// +/// Calls +/// `GET {org_url}/{project}/_apis/git/repositories/{repo}/items?path={item_path} +/// &versionDescriptor.version={commit_sha}&versionDescriptor.versionType=commit +/// &includeContent=true&api-version=7.1` +/// and returns the file `content` as UTF-8 bytes. +/// +/// `org_url` is a full organization collection URL (e.g. +/// `https://dev.azure.com/myorg`); for cross-organization imports pass the +/// target org's URL. Detects the ADO AAD sign-in HTML response (see +/// [`looks_like_ado_signin`]) before attempting to parse JSON so an auth +/// failure surfaces an actionable error rather than a JSON-parse error. +pub async fn fetch_git_item( + client: &reqwest::Client, + org_url: &str, + project: &str, + repo: &str, + item_path: &str, + commit_sha: &str, + auth: &AdoAuth, +) -> Result> { + let url = format!( + "{}/{}/_apis/git/repositories/{}/items?path={}\ + &versionDescriptor.version={}&versionDescriptor.versionType=commit\ + &includeContent=true&api-version=7.1", + org_url.trim_end_matches('/'), + percent_encoding::utf8_percent_encode(project, PATH_SEGMENT), + percent_encoding::utf8_percent_encode(repo, PATH_SEGMENT), + percent_encoding::utf8_percent_encode(item_path, PATH_SEGMENT), + percent_encoding::utf8_percent_encode(commit_sha, PATH_SEGMENT), + ); + + debug!("Fetching Azure Repos item: {}", url); + + let resp = auth + .apply(client.get(&url)) + .header(reqwest::header::ACCEPT, "application/json") + .send() + .await + .with_context(|| format!("Failed to request Azure Repos item '{}'", item_path))?; + + let status = resp.status(); + let content_type = resp + .headers() + .get(reqwest::header::CONTENT_TYPE) + .and_then(|v| v.to_str().ok()) + .map(str::to_string); + let body = resp + .text() + .await + .with_context(|| format!("Failed to read Azure Repos item response for '{}'", item_path))?; + + if looks_like_ado_signin(status, content_type.as_deref(), &body) { + return Err(ado_signin_error(auth)); + } + if !status.is_success() { + anyhow::bail!( + "ADO Git Items API returned {} for '{}': {}", + status, + item_path, + body.trim() + ); + } + + #[derive(Deserialize)] + struct GitItem { + content: Option, + } + + let item: GitItem = serde_json::from_str(&body).with_context(|| { + format!("Failed to parse Azure Repos item response for '{}'", item_path) + })?; + let content = item.content.with_context(|| { + format!( + "Azure Repos item '{}' response contained no `content` (is it a directory?)", + item_path + ) + })?; + Ok(content.into_bytes()) +} + /// Resolve a GitHub service-connection identifier to its GUID. /// /// Accepts either a raw UUID (returned unchanged — no API call) or a diff --git a/src/audit/analyzers/safe_outputs.rs b/src/audit/analyzers/safe_outputs.rs index 0d72a722..a861d344 100644 --- a/src/audit/analyzers/safe_outputs.rs +++ b/src/audit/analyzers/safe_outputs.rs @@ -9,8 +9,8 @@ use std::path::{Path, PathBuf}; use tokio::fs; use crate::audit::model::{ - CreatedItemReport, Finding, RejectedSafeOutputsRollup, SafeOutputExecution, - SafeOutputExecutionItem, SafeOutputStatus, SafeOutputSummary, Severity, + AwInfo, ComponentProvenance, CreatedItemReport, Finding, RejectedSafeOutputsRollup, + SafeOutputExecution, SafeOutputExecutionItem, SafeOutputStatus, SafeOutputSummary, Severity, }; use crate::ndjson::{EXECUTED_NDJSON_FILENAME, SAFE_OUTPUT_FILENAME, read_ndjson_file}; @@ -41,8 +41,19 @@ struct ExecutionRecord { name: String, status: String, context: Option, + proposal_index: Option, result: Option, error: Option, + component: Option, +} + +#[derive(Debug, Clone, Default, Deserialize)] +#[serde(default)] +struct ExecutionComponentProvenance { + source: Option, + sha: Option, + manifest_digest: Option, + schema_digest: Option, } #[derive(Debug, Clone)] @@ -70,6 +81,7 @@ pub async fn analyze_safe_outputs( let proposals = load_proposals(proposals_path.as_deref()).await?; let detection = load_detection_verdict(detection_path.as_deref()).await?; let executions = load_execution_records(&executions_paths).await?; + let marker_components = load_marker_custom_components(download_root).await?; let detection_gate_fired = detection.as_ref().is_some_and(DetectionVerdict::gate_fired); let items = if detection_gate_fired { @@ -132,7 +144,7 @@ pub async fn analyze_safe_outputs( }) .unwrap_or_default(); let rollup = build_rollup(summary.as_ref(), execution.as_ref(), detection.as_ref()); - let findings = if detection_gate_fired && proposed_count > 0 { + let mut findings = if detection_gate_fired && proposed_count > 0 { vec![build_detection_finding( detection.as_ref().expect("gate-fired verdict"), proposed_count, @@ -140,6 +152,9 @@ pub async fn analyze_safe_outputs( } else { Vec::new() }; + if let Some(marker_components) = marker_components.as_deref() { + findings.extend(build_provenance_findings(&executions, marker_components)); + } Ok(SafeOutputAnalysis { summary, @@ -201,9 +216,7 @@ async fn load_detection_verdict(path: Option<&Path>) -> anyhow::Result anyhow::Result> { +async fn load_execution_records(paths: &[PathBuf]) -> anyhow::Result> { let mut records = Vec::new(); for path in paths { let values = read_ndjson_file(path).await?; @@ -229,6 +242,58 @@ async fn load_execution_records( Ok(records) } +async fn load_marker_custom_components( + download_root: &Path, +) -> anyhow::Result>> { + // Phase F1 mirrors the compile-time `# ado-aw-metadata` marker into + // staging/aw_info.json, so audit can cross-check provenance locally without + // fetching component sources from the network. + for directory in top_level_dirs_with_prefix(download_root, "agent_outputs_").await? { + for candidate in [ + directory.join("staging").join("aw_info.json"), + directory.join("aw_info.json"), + ] { + if !fs::metadata(&candidate) + .await + .map(|m| m.is_file()) + .unwrap_or(false) + { + continue; + } + let contents = tokio::fs::read_to_string(&candidate) + .await + .with_context(|| format!("Failed to read aw_info file: {}", candidate.display()))?; + let aw_info = serde_json::from_str::(&contents).with_context(|| { + format!("Failed to parse aw_info file: {}", candidate.display()) + })?; + return Ok(Some(aw_info.custom_components)); + } + } + Ok(None) +} + +impl ExecutionComponentProvenance { + fn to_model(&self) -> ComponentProvenance { + ComponentProvenance { + tool: String::new(), + source: normalize_optional_string(self.source.clone()).unwrap_or_default(), + sha: normalize_optional_string(self.sha.clone()).unwrap_or_default(), + manifest_digest: normalize_optional_string(self.manifest_digest.clone()) + .unwrap_or_default(), + schema_digest: normalize_optional_string(self.schema_digest.clone()) + .unwrap_or_default(), + } + } +} + +impl ExecutionRecord { + fn component_provenance(&self) -> Option { + self.component + .as_ref() + .map(ExecutionComponentProvenance::to_model) + } +} + fn build_execution_items( proposals: &[ProposalRecord], executions: &[IndexedExecutionRecord], @@ -236,15 +301,20 @@ fn build_execution_items( let mut proposal_to_execution = vec![None; proposals.len()]; let mut execution_matched = vec![false; executions.len()]; let mut context_index = BTreeMap::<(String, String), VecDeque>::new(); + let mut context_free_index = BTreeMap::>::new(); for proposal in proposals { - let Some(context) = proposal.context.clone() else { - continue; - }; - context_index - .entry((proposal.name.clone(), context)) - .or_default() - .push_back(proposal.index); + if let Some(context) = proposal.context.clone() { + context_index + .entry((proposal.name.clone(), context)) + .or_default() + .push_back(proposal.index); + } else { + context_free_index + .entry(proposal.name.clone()) + .or_default() + .push_back(proposal.index); + } } for execution in executions { @@ -270,15 +340,29 @@ fn build_execution_items( continue; } - let Some(proposal) = proposals.get(execution.index) else { - continue; - }; - if proposal_to_execution[proposal.index].is_some() { + if let Some(proposal_index) = execution.record.proposal_index { + let Some(proposal) = proposals.get(proposal_index) else { + continue; + }; + if proposal_to_execution[proposal.index].is_none() + && proposal.context.is_none() + && proposal.name == execution.record.name + { + proposal_to_execution[proposal.index] = Some(execution.index); + execution_matched[execution.index] = true; + } continue; } - if proposal.context.is_none() && proposal.name == execution.record.name { - proposal_to_execution[proposal.index] = Some(execution.index); - execution_matched[execution.index] = true; + + let Some(proposal_indexes) = context_free_index.get_mut(&execution.record.name) else { + continue; + }; + while let Some(proposal_index) = proposal_indexes.pop_front() { + if proposal_to_execution[proposal_index].is_none() { + proposal_to_execution[proposal_index] = Some(execution.index); + execution_matched[execution.index] = true; + break; + } } } @@ -323,6 +407,7 @@ fn build_item_from_execution( proposal: proposal.proposal.clone(), error: error.clone(), result: execution.result.clone(), + component_provenance: execution.component_provenance(), rejection_reason: rejection_reason_for_status(status, error), applies_to_whole_batch: false, } @@ -337,6 +422,7 @@ fn build_missing_execution_item(proposal: &ProposalRecord) -> SafeOutputExecutio proposal: proposal.proposal.clone(), error: error.clone(), result: None, + component_provenance: None, rejection_reason: error, applies_to_whole_batch: false, } @@ -352,6 +438,7 @@ fn build_unmatched_execution_item(execution: &ExecutionRecord) -> SafeOutputExec proposal: Value::Null, error: error.clone(), result: execution.result.clone(), + component_provenance: execution.component_provenance(), rejection_reason: rejection_reason_for_status(status, error), applies_to_whole_batch: false, } @@ -368,6 +455,7 @@ fn build_gate_rejected_item( proposal: proposal.proposal.clone(), error: None, result: None, + component_provenance: None, rejection_reason: Some(aggregate_reason_key(detection)), applies_to_whole_batch: true, } @@ -470,6 +558,77 @@ fn build_detection_finding(detection: &DetectionVerdict, proposed_count: u64) -> } } +fn build_provenance_findings( + executions: &[IndexedExecutionRecord], + marker_components: &[ComponentProvenance], +) -> Vec { + executions + .iter() + .filter_map(|execution| { + let component = execution.record.component_provenance(); + let marker_declares_tool = marker_components + .iter() + .any(|marker| marker.tool == execution.record.name); + let legacy_marker_matches_source = component.as_ref().is_some_and(|runtime| { + !runtime.source.is_empty() + && marker_components + .iter() + .any(|marker| marker.tool.is_empty() && marker.source == runtime.source) + }); + if !marker_declares_tool && !legacy_marker_matches_source { + return None; + } + let component = component.unwrap_or_default(); + cross_check_provenance(&execution.record.name, &component, marker_components) + }) + .collect() +} + +fn cross_check_provenance( + tool: &str, + record_component: &ComponentProvenance, + marker_components: &[ComponentProvenance], +) -> Option { + let Some(expected) = marker_components.iter().find(|component| { + (component.tool.is_empty() || component.tool == tool) + && (record_component.source.is_empty() || component.source == record_component.source) + }) else { + return Some( + crate::audit::findings::custom_component_provenance_mismatch_finding( + tool, + &record_component.source, + None, + record_component, + &["source"], + ), + ); + }; + + let mut mismatched_fields = Vec::new(); + if expected.source != record_component.source { + mismatched_fields.push("source"); + } + if expected.sha != record_component.sha { + mismatched_fields.push("sha"); + } + if expected.manifest_digest != record_component.manifest_digest { + mismatched_fields.push("manifest_digest"); + } + if expected.schema_digest != record_component.schema_digest { + mismatched_fields.push("schema_digest"); + } + + (!mismatched_fields.is_empty()).then(|| { + crate::audit::findings::custom_component_provenance_mismatch_finding( + tool, + &record_component.source, + Some(expected), + record_component, + &mismatched_fields, + ) + }) +} + fn created_item_from_execution_item(item: &SafeOutputExecutionItem) -> Option { if item.status != SafeOutputStatus::Executed { return None; @@ -661,8 +820,10 @@ fn collect_named_files<'a>( #[cfg(test)] mod tests { use super::{ - CreatedItemReport, EXECUTED_NDJSON_FILENAME, SAFE_OUTPUT_FILENAME, SafeOutputStatus, - Severity, analyze_safe_outputs, + ComponentProvenance, CreatedItemReport, EXECUTED_NDJSON_FILENAME, + ExecutionComponentProvenance, ExecutionRecord, IndexedExecutionRecord, ProposalRecord, + SAFE_OUTPUT_FILENAME, SafeOutputStatus, Severity, analyze_safe_outputs, + build_execution_items, build_provenance_findings, cross_check_provenance, }; use serde_json::{Value, json}; use std::fs; @@ -727,10 +888,284 @@ mod tests { .iter() .all(|item| item.status == SafeOutputStatus::Executed) ); + assert!( + execution + .items + .iter() + .all(|item| item.component_provenance.is_none()), + "built-in records must not gain custom component provenance" + ); assert!(analysis.rollup.is_none()); assert!(analysis.findings.is_empty()); } + #[tokio::test] + async fn custom_execution_record_populates_component_provenance() { + let temp_dir = TempDir::new().expect("create temp dir"); + write_ndjson( + &temp_dir + .path() + .join("agent_outputs_42") + .join("staging") + .join(SAFE_OUTPUT_FILENAME), + &[json!({"name": "custom_notify", "context": "custom-1"})], + ); + write_json( + &temp_dir + .path() + .join("agent_outputs_42") + .join("staging") + .join("aw_info.json"), + &json!({ + "schema": "ado-aw/aw_info/1", + "custom_components": [{ + "tool": "custom_notify", + "source": "org/repo/components/notify", + "sha": "0123456789abcdef0123456789abcdef01234567", + "manifest_digest": "sha256:manifest", + "schema_digest": "sha256:schema" + }] + }), + ); + write_ndjson( + &temp_dir + .path() + .join("safe_outputs") + .join(EXECUTED_NDJSON_FILENAME), + &[json!({ + "schema_version": 1, + "tool": "custom_notify", + "proposal_id": "custom-1", + "proposal_index": 0, + "name": "custom_notify", + "status": "succeeded", + "context": "custom-1", + "message": "ok", + "component": { + "source": "org/repo/components/notify", + "sha": "0123456789abcdef0123456789abcdef01234567", + "manifest_digest": "sha256:manifest", + "schema_digest": "sha256:schema" + }, + "attempt": { + "number": 1, + "staged": false, + "started_at": "2026-07-13T00:00:00Z", + "ended_at": "2026-07-13T00:00:01Z" + }, + "result": {"status": "ok"} + })], + ); + + let analysis = analyze_safe_outputs(temp_dir.path()) + .await + .expect("analyze custom safe output"); + + let execution = analysis.execution.expect("execution"); + assert_eq!(execution.items.len(), 1); + assert_eq!( + execution.items[0].component_provenance, + Some(ComponentProvenance { + tool: String::new(), + source: String::from("org/repo/components/notify"), + sha: String::from("0123456789abcdef0123456789abcdef01234567"), + manifest_digest: String::from("sha256:manifest"), + schema_digest: String::from("sha256:schema"), + }) + ); + assert!( + analysis.findings.is_empty(), + "matching marker/runtime provenance should not emit findings" + ); + } + + #[test] + fn cross_check_provenance_accepts_matching_marker_component() { + let component = ComponentProvenance { + tool: String::from("custom_notify"), + source: String::from("org/repo/components/notify"), + sha: String::from("sha-a"), + manifest_digest: String::from("manifest-a"), + schema_digest: String::from("schema-a"), + }; + + assert!( + cross_check_provenance( + "custom_notify", + &component, + std::slice::from_ref(&component) + ) + .is_none() + ); + } + + #[test] + fn cross_check_provenance_reports_mismatched_sha() { + let marker = ComponentProvenance { + tool: String::from("custom_notify"), + source: String::from("org/repo/components/notify"), + sha: String::from("sha-expected"), + manifest_digest: String::from("manifest-a"), + schema_digest: String::from("schema-a"), + }; + let runtime = ComponentProvenance { + tool: String::new(), + source: String::from("org/repo/components/notify"), + sha: String::from("sha-actual"), + manifest_digest: String::from("manifest-a"), + schema_digest: String::from("schema-a"), + }; + + let finding = cross_check_provenance("custom_notify", &runtime, &[marker]) + .expect("mismatched sha should emit finding"); + assert_eq!(finding.severity, Severity::High); + assert!(finding.title.contains("custom_notify")); + assert!(finding.description.contains("sha-expected")); + assert!(finding.description.contains("sha-actual")); + } + + #[test] + fn cross_check_provenance_matches_tool_when_components_share_source() { + let runtime = ComponentProvenance { + tool: String::new(), + source: String::from("org/repo/components/shared"), + sha: String::from("sha-notify"), + manifest_digest: String::from("manifest"), + schema_digest: String::from("schema-notify"), + }; + let markers = vec![ + ComponentProvenance { + tool: String::from("other-tool"), + source: runtime.source.clone(), + sha: String::from("sha-other"), + manifest_digest: runtime.manifest_digest.clone(), + schema_digest: String::from("schema-other"), + }, + ComponentProvenance { + tool: String::from("custom_notify"), + source: runtime.source.clone(), + sha: runtime.sha.clone(), + manifest_digest: runtime.manifest_digest.clone(), + schema_digest: runtime.schema_digest.clone(), + }, + ]; + + assert!(cross_check_provenance("custom_notify", &runtime, &markers).is_none()); + } + + #[test] + fn provenance_findings_reject_missing_runtime_component_for_marker_tool() { + let executions = vec![IndexedExecutionRecord { + index: 0, + record: ExecutionRecord { + name: String::from("custom_notify"), + status: String::from("succeeded"), + component: None, + ..ExecutionRecord::default() + }, + }]; + let markers = vec![ComponentProvenance { + tool: String::from("custom_notify"), + source: String::from("org/repo/components/notify"), + sha: String::from("sha"), + manifest_digest: String::from("manifest"), + schema_digest: String::from("schema"), + }]; + + let findings = build_provenance_findings(&executions, &markers); + assert_eq!(findings.len(), 1); + assert_eq!(findings[0].severity, Severity::High); + assert!(findings[0].description.contains("source")); + } + + #[test] + fn provenance_findings_reject_partial_runtime_component() { + let executions = vec![IndexedExecutionRecord { + index: 0, + record: ExecutionRecord { + name: String::from("custom_notify"), + status: String::from("succeeded"), + component: Some(ExecutionComponentProvenance { + source: Some(String::from("org/repo/components/notify")), + sha: None, + manifest_digest: Some(String::from("manifest")), + schema_digest: Some(String::from("schema")), + }), + ..ExecutionRecord::default() + }, + }]; + let markers = vec![ComponentProvenance { + tool: String::from("custom_notify"), + source: String::from("org/repo/components/notify"), + sha: String::from("sha"), + manifest_digest: String::from("manifest"), + schema_digest: String::from("schema"), + }]; + + let findings = build_provenance_findings(&executions, &markers); + assert_eq!(findings.len(), 1); + assert!(findings[0].description.contains("sha")); + } + + #[test] + fn provenance_findings_ignore_local_custom_tools_without_marker_component() { + let executions = vec![IndexedExecutionRecord { + index: 0, + record: ExecutionRecord { + name: String::from("local_custom"), + status: String::from("succeeded"), + component: Some(ExecutionComponentProvenance::default()), + ..ExecutionRecord::default() + }, + }]; + + assert!(build_provenance_findings(&executions, &[]).is_empty()); + } + + #[test] + fn split_artifacts_match_custom_index_and_legacy_tool_fifo() { + let proposals = vec![ + ProposalRecord { + index: 0, + name: String::from("noop"), + context: None, + proposal: json!({"name": "noop"}), + }, + ProposalRecord { + index: 1, + name: String::from("custom-tool"), + context: None, + proposal: json!({"name": "custom-tool"}), + }, + ]; + let executions = vec![ + IndexedExecutionRecord { + index: 0, + record: ExecutionRecord { + name: String::from("custom-tool"), + status: String::from("succeeded"), + proposal_index: Some(1), + result: Some(json!({"matched": "custom"})), + ..ExecutionRecord::default() + }, + }, + IndexedExecutionRecord { + index: 1, + record: ExecutionRecord { + name: String::from("noop"), + status: String::from("succeeded"), + proposal_index: None, + result: Some(json!({"matched": "builtin"})), + ..ExecutionRecord::default() + }, + }, + ]; + + let items = build_execution_items(&proposals, &executions); + assert_eq!(items[0].result, Some(json!({"matched": "builtin"}))); + assert_eq!(items[1].result, Some(json!({"matched": "custom"}))); + } + #[tokio::test] async fn execution_records_aggregate_across_split_artifacts() { // Manual-review split: automatic outputs land in `safe_outputs/`, @@ -753,14 +1188,18 @@ mod tests { .path() .join("safe_outputs") .join(EXECUTED_NDJSON_FILENAME), - &[json!({"name": "add_pr_comment", "status": "succeeded", "context": "c-1", "result": {"status": "ok"}})], + &[ + json!({"name": "add_pr_comment", "status": "succeeded", "context": "c-1", "result": {"status": "ok"}}), + ], ); write_ndjson( &temp_dir .path() .join("safe_outputs_reviewed") .join(EXECUTED_NDJSON_FILENAME), - &[json!({"name": "create_pull_request", "status": "succeeded", "context": "pr-1", "result": {"number": 9}})], + &[ + json!({"name": "create_pull_request", "status": "succeeded", "context": "pr-1", "result": {"number": 9}}), + ], ); let analysis = analyze_safe_outputs(temp_dir.path()) diff --git a/src/audit/cli.rs b/src/audit/cli.rs index 05617a7b..383243fb 100644 --- a/src/audit/cli.rs +++ b/src/audit/cli.rs @@ -758,6 +758,8 @@ fn artifact_name_to_prefix(name: &str) -> Option<&'static str> { Some("analyzed_outputs") } else if name == "safe_outputs" || name.starts_with("safe_outputs_") { Some("safe_outputs") + } else if name.starts_with("custom_safe_output_") { + Some("safe_outputs") } else { None } @@ -1041,6 +1043,10 @@ mod tests { assert!(artifact_matches_selected("agent_outputs_42", normalized)); assert!(artifact_matches_selected("analyzed_outputs_42", normalized)); assert!(artifact_matches_selected("safe_outputs", normalized)); + assert!(artifact_matches_selected( + "custom_safe_output_notify_42", + normalized + )); let agent_only = vec![String::from("agent")]; let agent_only = @@ -1052,6 +1058,18 @@ mod tests { agent_only )); assert!(!artifact_matches_selected("safe_outputs", agent_only)); + assert!(!artifact_matches_selected( + "custom_safe_output_notify_42", + agent_only + )); + + let safe_outputs_only = vec![String::from("safe-outputs")]; + let safe_outputs_only = normalize_artifact_filters(Some(&safe_outputs_only)) + .expect("normalize safe-output filter"); + assert!(artifact_matches_selected( + "custom_safe_output_notify_42", + safe_outputs_only.as_deref() + )); } // ── validate_audit_url_host: PAT exfiltration guard ─────────────────── diff --git a/src/audit/findings.rs b/src/audit/findings.rs index 4f50ebbc..2986f88b 100644 --- a/src/audit/findings.rs +++ b/src/audit/findings.rs @@ -1,6 +1,46 @@ use std::collections::BTreeMap; -use crate::audit::model::{AuditData, Finding, JobData, Recommendation, Severity}; +use crate::audit::model::{ + AuditData, ComponentProvenance, Finding, JobData, Recommendation, Severity, +}; + +/// Build a finding for a custom safe-output component whose runtime +/// provenance disagrees with the compile-time ado-aw metadata marker. +pub fn custom_component_provenance_mismatch_finding( + tool: &str, + source: &str, + expected: Option<&ComponentProvenance>, + actual: &ComponentProvenance, + mismatched_fields: &[&str], +) -> Finding { + let expected_text = expected.map_or_else( + || String::from("no compile-time marker entry"), + |expected| { + format!( + "sha={}, manifest_digest={}, schema_digest={}", + expected.sha, expected.manifest_digest, expected.schema_digest + ) + }, + ); + let fields = if mismatched_fields.is_empty() { + String::from("source") + } else { + mismatched_fields.join(", ") + }; + + Finding { + category: String::from("safe_outputs"), + severity: Severity::High, + title: format!("Custom component provenance mismatch for {tool}"), + description: format!( + "Runtime provenance for custom safe-output tool '{tool}' from '{source}' does not match the compile-time ado-aw metadata marker ({fields}). Expected: {expected_text}. Actual: sha={}, manifest_digest={}, schema_digest={}.", + actual.sha, actual.manifest_digest, actual.schema_digest + ), + impact: Some(String::from( + "A custom safe-output executor may have run different component code or schema than the compiled workflow declared.", + )), + } +} /// Aggregate findings + recommendations from every populated section /// of `AuditData`. Pure function; does not mutate the input. diff --git a/src/audit/model.rs b/src/audit/model.rs index 07b96931..ee305007 100644 --- a/src/audit/model.rs +++ b/src/audit/model.rs @@ -176,6 +176,30 @@ pub struct AwInfo { /// Compiler version that produced the workflow. #[serde(skip_serializing_if = "Option::is_none")] pub compiler_version: Option, + /// Compile-time custom component provenance emitted by ado-aw metadata. + #[serde(default, skip_serializing_if = "Vec::is_empty")] + pub custom_components: Vec, +} + +/// Custom safe-output component provenance captured at compile and execution time. +#[derive(Debug, Clone, Default, PartialEq, Eq, Serialize, Deserialize)] +#[serde(default)] +pub struct ComponentProvenance { + /// Custom safe-output tool associated with this component. + #[serde(default, skip_serializing_if = "String::is_empty")] + pub tool: String, + /// Import source, for example `org/repo/path`. + #[serde(default, skip_serializing_if = "String::is_empty")] + pub source: String, + /// Resolved component commit SHA. + #[serde(default, skip_serializing_if = "String::is_empty")] + pub sha: String, + /// SHA-256 digest of the component manifest. + #[serde(default, skip_serializing_if = "String::is_empty")] + pub manifest_digest: String, + /// SHA-256 digest of the component schema. + #[serde(default, skip_serializing_if = "String::is_empty")] + pub schema_digest: String, } /// Aggregate numeric metrics for the audited run. @@ -755,6 +779,9 @@ pub struct SafeOutputExecutionItem { /// Optional execution result payload. #[serde(skip_serializing_if = "Option::is_none")] pub result: Option, + /// Custom component provenance for custom safe-output tools. + #[serde(skip_serializing_if = "Option::is_none")] + pub component_provenance: Option, /// Optional rejection reason emitted by detection or execution. #[serde(skip_serializing_if = "Option::is_none")] pub rejection_reason: Option, @@ -919,6 +946,7 @@ mod tests { source: Some(String::from("agents/security-scan.md")), target: Some(String::from("standalone")), compiler_version: Some(String::from("0.30.2")), + custom_components: Vec::new(), }), }, task_domain: Some(TaskDomainInfo { @@ -982,6 +1010,7 @@ mod tests { proposal: json!({"title": "Fix pipeline", "repository": "repo"}), error: Some(String::from("Batch blocked by detection gate")), result: Some(json!({"status": "blocked"})), + component_provenance: None, rejection_reason: Some(String::from("prompt_injection")), applies_to_whole_batch: true, }], @@ -1139,6 +1168,40 @@ mod tests { assert_eq!(keys_sorted, vec!["downloaded_files", "metrics", "overview"]); } + #[test] + fn safe_output_execution_item_serializes_component_provenance_only_when_present() { + let without_component = SafeOutputExecutionItem { + tool: String::from("noop"), + status: SafeOutputStatus::Executed, + proposal: json!({"name": "noop"}), + ..Default::default() + }; + let value = serde_json::to_value(&without_component).expect("serialize without component"); + assert!( + value.get("component_provenance").is_none(), + "None provenance must not change built-in audit JSON" + ); + + let with_component = SafeOutputExecutionItem { + tool: String::from("custom_notify"), + status: SafeOutputStatus::Executed, + proposal: json!({"message": "done"}), + component_provenance: Some(ComponentProvenance { + tool: String::new(), + source: String::from("org/repo/components/notify"), + sha: String::from("0123456789abcdef0123456789abcdef01234567"), + manifest_digest: String::from("sha256:manifest"), + schema_digest: String::from("sha256:schema"), + }), + ..Default::default() + }; + let json = serde_json::to_string(&with_component).expect("serialize with component"); + assert!(json.contains("component_provenance")); + let round_tripped: SafeOutputExecutionItem = + serde_json::from_str(&json).expect("deserialize with component"); + assert_eq!(round_tripped, with_component); + } + #[test] fn matches_ir_id_accepts_bare_and_single_level_qualified_names() { let bare = JobData { diff --git a/src/audit/render/console.rs b/src/audit/render/console.rs index 76f37cd5..941d9a3a 100644 --- a/src/audit/render/console.rs +++ b/src/audit/render/console.rs @@ -1171,6 +1171,7 @@ By threat: source: Some("agents/my-agent.md".to_string()), target: Some("standalone".to_string()), compiler_version: Some("0.30.2".to_string()), + custom_components: Vec::new(), }), }, task_domain: Some(TaskDomainInfo { @@ -1235,6 +1236,7 @@ By threat: proposal: json!({"title": "Fix bug"}), error: Some("Blocked by detection gate".to_string()), result: Some(json!({"status": "blocked"})), + component_provenance: None, rejection_reason: Some("prompt_injection".to_string()), applies_to_whole_batch: true, }], diff --git a/src/audit/render/json.rs b/src/audit/render/json.rs index 24b279d7..09b3e229 100644 --- a/src/audit/render/json.rs +++ b/src/audit/render/json.rs @@ -65,6 +65,7 @@ mod tests { source: Some(String::from("agents/security-scan.md")), target: Some(String::from("standalone")), compiler_version: Some(String::from("0.30.2")), + custom_components: Vec::new(), }), }, task_domain: Some(TaskDomainInfo { @@ -128,6 +129,7 @@ mod tests { proposal: json!({"title": "Fix pipeline", "repository": "repo"}), error: Some(String::from("Batch blocked by detection gate")), result: Some(json!({"status": "blocked"})), + component_provenance: None, rejection_reason: Some(String::from("prompt_injection")), applies_to_whole_batch: true, }], diff --git a/src/compile/ado_bundle.rs b/src/compile/ado_bundle.rs index aea2ae6a..1659b461 100644 --- a/src/compile/ado_bundle.rs +++ b/src/compile/ado_bundle.rs @@ -63,6 +63,12 @@ pub enum Bundle { /// containerized SafeOutputs MCP server can compute a diff base on /// shallow-default pools. PreparePrBase, + /// SHA-pinned component checkout for custom safe-output jobs (#1473). Runs + /// in the isolated custom safe-output job after the component repository + /// resource is checked out. Fetches the pinned commit using either the ADO + /// bearer (same-org Azure Repos) or credentials persisted by the repository + /// checkout (GitHub/GHE/cross-org), then verifies HEAD equals the pin. + CheckoutComponent, } /// The auth contract a bundle requires from the step that invokes it. @@ -77,6 +83,9 @@ pub enum BundleAuth { /// URI, by contrast, comes from the auto-injected `SYSTEM_COLLECTIONURI` /// and is therefore *not* part of this contract (see #1307). Bearer, + /// The bundle can use a projected ADO bearer, but may instead rely on + /// credentials persisted by a repository-resource checkout. + OptionalBearer, /// The bundle needs no bearer (pure filesystem / git-without-auth / argv). None, } @@ -142,6 +151,7 @@ impl Bundle { Bundle::Conclusion, Bundle::GithubAppToken, Bundle::PreparePrBase, + Bundle::CheckoutComponent, ]; /// The bundle's unpacked on-disk path inside the runtime VM. The Conclusion @@ -168,6 +178,7 @@ impl Bundle { Bundle::Conclusion => paths::CONCLUSION_PATH, Bundle::GithubAppToken => paths::GITHUB_APP_TOKEN_PATH, Bundle::PreparePrBase => paths::PREPARE_PR_BASE_PATH, + Bundle::CheckoutComponent => paths::CHECKOUT_COMPONENT_PATH, } } @@ -186,7 +197,11 @@ impl Bundle { | Bundle::ExecContextSchedule | Bundle::Conclusion // Fetches/deepens the target branch over the ADO bearer (bearerEnv). - | Bundle::PreparePrBase => BundleAuth::Bearer, + | Bundle::PreparePrBase + => BundleAuth::Bearer, + // Same-org Azure Repos uses the ADO bearer; external repository + // resources reuse credentials persisted by their checkout. + Bundle::CheckoutComponent => BundleAuth::OptionalBearer, // Pure filesystem / git-without-auth / argv — no bearer. Bundle::Import | Bundle::ExecContextManual @@ -208,10 +223,23 @@ impl Bundle { /// guarantee that every bearer-requiring bundle step carries a token — the /// structural fix for the class of bug behind #1307. pub fn apply_bundle_auth(step: BashStep, bundle: Bundle, token: TokenSource) -> BashStep { + apply_bundle_auth_optional(step, bundle, Some(token)) +} + +pub fn apply_bundle_auth_optional( + step: BashStep, + bundle: Bundle, + token: Option, +) -> BashStep { match bundle.auth() { - BundleAuth::Bearer => { - step.with_env("SYSTEM_ACCESSTOKEN", EnvValue::secret(token.variable())) - } + BundleAuth::Bearer => step.with_env( + "SYSTEM_ACCESSTOKEN", + EnvValue::secret(token.expect("required bundle bearer").variable()), + ), + BundleAuth::OptionalBearer => match token { + Some(token) => step.with_env("SYSTEM_ACCESSTOKEN", EnvValue::secret(token.variable())), + None => step, + }, BundleAuth::None => step, } } @@ -254,8 +282,7 @@ mod tests { fn every_bundle_path_is_under_the_unpack_dir() { for b in Bundle::ALL { assert!( - b.path() - .starts_with("/tmp/ado-aw-scripts/ado-script/"), + b.path().starts_with("/tmp/ado-aw-scripts/ado-script/"), "{b:?} path must live under the unzip destination" ); assert!(b.path().ends_with(".js"), "{b:?} path must be a .js bundle"); @@ -269,9 +296,9 @@ mod tests { let out = apply_bundle_auth(step, *b, TokenSource::SystemAccessToken); let has_token = out.env.contains_key("SYSTEM_ACCESSTOKEN"); match b.auth() { - BundleAuth::Bearer => assert!( + BundleAuth::Bearer | BundleAuth::OptionalBearer => assert!( has_token, - "{b:?} is Bearer and must carry SYSTEM_ACCESSTOKEN" + "{b:?} accepts a bearer and apply_bundle_auth must project it" ), BundleAuth::None => assert!( !has_token, @@ -281,6 +308,16 @@ mod tests { } } + #[test] + fn optional_bundle_auth_can_rely_on_persisted_credentials() { + let step = apply_bundle_auth_optional( + BashStep::new("t", "node x\n"), + Bundle::CheckoutComponent, + None, + ); + assert!(!step.env.contains_key("SYSTEM_ACCESSTOKEN")); + } + #[test] fn token_source_selection_matches_write_service_connection() { assert_eq!(token_source_for(None), TokenSource::SystemAccessToken); @@ -289,10 +326,7 @@ mod tests { token_source_for(Some("my-sc")), TokenSource::WriteServiceConnection ); - assert_eq!( - token_source_for(Some("my-sc")).variable(), - "SC_WRITE_TOKEN" - ); + assert_eq!(token_source_for(Some("my-sc")).variable(), "SC_WRITE_TOKEN"); } #[test] diff --git a/src/compile/agentic_pipeline.rs b/src/compile/agentic_pipeline.rs index 0a978f35..0ecd45fe 100644 --- a/src/compile/agentic_pipeline.rs +++ b/src/compile/agentic_pipeline.rs @@ -60,7 +60,7 @@ //! - `Teardown` (optional): user `teardown:` steps. //! - `Conclusion` (optional): post-run reporting / work-item filing. -use anyhow::Result; +use anyhow::{Context, Result}; use std::path::Path; use super::common::PerJobPools; @@ -68,6 +68,10 @@ use super::common::{ self, ADO_BUILD_ID_SUFFIX, AWF_VERSION, HEADER_MARKER, MCPG_CONTAINER_NAME, MCPG_DOMAIN, MCPG_IMAGE, MCPG_PORT, MCPG_VERSION, image_ref, }; +use super::custom_tools::{ + CustomComponentDefinition, CustomToolDefinition, CustomToolKind, + collect_custom_tool_definitions, +}; use super::extensions::{CompileContext, CompilerExtension, Declarations, Extension, McpgConfig}; use super::ir::condition::{Condition, Expr}; use super::ir::env::EnvValue; @@ -98,6 +102,7 @@ use super::types::{ /// The `safe-outputs:` key for the create-pull-request tool. Matches the kebab /// name `FrontMatter::create_pr_config`/`partition_safe_outputs_by_approval` use. const CREATE_PULL_REQUEST_TOOL: &str = "create-pull-request"; +const CUSTOM_PROPOSALS_STEP_ID: &str = "customProposals"; /// Built pipeline context — the result of running every validation, /// scalar computation, extension declaration fanout, and canonical- @@ -243,6 +248,15 @@ pub(crate) fn build_pipeline_context( if !agent_env.is_empty() { engine_env = format!("{engine_env}\n{agent_env}"); } + let agent_ado_env = common::generate_agent_ado_env( + front_matter + .permissions + .as_ref() + .and_then(|permissions| permissions.read.as_deref()), + ); + if !agent_ado_env.is_empty() { + engine_env = format!("{engine_env}\n{agent_ado_env}"); + } // AWF mounts + allowlist let allowed_domains = @@ -250,7 +264,15 @@ pub(crate) fn build_pipeline_context( let awf_mounts = common::generate_awf_mounts(extensions, &extension_declarations); let awf_path_step_yaml = common::generate_awf_path_step(&awf_paths); - // MCPG config + // MCPG config + compiler-generated dynamic SafeOutputs tool definitions. + let custom_tool_schemas = super::custom_tools::generate_custom_tool_schemas(front_matter)?; + let custom_tools_json = if custom_tool_schemas.is_empty() { + None + } else { + Some(super::custom_tools::custom_tools_json( + &custom_tool_schemas, + )?) + }; let mcpg_config_obj = common::generate_mcpg_config(front_matter, &extension_declarations)?; let mcpg_config_json = serde_json::to_string_pretty(&mcpg_config_obj) .map_err(|e| anyhow::anyhow!("Failed to serialize MCPG config: {e}"))?; @@ -303,13 +325,24 @@ pub(crate) fn build_pipeline_context( front_matter, input_path, markdown_body, + &ctx.imported_prompt_body, &source_path, &trigger_repo_directory, )?; // ─── Top-level pipeline fields ──────────────────────────────── let parameters = build_parameters(front_matter)?; - let resources = build_resources(&front_matter.repositories, &front_matter.on_config); + let resources = build_resources( + front_matter, + &front_matter.repositories, + &front_matter.on_config, + )?; + // P7: template targets (job/stage) can't own top-level repository resources, + // so a remote custom-component import must be declared + authorized by the + // parent pipeline. Surface that requirement (no-op for standalone/1es). + if let Some(diagnostic) = custom_import_parent_diagnostic(front_matter)? { + eprintln!("Warning: {diagnostic}"); + } let triggers = build_triggers(&front_matter.on_config, front_matter)?; // ─── Extension declaration fanout ───────────────────────────── @@ -353,6 +386,7 @@ pub(crate) fn build_pipeline_context( awf_mounts, awf_path_step_yaml, mcpg_config_json, + custom_tools_json, mcpg_docker_env, mcpg_step_env, source_path, @@ -428,32 +462,56 @@ pub(crate) fn build_canonical_jobs( if let Some(review) = build_manual_review_job(front_matter, cfg, &p)? { jobs.push(review); } + let custom_defs = collect_custom_safe_output_job_defs(front_matter, &p)?; + let custom_job_ids: Vec = custom_defs.iter().map(|d| d.job_id.clone()).collect(); + let custom_reviewed_job_ids: Vec = custom_defs + .iter() + .filter(|d| d.reviewed) + .map(|d| d.job_id.clone()) + .collect(); + for def in &custom_defs { + jobs.push(build_custom_safe_output_job(def, front_matter, cfg)?); + } // Safe-outputs execution. With manual review, execution may split into an // automatic job (runs immediately) and a reviewed job (gated behind the // ManualReview approval). Partition decides the shape: // - no reviewed tools → single default job (unchanged) // - all reviewed tools → single default job, gated by ManualReview // - mixed (auto + reviewed) → auto job + reviewed job - let (auto, reviewed) = front_matter.partition_safe_outputs_by_approval(); + let (auto_all, reviewed_all) = front_matter.partition_safe_outputs_by_approval(); + let custom_tool_names = front_matter.custom_safe_output_tool_names(); + let custom_tool_set: std::collections::HashSet<&str> = + custom_tool_names.iter().map(String::as_str).collect(); + let auto: Vec = auto_all + .into_iter() + .filter(|tool| !custom_tool_set.contains(tool.as_str())) + .collect(); + let reviewed: Vec = reviewed_all + .into_iter() + .filter(|tool| !custom_tool_set.contains(tool.as_str())) + .collect(); // Which variant actually runs `create-pull-request` (and thus needs the // `prepare-pr-base` fetch/deepen — issue #1453). In a split it lives in // exactly one variant; the other filters it out, so only the running // variant should pay for the bundle download + prepare step. let create_pr_configured = front_matter.create_pr_config().is_some(); let create_pr_reviewed = reviewed.iter().any(|t| t == CREATE_PULL_REQUEST_TOOL); + let safeoutputs_waits_for_review = !reviewed.is_empty() && auto.is_empty(); if reviewed.is_empty() || auto.is_empty() { jobs.push(build_safeoutputs_job( front_matter, cfg, &p, - &SafeOutputsVariant::default_single(create_pr_configured), + &SafeOutputsVariant::default_single(create_pr_configured) + .with_excluded_tools(&custom_tool_names), )?); } else { jobs.push(build_safeoutputs_job( front_matter, cfg, &p, - &SafeOutputsVariant::automatic(&reviewed, create_pr_configured && !create_pr_reviewed), + &SafeOutputsVariant::automatic(&reviewed, create_pr_configured && !create_pr_reviewed) + .with_excluded_tools(&custom_tool_names), )?); jobs.push(build_safeoutputs_job( front_matter, @@ -471,7 +529,13 @@ pub(crate) fn build_canonical_jobs( // Wire dependsOn between jobs (graph pass also derives but // explicit edges make the YAML match committed lock files). - wire_explicit_dependencies(&mut jobs, &p)?; + wire_explicit_dependencies( + &mut jobs, + &p, + &custom_reviewed_job_ids, + &custom_job_ids, + safeoutputs_waits_for_review, + )?; Ok(jobs) } @@ -495,6 +559,14 @@ impl<'a> JobPrefix<'a> { _ => JobId::new(base), } } + + fn custom_id(&self, tool: &str) -> Result { + let base = format!("Custom_{}", ado_identifier_suffix(tool)); + match self.0 { + Some(prefix) => JobId::new(format!("{prefix}_{base}")), + None => JobId::new(base), + } + } } /// Aggregates the precomputed scalars + YAML fragments threaded into @@ -527,6 +599,10 @@ pub(crate) struct StandaloneCtx { /// `awf_path_step` YAML body (or empty when no path prepends). pub(crate) awf_path_step_yaml: String, pub(crate) mcpg_config_json: String, + /// Compiler-generated dynamic SafeOutputs tool definitions. When present, + /// the Agent job stages this beside the MCPG config and the hardened + /// SafeOutputs stdio container reads it through its `/safeoutputs` mount. + pub(crate) custom_tools_json: Option, /// `-e KEY=...` docker flags for MCPG. pub(crate) mcpg_docker_env: String, /// `env:` block for the MCPG step (`env:\n KEY: ...`). @@ -638,7 +714,11 @@ fn yaml_value_as_string(v: &serde_yaml::Value) -> String { } } -fn build_resources(repos: &[RepoCfg], on: &Option) -> Resources { +fn build_resources( + front_matter: &FrontMatter, + repos: &[RepoCfg], + on: &Option, +) -> Result { let mut repositories: Vec = vec![RepositoryResource::SelfRepo { clean: true, submodules: true, @@ -649,8 +729,10 @@ fn build_resources(repos: &[RepoCfg], on: &Option) -> Resources { kind: r.repo_type.clone(), name: r.name.clone(), r#ref: Some(r.repo_ref.clone()), + endpoint: r.endpoint.clone(), }); } + repositories.extend(custom_repository_resources(front_matter)?); // Pipeline-completion triggers surface as `resources.pipelines[]`. // Mirrors legacy `generate_pipeline_resources`. let mut pipelines: Vec = Vec::new(); @@ -675,10 +757,10 @@ fn build_resources(repos: &[RepoCfg], on: &Option) -> Resources { trigger: true, }); } - Resources { + Ok(Resources { repositories, pipelines, - } + }) } fn build_triggers(on: &Option, front_matter: &FrontMatter) -> Result { @@ -858,6 +940,7 @@ fn build_agent_job( fetch_depth: fetch.depth_for_emit(), fetch_tags: fetch.fetch_tags, persist_credentials: None, + path: None, })); } @@ -891,7 +974,10 @@ fn build_agent_job( )?; // 7. Prepare tooling (generates MCPG API key, writes MCPG config to staging) - steps.push(Step::Bash(prepare_mcpg_config_step(&cfg.mcpg_config_json))); + steps.push(Step::Bash(prepare_mcpg_config_step( + &cfg.mcpg_config_json, + cfg.custom_tools_json.as_deref(), + )?)); // 8. Prepare tooling - copy binary + config to /tmp steps.push(Step::Bash(prepare_tooling_step())); @@ -1252,6 +1338,13 @@ fn build_detection_job( &reviewed_tools, ))); } + let custom_tools = front_matter.custom_safe_output_tool_names(); + if !custom_tools.is_empty() { + steps.push(Step::Bash(detect_custom_proposals_step( + &cfg.working_directory, + &custom_tools, + )?)); + } // Copy logs steps.push(Step::Bash(copy_logs_step(&cfg.engine_log_dir, true))); // Publish @@ -1334,6 +1427,16 @@ impl SafeOutputsVariant { is_reviewed: true, } } + + fn with_excluded_tools(mut self, excluded: &[String]) -> Self { + if excluded.is_empty() { + return self; + } + let mut flags = self.filter_args; + flags.push_str(&filter_flags("--exclude", excluded)); + self.filter_args = flags; + self + } } /// Build a ` -- ` run for `ado-aw execute` (leading space so it @@ -1352,6 +1455,482 @@ fn filter_flags(flag: &str, tools: &[String]) -> String { s } +#[derive(Debug, Clone)] +struct CustomSafeOutputJobDef { + name: String, + job_id: JobId, + reviewed: bool, + max: usize, + env: Vec<(String, String)>, + kind: CustomToolKind, + component: Option, + schema_digest: String, +} + +fn ado_identifier_suffix(raw: &str) -> String { + let mut out = String::with_capacity(raw.len()); + for ch in raw.chars() { + if ch.is_ascii_alphanumeric() { + out.push(ch); + } else { + out.push('_'); + } + } + if out + .chars() + .next() + .is_some_and(|ch| ch.is_ascii_alphabetic() || ch == '_') + { + out + } else { + format!("_{out}") + } +} + +fn custom_tool_output_var(tool: &str) -> String { + format!("HasCustom_{}", ado_identifier_suffix(tool)) +} + +fn collect_custom_safe_output_job_defs( + front_matter: &FrontMatter, + prefix: &JobPrefix<'_>, +) -> Result> { + let (_, reviewed) = front_matter.partition_safe_outputs_by_approval(); + let reviewed: std::collections::HashSet<&str> = reviewed.iter().map(String::as_str).collect(); + collect_custom_tool_definitions(front_matter)? + .into_iter() + .map(|definition| custom_job_def(definition, &reviewed, prefix)) + .collect() +} + +fn custom_job_def( + definition: CustomToolDefinition, + reviewed: &std::collections::HashSet<&str>, + prefix: &JobPrefix<'_>, +) -> Result { + if let CustomToolKind::Jobs { steps } = &definition.kind { + for step in steps { + let step = serde_yaml::to_value(step) + .context("failed to convert custom job step for validation")?; + if let Some(Err(message)) = super::ir::tasks::parse::validate_task_step(&step) { + eprintln!( + "Warning: safe-outputs.jobs.{}.steps contains an invalid task input: {message}", + definition.name + ); + } + } + } + Ok(CustomSafeOutputJobDef { + job_id: prefix.custom_id(&definition.name)?, + reviewed: reviewed.contains(definition.name.as_str()), + name: definition.name, + max: definition.max, + env: definition.env, + kind: definition.kind, + component: definition.component, + schema_digest: definition.schema_digest, + }) +} + +fn custom_repository_resources(front_matter: &FrontMatter) -> Result> { + let mut resources = Vec::new(); + let mut seen = std::collections::HashSet::new(); + for definition in collect_custom_tool_definitions(front_matter)? { + let Some(component) = definition.component else { + continue; + }; + if !seen.insert(component.alias.clone()) { + continue; + } + let mut parts = component.source.splitn(3, '/'); + let owner = parts.next().unwrap_or_default(); + let repo = parts.next().unwrap_or_default(); + resources.push(RepositoryResource::Named { + identifier: component.alias, + kind: component.repo_type, + name: format!("{owner}/{repo}"), + r#ref: None, + endpoint: component.endpoint, + }); + } + Ok(resources) +} + +/// P7: `target: job` / `target: stage` compile to Azure DevOps *templates*, which +/// cannot declare top-level `resources.repositories`. When such a workflow +/// imports a remote custom safe-output component (which needs a runtime +/// checkout), the **parent** pipeline must declare + authorize that repository — +/// the compiler must not broaden access silently. Returns the human-readable +/// requirement, or `None` for resource-owning targets (standalone / 1es) or when +/// no imported component repositories are present. +fn custom_import_parent_diagnostic(front_matter: &FrontMatter) -> Result> { + let aliases: Vec = custom_repository_resources(front_matter)? + .into_iter() + .filter_map(|resource| match resource { + RepositoryResource::Named { identifier, .. } => Some(identifier), + _ => None, + }) + .collect(); + Ok(super::imports::alias::import_resource_parent_diagnostic( + front_matter.target.clone(), + &aliases, + )) +} + +fn build_custom_safe_output_job( + def: &CustomSafeOutputJobDef, + front_matter: &FrontMatter, + cfg: &StandaloneCtx, +) -> Result { + let mut steps = Vec::new(); + // ADO moves `self` to `s/` when a second repository is + // checked out unless `path` is explicit. Keep the trigger repo at the + // canonical `$(Build.SourcesDirectory)` root because `cfg.source_path` + // and every custom wrapper invocation are anchored there. + steps.push(checkout_self_step_at_sources_root(&cfg.self_checkout_fetch)); + if let Some(component) = &def.component { + let use_system_access_token = component.repo_type == "git" && component.endpoint.is_none(); + steps.push(Step::Checkout(CheckoutStep { + repository: CheckoutRepo::Named(component.alias.clone()), + clean: None, + submodules: None, + fetch_depth: Some(1), + fetch_tags: Some(false), + persist_credentials: Some(!use_system_access_token), + path: Some(format!("s/{}", component.alias)), + })); + // Install Node + download the ado-script bundle, then fetch/verify the + // pinned component commit via the checkout-component bundle. An ADO + // repository-resource `ref` cannot be a commit SHA, so the shallow + // resource checkout above lacks the pinned object; the bundle obtains it + // (direct by-SHA fetch → progressive deepening) and verifies a detached + // checkout of it, failing closed on any mismatch. + steps.extend( + super::extensions::ado_script::install_and_download_steps_typed( + front_matter.supply_chain.as_ref(), + ), + ); + steps.push( + super::extensions::ado_script::checkout_component_step_typed( + &component.checkout_dir, + component.sha.as_str(), + use_system_access_token, + ), + ); + } + steps.push(Step::Download(DownloadStep { + source: "current".to_string(), + artifact: "analyzed_outputs_$(Build.BuildId)".to_string(), + condition: None, + })); + if let Some(auth) = feed_auth_step(front_matter.supply_chain()) { + steps.push(auth); + } + steps.extend(download_compiler_step( + &cfg.compiler_version, + front_matter.supply_chain(), + )); + steps.push(Step::Bash(prepare_custom_executor_binary_step())); + + match &def.kind { + CustomToolKind::Scripts { + entrypoint, + timeout_minutes, + } => { + let config_path = format!("$(Agent.TempDirectory)/ado-aw-custom/{}.json", def.name); + let cwd = def + .component + .as_ref() + .map(|c| c.checkout_dir.clone()) + .unwrap_or_else(|| cfg.working_directory.clone()); + steps.push(Step::Bash(write_custom_scripts_config_step( + def, + entrypoint, + *timeout_minutes, + &cwd, + &config_path, + )?)); + steps.push(Step::Bash(custom_scripts_execute_step( + &cfg.source_path, + &config_path, + def, + ))); + } + CustomToolKind::Jobs { + steps: component_steps, + } => { + let proposals_path = format!("$(Agent.TempDirectory)/proposals-{}.json", def.name); + let results_path = format!("$(Agent.TempDirectory)/results-{}.json", def.name); + steps.push(Step::Bash(custom_jobs_pre_step( + &cfg.source_path, + &def.name, + def.max, + &proposals_path, + &def.env, + ))); + for step in component_steps { + let step = + serde_yaml::to_value(step).context("failed to convert custom job step")?; + steps.push(Step::RawYaml(component_step_with_custom_env( + &step, + &def.env, + &proposals_path, + &results_path, + )?)); + } + steps.push(Step::Bash(custom_jobs_post_step( + &cfg.source_path, + def, + &proposals_path, + &results_path, + ))); + } + } + steps.push(Step::Publish(PublishStep { + path: "$(Pipeline.Workspace)/analyzed_outputs_$(Build.BuildId)".to_string(), + artifact: format!( + "custom_safe_output_{}_$(Build.BuildId)", + ado_identifier_suffix(&def.name) + ), + condition: Some(Condition::Always), + })); + + let custom_pool = if def.reviewed { + cfg.pools.safe_outputs_reviewed.clone() + } else { + cfg.pools.safe_outputs.clone() + }; + let mut job = Job::new( + def.job_id.clone(), + format!("Custom safe output: {}", def.name), + custom_pool, + ); + job.steps = steps; + job.condition = Some(custom_job_condition(def)?); + Ok(job) +} + +fn custom_job_condition(def: &CustomSafeOutputJobDef) -> Result { + let mut parts = vec![ + Condition::Succeeded, + Condition::Eq( + Expr::StepOutput(OutputRef::new( + StepId::new("threatAnalysis")?, + "SafeToProcess", + )), + Expr::Literal("true".to_string()), + ), + Condition::Eq( + Expr::StepOutput(OutputRef::new( + StepId::new(CUSTOM_PROPOSALS_STEP_ID)?, + custom_tool_output_var(&def.name), + )), + Expr::Literal("true".to_string()), + ), + ]; + if def.reviewed { + parts.push(Condition::Eq( + Expr::StepOutput(OutputRef::new( + StepId::new("reviewedProposals")?, + "HasReviewedProposals", + )), + Expr::Literal("true".to_string()), + )); + } + Ok(Condition::And(parts)) +} + +fn prepare_custom_executor_binary_step() -> BashStep { + bash( + "Prepare custom safe-output executor", + "mkdir -p /tmp/awf-tools\n\ + AGENTIC_PIPELINES_PATH=\"$(Pipeline.Workspace)/agentic-pipeline-compiler/ado-aw\"\n\ + chmod +x \"$AGENTIC_PIPELINES_PATH\"\n\ + cp \"$AGENTIC_PIPELINES_PATH\" /tmp/awf-tools/ado-aw\n\ + chmod +x /tmp/awf-tools/ado-aw\n", + ) +} + +fn write_custom_scripts_config_step( + def: &CustomSafeOutputJobDef, + entrypoint: &str, + timeout_minutes: u32, + cwd: &str, + config_path: &str, +) -> Result { + let mut tool = serde_json::Map::new(); + tool.insert( + "entrypoint".to_string(), + serde_json::Value::String(entrypoint.to_string()), + ); + tool.insert( + "cwd".to_string(), + serde_json::Value::String(cwd.to_string()), + ); + tool.insert( + "max".to_string(), + serde_json::Value::Number(serde_json::Number::from(def.max)), + ); + tool.insert( + "timeout_minutes".to_string(), + serde_json::Value::Number(serde_json::Number::from(timeout_minutes)), + ); + let mut tools = serde_json::Map::new(); + tools.insert(def.name.clone(), serde_json::Value::Object(tool)); + let mut root = serde_json::Map::new(); + root.insert("tools".to_string(), serde_json::Value::Object(tools)); + let config = serde_json::Value::Object(root); + let json = serde_json::to_string_pretty(&config) + .context("failed to serialize custom scripts config")?; + let script = format!( + "mkdir -p \"$(Agent.TempDirectory)/ado-aw-custom\"\n\ + # shellcheck disable=SC2016 # ADO expands $(Agent.TempDirectory) before bash evaluates the quoted path.\n\ + cat > {config_path} << 'ADO_AW_CUSTOM_CONFIG_JSON'\n\ +{json}\n\ + ADO_AW_CUSTOM_CONFIG_JSON\n\ + # shellcheck disable=SC2016 # ADO expands $(Agent.TempDirectory) before bash evaluates the quoted path.\n\ + python3 -m json.tool {config_path} > /dev/null\n", + config_path = shell_quote(config_path) + ); + Ok(bash("Write custom safe-output config", script)) +} + +fn custom_scripts_execute_step( + source_path: &str, + config_path: &str, + def: &CustomSafeOutputJobDef, +) -> BashStep { + let provenance_args = custom_provenance_args(def); + let script = format!( + "# shellcheck disable=SC2016 # ADO expands path macros before bash evaluates the single-quoted arguments.\n\ + /tmp/awf-tools/ado-aw execute --source {source} --safe-output-dir \"$(Pipeline.Workspace)/analyzed_outputs_$(Build.BuildId)\" --custom-config {config}{provenance_args}\n", + source = shell_quote(source_path), + config = shell_quote(config_path), + ); + with_custom_secret_env(bash("Execute custom safe output", script), &def.env) +} + +fn custom_jobs_pre_step( + source_path: &str, + tool: &str, + max: usize, + proposals_path: &str, + env: &[(String, String)], +) -> BashStep { + let script = format!( + "# shellcheck disable=SC2016 # ADO expands path macros before bash evaluates the single-quoted arguments.\n\ + /tmp/awf-tools/ado-aw execute --source {source} --safe-output-dir \"$(Pipeline.Workspace)/analyzed_outputs_$(Build.BuildId)\" --custom-phase pre --tool {tool} --max {max} --proposals-out {proposals}\n", + source = shell_quote(source_path), + tool = shell_quote(tool), + max = max, + proposals = shell_quote(proposals_path) + ); + with_custom_secret_env(bash("Prepare custom safe-output proposals", script), env) +} + +fn custom_jobs_post_step( + source_path: &str, + def: &CustomSafeOutputJobDef, + proposals_path: &str, + results_path: &str, +) -> BashStep { + let provenance_args = custom_provenance_args(def); + let script = format!( + "# shellcheck disable=SC2016 # ADO expands path macros before bash evaluates the single-quoted arguments.\n\ + /tmp/awf-tools/ado-aw execute --source {source} --safe-output-dir \"$(Pipeline.Workspace)/analyzed_outputs_$(Build.BuildId)\" --custom-phase post --tool {tool} --max {max} --results-in {results}{provenance_args}\n", + source = shell_quote(source_path), + tool = shell_quote(&def.name), + max = def.max, + results = shell_quote(results_path) + ); + let step = with_custom_secret_env( + bash("Finalize custom safe-output results", script) + .with_env( + "ADO_AW_SAFE_OUTPUT_PROPOSALS", + EnvValue::literal(proposals_path.to_string()), + ) + .with_env( + "ADO_AW_SAFE_OUTPUT_RESULTS", + EnvValue::literal(results_path.to_string()), + ), + &def.env, + ); + BashStep { + condition: Some(Condition::Always), + ..step + } +} + +fn custom_provenance_args(def: &CustomSafeOutputJobDef) -> String { + let mut provenance_args = String::new(); + if let Some(component) = &def.component { + provenance_args.push_str(&format!( + " --component-sha {} --component-source {}", + shell_quote(component.sha.as_str()), + shell_quote(&component.source) + )); + if let Some(digest) = &component.manifest_digest { + provenance_args.push_str(&format!(" --manifest-digest {}", shell_quote(digest))); + } + } + provenance_args.push_str(&format!( + " --schema-digest {}", + shell_quote(&def.schema_digest) + )); + provenance_args +} + +fn with_custom_secret_env(mut step: BashStep, env: &[(String, String)]) -> BashStep { + for (name, var) in env { + step = step.with_env(name.clone(), EnvValue::secret(var.clone())); + } + step +} + +fn component_step_with_custom_env( + step: &serde_yaml::Value, + custom_env: &[(String, String)], + proposals_path: &str, + results_path: &str, +) -> Result { + let mut step = step.clone(); + let mapping = step.as_mapping_mut().ok_or_else(|| { + anyhow::anyhow!("safe-outputs.jobs..steps entries must be YAML mappings") + })?; + let env_key = serde_yaml::Value::String("env".to_string()); + if !mapping.contains_key(&env_key) { + mapping.insert( + env_key.clone(), + serde_yaml::Value::Mapping(serde_yaml::Mapping::new()), + ); + } + let env_value = mapping + .get_mut(&env_key) + .expect("env key inserted above when absent"); + let env_map = env_value.as_mapping_mut().ok_or_else(|| { + anyhow::anyhow!("safe-outputs.jobs..steps env blocks must be mappings") + })?; + env_map.insert( + serde_yaml::Value::String("ADO_AW_SAFE_OUTPUT_PROPOSALS".to_string()), + serde_yaml::Value::String(proposals_path.to_string()), + ); + env_map.insert( + serde_yaml::Value::String("ADO_AW_SAFE_OUTPUT_RESULTS".to_string()), + serde_yaml::Value::String(results_path.to_string()), + ); + for (name, var) in custom_env { + env_map.insert( + serde_yaml::Value::String(name.clone()), + serde_yaml::Value::String(format!("$({var})")), + ); + } + step_to_raw_yaml_string(&step) +} + +fn shell_quote(raw: &str) -> String { + format!("'{}'", raw.replace('\'', "'\\''")) +} + /// Build the `(dir, target-branch)` pairs the `prepare-pr-base` bundle must /// fetch/deepen — one per allowed `create-pull-request` repo, mirroring /// `mcp.rs::resolve_git_dir_for_patch`: `working_directory` (for `self`) and @@ -2035,7 +2614,13 @@ fi\n" /// from the same `prefix`, so a failure here would indicate an /// invalid `JobPrefix` reaching this function — the typed error is /// preferable to a panic for any future caller. -fn wire_explicit_dependencies(jobs: &mut [Job], prefix: &JobPrefix<'_>) -> Result<()> { +fn wire_explicit_dependencies( + jobs: &mut [Job], + prefix: &JobPrefix<'_>, + custom_reviewed_job_ids: &[JobId], + custom_job_ids: &[JobId], + safeoutputs_waits_for_review: bool, +) -> Result<()> { let setup_id = prefix.id("Setup")?; let agent_id = prefix.id("Agent")?; let detection_id = prefix.id("Detection")?; @@ -2046,7 +2631,6 @@ fn wire_explicit_dependencies(jobs: &mut [Job], prefix: &JobPrefix<'_>) -> Resul let conclusion_id = prefix.id("Conclusion")?; let has_setup = jobs.iter().any(|j| j.id == setup_id); let has_teardown = jobs.iter().any(|j| j.id == teardown_id); - let has_review = jobs.iter().any(|j| j.id == manualreview_id); // The reviewed execution job only exists in the mixed (split) case. let has_reviewed_job = jobs.iter().any(|j| j.id == reviewed_id); for j in jobs.iter_mut() { @@ -2058,12 +2642,22 @@ fn wire_explicit_dependencies(jobs: &mut [Job], prefix: &JobPrefix<'_>) -> Resul // Agentless gate: depends on Detection (its condition reads // Detection's threatAnalysis.SafeToProcess output). j.depends_on = vec![agent_id.clone(), detection_id.clone()]; + } else if custom_job_ids.iter().any(|id| id == &j.id) { + j.depends_on = if custom_reviewed_job_ids.iter().any(|id| id == &j.id) { + vec![ + agent_id.clone(), + detection_id.clone(), + manualreview_id.clone(), + ] + } else { + vec![agent_id.clone(), detection_id.clone()] + }; } else if j.id == safeoutputs_id { // The "SafeOutputs" job is the automatic path. It is gated behind // ManualReview only when it is the *sole* execution job (all tools // reviewed); in the mixed split it runs immediately after Detection // alongside the separate reviewed job. - j.depends_on = if has_review && !has_reviewed_job { + j.depends_on = if safeoutputs_waits_for_review { vec![ agent_id.clone(), detection_id.clone(), @@ -2106,6 +2700,7 @@ fn wire_explicit_dependencies(jobs: &mut [Job], prefix: &JobPrefix<'_>) -> Resul // skipped or fails. deps.push(reviewed_id.clone()); } + deps.extend(custom_job_ids.iter().cloned()); if has_teardown { deps.push(teardown_id.clone()); } @@ -2120,6 +2715,14 @@ fn wire_explicit_dependencies(jobs: &mut [Job], prefix: &JobPrefix<'_>) -> Resul // ───────────────────────────────────────────────────────────────────── fn checkout_self_step(fetch: &CheckoutFetchOpts) -> Step { + checkout_self_step_with_path(fetch, None) +} + +fn checkout_self_step_at_sources_root(fetch: &CheckoutFetchOpts) -> Step { + checkout_self_step_with_path(fetch, Some("s")) +} + +fn checkout_self_step_with_path(fetch: &CheckoutFetchOpts, path: Option<&str>) -> Step { Step::Checkout(CheckoutStep { repository: CheckoutRepo::Self_, clean: None, @@ -2127,6 +2730,7 @@ fn checkout_self_step(fetch: &CheckoutFetchOpts) -> Step { fetch_depth: fetch.depth_for_emit(), fetch_tags: fetch.fetch_tags, persist_credentials: None, + path: path.map(str::to_string), }) } @@ -2138,6 +2742,7 @@ fn checkout_none_step() -> Step { fetch_depth: None, fetch_tags: None, persist_credentials: None, + path: None, }) } @@ -2502,10 +3107,26 @@ fn substitute_integrity_check(yaml: &str, pipeline_path: &str, trigger_repo_dir: .replace("{{ trigger_repo_directory }}", trigger_repo_dir) } -fn prepare_mcpg_config_step(mcpg_config_json: &str) -> BashStep { +fn prepare_mcpg_config_step( + mcpg_config_json: &str, + custom_tools_json: Option<&str>, +) -> Result { // mcpg_config_json is pretty-printed JSON. We want `{` to align with // the surrounding `cat`/`echo` lines (no extra leading indent) so the // emitted block-scalar bash body matches base.yml. + let custom_tools_script = if let Some(custom_tools_json) = custom_tools_json { + let sentinel = super::common::heredoc_sentinel("CUSTOM_TOOLS_JSON_EOF", custom_tools_json)?; + format!( + "# Write compiler-generated dynamic SafeOutputs tool definitions\n\ + cat > \"$(Agent.TempDirectory)/staging/custom-tools.json\" << '{sentinel}'\n\ +{custom_tools_json}\n\ + {sentinel}\n\ + python3 -m json.tool \"$(Agent.TempDirectory)/staging/custom-tools.json\" > /dev/null\n\ + \n" + ) + } else { + String::new() + }; let script = format!( "mkdir -p \"$(Agent.TempDirectory)/staging\"\n\ \n\ @@ -2526,13 +3147,14 @@ fn prepare_mcpg_config_step(mcpg_config_json: &str) -> BashStep { {mcpg_config_json}\n\ MCPG_CONFIG_EOF\n\ \n\ +{custom_tools_script}\ echo \"MCPG config:\"\n\ cat \"$(Agent.TempDirectory)/staging/mcpg-config.json\"\n\ \n\ # Validate JSON\n\ python3 -m json.tool \"$(Agent.TempDirectory)/staging/mcpg-config.json\" > /dev/null && echo \"JSON is valid\"\n" ); - bash("Prepare MCPG config", script) + Ok(bash("Prepare MCPG config", script)) } fn prepare_tooling_step() -> BashStep { @@ -2554,7 +3176,10 @@ fn prepare_tooling_step() -> BashStep { chmod +x /tmp/awf-tools/ado-aw\n\ \n\ # Copy MCPG config to /tmp\n\ - cp \"$(Agent.TempDirectory)/staging/mcpg-config.json\" /tmp/awf-tools/staging/mcpg-config.json\n"; + cp \"$(Agent.TempDirectory)/staging/mcpg-config.json\" /tmp/awf-tools/staging/mcpg-config.json\n\ + if [ -f \"$(Agent.TempDirectory)/staging/custom-tools.json\" ]; then\n\ + cp \"$(Agent.TempDirectory)/staging/custom-tools.json\" /tmp/awf-tools/staging/custom-tools.json\n\ + fi\n"; bash("Prepare tooling", script) } @@ -3360,6 +3985,47 @@ fn detect_reviewed_proposals_step(working_directory: &str, reviewed: &[String]) .with_condition(Condition::Always) } +/// Scan the analyzed proposal NDJSON once and publish one output variable per +/// custom tool. Custom executor jobs use these booleans in their job-level +/// `condition:` so an empty/no-op custom proposal set does not start a job. +fn detect_custom_proposals_step(working_directory: &str, tools: &[String]) -> Result { + let mut script = format!( + "PROPOSALS=$(find \"{working_directory}/safe_outputs\" -name \"safe_outputs.ndjson\" 2>/dev/null | head -n 1)\n\ + NAMES=\"\"\n\ + RAW_SCAN=\"false\"\n\ + if [ -n \"$PROPOSALS\" ] && [ -f \"$PROPOSALS\" ]; then\n \ + if command -v jq >/dev/null 2>&1; then\n \ + if ! NAMES=$(jq -r 'select(type==\"object\") | .name // empty' \"$PROPOSALS\" 2>/dev/null); then\n \ + echo \"##vso[task.logissue type=warning]custom-proposals: jq failed to parse $PROPOSALS; using raw scan\"\n \ + RAW_SCAN=\"true\"\n \ + fi\n \ + else\n \ + RAW_SCAN=\"true\"\n \ + fi\n\ + fi\n" + ); + let mut step = bash("Detect custom proposals", ""); + for tool in tools { + let output = custom_tool_output_var(tool); + script.push_str(&format!( + "{output}=\"false\"\n\ + if [ -n \"$NAMES\" ] && printf '%s\\n' \"$NAMES\" | grep -Fxq {tool_q}; then\n \ + {output}=\"true\"\n\ + elif [ \"$RAW_SCAN\" = \"true\" ] && [ -n \"$PROPOSALS\" ] && grep -Eq '\"name\"[[:space:]]*:[[:space:]]*\"{tool}\"' \"$PROPOSALS\"; then\n \ + {output}=\"true\"\n\ + fi\n\ + echo \"##vso[task.setvariable variable={output};isOutput=true]${output}\"\n\ + echo \"{output} set to: ${output}\"\n", + tool_q = shell_quote(tool), + )); + step = step.with_output(OutputDecl::new(output)); + } + step.script = dedent(&script); + Ok(step + .with_id(StepId::new(CUSTOM_PROPOSALS_STEP_ID)?) + .with_condition(Condition::Always)) +} + fn verify_mcp_backends_step() -> BashStep { // Debug-only probe (emitted when --debug-pipeline is on). Probes every // MCPG backend via MCP initialize + tools/list to surface broken @@ -3682,12 +4348,21 @@ fn step_value_to_dash_yaml(v: serde_yaml::Value) -> Result { Ok(out) } -/// Build the agent prompt body — either inlined imports or a -/// runtime-import marker. Mirrors `compile_shared`'s logic. +/// Build the agent prompt body. +/// +/// In `inlined-imports: true` mode the entire body (imported + consumer) is +/// already in `markdown_body`, so it is resolved inline verbatim. In the +/// default mode the consumer body is delivered by a `{{#runtime-import}}` +/// marker (so authors can edit it without recompiling), but any imported +/// component bodies (`imported_prompt_body`) are inlined **ahead** of that +/// marker: they were substituted at compile time and cannot be re-derived at +/// runtime from the consumer's own source. Mirrors gh-aw, which compile-inlines +/// input-bearing imports and runtime-imports only the main body. fn build_agent_content( front_matter: &FrontMatter, input_path: &Path, markdown_body: &str, + imported_prompt_body: &str, source_path: &str, trigger_repo_directory: &str, ) -> Result { @@ -3718,7 +4393,15 @@ fn build_agent_content( "runtime-import: agent source path '{}' contains '}}', which is not supported by the runtime resolver (rename the path to remove '}}' characters, or set `inlined-imports: true`)", marker_path ); - Ok(format!("{{{{#runtime-import {}}}}}", marker_path)) + let consumer_marker = format!("{{{{#runtime-import {}}}}}", marker_path); + + // Prepend the compile-time-substituted imported component bodies (if any) + // ahead of the consumer's runtime-import marker (imports-first ordering). + if imported_prompt_body.trim().is_empty() { + Ok(consumer_marker) + } else { + Ok(format!("{imported_prompt_body}\n\n{consumer_marker}")) + } } // Suppress unused warnings on imports retained for clarity / future use. @@ -3739,6 +4422,490 @@ const _SUBMODULES_OPT_BIND: Option = None; mod tests { use super::*; + fn test_front_matter(yaml: &str) -> FrontMatter { + serde_yaml::from_str(yaml).expect("front matter should parse") + } + + fn test_ctx() -> StandaloneCtx { + let test_pool = Pool::VmImage("ubuntu-latest".to_string()); + StandaloneCtx { + pools: PerJobPools { + setup: test_pool.clone(), + agent: test_pool.clone(), + detection: test_pool.clone(), + safe_outputs: test_pool.clone(), + safe_outputs_reviewed: test_pool.clone(), + teardown: test_pool.clone(), + conclusion: test_pool.clone(), + }, + agent_display_name: "Test".to_string(), + self_checkout_fetch: CheckoutFetchOpts::default(), + working_directory: "$(Build.SourcesDirectory)".to_string(), + trigger_repo_directory: "$(Build.SourcesDirectory)".to_string(), + compiler_version: "0.0.0-test".to_string(), + engine_install_steps_yaml: String::new(), + engine_run: "echo agent".to_string(), + engine_run_detection: "echo detection".to_string(), + engine_env: "GITHUB_READ_ONLY: 1".to_string(), + engine_log_dir: "/tmp/logs".to_string(), + allowed_domains: "example.com".to_string(), + awf_mounts: "\\".to_string(), + awf_path_step_yaml: String::new(), + mcpg_config_json: "{}".to_string(), + custom_tools_json: None, + mcpg_docker_env: String::new(), + mcpg_step_env: String::new(), + source_path: "$(Build.SourcesDirectory)/agents/test.md".to_string(), + pipeline_path: "$(Build.SourcesDirectory)/agents/test.lock.yml".to_string(), + acquire_read_token: String::new(), + acquire_write_token: String::new(), + executor_ado_env: "env:\n SYSTEM_ACCESSTOKEN: $(System.AccessToken)\n".to_string(), + integrity_check_yaml: String::new(), + agent_content_value: "Test prompt".to_string(), + debug_pipeline: false, + byom_exclude_keys: Vec::new(), + detection_provider_env: Vec::new(), + } + } + + fn canonical_jobs_for(yaml: &str) -> Vec { + let fm = test_front_matter(yaml); + let cfg = test_ctx(); + build_canonical_jobs(&fm, &[], &cfg, &[], &[], &[], None).unwrap() + } + + fn job_by_id<'a>(jobs: &'a [Job], id: &str) -> &'a Job { + jobs.iter().find(|job| job.id.as_str() == id).unwrap() + } + + fn step_env_has_secret(step: &Step, name: &str, var: &str) -> bool { + let env = match step { + Step::Bash(s) => &s.env, + Step::Task(s) => &s.env, + _ => return false, + }; + matches!(env.get(name), Some(EnvValue::Secret(v)) if v == var) + } + + #[test] + fn custom_component_repo_resource_omits_ref_for_default_branch() { + // Regression: the imported-component repository resource must NOT pin a + // hardcoded `refs/heads/main` ref (ADO hard-fails the checkout for repos + // whose default branch differs). Omitting `ref` makes ADO use the repo's + // actual default branch; the exact SHA is pinned at runtime by the + // checkout-component bundle. + let fm = test_front_matter( + r#" +name: Test +description: Test +safe-outputs: + scripts: + notify-team: + run: node notify.js + component-source: octo/tools/components/notify.md + component-sha: 0123456789012345678901234567890123456789 + manifest-digest: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa +"#, + ); + let resources = custom_repository_resources(&fm).unwrap(); + assert_eq!(resources.len(), 1); + match &resources[0] { + RepositoryResource::Named { + identifier, + kind, + name, + r#ref, + endpoint, + } => { + assert!(identifier.starts_with("import_octo_tools_"), "{identifier}"); + assert_eq!(kind, "git"); + assert_eq!(name, "octo/tools"); + assert_eq!( + *r#ref, None, + "ref must be omitted so ADO uses the component repo's default branch" + ); + assert_eq!(*endpoint, None); + } + other => panic!("expected a Named repository resource, got {other:?}"), + } + } + + #[test] + fn custom_component_repo_resource_maps_typed_endpoint_to_kind_and_connection() { + // A component imported through a GitHub endpoint must produce a + // `github`-typed repository resource wired to its service connection — + // not the hardcoded `git` / no-endpoint. `component-repo-type` and + // `component-endpoint` are stamped from the typed endpoint at merge time. + let fm = test_front_matter( + r#" +name: Test +description: Test +safe-outputs: + scripts: + notify-team: + run: node notify.js + component-source: octo/tools/components/notify.md + component-sha: 0123456789012345678901234567890123456789 + component-repo-type: github + component-endpoint: gh-shared-conn +"#, + ); + let resources = custom_repository_resources(&fm).unwrap(); + assert_eq!(resources.len(), 1); + match &resources[0] { + RepositoryResource::Named { + kind, + name, + r#ref, + endpoint, + .. + } => { + assert_eq!(kind, "github"); + assert_eq!(name, "octo/tools"); + assert_eq!(*r#ref, None); + assert_eq!(endpoint.as_deref(), Some("gh-shared-conn")); + } + other => panic!("expected a Named repository resource, got {other:?}"), + } + } + + #[test] + fn custom_component_repo_resource_ghe_kind_maps_to_githubenterprise() { + let fm = test_front_matter( + r#" +name: Test +description: Test +safe-outputs: + jobs: + ticket: + steps: + - bash: echo hi + component-source: octo/tools/components/ticket.md + component-sha: 0123456789012345678901234567890123456789 + component-repo-type: githubenterprise + component-endpoint: ghe-conn +"#, + ); + let resources = custom_repository_resources(&fm).unwrap(); + assert_eq!(resources.len(), 1); + match &resources[0] { + RepositoryResource::Named { kind, endpoint, .. } => { + assert_eq!(kind, "githubenterprise"); + assert_eq!(endpoint.as_deref(), Some("ghe-conn")); + } + other => panic!("expected a Named repository resource, got {other:?}"), + } + } + + #[test] + fn custom_import_parent_diagnostic_fires_for_template_targets_with_imports() { + // A `target: job` / `target: stage` workflow that imports a remote + // custom component cannot own the repository resource, so the parent + // pipeline must declare + authorize it — the compiler surfaces that. + for target in ["job", "stage"] { + let fm = test_front_matter(&format!( + r#" +name: Test +description: Test +target: {target} +safe-outputs: + scripts: + notify-team: + run: node notify.js + component-source: octo/tools/components/notify.md + component-sha: 0123456789012345678901234567890123456789 + manifest-digest: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa +"# + )); + let diagnostic = custom_import_parent_diagnostic(&fm) + .unwrap() + .unwrap_or_else(|| panic!("target `{target}` should require a parent diagnostic")); + assert!( + diagnostic.contains("parent pipeline"), + "target `{target}`: {diagnostic}" + ); + assert!( + diagnostic.contains("import_octo_tools_"), + "target `{target}` should name the alias: {diagnostic}" + ); + } + } + + #[test] + fn custom_import_parent_diagnostic_absent_for_owning_target_or_no_imports() { + // Standalone owns its resources → no diagnostic even with an import. + let with_import = r#" +name: Test +description: Test +target: standalone +safe-outputs: + scripts: + notify-team: + run: node notify.js + component-source: octo/tools/components/notify.md + component-sha: 0123456789012345678901234567890123456789 + manifest-digest: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa +"#; + assert!( + custom_import_parent_diagnostic(&test_front_matter(with_import)) + .unwrap() + .is_none() + ); + + // Template target but no imported component (no `component-source`) → + // nothing for the parent to authorize. + let job_no_import = r#" +name: Test +description: Test +target: job +safe-outputs: + create-issue: + max: 1 +"#; + assert!( + custom_import_parent_diagnostic(&test_front_matter(job_no_import)) + .unwrap() + .is_none() + ); + } + + #[test] + fn custom_job_scripts_tool_emits_job_config_execute_secret_and_remote_checkout() { + let jobs = canonical_jobs_for( + r#" +name: Test +description: Test +safe-outputs: + scripts: + notify-team: + run: node notify.js + max: 2 + timeout-minutes: 7 + component-source: octo/tools/components/notify.md + component-sha: 0123456789012345678901234567890123456789 + manifest-digest: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa + env: + API_TOKEN: NOTIFY_TOKEN +"#, + ); + let custom = job_by_id(&jobs, "Custom_notify_team"); + assert_eq!( + custom + .depends_on + .iter() + .map(|id| id.as_str()) + .collect::>(), + vec!["Agent", "Detection"] + ); + assert!(custom.steps.iter().any(|step| { + matches!( + step, + Step::Checkout(CheckoutStep { + repository: CheckoutRepo::Named(alias), + fetch_depth: Some(1), + path: Some(path), + .. + }) if alias.starts_with("import_octo_tools_") + && path == &format!("s/{alias}") + ) + })); + assert!(matches!( + custom.steps.first(), + Some(Step::Checkout(CheckoutStep { + repository: CheckoutRepo::Self_, + path: Some(path), + .. + })) if path == "s" + )); + // The pinned SHA is fetched + verified via the checkout-component + // ado-script bundle (not a raw `git checkout` that would fail on a + // shallow pool), with the ADO bearer projected for the fetch. + assert!(custom.steps.iter().any(|step| { + matches!(step, Step::Bash(s) + if s.script.contains("checkout-component.js") + && s.script.contains("--sha '0123456789012345678901234567890123456789'") + && s.env.get("SYSTEM_ACCESSTOKEN").is_some()) + })); + // The old raw-git verify approach must be gone. + assert!(!custom.steps.iter().any(|step| { + matches!(step, Step::Bash(s) if s.script.contains("checkout --detach") + && !s.script.contains("checkout-component.js")) + })); + assert!(custom.steps.iter().any(|step| { + matches!(step, Step::Bash(s) if s.script.contains("\"notify-team\"") + && s.script.contains("\"entrypoint\": \"node notify.js\"") + && s.script.contains("\"max\": 2") + && s.script.contains("\"timeout_minutes\": 7")) + })); + assert!(custom.steps.iter().any(|step| { + matches!(step, Step::Bash(s) + if s.script.contains("--custom-config") + && s.script.contains("--component-sha '0123456789012345678901234567890123456789'") + && s.script.contains("--component-source 'octo/tools/components/notify.md'") + && s.script.contains("--manifest-digest 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa'") + && s.script.contains("--schema-digest")) + && step_env_has_secret(step, "API_TOKEN", "NOTIFY_TOKEN") + })); + let safeoutputs = job_by_id(&jobs, "SafeOutputs"); + assert!(safeoutputs.steps.iter().any(|step| { + matches!(step, Step::Bash(s) if s.script.contains("--exclude notify-team")) + })); + let agent = job_by_id(&jobs, "Agent"); + let detection = job_by_id(&jobs, "Detection"); + assert!(!agent.steps.iter().any(|step| step_env_has_secret( + step, + "API_TOKEN", + "NOTIFY_TOKEN" + ))); + assert!(!detection.steps.iter().any(|step| step_env_has_secret( + step, + "API_TOKEN", + "NOTIFY_TOKEN" + ))); + assert!(custom.steps.iter().any(|step| { + matches!( + step, + Step::Publish(PublishStep { + artifact, + condition: Some(Condition::Always), + .. + }) if artifact == "custom_safe_output_notify_team_$(Build.BuildId)" + ) + })); + } + + #[test] + fn external_component_checkout_reuses_persisted_endpoint_credentials() { + let jobs = canonical_jobs_for( + r#" +name: Test +description: Test +safe-outputs: + scripts: + notify-team: + run: node notify.js + component-source: octo/tools/components/notify.md + component-sha: 0123456789012345678901234567890123456789 + component-repo-type: github + component-endpoint: gh-shared-conn +"#, + ); + let custom = job_by_id(&jobs, "Custom_notify_team"); + assert!(custom.steps.iter().any(|step| { + matches!( + step, + Step::Checkout(CheckoutStep { + repository: CheckoutRepo::Named(_), + persist_credentials: Some(true), + .. + }) + ) + })); + assert!(custom.steps.iter().any(|step| { + matches!( + step, + Step::Bash(step) + if step.display_name == "Checkout pinned custom component" + && !step.env.contains_key("SYSTEM_ACCESSTOKEN") + ) + })); + } + + #[test] + fn custom_job_jobs_tool_emits_pre_component_post_in_order() { + let jobs = canonical_jobs_for( + r#" +name: Test +description: Test +safe-outputs: + jobs: + deploy-thing: + max: 1 + steps: + - bash: echo deploy + displayName: Component deploy +"#, + ); + let custom = job_by_id(&jobs, "Custom_deploy_thing"); + let pre = custom + .steps + .iter() + .position( + |step| matches!(step, Step::Bash(s) if s.script.contains("--custom-phase pre")), + ) + .unwrap(); + let component = custom + .steps + .iter() + .position(|step| matches!(step, Step::RawYaml(yaml) if yaml.contains("Component deploy") && yaml.contains("ADO_AW_SAFE_OUTPUT_PROPOSALS"))) + .unwrap(); + let post = custom + .steps + .iter() + .position( + |step| matches!(step, Step::Bash(s) if s.script.contains("--custom-phase post")), + ) + .unwrap(); + assert!(pre < component && component < post); + assert!(matches!( + &custom.steps[pre], + Step::Bash(step) if step.script.contains("--max 1") + )); + assert!(matches!( + &custom.steps[post], + Step::Bash(step) + if step.script.contains("--max 1") + && step.condition == Some(Condition::Always) + )); + } + + #[test] + fn custom_job_reviewed_tool_depends_on_manual_review() { + let jobs = canonical_jobs_for( + r#" +name: Test +description: Test +safe-outputs: + require-approval: true + scripts: + gated-tool: + run: ./gated +"#, + ); + let custom = job_by_id(&jobs, "Custom_gated_tool"); + assert_eq!( + custom + .depends_on + .iter() + .map(|id| id.as_str()) + .collect::>(), + vec!["Agent", "Detection", "ManualReview"] + ); + let safeoutputs = job_by_id(&jobs, "SafeOutputs"); + assert_eq!( + safeoutputs + .depends_on + .iter() + .map(|id| id.as_str()) + .collect::>(), + vec!["Agent", "Detection"] + ); + } + + #[test] + fn custom_job_no_custom_tools_leaves_canonical_job_names_unchanged() { + let jobs = canonical_jobs_for( + r#" +name: Test +description: Test +"#, + ); + assert_eq!( + jobs.iter().map(|job| job.id.as_str()).collect::>(), + vec!["Agent", "Detection", "SafeOutputs"] + ); + } + // ── fold_agent_conditions (issue #987) ───────────────────────────────── #[test] @@ -3997,6 +5164,7 @@ mod tests { awf_mounts: "\\".to_string(), awf_path_step_yaml: String::new(), mcpg_config_json: "{}".to_string(), + custom_tools_json: None, mcpg_docker_env: String::new(), mcpg_step_env: String::new(), source_path: "source.md".to_string(), @@ -4108,4 +5276,58 @@ mod tests { "ubuntu-22.04" ); } + + // ─── build_agent_content: imported-body delivery ───────────────────────── + + #[test] + fn build_agent_content_default_mode_inlines_imported_body_before_marker() { + let fm = test_front_matter("name: t\ndescription: d\n"); + let out = build_agent_content( + &fm, + std::path::Path::new("agents/test.md"), + // markdown_body (combined) is ignored in default mode. + "IGNORED COMBINED BODY", + "Imported guidance line.", + "$(Build.SourcesDirectory)/agents/test.md", + "$(Build.SourcesDirectory)", + ) + .unwrap(); + assert_eq!( + out, + "Imported guidance line.\n\n{{#runtime-import agents/test.md}}" + ); + } + + #[test] + fn build_agent_content_default_mode_without_imports_is_marker_only() { + let fm = test_front_matter("name: t\ndescription: d\n"); + let out = build_agent_content( + &fm, + std::path::Path::new("agents/test.md"), + "IGNORED", + "", + "$(Build.SourcesDirectory)/agents/test.md", + "$(Build.SourcesDirectory)", + ) + .unwrap(); + assert_eq!(out, "{{#runtime-import agents/test.md}}"); + } + + #[test] + fn build_agent_content_inlined_mode_uses_combined_body() { + // In inlined mode the combined body (imported + consumer) is already in + // markdown_body and is emitted verbatim; the separate + // imported_prompt_body arg is not appended a second time. + let fm = test_front_matter("name: t\ndescription: d\ninlined-imports: true\n"); + let out = build_agent_content( + &fm, + std::path::Path::new("agents/test.md"), + "Imported guidance line.\n\nConsumer body.", + "Imported guidance line.", + "$(Build.SourcesDirectory)/agents/test.md", + "$(Build.SourcesDirectory)", + ) + .unwrap(); + assert_eq!(out, "Imported guidance line.\n\nConsumer body."); + } } diff --git a/src/compile/common.rs b/src/compile/common.rs index 72305c72..b0d9056b 100644 --- a/src/compile/common.rs +++ b/src/compile/common.rs @@ -127,6 +127,66 @@ pub struct ParsedSource { pub source_sha256: [u8; 32], } +/// Raw markdown split result used by both the typed workflow parser and +/// import resolution for component manifests that may omit typed fields. +pub(crate) struct MarkdownFrontMatterParts { + pub leading_whitespace: String, + pub yaml_raw: Option, + pub body_raw: String, + pub markdown_body: String, +} + +/// Split optional YAML front matter from a markdown document. +/// +/// When `require_front_matter` is true this preserves the historical workflow +/// parser behavior and errors unless the file starts (modulo leading +/// whitespace) with `---`. +pub(crate) fn split_markdown_front_matter( + content: &str, + require_front_matter: bool, +) -> Result { + // Allow leading whitespace before the opening fence (preserves + // historical leniency). We compute a byte offset into `content` so + // that `body_raw` extraction is purely byte-faithful, and we keep + // the whitespace prefix around so that source rewrites preserve + // anything the user (or their editor) put before the opening + // fence. + let leading_ws = content + .bytes() + .take_while(|b| b.is_ascii_whitespace()) + .count(); + let leading_whitespace = content[..leading_ws].to_string(); + let after_lead = &content[leading_ws..]; + if !after_lead.starts_with("---") { + if require_front_matter { + anyhow::bail!("Markdown file must start with YAML front matter (---)"); + } + return Ok(MarkdownFrontMatterParts { + leading_whitespace, + yaml_raw: None, + body_raw: content.to_string(), + markdown_body: content.trim().to_string(), + }); + } + + let after_open = &after_lead[3..]; + let end_idx = after_open + .find("\n---") + .context("Could not find closing --- for front matter")?; + + let yaml_raw = after_open[..end_idx].to_string(); + let body_raw_slice = &after_open[end_idx + 4..]; + let body_raw = body_raw_slice.to_string(); + let markdown_body = body_raw_slice.trim().to_string(); + + Ok(MarkdownFrontMatterParts { + leading_whitespace, + yaml_raw: Some(yaml_raw), + body_raw, + markdown_body, + }) +} + /// Parse the markdown file, run the codemod registry on the front /// matter in memory, and return both the typed `FrontMatter` and the /// raw fragments needed to rewrite the source on disk byte-faithfully. @@ -154,31 +214,11 @@ pub(crate) fn parse_markdown_detailed_with_registry( hasher.update(content.as_bytes()); let source_sha256: [u8; 32] = hasher.finalize().into(); - // Allow leading whitespace before the opening fence (preserves - // historical leniency). We compute a byte offset into `content` so - // that `body_raw` extraction is purely byte-faithful, and we keep - // the whitespace prefix around so that source rewrites preserve - // anything the user (or their editor) put before the opening - // fence. - let leading_ws = content - .bytes() - .take_while(|b| b.is_ascii_whitespace()) - .count(); - let leading_whitespace = content[..leading_ws].to_string(); - let after_lead = &content[leading_ws..]; - if !after_lead.starts_with("---") { - anyhow::bail!("Markdown file must start with YAML front matter (---)"); - } - - let after_open = &after_lead[3..]; - let end_idx = after_open - .find("\n---") - .context("Could not find closing --- for front matter")?; - - let yaml_str = &after_open[..end_idx]; - let body_raw_slice = &after_open[end_idx + 4..]; - let body_raw = body_raw_slice.to_string(); - let markdown_body = body_raw_slice.trim().to_string(); + let parts = split_markdown_front_matter(content, true)?; + let yaml_str = parts + .yaml_raw + .as_deref() + .expect("required front matter split must return YAML"); // Stage 1: parse to untyped Value, reject non-mapping at top level. let parsed_value: serde_yaml::Value = @@ -217,11 +257,11 @@ pub(crate) fn parse_markdown_detailed_with_registry( Ok(ParsedSource { front_matter, - markdown_body, + markdown_body: parts.markdown_body, codemods: report, front_matter_mapping: mapping, - leading_whitespace, - body_raw, + leading_whitespace: parts.leading_whitespace, + body_raw: parts.body_raw, source_sha256, }) } @@ -620,7 +660,44 @@ pub fn build_parameters( /// The result of lowering a `repos:` list: the ADO repository resources, the /// checkout-alias list, and per-checkout fetch tuning keyed by alias (plus /// [`SELF_CHECKOUT_ALIAS`] for the trigger repo). -pub type LoweredRepos = (Vec, Vec, HashMap); +pub type LoweredRepos = ( + Vec, + Vec, + HashMap, +); + +/// Repository resource types that are backed by an Azure DevOps **service +/// connection** and therefore require an `endpoint:` on the repository +/// resource. Azure Repos (`git`) is same-organization and needs no endpoint. +pub fn repo_type_requires_endpoint(repo_type: &str) -> bool { + matches!(repo_type, "github" | "githubenterprise" | "bitbucket") +} + +/// Validate that a repository resource of a service-connection-backed type +/// (`github` / `githubenterprise` / `bitbucket`) carries a non-empty +/// `endpoint:`. Azure Repos (`git`) resources are exempt. +/// +/// This closes a real gap: `repo_type` was previously an untested passthrough, +/// so a `type: github` resource without an `endpoint:` compiled to invalid +/// Azure DevOps YAML. Fail fast at compile time with an actionable message. +pub fn validate_repo_endpoint( + repo_type: &str, + endpoint: &Option, + name: &str, +) -> Result<()> { + let has_endpoint = endpoint + .as_deref() + .map(|e| !e.trim().is_empty()) + .unwrap_or(false); + if repo_type_requires_endpoint(repo_type) && !has_endpoint { + anyhow::bail!( + "Repository '{name}' has type '{repo_type}', which requires an `endpoint:` \ + (an Azure DevOps service connection) to authenticate. Add \ + `endpoint: ` to this repository." + ); + } + Ok(()) +} /// Lower a `repos:` list into the internal [`LoweredRepos`] triple consumed by /// the rest of the compiler. A reserved `self` entry (an entry whose *name* is @@ -652,7 +729,7 @@ pub fn lower_repos(items: &[ReposItem]) -> Result { continue; } - let (name, alias, repo_type, repo_ref, do_checkout, fetch_opts) = match item { + let (name, alias, repo_type, repo_ref, endpoint, do_checkout, fetch_opts) = match item { ReposItem::Shorthand(s) => { let (alias, name) = parse_shorthand(s)?; ( @@ -660,6 +737,7 @@ pub fn lower_repos(items: &[ReposItem]) -> Result { alias, "git".to_string(), "refs/heads/main".to_string(), + None, true, CheckoutFetchOpts::default(), ) @@ -674,6 +752,7 @@ pub fn lower_repos(items: &[ReposItem]) -> Result { alias, entry.repo_type.clone(), entry.repo_ref.clone(), + entry.endpoint.clone(), entry.checkout, CheckoutFetchOpts { fetch_depth: entry.fetch_depth, @@ -730,11 +809,14 @@ pub fn lower_repos(items: &[ReposItem]) -> Result { ); } + validate_repo_endpoint(&repo_type, &endpoint, &name)?; + repositories.push(Repository { repository: alias.clone(), repo_type, name, repo_ref, + endpoint, }); if do_checkout { @@ -1091,8 +1173,7 @@ pub(crate) fn contains_template_marker(input: &str, name: &str) -> bool { // legacy marker even if its inner content trims to `name`. Matching it // here would both raise a false positive and — in `replace_marker` — // splice `repl` after the `$`, corrupting the output. - let preceded_by_dollar = - marker_start > 0 && input.as_bytes()[marker_start - 1] == b'$'; + let preceded_by_dollar = marker_start > 0 && input.as_bytes()[marker_start - 1] == b'$'; if !preceded_by_dollar && let Some(close) = input[start..].find("}}") && input[start..start + close].trim() == name @@ -1205,7 +1286,8 @@ pub fn resolve_pool_typed( (Some(name), Some(vm_image)) => { anyhow::bail!( "pool cannot specify both `name` and `vmImage` (got name='{}', vmImage='{}')", - name, vm_image + name, + vm_image ); } (_, Some(vm_image)) => { @@ -1800,6 +1882,23 @@ pub fn generate_acquire_ado_token(service_connection: Option<&str>, variable_nam } } +/// Generate the Agent-step ADO environment entry for `permissions.read`. +/// +/// The read-scoped token is exposed as `AZURE_DEVOPS_EXT_PAT`, the standard +/// variable consumed by `az devops` and other read-only ADO clients inside the +/// AWF sandbox. The write-scoped token and `System.AccessToken` are deliberately +/// absent: Stage 1 never receives either write-capable credential. +/// +/// Returns entries without an `env:` header because they are composed into the +/// engine environment before the typed Agent step is built. +pub fn generate_agent_ado_env(read_service_connection: Option<&str>) -> String { + if read_service_connection.is_some() { + "AZURE_DEVOPS_EXT_PAT: $(SC_READ_TOKEN)".to_string() + } else { + String::new() + } +} + /// Generate the env block entries for the executor step (Stage 3 Execution). /// /// Always emits a non-empty `env:` block containing at minimum @@ -2133,11 +2232,24 @@ pub fn validate_safe_outputs_keys(front_matter: &FrontMatter) -> Result<()> { let mut unknown: Vec<(String, Vec<&'static str>)> = Vec::new(); let mut invalid_names: Vec = Vec::new(); + // Custom tools declared under `safe-outputs.scripts` / `safe-outputs.jobs` + // (decision D16). A top-level key that matches a custom tool name is that + // tool's *configuration* (e.g. `require-approval`) and must be accepted + // rather than treated as an unknown built-in. + let custom_names: std::collections::HashSet = front_matter + .custom_safe_output_tool_names() + .into_iter() + .collect(); + for key in front_matter.safe_output_tool_names() { if !validate::is_safe_tool_name(key) { invalid_names.push(key.clone()); continue; } + if custom_names.contains(key) { + // Consumer configuration for an imported custom tool. + continue; + } if NON_MCP_SAFE_OUTPUT_KEYS.contains(&key.as_str()) { continue; } @@ -2199,6 +2311,46 @@ pub fn validate_safe_outputs_keys(front_matter: &FrontMatter) -> Result<()> { anyhow::bail!("{}", msg); } + validate_custom_safe_output_tools(front_matter)?; + + Ok(()) +} + +/// Validate custom safe-output tool definitions under `safe-outputs.scripts` +/// and `safe-outputs.jobs` (decision D16): each tool name must be a valid tool +/// name and must not collide with a built-in safe-output tool. The `scripts` +/// and `jobs` sections themselves must be mappings. +pub fn validate_custom_safe_output_tools(front_matter: &FrontMatter) -> Result<()> { + use crate::safe_outputs::ALL_KNOWN_SAFE_OUTPUTS; + + for section in ["scripts", "jobs"] { + let Some(value) = front_matter.safe_outputs.get(section) else { + continue; + }; + if value.is_null() { + continue; + } + let Some(map) = value.as_object() else { + anyhow::bail!( + "safe-outputs.{section} must be a mapping of tool-name to definition. Example:\n\n \ + safe-outputs:\n {section}:\n my-tool:\n description: ...\n" + ); + }; + for name in map.keys() { + if !validate::is_safe_tool_name(name) { + anyhow::bail!( + "safe-outputs.{section}.{name} has an invalid tool name. \ + Tool names must contain only ASCII letters, digits, and hyphens." + ); + } + if ALL_KNOWN_SAFE_OUTPUTS.contains(&name.as_str()) { + anyhow::bail!( + "safe-outputs.{section}.{name} collides with the built-in safe-output \ + tool '{name}'. Custom tool names must not shadow built-ins; rename it." + ); + } + } + } Ok(()) } @@ -2420,6 +2572,12 @@ pub fn generate_mcpg_config( .split_whitespace() .map(str::to_string), ); + if !front_matter.custom_safe_output_tool_names().is_empty() { + safeoutputs_entrypoint_args.extend([ + "--custom-tools".to_string(), + "/safeoutputs/custom-tools.json".to_string(), + ]); + } safeoutputs_entrypoint_args.extend([ "/safeoutputs".to_string(), working_directory.clone(), @@ -2625,10 +2783,7 @@ fn add_extension_network_hosts( /// Ecosystem identifiers (e.g., `"python"`) are expanded to their domain /// lists. Raw domain names are validated against DNS-safe characters before /// insertion; an invalid name causes this function to return an error. -fn add_user_network_hosts( - user_hosts: &[String], - hosts: &mut HashSet, -) -> Result<()> { +fn add_user_network_hosts(user_hosts: &[String], hosts: &mut HashSet) -> Result<()> { for host in user_hosts { if is_ecosystem_identifier(host) { let domains = get_ecosystem_domains(host); @@ -2989,7 +3144,10 @@ mod tests { let err = resolve_pool_typed(CompileTarget::Standalone, Some(&pool)) .unwrap_err() .to_string(); - assert!(err.contains("pool.demands requires `pool.name`"), "err: {err}"); + assert!( + err.contains("pool.demands requires `pool.name`"), + "err: {err}" + ); } #[test] @@ -3104,9 +3262,8 @@ mod tests { // name-validation stage (before the duplicate check), because the raw // string is emitted verbatim into the YAML and would not match the ADO // group. - let src = - "---\nname: t\ndescription: d\nvariable-groups:\n - \" Shared Secrets \"\n---\n" - .to_string(); + let src = "---\nname: t\ndescription: d\nvariable-groups:\n - \" Shared Secrets \"\n---\n" + .to_string(); let (fm, _) = parse_markdown(&src).unwrap(); let err = validate_variable_groups(&fm).unwrap_err().to_string(); assert!(err.contains("is not a valid"), "err: {err}"); @@ -3560,6 +3717,7 @@ mod tests { repo_type: "git".to_string(), name: "org/my-repo".to_string(), repo_ref: "refs/heads/main".to_string(), + endpoint: None, }]; let checkout = vec!["my-repo".to_string()]; let result = validate_checkout_list(&repos, &checkout); @@ -3573,6 +3731,7 @@ mod tests { repo_type: "git".to_string(), name: "org/my-repo".to_string(), repo_ref: "refs/heads/main".to_string(), + endpoint: None, }]; let checkout = vec!["unknown-alias".to_string()]; let result = validate_checkout_list(&repos, &checkout); @@ -3587,6 +3746,7 @@ mod tests { repo_type: "git".to_string(), name: "org/my-repo".to_string(), repo_ref: "refs/heads/main".to_string(), + endpoint: None, }]; let result = validate_checkout_list(&repos, &[]); assert!(result.is_ok()); @@ -3601,6 +3761,7 @@ mod tests { repo_type: "git".to_string(), name: "org/repo".to_string(), repo_ref: "refs/heads/main".to_string(), + endpoint: None, }]; let checkout = vec!["repo".to_string()]; let err = validate_checkout_list(&repos, &checkout).unwrap_err(); @@ -3621,6 +3782,7 @@ mod tests { repo_type: "git".to_string(), name: "some-org/my-repo".to_string(), repo_ref: "refs/heads/main".to_string(), + endpoint: None, }]; let checkout = vec!["my-repo".to_string()]; let err = validate_checkout_self_collision(&repos, &checkout, Some("my-repo")).unwrap_err(); @@ -3638,6 +3800,7 @@ mod tests { repo_type: "git".to_string(), name: "some-org/other".to_string(), repo_ref: "refs/heads/main".to_string(), + endpoint: None, }]; let checkout = vec!["other".to_string()]; let result = validate_checkout_self_collision(&repos, &checkout, Some("my-repo")); @@ -3653,6 +3816,7 @@ mod tests { repo_type: "git".to_string(), name: "Some-Org/My-Repo".to_string(), repo_ref: "refs/heads/main".to_string(), + endpoint: None, }]; let checkout = vec!["my-repo".to_string()]; let err = validate_checkout_self_collision(&repos, &checkout, Some("my-repo")).unwrap_err(); @@ -3667,6 +3831,7 @@ mod tests { repo_type: "git".to_string(), name: "org/my-repo".to_string(), repo_ref: "refs/heads/main".to_string(), + endpoint: None, }]; let checkout = vec!["my-repo".to_string()]; let result = validate_checkout_self_collision(&repos, &checkout, None); @@ -3680,6 +3845,7 @@ mod tests { repo_type: "git".to_string(), name: "org/my-repo".to_string(), repo_ref: "refs/heads/main".to_string(), + endpoint: None, }]; let result = validate_checkout_self_collision(&repos, &[], Some("my-repo")); assert!(result.is_ok()); @@ -4867,6 +5033,55 @@ safe-outputs: ); } + #[test] + fn test_validate_safe_outputs_keys_accepts_custom_tool_config() { + // A custom tool declared under scripts/jobs, with a top-level config key + // of the same name, must validate. + let yaml = r#"--- +name: test +description: test +safe-outputs: + scripts: + send-notification: + run: node notify.js + send-notification: + require-approval: true +--- +"#; + let (fm, _) = parse_markdown(yaml).unwrap(); + assert!(validate_safe_outputs_keys(&fm).is_ok()); + } + + #[test] + fn test_validate_custom_tool_rejects_builtin_collision() { + let yaml = r#"--- +name: test +description: test +safe-outputs: + scripts: + create-pull-request: + run: evil.js +--- +"#; + let (fm, _) = parse_markdown(yaml).unwrap(); + let err = validate_safe_outputs_keys(&fm).unwrap_err().to_string(); + assert!(err.contains("collides with the built-in"), "{err}"); + } + + #[test] + fn test_validate_custom_tool_section_must_be_mapping() { + let yaml = r#"--- +name: test +description: test +safe-outputs: + scripts: "not-a-map" +--- +"#; + let (fm, _) = parse_markdown(yaml).unwrap(); + let err = validate_safe_outputs_keys(&fm).unwrap_err().to_string(); + assert!(err.contains("must be a mapping"), "{err}"); + } + #[test] fn test_validate_safe_outputs_keys_rejects_unknown_no_close_match() { let yaml = r#"--- @@ -5175,7 +5390,7 @@ safe-outputs: assert!(!result.contains("SC_READ_TOKEN")); } - // ─── engine env / generate_executor_ado_env ──────────────────────────── + // ─── engine env / agent + executor ADO env ───────────────────────────── #[test] fn test_engine_env() { @@ -5188,10 +5403,23 @@ safe-outputs: ); assert!( !result.contains("AZURE_DEVOPS_EXT_PAT"), - "ADO token is handled by MCPG, not engine env" + "base engine env must not own the ADO token; the pipeline adds it after resolving permissions.read" ); } + #[test] + fn test_generate_agent_ado_env_with_read_connection() { + let result = generate_agent_ado_env(Some("read-sc")); + assert_eq!(result, "AZURE_DEVOPS_EXT_PAT: $(SC_READ_TOKEN)"); + assert!(!result.contains("SC_WRITE_TOKEN")); + assert!(!result.contains("SYSTEM_ACCESSTOKEN")); + } + + #[test] + fn test_generate_agent_ado_env_without_read_connection() { + assert!(generate_agent_ado_env(None).is_empty()); + } + #[test] fn test_generate_executor_ado_env_with_connection() { let result = generate_executor_ado_env(Some("my-sc"), false); @@ -5273,8 +5501,8 @@ safe-outputs: #[test] fn test_model_name_rejects_single_quote() { let mut fm = minimal_front_matter(); - fm.engine = - crate::compile::types::EngineConfig::Full(Box::new(crate::compile::types::EngineOptions { + fm.engine = crate::compile::types::EngineConfig::Full(Box::new( + crate::compile::types::EngineOptions { id: Some("copilot".to_string()), model: Some("model' && echo pwned".to_string()), version: None, @@ -5286,7 +5514,8 @@ safe-outputs: timeout_minutes: None, github_app_token: None, provider: None, - })); + }, + )); let result = engine_args_for(&fm); assert!(result.is_err()); assert!( @@ -5300,8 +5529,8 @@ safe-outputs: #[test] fn test_model_name_rejects_space() { let mut fm = minimal_front_matter(); - fm.engine = - crate::compile::types::EngineConfig::Full(Box::new(crate::compile::types::EngineOptions { + fm.engine = crate::compile::types::EngineConfig::Full(Box::new( + crate::compile::types::EngineOptions { id: Some("copilot".to_string()), model: Some("model && curl evil.com".to_string()), version: None, @@ -5313,7 +5542,8 @@ safe-outputs: timeout_minutes: None, github_app_token: None, provider: None, - })); + }, + )); let result = engine_args_for(&fm); assert!(result.is_err()); } @@ -5327,8 +5557,8 @@ safe-outputs: "my_model:latest", ] { let mut fm = minimal_front_matter(); - fm.engine = - crate::compile::types::EngineConfig::Full(Box::new(crate::compile::types::EngineOptions { + fm.engine = crate::compile::types::EngineConfig::Full(Box::new( + crate::compile::types::EngineOptions { id: Some("copilot".to_string()), model: Some(name.to_string()), version: None, @@ -5340,7 +5570,8 @@ safe-outputs: timeout_minutes: None, github_app_token: None, provider: None, - })); + }, + )); let result = engine_args_for(&fm); assert!(result.is_ok(), "Model name '{}' should be valid", name); } @@ -5941,6 +6172,44 @@ safe-outputs: ); } + #[test] + fn test_generate_mcpg_config_safeoutputs_receives_custom_tool_definitions() { + let mut fm = minimal_front_matter(); + fm.safe_outputs.insert( + "scripts".to_string(), + serde_json::json!({ + "notify": { + "description": "Send a notification", + "run": "node notify.js", + "inputs": { + "message": { + "type": "string", + "required": true + } + } + } + }), + ); + + let config = generate_mcpg_config(&fm, &collect_exts_and_decls(&fm).1).unwrap(); + let args = config + .mcp_servers + .get("safeoutputs") + .unwrap() + .entrypoint_args + .as_ref() + .unwrap(); + assert!( + args.windows(2).any(|pair| { + pair == [ + "--custom-tools".to_string(), + "/safeoutputs/custom-tools.json".to_string(), + ] + }), + "SafeOutputs must load the staged custom tool definitions: {args:?}" + ); + } + #[test] fn test_generate_mcpg_config_safeoutputs_is_hardened_stdio_container() { let fm = minimal_front_matter(); @@ -6860,6 +7129,7 @@ safe-outputs: alias: None, repo_type: "git".to_string(), repo_ref: "refs/heads/main".to_string(), + endpoint: None, checkout: true, fetch_depth: None, fetch_tags: None, @@ -6877,6 +7147,7 @@ safe-outputs: alias: None, repo_type: "git".to_string(), repo_ref: "refs/heads/main".to_string(), + endpoint: None, checkout: false, fetch_depth: None, fetch_tags: None, @@ -6894,6 +7165,7 @@ safe-outputs: alias: Some("docs-v2".to_string()), repo_type: "git".to_string(), repo_ref: "refs/heads/release/2.x".to_string(), + endpoint: None, checkout: true, fetch_depth: None, fetch_tags: None, @@ -6904,6 +7176,49 @@ safe-outputs: assert_eq!(checkout, vec!["docs-v2"]); } + #[test] + fn test_repos_github_type_requires_endpoint() { + let items = vec![ReposItem::Full(RepoEntry { + name: "acme/shared".to_string(), + alias: Some("shared".to_string()), + repo_type: "github".to_string(), + repo_ref: "refs/heads/main".to_string(), + endpoint: None, + checkout: true, + fetch_depth: None, + fetch_tags: None, + })]; + let err = lower_repos(&items).unwrap_err(); + assert!(err.to_string().contains("requires an `endpoint:`"), "{err}"); + assert!(err.to_string().contains("acme/shared"), "{err}"); + } + + #[test] + fn test_repos_github_type_with_endpoint_ok() { + let items = vec![ReposItem::Full(RepoEntry { + name: "acme/shared".to_string(), + alias: Some("shared".to_string()), + repo_type: "githubenterprise".to_string(), + repo_ref: "refs/heads/main".to_string(), + endpoint: Some("shared-conn".to_string()), + checkout: true, + fetch_depth: None, + fetch_tags: None, + })]; + let (repos, _checkout, _fetch) = lower_repos(&items).unwrap(); + assert_eq!(repos[0].repo_type, "githubenterprise"); + assert_eq!(repos[0].endpoint.as_deref(), Some("shared-conn")); + } + + #[test] + fn test_repos_git_type_needs_no_endpoint() { + // Same-org Azure Repos (`git`) must NOT require an endpoint. + assert!(validate_repo_endpoint("git", &None, "proj/repo").is_ok()); + assert!(repo_type_requires_endpoint("github")); + assert!(repo_type_requires_endpoint("githubenterprise")); + assert!(!repo_type_requires_endpoint("git")); + } + #[test] fn test_repos_rejects_duplicate_aliases() { let items = vec![ @@ -6939,6 +7254,7 @@ safe-outputs: alias: Some("root".to_string()), repo_type: "git".to_string(), repo_ref: "refs/heads/main".to_string(), + endpoint: None, checkout: true, fetch_depth: None, fetch_tags: None, @@ -6958,6 +7274,7 @@ safe-outputs: alias: Some(bad.to_string()), repo_type: "git".to_string(), repo_ref: "refs/heads/main".to_string(), + endpoint: None, checkout: true, fetch_depth: None, fetch_tags: None, @@ -6975,6 +7292,7 @@ safe-outputs: alias: Some("my-tools_2".to_string()), repo_type: "git".to_string(), repo_ref: "refs/heads/main".to_string(), + endpoint: None, checkout: true, fetch_depth: None, fetch_tags: None, @@ -6992,6 +7310,7 @@ safe-outputs: alias: None, repo_type: "git".to_string(), repo_ref: "refs/heads/main".to_string(), + endpoint: None, checkout: false, fetch_depth: None, fetch_tags: None, @@ -7011,6 +7330,7 @@ safe-outputs: alias: None, repo_type: "git".to_string(), repo_ref: "refs/heads/main".to_string(), + endpoint: None, checkout: true, fetch_depth: Some(1), fetch_tags: Some(false), @@ -7036,6 +7356,7 @@ safe-outputs: alias: None, repo_type: "git".to_string(), repo_ref: "refs/heads/main".to_string(), + endpoint: None, checkout: true, fetch_depth: Some(0), fetch_tags: Some(false), @@ -7059,6 +7380,7 @@ safe-outputs: alias: None, repo_type: "git".to_string(), repo_ref: "refs/heads/main".to_string(), + endpoint: None, checkout: true, fetch_depth: Some(1), fetch_tags: None, @@ -7068,6 +7390,7 @@ safe-outputs: alias: None, repo_type: "git".to_string(), repo_ref: "refs/heads/main".to_string(), + endpoint: None, checkout: true, fetch_depth: None, fetch_tags: Some(true), @@ -7086,6 +7409,7 @@ safe-outputs: alias: Some("mine".to_string()), repo_type: "git".to_string(), repo_ref: "refs/heads/feature".to_string(), + endpoint: None, checkout: false, fetch_depth: Some(1), fetch_tags: None, @@ -7122,6 +7446,7 @@ safe-outputs: alias: None, repo_type: "git".to_string(), repo_ref: "refs/heads/main".to_string(), + endpoint: None, checkout: false, fetch_depth: Some(1), fetch_tags: Some(false), diff --git a/src/compile/custom_tools.rs b/src/compile/custom_tools.rs new file mode 100644 index 00000000..78113b10 --- /dev/null +++ b/src/compile/custom_tools.rs @@ -0,0 +1,855 @@ +//! Compile-time schema generation for config-driven custom safe-output tools. + +use std::collections::HashSet; + +use anyhow::{Context, Result, anyhow, bail, ensure}; +use serde::Serialize; +use serde_json::{Map, Value, json}; + +use crate::compile::types::FrontMatter; +use crate::secure::CommitSha; + +const CUSTOM_TOOL_LIMIT: usize = 10; +const DEFAULT_STRING_MAX_LENGTH: u64 = 4_000; +const HARD_STRING_MAX_LENGTH: u64 = 8_000; +pub const DEFAULT_CUSTOM_MAX: usize = 3; +pub const DEFAULT_CUSTOM_SCRIPT_TIMEOUT_MINUTES: u32 = 10; +pub const MAX_CUSTOM_SCRIPT_TIMEOUT_MINUTES: u32 = 60; + +pub const COMPONENT_PROVENANCE_KEYS: [&str; 5] = [ + "component-source", + "component-sha", + "manifest-digest", + "component-repo-type", + "component-endpoint", +]; + +/// A compiler-generated custom MCP tool definition. +#[derive(Debug, Clone, PartialEq)] +pub struct CustomToolSchema { + pub name: String, + pub description: String, + pub input_schema: Map, +} + +#[derive(Debug, Clone, PartialEq)] +pub enum CustomToolKind { + Scripts { + entrypoint: String, + timeout_minutes: u32, + }, + Jobs { + steps: Vec, + }, +} + +#[derive(Debug, Clone, PartialEq, Eq)] +pub struct CustomComponentDefinition { + pub alias: String, + pub checkout_dir: String, + pub source: String, + pub sha: CommitSha, + pub manifest_digest: Option, + pub repo_type: String, + pub endpoint: Option, +} + +#[derive(Debug, Clone, PartialEq)] +pub struct CustomToolDefinition { + pub name: String, + pub description: String, + pub input_schema: Map, + pub schema_digest: String, + pub max: usize, + pub env: Vec<(String, String)>, + pub kind: CustomToolKind, + pub component: Option, +} + +pub fn collect_custom_tool_definitions( + front_matter: &FrontMatter, +) -> Result> { + let mut definitions = Vec::new(); + let mut seen = HashSet::new(); + + for section in ["scripts", "jobs"] { + let Some(section_value) = front_matter.safe_outputs.get(section) else { + continue; + }; + let section_obj = section_value.as_object().ok_or_else(|| { + anyhow!("safe-outputs.{section} must be a mapping of tool name to tool definition") + })?; + let mut names: Vec<&String> = section_obj.keys().collect(); + names.sort(); + + for tool_name in names { + let tool_def = section_obj + .get(tool_name) + .expect("tool name collected from map keys"); + validate_tool_name(section, tool_name)?; + ensure!( + seen.insert(tool_name.clone()), + "custom safe-output tool '{tool_name}' is declared more than once" + ); + + let tool_obj = tool_def.as_object().ok_or_else(|| { + anyhow!( + "safe-outputs.{section}.{tool_name} must be a mapping with optional \ + description/max/executor fields and inputs" + ) + })?; + let description = optional_string(tool_obj, "description") + .with_context(|| format!("safe-outputs.{section}.{tool_name}.description"))? + .unwrap_or_default(); + let input_schema = build_input_schema(section, tool_name, tool_obj.get("inputs"))?; + let schema_digest = crate::hash::sha256_hex( + &serde_json::to_vec(&input_schema) + .context("failed to serialize custom tool schema for digest")?, + ); + let max = parse_max(section, tool_name, tool_obj.get("max"))?; + let env = parse_env(section, tool_name, tool_obj.get("env"))?; + let component = parse_component(tool_obj, section, tool_name)?; + let kind = if section == "scripts" { + let entrypoint = tool_obj + .get("entrypoint") + .or_else(|| tool_obj.get("run")) + .and_then(Value::as_str) + .ok_or_else(|| { + anyhow!("safe-outputs.scripts.{tool_name} requires `run` or `entrypoint`") + })? + .to_string(); + CustomToolKind::Scripts { + entrypoint, + timeout_minutes: parse_script_timeout( + section, + tool_name, + tool_obj.get("timeout-minutes"), + )?, + } + } else { + ensure!( + !tool_obj.contains_key("timeout-minutes"), + "safe-outputs.jobs.{tool_name}.timeout-minutes is not supported; \ + set timeouts on the authored ADO steps or job instead" + ); + let steps = tool_obj + .get("steps") + .and_then(Value::as_array) + .ok_or_else(|| anyhow!("safe-outputs.jobs.{tool_name}.steps must be a list"))? + .clone(); + CustomToolKind::Jobs { steps } + }; + + definitions.push(CustomToolDefinition { + name: tool_name.clone(), + description, + input_schema, + schema_digest, + max, + env, + kind, + component, + }); + ensure!( + definitions.len() <= CUSTOM_TOOL_LIMIT, + "custom safe-output tools per workflow must be <= {CUSTOM_TOOL_LIMIT}" + ); + } + } + + Ok(definitions) +} + +/// Reject compiler-owned component provenance in the consumer's authored front +/// matter before imports are resolved. Remote-import provenance is stamped +/// later by the merge pass; accepting these fields here would let an ordinary +/// workflow forge a repository checkout and audit identity. +pub fn reject_author_component_provenance(front_matter: &FrontMatter) -> Result<()> { + for section in ["scripts", "jobs"] { + let Some(section_value) = front_matter.safe_outputs.get(section) else { + continue; + }; + let Some(tools) = section_value.as_object() else { + continue; + }; + for (tool_name, tool_value) in tools { + let Some(tool) = tool_value.as_object() else { + continue; + }; + for key in COMPONENT_PROVENANCE_KEYS { + ensure!( + !tool.contains_key(key), + "safe-outputs.{section}.{tool_name}.{key} is compiler-owned and may only \ + be supplied by a resolved remote import" + ); + } + } + } + Ok(()) +} + +/// Generate closed JSON Schemas for custom tools under +/// `safe-outputs.scripts` and `safe-outputs.jobs`. +pub fn generate_custom_tool_schemas(front_matter: &FrontMatter) -> Result> { + collect_custom_tool_definitions(front_matter).map(|definitions| { + definitions + .into_iter() + .map(|definition| CustomToolSchema { + name: definition.name, + description: definition.description, + input_schema: definition.input_schema, + }) + .collect() + }) +} + +/// Serialize schemas to the JSON array shape consumed by the SafeOutputs MCP +/// server's `--custom-tools` loader: +/// `[{ "name": ..., "description": ..., "inputSchema": ... }]`. +pub fn custom_tools_json(schemas: &[CustomToolSchema]) -> Result { + #[derive(Serialize)] + struct CustomToolDef<'a> { + name: &'a str, + description: &'a str, + #[serde(rename = "inputSchema")] + input_schema: &'a Map, + } + + let defs: Vec<_> = schemas + .iter() + .map(|schema| CustomToolDef { + name: &schema.name, + description: &schema.description, + input_schema: &schema.input_schema, + }) + .collect(); + + serde_json::to_string(&defs).context("failed to serialize custom tool schemas") +} + +fn validate_tool_name(section: &str, tool_name: &str) -> Result<()> { + ensure!( + crate::validate::is_safe_tool_name(tool_name), + "safe-outputs.{section}.{tool_name}: invalid custom tool name \ + (must be ASCII alphanumeric/hyphens only)" + ); + ensure!( + !crate::safe_outputs::ALL_KNOWN_SAFE_OUTPUTS.contains(&tool_name), + "safe-outputs.{section}.{tool_name}: custom tool name collides with a built-in \ + safe-output tool" + ); + ensure!( + !matches!(tool_name, "scripts" | "jobs"), + "safe-outputs.{section}.{tool_name}: custom tool name is reserved for a \ + safe-outputs structural section" + ); + Ok(()) +} + +fn validate_input_name(section: &str, tool_name: &str, input_name: &str) -> Result<()> { + ensure!( + crate::validate::is_valid_parameter_name(input_name), + "safe-outputs.{section}.{tool_name}.inputs.{input_name}: invalid input name \ + (must match [A-Za-z_][A-Za-z0-9_]*)" + ); + Ok(()) +} + +fn build_input_schema( + section: &str, + tool_name: &str, + inputs: Option<&Value>, +) -> Result> { + let mut schema = Map::new(); + schema.insert("type".to_string(), Value::String("object".to_string())); + schema.insert("additionalProperties".to_string(), Value::Bool(false)); + + let mut required = Vec::new(); + let mut properties = Map::new(); + + if let Some(inputs_value) = inputs { + let inputs_obj = inputs_value.as_object().ok_or_else(|| { + anyhow!("safe-outputs.{section}.{tool_name}.inputs must be a mapping") + })?; + + for (input_name, input_def) in inputs_obj { + validate_input_name(section, tool_name, input_name)?; + let input_obj = input_def.as_object().ok_or_else(|| { + anyhow!("safe-outputs.{section}.{tool_name}.inputs.{input_name} must be a mapping") + })?; + + if required_flag(input_obj, section, tool_name, input_name)? { + required.push(Value::String(input_name.clone())); + } + properties.insert( + input_name.clone(), + scalar_schema(section, tool_name, input_name, input_obj)?, + ); + } + } + + schema.insert("required".to_string(), Value::Array(required)); + schema.insert("properties".to_string(), Value::Object(properties)); + Ok(schema) +} + +fn scalar_schema( + section: &str, + tool_name: &str, + input_name: &str, + input_obj: &Map, +) -> Result { + let input_type = input_obj + .get("type") + .and_then(Value::as_str) + .ok_or_else(|| { + anyhow!("safe-outputs.{section}.{tool_name}.inputs.{input_name}.type is required") + })?; + + match input_type { + "string" => Ok(json!({ + "type": "string", + "maxLength": string_max_length(input_obj, section, tool_name, input_name)?, + })), + "number" => Ok(json!({ "type": "number" })), + "boolean" => Ok(json!({ "type": "boolean" })), + "choice" => Ok(json!({ + "type": "string", + "enum": choice_options(input_obj, section, tool_name, input_name)?, + })), + "array" | "object" => bail!( + "safe-outputs.{section}.{tool_name}.inputs.{input_name}: agent-facing \ + custom tool inputs are scalar-only; type '{input_type}' is not supported \ + (use string, number, boolean, or choice)" + ), + other => bail!( + "safe-outputs.{section}.{tool_name}.inputs.{input_name}: unknown input type \ + '{other}' (expected string, number, boolean, or choice)" + ), + } +} + +fn string_max_length( + input_obj: &Map, + section: &str, + tool_name: &str, + input_name: &str, +) -> Result { + let Some(value) = input_obj.get("max-length") else { + return Ok(DEFAULT_STRING_MAX_LENGTH); + }; + let max = value.as_u64().ok_or_else(|| { + anyhow!( + "safe-outputs.{section}.{tool_name}.inputs.{input_name}.max-length must be \ + a positive integer" + ) + })?; + ensure!( + max > 0, + "safe-outputs.{section}.{tool_name}.inputs.{input_name}.max-length must be > 0" + ); + ensure!( + max <= HARD_STRING_MAX_LENGTH, + "safe-outputs.{section}.{tool_name}.inputs.{input_name}.max-length must be <= \ + {HARD_STRING_MAX_LENGTH}" + ); + Ok(max) +} + +fn choice_options( + input_obj: &Map, + section: &str, + tool_name: &str, + input_name: &str, +) -> Result> { + let options = input_obj.get("options").ok_or_else(|| { + anyhow!("safe-outputs.{section}.{tool_name}.inputs.{input_name}.options is required") + })?; + let options = options.as_array().ok_or_else(|| { + anyhow!("safe-outputs.{section}.{tool_name}.inputs.{input_name}.options must be a list") + })?; + ensure!( + !options.is_empty(), + "safe-outputs.{section}.{tool_name}.inputs.{input_name}.options must not be empty" + ); + + options + .iter() + .map(|option| { + option.as_str().map(str::to_string).ok_or_else(|| { + anyhow!( + "safe-outputs.{section}.{tool_name}.inputs.{input_name}.options entries \ + must be strings" + ) + }) + }) + .collect() +} + +fn required_flag( + input_obj: &Map, + section: &str, + tool_name: &str, + input_name: &str, +) -> Result { + match input_obj.get("required") { + None => Ok(false), + Some(Value::Bool(required)) => Ok(*required), + Some(_) => { + bail!("safe-outputs.{section}.{tool_name}.inputs.{input_name}.required must be boolean") + } + } +} + +fn optional_string(obj: &Map, key: &str) -> Result> { + match obj.get(key) { + None | Some(Value::Null) => Ok(None), + Some(Value::String(s)) => Ok(Some(s.clone())), + Some(_) => bail!("must be a string when present"), + } +} + +fn parse_max(section: &str, tool_name: &str, value: Option<&Value>) -> Result { + let Some(value) = value else { + return Ok(DEFAULT_CUSTOM_MAX); + }; + let max = value.as_u64().ok_or_else(|| { + anyhow!("safe-outputs.{section}.{tool_name}.max must be a positive integer") + })?; + ensure!( + max > 0, + "safe-outputs.{section}.{tool_name}.max must be a positive integer" + ); + usize::try_from(max) + .with_context(|| format!("safe-outputs.{section}.{tool_name}.max is too large")) +} + +fn parse_script_timeout(section: &str, tool_name: &str, value: Option<&Value>) -> Result { + let Some(value) = value else { + return Ok(DEFAULT_CUSTOM_SCRIPT_TIMEOUT_MINUTES); + }; + let timeout = value.as_u64().ok_or_else(|| { + anyhow!( + "safe-outputs.{section}.{tool_name}.timeout-minutes must be an integer from 1 to \ + {MAX_CUSTOM_SCRIPT_TIMEOUT_MINUTES}" + ) + })?; + ensure!( + (1..=u64::from(MAX_CUSTOM_SCRIPT_TIMEOUT_MINUTES)).contains(&timeout), + "safe-outputs.{section}.{tool_name}.timeout-minutes must be from 1 to \ + {MAX_CUSTOM_SCRIPT_TIMEOUT_MINUTES}" + ); + Ok(timeout as u32) +} + +fn parse_env( + section: &str, + tool_name: &str, + value: Option<&Value>, +) -> Result> { + let Some(value) = value else { + return Ok(Vec::new()); + }; + let env = value + .as_object() + .ok_or_else(|| anyhow!("safe-outputs.{section}.{tool_name}.env must be a mapping"))?; + let mut pairs = Vec::new(); + for (name, value) in env { + ensure!( + crate::validate::is_valid_env_var_name(name), + "safe-outputs.{section}.{tool_name}.env key `{name}` is not a valid environment variable name" + ); + let variable = value.as_str().ok_or_else(|| { + anyhow!("safe-outputs.{section}.{tool_name}.env.{name} must name an ADO variable") + })?; + ensure!( + crate::validate::is_valid_ado_variable_name(variable), + "safe-outputs.{section}.{tool_name}.env.{name} must be a valid ADO variable name" + ); + pairs.push((name.clone(), variable.to_string())); + } + pairs.sort_by(|a, b| a.0.cmp(&b.0)); + Ok(pairs) +} + +fn parse_component( + tool_obj: &Map, + section: &str, + tool_name: &str, +) -> Result> { + let source = tool_obj.get("component-source").and_then(Value::as_str); + let sha = tool_obj.get("component-sha").and_then(Value::as_str); + let has_provenance = source.is_some() + || sha.is_some() + || COMPONENT_PROVENANCE_KEYS + .iter() + .any(|key| tool_obj.contains_key(*key)); + if !has_provenance { + return Ok(None); + } + let source = source.ok_or_else(|| { + anyhow!( + "safe-outputs.{section}.{tool_name} has incomplete component provenance: \ + component-source and component-sha must be present together" + ) + })?; + let sha = sha.ok_or_else(|| { + anyhow!( + "safe-outputs.{section}.{tool_name} has incomplete component provenance: \ + component-source and component-sha must be present together" + ) + })?; + let sha = CommitSha::parse(sha) + .with_context(|| format!("safe-outputs.{section}.{tool_name}.component-sha"))?; + let mut parts = source.splitn(3, '/'); + let owner = parts.next().unwrap_or_default(); + let repo = parts.next().unwrap_or_default(); + let path = parts.next().unwrap_or_default(); + ensure!( + !owner.is_empty() && !repo.is_empty() && !path.is_empty(), + "safe-outputs.{section}.{tool_name}.component-source must be owner/repo/path" + ); + let repo_type = tool_obj + .get("component-repo-type") + .and_then(Value::as_str) + .unwrap_or("git"); + ensure!( + matches!(repo_type, "git" | "github" | "githubenterprise"), + "safe-outputs.{section}.{tool_name}.component-repo-type must be git, github, or githubenterprise" + ); + let endpoint = tool_obj + .get("component-endpoint") + .and_then(Value::as_str) + .map(str::to_string); + let alias = super::imports::alias::component_alias_identifier( + owner, + repo, + repo_type, + endpoint.as_deref(), + ); + Ok(Some(CustomComponentDefinition { + checkout_dir: format!("$(Build.SourcesDirectory)/{alias}"), + alias, + source: source.to_string(), + sha, + manifest_digest: tool_obj + .get("manifest-digest") + .and_then(Value::as_str) + .map(str::to_string), + repo_type: repo_type.to_string(), + endpoint, + })) +} + +#[cfg(test)] +mod tests { + use super::*; + use serde::Deserialize; + + fn parse_front_matter(yaml: &str) -> FrontMatter { + serde_yaml::from_str(yaml).unwrap() + } + + fn schema_by_name<'a>(schemas: &'a [CustomToolSchema], name: &str) -> &'a CustomToolSchema { + schemas.iter().find(|schema| schema.name == name).unwrap() + } + + #[test] + fn scripts_and_jobs_generate_closed_scalar_schemas() { + let fm = parse_front_matter( + r#" +name: Test +description: Test +safe-outputs: + scripts: + send-notification: + description: Send a structured notification. + max: 3 + run: node notify.js + inputs: + title: { type: string, required: true, max-length: 120 } + severity: { type: choice, options: [info, warning, critical], required: true } + jobs: + deploy-thing: + description: Deploy via ADO steps. + steps: [] + inputs: + target: { type: string, required: true } +"#, + ); + + let schemas = generate_custom_tool_schemas(&fm).unwrap(); + assert_eq!(schemas.len(), 2); + + let send = schema_by_name(&schemas, "send-notification"); + assert_eq!(send.description, "Send a structured notification."); + assert_eq!(send.input_schema["type"], "object"); + assert_eq!(send.input_schema["additionalProperties"], false); + assert_eq!( + send.input_schema["properties"]["title"], + json!({ "type": "string", "maxLength": 120 }) + ); + assert_eq!( + send.input_schema["properties"]["severity"], + json!({ "type": "string", "enum": ["info", "warning", "critical"] }) + ); + let required = send.input_schema["required"].as_array().unwrap(); + assert!(required.contains(&Value::String("title".to_string()))); + assert!(required.contains(&Value::String("severity".to_string()))); + + let deploy = schema_by_name(&schemas, "deploy-thing"); + assert_eq!( + deploy.input_schema["properties"]["target"], + json!({ "type": "string", "maxLength": DEFAULT_STRING_MAX_LENGTH }) + ); + + let definitions = collect_custom_tool_definitions(&fm).unwrap(); + let send = definitions + .iter() + .find(|definition| definition.name == "send-notification") + .unwrap(); + assert_eq!(send.max, 3); + assert!(matches!( + send.kind, + CustomToolKind::Scripts { + timeout_minutes: DEFAULT_CUSTOM_SCRIPT_TIMEOUT_MINUTES, + .. + } + )); + let deploy = definitions + .iter() + .find(|definition| definition.name == "deploy-thing") + .unwrap(); + assert_eq!(deploy.max, DEFAULT_CUSTOM_MAX); + } + + #[test] + fn script_timeout_is_bounded_and_jobs_reject_it() { + for timeout in [0, 61] { + let fm = parse_front_matter(&format!( + r#" +name: Test +description: Test +safe-outputs: + scripts: + notify: + run: ./notify + timeout-minutes: {timeout} +"# + )); + let err = collect_custom_tool_definitions(&fm).unwrap_err(); + assert!(err.to_string().contains("timeout-minutes"), "{err:#}"); + } + + let fm = parse_front_matter( + r#" +name: Test +description: Test +safe-outputs: + scripts: + notify: + run: ./notify + timeout-minutes: 60 + jobs: + deploy: + timeout-minutes: 10 + steps: [] +"#, + ); + let err = collect_custom_tool_definitions(&fm).unwrap_err(); + assert!(err.to_string().contains("not supported"), "{err:#}"); + } + + #[test] + fn component_sha_is_typed_and_partial_provenance_fails_closed() { + let fm = parse_front_matter( + r#" +name: Test +description: Test +safe-outputs: + scripts: + notify: + run: ./notify + component-source: octo/tools/components/notify.md + component-sha: not-a-full-sha +"#, + ); + let err = collect_custom_tool_definitions(&fm).unwrap_err(); + assert!(err.to_string().contains("component-sha"), "{err:#}"); + + let fm = parse_front_matter( + r#" +name: Test +description: Test +safe-outputs: + scripts: + notify: + run: ./notify + component-source: octo/tools/components/notify.md +"#, + ); + let err = collect_custom_tool_definitions(&fm).unwrap_err(); + assert!(err.to_string().contains("incomplete"), "{err:#}"); + + let fm = parse_front_matter( + r#" +name: Test +description: Test +safe-outputs: + scripts: + notify: + run: ./notify + source: attacker/repo/component.md + sha: 0123456789abcdef0123456789abcdef01234567 +"#, + ); + let definitions = collect_custom_tool_definitions(&fm).unwrap(); + assert!( + definitions[0].component.is_none(), + "legacy source/sha fields must not synthesize component provenance" + ); + } + + #[test] + fn authored_component_provenance_is_rejected_before_import_merge() { + for key in COMPONENT_PROVENANCE_KEYS { + let fm = parse_front_matter(&format!( + r#" +name: Test +description: Test +safe-outputs: + scripts: + notify: + run: ./notify + {key}: forged +"# + )); + let err = reject_author_component_provenance(&fm).unwrap_err(); + let message = err.to_string(); + assert!(message.contains("compiler-owned"), "{message}"); + assert!(message.contains(key), "{message}"); + } + } + + #[test] + fn array_and_object_agent_inputs_are_rejected() { + for input_type in ["array", "object"] { + let fm = parse_front_matter(&format!( + r#" +name: Test +description: Test +safe-outputs: + scripts: + bad-tool: + run: ./tool + inputs: + payload: {{ type: {input_type} }} +"# + )); + + let err = generate_custom_tool_schemas(&fm).unwrap_err(); + assert!(err.to_string().contains("scalar-only")); + } + } + + #[test] + fn built_in_tool_name_collisions_are_rejected() { + let fm = parse_front_matter( + r#" +name: Test +description: Test +safe-outputs: + scripts: + create-work-item: + run: ./tool + inputs: {} +"#, + ); + + let err = generate_custom_tool_schemas(&fm).unwrap_err(); + assert!(err.to_string().contains("collides with a built-in")); + } + + #[test] + fn structural_section_names_are_rejected_as_custom_tools() { + for name in ["scripts", "jobs"] { + let fm = parse_front_matter(&format!( + r#" +name: Test +description: Test +safe-outputs: + scripts: + {name}: + run: ./tool +"# + )); + let err = collect_custom_tool_definitions(&fm).unwrap_err(); + assert!(err.to_string().contains("reserved"), "{err:#}"); + } + } + + #[test] + fn more_than_ten_custom_tools_is_rejected() { + let mut yaml = String::from("name: Test\ndescription: Test\nsafe-outputs:\n scripts:\n"); + for i in 0..11 { + yaml.push_str(&format!( + " tool-{i}:\n run: ./tool\n inputs: {{}}\n" + )); + } + let fm = parse_front_matter(&yaml); + + let err = generate_custom_tool_schemas(&fm).unwrap_err(); + assert!( + err.to_string() + .contains("custom safe-output tools per workflow") + ); + } + + #[test] + fn custom_tools_json_uses_camel_case_input_schema() { + #[derive(Deserialize)] + struct MirrorCustomToolDef { + name: String, + description: String, + #[serde(rename = "inputSchema")] + input_schema: Map, + } + + let fm = parse_front_matter( + r#" +name: Test +description: Test +safe-outputs: + scripts: + send-notification: + description: Send a structured notification. + run: node notify.js + inputs: + title: { type: string, required: true } +"#, + ); + let schemas = generate_custom_tool_schemas(&fm).unwrap(); + let json = custom_tools_json(&schemas).unwrap(); + + assert!(json.contains("\"inputSchema\"")); + assert!(!json.contains("input_schema")); + let defs: Vec = serde_json::from_str(&json).unwrap(); + assert_eq!(defs.len(), 1); + assert_eq!(defs[0].name, "send-notification"); + assert_eq!(defs[0].description, "Send a structured notification."); + assert_eq!(defs[0].input_schema["additionalProperties"], false); + } + + #[test] + fn no_scripts_or_jobs_returns_empty_vec() { + let fm = parse_front_matter( + r#" +name: Test +description: Test +"#, + ); + + let schemas = generate_custom_tool_schemas(&fm).unwrap(); + assert!(schemas.is_empty()); + } +} diff --git a/src/compile/extensions/ado_aw_marker.rs b/src/compile/extensions/ado_aw_marker.rs index 55b8fecc..06ff0f2c 100644 --- a/src/compile/extensions/ado_aw_marker.rs +++ b/src/compile/extensions/ado_aw_marker.rs @@ -25,6 +25,7 @@ use super::{CompileContext, CompilerExtension, Declarations, ExtensionPhase}; use crate::compile::ir::condition::Condition; use crate::compile::ir::step::{BashStep, Step}; +use serde::Serialize; // ─── ado-aw marker (always-on, internal) ───────────────────────────── @@ -36,7 +37,36 @@ use crate::compile::ir::step::{BashStep, Step}; /// project-scope discovery in [`crate::ado`]. Discovery enumerates ADO /// definitions, expands each via the Pipeline Preview API, and greps /// the result for this marker. -pub struct AdoAwMarkerExtension; +#[derive(Debug, Clone, Default)] +pub struct AdoAwMarkerExtension { + custom_components: Option>, +} + +/// Provenance for a safe-output custom component imported at compile time. +/// +/// The later import-resolution plumbing is responsible for computing the +/// digest strings with [`crate::hash::sha256_hex`]. The marker extension only +/// carries and emits the resolved values. +#[derive(Debug, Clone, Serialize)] +pub struct CustomComponentProvenance { + /// Custom safe-output tool that consumes this component. + pub tool: String, + /// Import source, for example `org/repo/path`. + pub source: String, + /// Full 40-character commit SHA that the component resolved to. + pub sha: String, + pub manifest_digest: String, + pub schema_digest: String, +} + +impl AdoAwMarkerExtension { + #[cfg(test)] + pub fn new(custom_components: Vec) -> Self { + Self { + custom_components: Some(custom_components), + } + } +} impl CompilerExtension for AdoAwMarkerExtension { fn name(&self) -> &str { @@ -54,7 +84,11 @@ impl CompilerExtension for AdoAwMarkerExtension { /// Returns the two Agent-job prepare steps as typed /// `Step::Bash(BashStep)` values. fn declarations(&self, ctx: &CompileContext) -> anyhow::Result { - let Some(metadata) = CompileMetadata::from_ctx(ctx) else { + let custom_components = match &self.custom_components { + Some(custom_components) => custom_components.clone(), + None => resolved_custom_components(ctx.front_matter)?, + }; + let Some(metadata) = CompileMetadata::from_ctx(ctx, custom_components) else { return Ok(Declarations::default()); }; let agent_prepare_steps = vec![ @@ -68,6 +102,27 @@ impl CompilerExtension for AdoAwMarkerExtension { } } +fn resolved_custom_components( + front_matter: &crate::compile::types::FrontMatter, +) -> anyhow::Result> { + Ok( + crate::compile::custom_tools::collect_custom_tool_definitions(front_matter)? + .into_iter() + .filter_map(|definition| { + definition + .component + .map(|component| CustomComponentProvenance { + tool: definition.name, + source: component.source, + sha: component.sha.as_str().to_string(), + manifest_digest: component.manifest_digest.unwrap_or_default(), + schema_digest: definition.schema_digest, + }) + }) + .collect(), + ) +} + /// Build the typed [`BashStep`] form of the `# ado-aw-metadata: …` /// marker step. fn marker_bash_step(metadata: &CompileMetadata) -> BashStep { @@ -116,10 +171,14 @@ struct CompileMetadata { engine: String, model: String, agent_name: String, + custom_components: Vec, } impl CompileMetadata { - fn from_ctx(ctx: &CompileContext) -> Option { + fn from_ctx( + ctx: &CompileContext, + custom_components: Vec, + ) -> Option { let input_path = ctx.input_path?; Some(Self { source: super::super::common::normalize_source_path(input_path), @@ -144,23 +203,24 @@ impl CompileMetadata { .to_string(), }, agent_name: ctx.agent_name.to_string(), + custom_components, }) } fn marker_json(&self) -> String { - serde_json::to_string(&serde_json::json!({ + let value = self.with_custom_components(serde_json::json!({ "schema": 1, "source": &self.source, "org": &self.org, "repo": &self.repo, "version": &self.compiler_version, "target": &self.target, - })) - .unwrap() + })); + serde_json::to_string(&value).unwrap() } fn aw_info_json(&self) -> String { - serde_json::to_string(&serde_json::json!({ + let value = self.with_custom_components(serde_json::json!({ "schema": "ado-aw/aw_info/1", "source": &self.source, "org": &self.org, @@ -174,8 +234,18 @@ impl CompileMetadata { "source_version": "$(Build.SourceVersion)", "source_branch": "$(Build.SourceBranch)", "build_definition_id": "$(System.DefinitionId)", - })) - .unwrap() + })); + serde_json::to_string(&value).unwrap() + } + + fn with_custom_components(&self, mut value: serde_json::Value) -> serde_json::Value { + if !self.custom_components.is_empty() { + value.as_object_mut().unwrap().insert( + "custom_components".to_string(), + serde_json::to_value(&self.custom_components).unwrap(), + ); + } + value } } @@ -198,7 +268,7 @@ mod tests { } fn agent_prepare_steps(ctx: &CompileContext<'_>) -> Vec { - AdoAwMarkerExtension + AdoAwMarkerExtension::default() .declarations(ctx) .unwrap() .agent_prepare_steps @@ -235,6 +305,7 @@ mod tests { engine: crate::engine::Engine::Copilot, compile_dir: None, input_path: Some(input_path), + imported_prompt_body: String::new(), }; let steps = agent_prepare_steps(&ctx); assert_eq!(steps.len(), 2); @@ -284,6 +355,7 @@ mod tests { engine: crate::engine::Engine::Copilot, compile_dir: None, input_path: Some(input_path), + imported_prompt_body: String::new(), }; let steps = agent_prepare_steps(&ctx); assert_eq!(steps.len(), 2); @@ -354,6 +426,136 @@ mod tests { ); } + #[test] + fn default_marker_and_aw_info_omit_custom_components() { + let fm = parse_fm("name: t\ndescription: x\n"); + let input_path = Path::new("agents/foo.md"); + let ctx = CompileContext { + agent_name: &fm.name, + front_matter: &fm, + ado_context: None, + engine: crate::engine::Engine::Copilot, + compile_dir: None, + input_path: Some(input_path), + imported_prompt_body: String::new(), + }; + let steps = agent_prepare_steps(&ctx); + assert_eq!(steps.len(), 2); + for step in steps.iter().map(bash_step) { + assert!( + !step.script.contains("\"custom_components\""), + "default marker extension must omit custom_components:\n{}", + step.script + ); + } + } + + #[test] + fn default_extension_derives_custom_components_from_front_matter() { + let fm = parse_fm( + r#" +name: t +description: x +safe-outputs: + scripts: + notify: + run: ./notify + component-source: org/repo/components/notify.md + component-sha: 0123456789abcdef0123456789abcdef01234567 + manifest-digest: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa +"#, + ); + let input_path = Path::new("agents/foo.md"); + let ctx = CompileContext { + agent_name: &fm.name, + front_matter: &fm, + ado_context: None, + engine: crate::engine::Engine::Copilot, + compile_dir: None, + input_path: Some(input_path), + imported_prompt_body: String::new(), + }; + let steps = AdoAwMarkerExtension::default() + .declarations(&ctx) + .unwrap() + .agent_prepare_steps; + for step in steps.iter().map(bash_step) { + assert!(step.script.contains("\"tool\":\"notify\"")); + assert!( + step.script + .contains("\"source\":\"org/repo/components/notify.md\"") + ); + } + } + + #[test] + fn emits_custom_component_provenance_when_configured() { + let fm = parse_fm("name: t\ndescription: x\n"); + let input_path = Path::new("agents/foo.md"); + let ctx = CompileContext { + agent_name: &fm.name, + front_matter: &fm, + ado_context: None, + engine: crate::engine::Engine::Copilot, + compile_dir: None, + input_path: Some(input_path), + imported_prompt_body: String::new(), + }; + let component = CustomComponentProvenance { + tool: "create-service-ticket".to_string(), + source: "org/repo/components/create-pr".to_string(), + sha: "0123456789abcdef0123456789abcdef01234567".to_string(), + manifest_digest: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa" + .to_string(), + schema_digest: "bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb" + .to_string(), + }; + let steps = AdoAwMarkerExtension::new(vec![component]) + .declarations(&ctx) + .unwrap() + .agent_prepare_steps; + assert_eq!(steps.len(), 2); + + for step in steps.iter().map(bash_step) { + assert!( + step.script.contains("\"custom_components\":["), + "step missing custom_components array:\n{}", + step.script + ); + assert!( + step.script + .contains("\"source\":\"org/repo/components/create-pr\""), + "step missing component source:\n{}", + step.script + ); + assert!( + step.script.contains("\"tool\":\"create-service-ticket\""), + "step missing component tool:\n{}", + step.script + ); + assert!( + step.script + .contains("\"sha\":\"0123456789abcdef0123456789abcdef01234567\""), + "step missing component sha:\n{}", + step.script + ); + assert!( + step.script.contains( + "\"manifest_digest\":\"aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa\"" + ), + "step missing component manifest digest:\n{}", + step.script + ); + assert!( + step.script.contains( + "\"schema_digest\":\"bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb\"" + ), + "step missing component schema digest:\n{}", + step.script + ); + } + } + #[test] fn org_and_repo_embed_from_ado_context_lowercased() { // When the compiler runs inside an ADO checkout (the production @@ -373,6 +575,7 @@ mod tests { engine: crate::engine::Engine::Copilot, compile_dir: None, input_path: Some(input_path), + imported_prompt_body: String::new(), }; let steps = agent_prepare_steps(&ctx); assert_eq!(steps.len(), 2); @@ -415,6 +618,7 @@ mod tests { engine: crate::engine::Engine::Copilot, compile_dir: None, input_path: Some(input_path), + imported_prompt_body: String::new(), }; let steps = agent_prepare_steps(&ctx); assert_eq!(steps.len(), 2, "target={raw_target}"); @@ -453,8 +657,9 @@ mod tests { engine: crate::engine::Engine::Copilot, compile_dir: None, input_path: Some(input_path), + imported_prompt_body: String::new(), }; - let decl = AdoAwMarkerExtension.declarations(&ctx).unwrap(); + let decl = AdoAwMarkerExtension::default().declarations(&ctx).unwrap(); assert_eq!(decl.agent_prepare_steps.len(), 2); match (&decl.agent_prepare_steps[0], &decl.agent_prepare_steps[1]) { (Step::Bash(marker), Step::Bash(aw_info)) => { @@ -489,6 +694,7 @@ mod tests { engine: crate::engine::Engine::Copilot, compile_dir: None, input_path: Some(input_path), + imported_prompt_body: String::new(), }; let steps = agent_prepare_steps(&ctx); assert_eq!(steps.len(), 2); @@ -529,6 +735,7 @@ mod tests { engine: crate::engine::Engine::Copilot, compile_dir: None, input_path: Some(input_path), + imported_prompt_body: String::new(), }; let steps = agent_prepare_steps(&ctx); assert_eq!(steps.len(), 2); @@ -579,6 +786,7 @@ mod tests { engine: crate::engine::Engine::Copilot, compile_dir: None, input_path: Some(input_path), + imported_prompt_body: String::new(), }; let steps = agent_prepare_steps(&ctx); assert_eq!(steps.len(), 2); diff --git a/src/compile/extensions/ado_script.rs b/src/compile/extensions/ado_script.rs index a3539325..58f89ee7 100644 --- a/src/compile/extensions/ado_script.rs +++ b/src/compile/extensions/ado_script.rs @@ -107,6 +107,12 @@ pub(crate) const GITHUB_APP_TOKEN_PATH: &str = "/tmp/ado-aw-scripts/ado-script/g /// the containerized SafeOutputs MCP server can compute a diff base on /// shallow-default agent pools. pub(crate) const PREPARE_PR_BASE_PATH: &str = "/tmp/ado-aw-scripts/ado-script/prepare-pr-base.js"; +/// Path to the checkout-component bundle inside the unpacked `ado-script.zip`. +/// Runs in the isolated custom safe-output job (#1473) after the component +/// repository resource is checked out, to fetch the pinned commit and verify a +/// detached checkout of it (fail-closed) on shallow-default agent pools. +pub(crate) const CHECKOUT_COMPONENT_PATH: &str = + "/tmp/ado-aw-scripts/ado-script/checkout-component.js"; const RELEASE_BASE_URL: &str = "https://github.com/githubnext/ado-aw/releases/download"; /// Single always-on extension that owns all `ado-script` bundle wiring. @@ -491,7 +497,7 @@ pub(crate) fn install_and_download_steps_typed( vec![Step::Task(install), Step::Bash(download)] } -/// Path-anchor ADO variables exposed to the agent prompt via the runtime +/// Non-secret ADO variables exposed to the agent prompt via the runtime /// import resolver. The compiler owns this allowlist; `import.js` /// substitutes only the `$(name)` tokens it is handed — it never reads /// these from the environment (see @@ -506,9 +512,14 @@ pub(crate) fn install_and_download_steps_typed( /// variable added to this list), it would break bash argument parsing in /// the resolver step — the same pre-existing exposure as the adjacent /// `--base "$(Build.SourcesDirectory)"`. The current entries are -/// ADO-controlled path anchors that cannot contain `"`, so this is safe; -/// re-quote (or shell-escape) before adding any user-influenced variable. -const PROMPT_ADO_VARS: &[&str] = &["Build.SourcesDirectory", "Build.Repository.Name"]; +/// ADO-controlled values that cannot contain `"`, so this is safe; re-quote +/// (or shell-escape) before adding any user-influenced variable. +const PROMPT_ADO_VARS: &[&str] = &[ + "Build.SourcesDirectory", + "Build.Repository.Name", + "Build.BuildId", + "System.CollectionUri", +]; /// The resolver step that expands runtime import markers in the agent prompt. fn resolver_step_typed() -> Step { @@ -670,9 +681,36 @@ pub fn prepare_pr_base_step_typed(mode: PreparePrBaseMode, repos: &[PreparePrBas Step::Bash(step) } -/// The GitHub App token **revocation** step (issue #1316). Runs after the -/// Copilot invocation in the Agent and Detection jobs (unless -/// `skip-token-revocation` is set) to delete the minted installation token +/// The SHA-pinned component checkout step (#1473). Runs in the isolated custom +/// safe-output job after the component repository resource is checked out, +/// invoking the `checkout-component` ado-script bundle to obtain the pinned +/// commit (direct by-SHA fetch, then progressive deepening) and verify a +/// detached checkout of it — failing closed if the exact pin cannot be +/// obtained and confirmed. +/// +/// `dir` is the compiler-generated ADO path macro for the component checkout +/// (`$(Build.SourcesDirectory)/`) — DOUBLE-quoted so ADO +/// substitutes the macro before bash runs (single quotes would trip shellcheck +/// SC2016). `sha` is a validated full 40-char commit SHA literal, single-quoted +/// (shadow-proof). Same-org Azure Repos projects the ADO bearer; GitHub/GHE and +/// cross-org Azure Repos reuse credentials persisted by the repository checkout. +/// The step runs OUTSIDE the AWF sandbox on the build agent's normal network, +/// so it needs no AWF allowlist entry. +pub fn checkout_component_step_typed(dir: &str, sha: &str, use_system_access_token: bool) -> Step { + let script = format!( + "set -eo pipefail\nnode '{CHECKOUT_COMPONENT_PATH}' --dir \"{dir}\" --sha {sha}\n", + dir = dir, + sha = sh_single_quote(sha), + ); + let step = crate::compile::ado_bundle::apply_bundle_auth_optional( + BashStep::new("Checkout pinned custom component", script) + .with_condition(Condition::Succeeded), + crate::compile::ado_bundle::Bundle::CheckoutComponent, + use_system_access_token + .then_some(crate::compile::ado_bundle::TokenSource::SystemAccessToken), + ); + Step::Bash(step) +} /// (`DELETE /installation/token`) so it does not remain valid for its full /// ~1h lifetime — matching `actions/create-github-app-token`'s default. /// @@ -1712,7 +1750,7 @@ mod tests { !resolver.script.contains("ADO_AW_IMPORT_BASE"), "resolver step must not export ADO_AW_IMPORT_BASE — base is passed via --base, not env" ); - // Each path-anchor var is passed as `--var "=$()"` so + // Each prompt var is passed as `--var "=$()"` so // ADO expands the macro at runtime and import.js substitutes the // concrete value into the prompt (consistent with inlined mode). assert!( @@ -1729,6 +1767,20 @@ mod tests { "resolver step must pass Build.Repository.Name as a --var, got: {}", resolver.script ); + assert!( + resolver + .script + .contains("--var \"Build.BuildId=$(Build.BuildId)\""), + "resolver step must pass Build.BuildId as a --var, got: {}", + resolver.script + ); + assert!( + resolver + .script + .contains("--var \"System.CollectionUri=$(System.CollectionUri)\""), + "resolver step must pass System.CollectionUri as a --var, got: {}", + resolver.script + ); } #[test] diff --git a/src/compile/extensions/mod.rs b/src/compile/extensions/mod.rs index b62b6081..534835b2 100644 --- a/src/compile/extensions/mod.rs +++ b/src/compile/extensions/mod.rs @@ -117,6 +117,11 @@ pub struct CompileContext<'a> { /// Consumed by the always-on `ado-aw-marker` compiler extension to /// embed source-path metadata in the compiled YAML. pub input_path: Option<&'a Path>, + /// Substituted, joined bodies of any imported components, inlined into the + /// agent prompt at compile time. Empty when the workflow declares no + /// imports (or on paths that do not resolve imports, e.g. `build_pipeline_ir` + /// for `inspect`/`graph`). Both `compile` and `check` resolve imports. + pub imported_prompt_body: String, } impl<'a> CompileContext<'a> { @@ -158,6 +163,7 @@ impl<'a> CompileContext<'a> { engine, compile_dir: Some(compile_dir), input_path: Some(input_path), + imported_prompt_body: String::new(), }) } @@ -234,6 +240,7 @@ impl<'a> CompileContext<'a> { engine: crate::engine::Engine::Copilot, compile_dir: None, input_path: None, + imported_prompt_body: String::new(), } } @@ -251,6 +258,7 @@ impl<'a> CompileContext<'a> { engine: crate::engine::Engine::Copilot, compile_dir: None, input_path: None, + imported_prompt_body: String::new(), } } @@ -264,6 +272,7 @@ impl<'a> CompileContext<'a> { engine: crate::engine::Engine::Copilot, compile_dir: Some(compile_dir), input_path: None, + imported_prompt_body: String::new(), } } } @@ -712,7 +721,7 @@ pub fn collect_extensions(front_matter: &FrontMatter) -> Vec { // `NodeExtension`). The user's pinned Node version then "wins last" // on PATH for the rest of the Agent job. let mut extensions = vec![ - Extension::AdoAwMarker(AdoAwMarkerExtension), + Extension::AdoAwMarker(AdoAwMarkerExtension::default()), Extension::GitHub(GitHubExtension), Extension::SafeOutputs(SafeOutputsExtension), Extension::AdoScript(Box::new({ diff --git a/src/compile/imports/alias.rs b/src/compile/imports/alias.rs new file mode 100644 index 00000000..774f6c1d --- /dev/null +++ b/src/compile/imports/alias.rs @@ -0,0 +1,195 @@ +//! Compiler-internal repository-resource identifiers for imported custom +//! safe-output components (for the runtime executor-job checkout), plus the +//! typed-endpoint → ADO repo-type/service-connection mapping and the P7 +//! parent-resource diagnostic for template targets. +//! +//! The compiler owns these aliases: authors write only `imports:` entries, and +//! the per-component provenance stamped during the merge pass carries the +//! `owner/repo` that [`alias_identifier`] turns into a stable, valid ADO +//! identifier: `import___`. The readable +//! parts replace non-ASCII-identifier characters with `_`; the fixed hash suffix +//! is derived from the original `owner/repo`, so repos that collide under simple +//! sanitization (for example `a-b/c` and `a_b/c`) still get distinct aliases. + +use crate::compile::types::{CompileTarget, ImportEndpoint}; +use crate::hash::sha256_hex; + +const HASH_SUFFIX_LEN: usize = 12; + +/// Return the stable compiler-generated repository-resource alias for a remote +/// import source. +/// +/// The alias always starts with `import_`, contains only ASCII alphanumeric +/// characters and underscores, and includes a short SHA-256 suffix of the +/// original `owner/repo` to avoid collisions from sanitization alone. +pub fn alias_identifier(owner: &str, repo: &str) -> String { + let digest = short_hash(&format!("{owner}/{repo}")); + render_alias(owner, repo, &digest) +} + +/// Return a repository-resource alias that also distinguishes provider and +/// service-connection identity. Same-org Azure Repos keeps the historical alias +/// for compatibility; every other provider/endpoint is included in the hash so +/// identical `owner/repo` text cannot collapse distinct repositories. +pub fn component_alias_identifier( + owner: &str, + repo: &str, + repo_type: &str, + endpoint: Option<&str>, +) -> String { + if repo_type == "git" && endpoint.is_none() { + return alias_identifier(owner, repo); + } + let digest = short_hash(&format!( + "{repo_type}\0{}\0{owner}/{repo}", + endpoint.unwrap_or_default() + )); + render_alias(owner, repo, &digest) +} + +fn render_alias(owner: &str, repo: &str, digest: &str) -> String { + let owner = sanitize_identifier_part(owner); + let repo = sanitize_identifier_part(repo); + format!("import_{owner}_{repo}_{digest}") +} + +/// Map a typed import [`ImportEndpoint`] to the ADO repository-resource `type` +/// and the backing service-connection name (`None` for same-org Azure Repos). +/// +/// Single source of truth for the compile-time component-provenance stamping in +/// [`crate::compile::imports::merge`], so the runtime component checkout uses +/// the correct repo type + service connection for GitHub / GitHub Enterprise / +/// cross-org Azure Repos components (not a hardcoded `git` / no-endpoint). +pub(crate) fn endpoint_repo_type_and_connection( + endpoint: Option<&ImportEndpoint>, +) -> (&'static str, Option) { + match endpoint { + None => ("git", None), + Some(ImportEndpoint::AzureReposCrossOrg { name, .. }) => ("git", Some(name.clone())), + Some(ImportEndpoint::GitHub { name }) => ("github", Some(name.clone())), + Some(ImportEndpoint::GitHubEnterprise { name, .. }) => { + ("githubenterprise", Some(name.clone())) + } + } +} + +fn sanitize_identifier_part(value: &str) -> String { + let sanitized: String = value + .chars() + .map(|ch| { + if ch.is_ascii_alphanumeric() || ch == '_' { + ch + } else { + '_' + } + }) + .collect(); + + if sanitized.is_empty() { + "_".to_string() + } else { + sanitized + } +} + +fn short_hash(value: &str) -> String { + sha256_hex(value.as_bytes())[..HASH_SUFFIX_LEN].to_string() +} + +/// Diagnostic (P7): job/stage compile targets are *templates* and cannot emit +/// top-level `resources.repositories`, so the **parent** pipeline must declare +/// and authorize the compiler-generated import repository aliases. Returns a +/// human-readable message listing the alias identifiers the parent must +/// declare, or `None` when the target owns its resources (standalone / 1es) or +/// there are no import aliases. +/// +/// The compiler must NOT broaden access automatically — this surfaces the +/// requirement to the pipeline administrator instead. +pub fn import_resource_parent_diagnostic( + target: CompileTarget, + aliases: &[String], +) -> Option { + if aliases.is_empty() { + return None; + } + match target { + CompileTarget::Job | CompileTarget::Stage => { + let target_name = match target { + CompileTarget::Job => "job", + CompileTarget::Stage => "stage", + _ => unreachable!(), + }; + Some(format!( + "target '{target_name}' is an Azure DevOps template and cannot declare \ + top-level repository resources; the parent pipeline must define and \ + authorize these imported component repositories: {}", + aliases.join(", ") + )) + } + CompileTarget::Standalone | CompileTarget::OneES => None, + } +} + +#[cfg(test)] +mod tests { + use super::*; + + fn assert_valid_alias(alias: &str) { + assert!(!alias.is_empty()); + assert!(!alias.as_bytes()[0].is_ascii_digit()); + assert!( + alias + .chars() + .all(|ch| ch.is_ascii_alphanumeric() || ch == '_'), + "invalid alias: {alias}" + ); + } + + #[test] + fn alias_identifier_is_stable_and_valid() { + let alias = alias_identifier("123-owner.with-dots", "repo/name"); + + assert_eq!(alias, alias_identifier("123-owner.with-dots", "repo/name")); + assert_valid_alias(&alias); + assert!(alias.starts_with("import_")); + } + + #[test] + fn component_alias_distinguishes_provider_and_endpoint() { + let same_org = component_alias_identifier("octo", "repo", "git", None); + assert_eq!(same_org, alias_identifier("octo", "repo")); + + let github = + component_alias_identifier("octo", "repo", "github", Some("github-connection")); + let ghe = + component_alias_identifier("octo", "repo", "githubenterprise", Some("ghe-connection")); + let cross_org = + component_alias_identifier("octo", "repo", "git", Some("other-org-connection")); + + assert_ne!(same_org, github); + assert_ne!(github, ghe); + assert_ne!(same_org, cross_org); + for alias in [github, ghe, cross_org] { + assert_valid_alias(&alias); + } + } + + #[test] + fn parent_diagnostic_emitted_for_template_targets() { + let aliases = vec!["import_owner_repo_abc".to_string()]; + let job = import_resource_parent_diagnostic(CompileTarget::Job, &aliases) + .expect("job target should require a parent diagnostic"); + assert!(job.contains("import_owner_repo_abc")); + assert!(job.contains("parent pipeline")); + assert!(import_resource_parent_diagnostic(CompileTarget::Stage, &aliases).is_some()); + } + + #[test] + fn parent_diagnostic_absent_for_owning_targets_or_no_aliases() { + let aliases = vec!["import_owner_repo_abc".to_string()]; + assert!(import_resource_parent_diagnostic(CompileTarget::Standalone, &aliases).is_none()); + assert!(import_resource_parent_diagnostic(CompileTarget::OneES, &aliases).is_none()); + // No aliases → no diagnostic even for template targets. + assert!(import_resource_parent_diagnostic(CompileTarget::Job, &[]).is_none()); + } +} diff --git a/src/compile/imports/integration_tests.rs b/src/compile/imports/integration_tests.rs new file mode 100644 index 00000000..8e7f3e90 --- /dev/null +++ b/src/compile/imports/integration_tests.rs @@ -0,0 +1,310 @@ +use std::fs; +use std::path::{Path, PathBuf}; + +use anyhow::Result; +use serde_yaml::{Mapping, Value}; + +use super::merge::merge_resolved; +use super::{ManifestFetcher, ResolvedImport, resolve_imports}; +use crate::compile::types::{ImportEntry, ParsedImportSpec}; + +struct PanicFetcher; + +#[async_trait::async_trait] +impl ManifestFetcher for PanicFetcher { + async fn fetch(&self, _spec: &ParsedImportSpec) -> Result> { + panic!("integration tests must not fetch remote imports") + } +} + +fn temp_repo() -> tempfile::TempDir { + let root = PathBuf::from(env!("CARGO_MANIFEST_DIR")) + .join("target") + .join("imports-integration-tmp"); + fs::create_dir_all(&root).expect("create integration temp root"); + tempfile::Builder::new() + .prefix("repo-") + .tempdir_in(root) + .expect("create temp repo") +} + +fn key(name: &str) -> Value { + Value::String(name.to_string()) +} + +fn ymap(yaml: &str) -> Mapping { + match serde_yaml::from_str::(yaml).expect("valid YAML") { + Value::Mapping(mapping) => mapping, + other => panic!("expected mapping, got {other:?}"), + } +} + +fn map_get<'a>(mapping: &'a Mapping, name: &str) -> &'a Value { + mapping + .get(key(name)) + .unwrap_or_else(|| panic!("expected mapping key `{name}`")) +} + +fn map_get_mapping<'a>(mapping: &'a Mapping, name: &str) -> &'a Mapping { + value_as_mapping(map_get(mapping, name)) +} + +fn value_as_mapping(value: &Value) -> &Mapping { + match value { + Value::Mapping(mapping) => mapping, + other => panic!("expected mapping value, got {other:?}"), + } +} + +fn import_entry(uses: &str) -> ImportEntry { + ImportEntry { + uses: uses.to_string(), + with: serde_json::Map::new(), + endpoint: None, + } +} + +fn parse_workflow(path: &Path) -> (Mapping, String, Vec) { + let content = fs::read_to_string(path).expect("read workflow"); + let parts = crate::compile::common::split_markdown_front_matter(&content, true) + .expect("split workflow front matter"); + let front_matter = match serde_yaml::from_str::( + parts.yaml_raw.as_deref().expect("front matter exists"), + ) + .expect("parse workflow front matter") + { + Value::Mapping(mapping) => mapping, + other => panic!("expected workflow mapping, got {other:?}"), + }; + let imports = front_matter + .get(key("imports")) + .map(|value| { + serde_yaml::from_value::>(value.clone()).expect("deserialize imports") + }) + .unwrap_or_default(); + + (front_matter, parts.markdown_body, imports) +} + +fn write_component(dir: &Path, name: &str, content: &str) { + let path = dir.join(name); + if let Some(parent) = path.parent() { + fs::create_dir_all(parent).expect("create component parent"); + } + fs::write(path, content).expect("write component"); +} + +async fn resolve_local(entries: &[ImportEntry], base_dir: &Path) -> Vec { + resolve_imports(entries, base_dir, &PanicFetcher) + .await + .expect("resolve local imports") +} + +#[tokio::test] +async fn imports_integration_local_resolve_then_merge_consumer_wins_and_unions() { + let repo = temp_repo(); + let workflow_dir = repo.path().join("workflows"); + fs::create_dir_all(&workflow_dir).expect("create workflow dir"); + write_component( + &workflow_dir, + "components/notify.md", + r#"--- +target: 1es +tools: + edit: {} +safe-outputs: + imported-notify: + run: node scripts/notify.js + inputs: + message: + type: string +--- +Imported guidance. +"#, + ); + let workflow_path = workflow_dir.join("agent.md"); + fs::write( + &workflow_path, + r#"--- +name: consumer +description: consumer workflow +target: standalone +imports: + - components/notify.md +tools: + bash: {} +--- +Consumer guidance. +"#, + ) + .expect("write workflow"); + + let (mut consumer_fm, consumer_body, entries) = parse_workflow(&workflow_path); + let resolved = resolve_local(&entries, &workflow_dir).await; + let merged_body = + merge_resolved(&mut consumer_fm, &consumer_body, &resolved).expect("merge imports"); + + assert_eq!( + map_get(&consumer_fm, "target"), + &Value::String("standalone".into()) + ); + assert!( + !consumer_fm.contains_key(key("imports")), + "imports key should be consumed" + ); + let tools = map_get_mapping(&consumer_fm, "tools"); + assert!(tools.contains_key(key("edit")), "imported tool missing"); + assert!(tools.contains_key(key("bash")), "consumer tool missing"); + let safe_outputs = map_get_mapping(&consumer_fm, "safe-outputs"); + assert!(safe_outputs.contains_key(key("imported-notify"))); + assert_eq!(merged_body, "Imported guidance.\n\nConsumer guidance."); +} + +#[tokio::test] +async fn imports_integration_schema_inputs_are_substituted_before_merge() { + let repo = temp_repo(); + let workflow_dir = repo.path().join("workflows"); + fs::create_dir_all(&workflow_dir).expect("create workflow dir"); + write_component( + &workflow_dir, + "components/deploy.md", + r#"--- +import-schema: + destination: + type: string + required: true +safe-outputs: + deploy: + run: "deploy --to {{ inputs.destination }}" + env: + DESTINATION: "{{ inputs.destination }}" +--- +Deploy to {{ inputs.destination }}. +"#, + ); + let workflow_path = workflow_dir.join("agent.md"); + fs::write( + &workflow_path, + r#"--- +name: consumer +description: consumer workflow +imports: + - uses: components/deploy.md + with: + destination: prod-west +--- +Consumer body. +"#, + ) + .expect("write workflow"); + + let (mut consumer_fm, consumer_body, entries) = parse_workflow(&workflow_path); + let resolved = resolve_local(&entries, &workflow_dir).await; + let merged_body = + merge_resolved(&mut consumer_fm, &consumer_body, &resolved).expect("merge imports"); + + assert!(!consumer_fm.contains_key(key("import-schema"))); + let safe_outputs = map_get_mapping(&consumer_fm, "safe-outputs"); + let deploy = value_as_mapping( + safe_outputs + .get(key("deploy")) + .expect("deploy safe-output present"), + ); + assert_eq!( + map_get(deploy, "run"), + &Value::String("deploy --to prod-west".into()) + ); + let env = map_get_mapping(deploy, "env"); + assert_eq!( + map_get(env, "DESTINATION"), + &Value::String("prod-west".into()) + ); + assert_eq!(merged_body, "Deploy to prod-west.\n\nConsumer body."); +} + +#[test] +fn imports_integration_remote_specs_must_be_sha_pinned() { + let err = ImportEntry { + uses: "o/r/p.md@main".to_string(), + with: serde_json::Map::new(), + endpoint: None, + } + .parse_source() + .expect_err("branch refs must be rejected"); + + assert!( + err.to_string().contains("full 40-character commit SHA"), + "{err}" + ); +} + +#[tokio::test] +async fn imports_integration_merge_conflicts_and_safe_output_configuration() { + let repo = temp_repo(); + let workflow_dir = repo.path().join("workflows"); + fs::create_dir_all(&workflow_dir).expect("create workflow dir"); + write_component( + &workflow_dir, + "tool-one.md", + "---\ntools:\n edit: {}\n---\none\n", + ); + write_component( + &workflow_dir, + "tool-two.md", + "---\ntools:\n edit: {}\n---\ntwo\n", + ); + let duplicate_tools = resolve_local( + &[import_entry("tool-one.md"), import_entry("tool-two.md")], + &workflow_dir, + ) + .await; + let err = merge_resolved(&mut ymap("name: consumer"), "", &duplicate_tools) + .expect_err("duplicate imported tools should fail"); + assert!(err.to_string().contains("tools.edit"), "{err}"); + + write_component( + &workflow_dir, + "notify.md", + "---\nsafe-outputs:\n notify:\n run: notify.js\n---\nnotify\n", + ); + let notify = resolve_local(&[import_entry("notify.md")], &workflow_dir).await; + let err = merge_resolved( + &mut ymap("safe-outputs:\n notify:\n run: consumer.js"), + "", + ¬ify, + ) + .expect_err("consumer executor redefinition should fail"); + assert!(err.to_string().contains("executor"), "{err}"); + + let mut consumer = ymap("safe-outputs:\n notify:\n require-approval: true"); + merge_resolved(&mut consumer, "", ¬ify).expect("configuration overlay should succeed"); + let safe_outputs = map_get_mapping(&consumer, "safe-outputs"); + let notify_cfg = value_as_mapping( + safe_outputs + .get(key("notify")) + .expect("notify safe-output present"), + ); + assert_eq!( + map_get(notify_cfg, "run"), + &Value::String("notify.js".into()) + ); + assert_eq!(map_get(notify_cfg, "require-approval"), &Value::Bool(true)); +} + +#[tokio::test] +async fn imports_integration_resolve_enforces_import_count_limit() { + let repo = temp_repo(); + let entries: Vec = (0..21) + .map(|idx| import_entry(&format!("missing-{idx}.md?"))) + .collect(); + + let err = resolve_imports(&entries, repo.path(), &PanicFetcher) + .await + .expect_err("more than 20 imports should fail before resolution"); + + assert!( + err.to_string() + .contains("imports per workflow must be <= 20"), + "{err}" + ); +} diff --git a/src/compile/imports/merge.rs b/src/compile/imports/merge.rs new file mode 100644 index 00000000..792aa1ba --- /dev/null +++ b/src/compile/imports/merge.rs @@ -0,0 +1,965 @@ +//! Consumer-wins front-matter merge for `imports:` (decision D9). +//! +//! Merges the front matter and body of resolved imported components into the +//! consumer workflow. Precedence is **consumer > later import > earlier +//! import**: +//! +//! * **Scalar / singleton keys** (`name`, `engine`, `target`, …): the +//! highest-precedence explicit setter wins. No error. +//! * **Collection keys** (`tools`, `mcp-servers`, `safe-outputs`, `runtimes`, +//! `env`): additive union by sub-key. A sub-key defined by **two different +//! imports** is a hard error. The consumer may **configure** an imported +//! `safe-outputs` tool (overlay non-executor config) but may **not** redefine +//! its executor (`steps`/`env`/`inputs`/`run`/`entrypoint`) — that is a hard +//! error. +//! * **Sequence keys** (`parameters`, `repos`, `variable-groups`): additive +//! concatenation (imports first, then consumer). +//! * **Body**: imported bodies are concatenated in declaration order, then the +//! consumer body. +//! +//! The merge runs only when the consumer declares `imports:`; with no imports +//! it is never invoked, so existing workflows are unaffected. + +use std::path::Path; + +use anyhow::{Context, Result}; +use serde_yaml::{Mapping, Value}; + +use super::schema::apply_import_inputs; +use super::{ManifestFetcher, ResolvedImport, resolve_imports_with_repo_root}; +use crate::compile::custom_tools::COMPONENT_PROVENANCE_KEYS; +use crate::compile::types::ImportEntry; + +/// Front-matter keys whose values are mappings merged additively by sub-key. +const COLLECTION_MAP_KEYS: &[&str] = &["tools", "mcp-servers", "safe-outputs", "runtimes", "env"]; + +/// Front-matter keys whose values are sequences merged by concatenation. +const SEQUENCE_KEYS: &[&str] = &["parameters", "repos", "variable-groups"]; + +/// `safe-outputs` sub-keys that define a tool's executor. A consumer may +/// configure an imported tool but may not redefine these. +const EXECUTOR_KEYS: &[&str] = &["steps", "env", "inputs", "run", "entrypoint"]; + +/// Resolve the consumer's imports, apply their `import-schema` inputs, and +/// merge their front matter + body into `consumer_fm` / the returned bodies. +/// +/// Returns `(imported_body, combined_body)`: +/// - `imported_body` is the substituted, joined bodies of the imported +/// components (declaration order), with the consumer body NOT appended. +/// This is inlined into the agent prompt at compile time because imported +/// component bodies are only substituted here — they cannot be delivered by +/// the default runtime-import path (which reads the consumer's own source). +/// - `combined_body` additionally appends the consumer body, and is what +/// `inlined-imports: true` folds into the compiled YAML. +/// +/// `consumer_fm` is mutated in place and its `imports:` key is removed (imports +/// are consumed by this pass). +pub async fn merge_imports( + consumer_fm: &mut Mapping, + consumer_body: &str, + entries: &[ImportEntry], + base_dir: &Path, + repo_root: &Path, + fetcher: &dyn ManifestFetcher, +) -> Result<(String, String)> { + let resolved = resolve_imports_with_repo_root(entries, base_dir, repo_root, fetcher).await?; + let imported_body = merge_resolved_imported_body(consumer_fm, &resolved)?; + let combined_body = join_bodies(&imported_body, consumer_body); + Ok((imported_body, combined_body)) +} + +/// Join the imported-body prefix with the consumer body using the same +/// `\n\n` separator as the merge accumulator, tolerating either side being +/// empty. +fn join_bodies(imported_body: &str, consumer_body: &str) -> String { + let consumer_trimmed = consumer_body.trim(); + match (imported_body.is_empty(), consumer_trimmed.is_empty()) { + (true, _) => consumer_trimmed.to_string(), + (false, true) => imported_body.to_string(), + (false, false) => format!("{imported_body}\n\n{consumer_trimmed}"), + } +} + +/// Merge already-resolved imports (test-friendly seam that takes no fetcher). +/// +/// Returns the combined body (imported bodies in declaration order, then the +/// consumer body). Front matter is merged into `consumer_fm`. +#[cfg(test)] +pub fn merge_resolved( + consumer_fm: &mut Mapping, + consumer_body: &str, + resolved: &[ResolvedImport], +) -> Result { + let imported_body = merge_resolved_imported_body(consumer_fm, resolved)?; + Ok(join_bodies(&imported_body, consumer_body)) +} + +/// Merge resolved imports' front matter into `consumer_fm` and return the +/// substituted, joined **imported** bodies (declaration order) — the consumer +/// body is NOT appended here (see [`merge_imports`] for why the imported body +/// is tracked separately from the consumer body). +pub fn merge_resolved_imported_body( + consumer_fm: &mut Mapping, + resolved: &[ResolvedImport], +) -> Result { + // Accumulate imported front matter in declaration order (import-vs-import + // rules), then overlay the consumer on top. + let mut acc = Mapping::new(); + let mut acc_provenance: std::collections::HashMap = + std::collections::HashMap::new(); + let mut body_parts: Vec = Vec::new(); + + for (idx, import) in resolved.iter().enumerate() { + let (mut sub_fm, sub_body) = + apply_import_inputs(&import.front_matter, &import.body, &import.entry.with) + .with_context(|| { + format!( + "failed to apply import inputs for '{}'", + import.provenance.source + ) + })?; + + // Stamp compile-time component provenance (source / sha / manifest + // digest + resolved repo-type/service-connection from the typed + // endpoint) onto this import's custom safe-output tools, so the runtime + // executor job can check the component repo out at the pinned SHA. Only + // remote imports have a repo to check out. + stamp_component_provenance(&mut sub_fm, import); + + if let Value::Mapping(component_map) = &sub_fm { + merge_import_into_acc( + &mut acc, + &mut acc_provenance, + component_map, + idx, + &import.provenance.source, + )?; + } + + let trimmed = sub_body.trim(); + if !trimmed.is_empty() { + body_parts.push(trimmed.to_string()); + } + } + + // Overlay the consumer front matter on top of the accumulated imports. + overlay_consumer(&mut acc, consumer_fm)?; + + // The merged mapping replaces the consumer mapping; drop the now-consumed + // `imports` key. + acc.remove(Value::String("imports".to_string())); + *consumer_fm = acc; + + Ok(body_parts.join("\n\n")) +} + +/// Stamp compiler-owned component provenance onto the custom safe-output tools +/// (`safe-outputs.scripts.*` / `safe-outputs.jobs.*`) declared by an import, so +/// the runtime executor job can check the component repository out at the pinned +/// commit — while ensuring the compiler **fully owns** these keys. +/// +/// The `component-*` keys are compiler-owned provenance. For **every** imported +/// component (local or remote) any author-provided `component-*` value is first +/// **stripped**, so a component cannot spoof its own checkout (e.g. inject a +/// `component-endpoint` service connection or redirect `component-source`). +/// Then, for **remote** imports only, the compiler-resolved values are stamped: +/// `component-source` (owner/repo/path), `component-sha`, `manifest-digest`, +/// `component-repo-type` (`git` | `github` | `githubenterprise`), and — when the +/// endpoint names a service connection — `component-endpoint`. Local imports +/// (whose components live in the consumer repo's own checkout) end up with no +/// provenance keys and thus synthesize no separate checkout resource. +fn stamp_component_provenance(component_fm: &mut Value, import: &ResolvedImport) { + use crate::compile::types::ImportSource; + + let Value::Mapping(fm_map) = component_fm else { + return; + }; + let Some(Value::Mapping(safe_outputs)) = fm_map.get_mut("safe-outputs") else { + return; + }; + + // Remote imports carry a repo + pinned SHA to check out; local imports do + // not (their components live in the consumer's own checkout). + let remote_sha = match &import.source { + ImportSource::Remote(_) => import.provenance.sha.as_deref(), + _ => None, + }; + let (repo_type, endpoint_name) = + crate::compile::imports::alias::endpoint_repo_type_and_connection( + import.entry.endpoint.as_ref(), + ); + + for section in ["scripts", "jobs"] { + let Some(Value::Mapping(tools)) = safe_outputs.get_mut(section) else { + continue; + }; + for (_tool_name, tool_cfg) in tools.iter_mut() { + let Value::Mapping(cfg) = tool_cfg else { + continue; + }; + + // Strip any author-provided provenance first (compiler fully owns + // these keys). + for key in COMPONENT_PROVENANCE_KEYS { + cfg.remove(Value::String(key.to_string())); + } + + // Stamp compiler-resolved provenance for remote imports only. + if let Some(sha) = remote_sha { + cfg.insert( + Value::String("component-source".into()), + Value::String(import.provenance.source.clone()), + ); + cfg.insert( + Value::String("component-sha".into()), + Value::String(sha.to_string()), + ); + cfg.insert( + Value::String("manifest-digest".into()), + Value::String(import.provenance.manifest_digest.clone()), + ); + cfg.insert( + Value::String("component-repo-type".into()), + Value::String(repo_type.to_string()), + ); + if let Some(name) = &endpoint_name { + cfg.insert( + Value::String("component-endpoint".into()), + Value::String(name.clone()), + ); + } + } + } + } +} + +/// Merge one import's mapping into the accumulator, enforcing import-vs-import +/// collision rules. +fn merge_import_into_acc( + acc: &mut Mapping, + provenance: &mut std::collections::HashMap, + component: &Mapping, + import_idx: usize, + source: &str, +) -> Result<()> { + for (key, value) in component { + let key_str = match key.as_str() { + Some(k) => k.to_string(), + None => continue, + }; + // `import-schema` is consumed by substitution and must never leak into + // the merged workflow. + if key_str == "import-schema" || key_str == "imports" { + continue; + } + + if is_collection_map_key(&key_str) { + merge_map_key( + acc, + &key_str, + value, + MergeSide::Import { + idx: import_idx, + source, + }, + provenance, + )?; + } else if is_sequence_key(&key_str) { + concat_sequence(acc, &key_str, value); + } else { + // Scalar/singleton: later import wins over earlier. + acc.insert(Value::String(key_str), value.clone()); + } + } + Ok(()) +} + +/// Overlay the consumer front matter on top of the accumulated imports. +fn overlay_consumer(acc: &mut Mapping, consumer: &Mapping) -> Result<()> { + for (key, value) in consumer { + let key_str = match key.as_str() { + Some(k) => k.to_string(), + None => continue, + }; + if key_str == "imports" { + continue; + } + + if is_collection_map_key(&key_str) { + merge_map_key( + acc, + &key_str, + value, + MergeSide::Consumer, + &mut Default::default(), + )?; + } else if is_sequence_key(&key_str) { + concat_sequence(acc, &key_str, value); + } else { + // Scalar/singleton: consumer wins. + acc.insert(Value::String(key_str), value.clone()); + } + } + Ok(()) +} + +enum MergeSide<'a> { + Import { idx: usize, source: &'a str }, + Consumer, +} + +/// Merge a collection-map key (e.g. `tools`) into the accumulator, applying the +/// per-sub-key collision rules. +fn merge_map_key( + acc: &mut Mapping, + key: &str, + incoming: &Value, + side: MergeSide<'_>, + provenance: &mut std::collections::HashMap, +) -> Result<()> { + let Value::Mapping(incoming_map) = incoming else { + // Non-mapping value under a collection key: treat as scalar overwrite. + acc.insert(Value::String(key.to_string()), incoming.clone()); + return Ok(()); + }; + + let entry = acc + .entry(Value::String(key.to_string())) + .or_insert_with(|| Value::Mapping(Mapping::new())); + let Value::Mapping(existing) = entry else { + // Existing non-mapping (unusual) — replace wholesale. + *entry = incoming.clone(); + return Ok(()); + }; + + for (sub_key, sub_val) in incoming_map { + let sub_name = match sub_key.as_str() { + Some(s) => s.to_string(), + None => continue, + }; + let prov_key = format!("{key}.{sub_name}"); + let already = existing.contains_key(sub_key); + + match &side { + MergeSide::Import { idx, source } => { + if key == "safe-outputs" && matches!(sub_name.as_str(), "scripts" | "jobs") { + merge_imported_custom_safe_output_section( + existing, sub_key, sub_val, &sub_name, *idx, source, provenance, + )?; + continue; + } + if already { + // Collision between two imports is a hard error. + let prev = provenance.get(&prov_key).copied(); + if prev.is_some() && prev != Some(*idx) { + anyhow::bail!( + "import conflict: '{key}.{sub_name}' is defined by more than one \ + imported component (latest from '{source}'). Imported \ + {key} entries must have unique names." + ); + } + } + existing.insert(sub_key.clone(), sub_val.clone()); + provenance.insert(prov_key, *idx); + } + MergeSide::Consumer => { + if already && key == "safe-outputs" { + // Consumer may configure an imported tool but not redefine + // its executor. + configure_safe_output(existing, sub_key, sub_val, &sub_name)?; + } else if already { + anyhow::bail!( + "import conflict: the consumer redefines '{key}.{sub_name}', which is \ + already provided by an imported component. Collections merge \ + additively; rename or remove the duplicate." + ); + } else { + existing.insert(sub_key.clone(), sub_val.clone()); + } + } + } + } + Ok(()) +} + +fn merge_imported_custom_safe_output_section( + safe_outputs: &mut Mapping, + section_key: &Value, + incoming: &Value, + section: &str, + import_idx: usize, + source: &str, + provenance: &mut std::collections::HashMap, +) -> Result<()> { + let Value::Mapping(incoming_tools) = incoming else { + anyhow::bail!("safe-outputs.{section} must be a mapping of tool definitions"); + }; + let entry = safe_outputs + .entry(section_key.clone()) + .or_insert_with(|| Value::Mapping(Mapping::new())); + let Value::Mapping(existing_tools) = entry else { + anyhow::bail!("safe-outputs.{section} must be a mapping of tool definitions"); + }; + + for (tool_key, tool_value) in incoming_tools { + let Some(tool_name) = tool_key.as_str() else { + continue; + }; + let provenance_key = format!("safe-outputs.{section}.{tool_name}"); + if existing_tools.contains_key(tool_key) { + let previous = provenance.get(&provenance_key).copied(); + if previous.is_some() && previous != Some(import_idx) { + anyhow::bail!( + "import conflict: 'safe-outputs.{section}.{tool_name}' is defined by more \ + than one imported component (latest from '{source}'). Imported custom \ + safe-output tools must have unique names." + ); + } + } + existing_tools.insert(tool_key.clone(), tool_value.clone()); + provenance.insert(provenance_key, import_idx); + } + Ok(()) +} + +/// Overlay consumer configuration onto an imported `safe-outputs` tool without +/// allowing executor redefinition. +fn configure_safe_output( + existing: &mut Mapping, + sub_key: &Value, + incoming: &Value, + sub_name: &str, +) -> Result<()> { + let existing_val = existing.get_mut(sub_key); + match (existing_val, incoming) { + (Some(Value::Mapping(existing_cfg)), Value::Mapping(incoming_cfg)) => { + if matches!(sub_name, "scripts" | "jobs") { + return configure_custom_safe_output_section(existing_cfg, incoming_cfg); + } + for (cfg_key, cfg_val) in incoming_cfg { + if let Some(name) = cfg_key.as_str() + && EXECUTOR_KEYS.contains(&name) + { + anyhow::bail!( + "import conflict: the consumer may configure the imported \ + safe-output '{sub_name}' but not redefine its executor \ + ('{name}' is executor-defining)." + ); + } + if let Some(name) = cfg_key.as_str() + && COMPONENT_PROVENANCE_KEYS.contains(&name) + { + anyhow::bail!( + "import conflict: the consumer may not override compiler-owned \ + component provenance for imported safe-output '{sub_name}' \ + ('{name}' is compiler-owned)." + ); + } + existing_cfg.insert(cfg_key.clone(), cfg_val.clone()); + } + Ok(()) + } + // The consumer provided a non-mapping value (e.g. `true`/`null`/a + // string) for an imported safe-output tool. Configuration must be a + // mapping overlay; a scalar would silently replace the tool wholesale, + // wiping the imported executor (`run`/`steps`/`entrypoint`/…) that the + // executor-redefinition guard above is meant to protect. Reject it. + (Some(_), _) => { + anyhow::bail!( + "import conflict: the consumer must provide a mapping to configure the \ + imported safe-output '{sub_name}' (got a non-mapping value); a scalar \ + would redefine its executor." + ); + } + (None, _) => { + existing.insert(sub_key.clone(), incoming.clone()); + Ok(()) + } + } +} + +fn configure_custom_safe_output_section(existing: &mut Mapping, incoming: &Mapping) -> Result<()> { + for (tool_key, tool_value) in incoming { + let Some(tool_name) = tool_key.as_str() else { + continue; + }; + if existing.contains_key(tool_key) { + configure_safe_output(existing, tool_key, tool_value, tool_name)?; + } else { + existing.insert(tool_key.clone(), tool_value.clone()); + } + } + Ok(()) +} + +/// Concatenate a sequence-valued key (imports first, then consumer). +fn concat_sequence(acc: &mut Mapping, key: &str, incoming: &Value) { + let Value::Sequence(incoming_seq) = incoming else { + acc.insert(Value::String(key.to_string()), incoming.clone()); + return; + }; + let entry = acc + .entry(Value::String(key.to_string())) + .or_insert_with(|| Value::Sequence(Vec::new())); + if let Value::Sequence(existing) = entry { + existing.extend(incoming_seq.iter().cloned()); + } else { + *entry = incoming.clone(); + } +} + +fn is_collection_map_key(key: &str) -> bool { + COLLECTION_MAP_KEYS.contains(&key) +} + +fn is_sequence_key(key: &str) -> bool { + SEQUENCE_KEYS.contains(&key) +} + +#[cfg(test)] +mod tests { + use super::super::ImportProvenance; + use super::*; + use crate::compile::types::ImportSource; + + fn ymap(yaml: &str) -> Mapping { + match serde_yaml::from_str::(yaml).unwrap() { + Value::Mapping(m) => m, + _ => panic!("expected mapping"), + } + } + + fn resolved(fm_yaml: &str, body: &str) -> ResolvedImport { + ResolvedImport { + entry: ImportEntry { + uses: "local.md".to_string(), + with: serde_json::Map::new(), + endpoint: None, + }, + source: ImportSource::Local { + path: "local.md".to_string(), + section: None, + optional: false, + }, + front_matter: serde_yaml::from_str(fm_yaml).unwrap(), + body: body.to_string(), + provenance: ImportProvenance { + source: "local.md".to_string(), + sha: None, + manifest_digest: "d".to_string(), + }, + } + } + + const REMOTE_SHA: &str = "0123456789abcdef0123456789abcdef01234567"; + + fn remote_resolved( + fm_yaml: &str, + endpoint: Option, + ) -> ResolvedImport { + use crate::compile::types::ParsedImportSpec; + use crate::secure::CommitSha; + ResolvedImport { + entry: ImportEntry { + uses: format!("octo/repo/notify.md@{REMOTE_SHA}"), + with: serde_json::Map::new(), + endpoint: endpoint.clone(), + }, + source: ImportSource::Remote(ParsedImportSpec { + owner: "octo".to_string(), + repo: "repo".to_string(), + path: "notify.md".to_string(), + sha: CommitSha::parse(REMOTE_SHA).unwrap(), + section: None, + optional: false, + endpoint, + }), + front_matter: serde_yaml::from_str(fm_yaml).unwrap(), + body: String::new(), + provenance: ImportProvenance { + source: "octo/repo/notify.md".to_string(), + sha: Some(REMOTE_SHA.to_string()), + manifest_digest: "digest123".to_string(), + }, + } + } + + fn scripts_notify_tool(consumer: &Mapping) -> &Mapping { + consumer + .get(Value::String("safe-outputs".into())) + .and_then(Value::as_mapping) + .and_then(|so| so.get(Value::String("scripts".into()))) + .and_then(Value::as_mapping) + .and_then(|s| s.get(Value::String("notify".into()))) + .and_then(Value::as_mapping) + .expect("safe-outputs.scripts.notify present") + } + + fn tool_str<'a>(tool: &'a Mapping, key: &str) -> Option<&'a str> { + tool.get(Value::String(key.into())).and_then(Value::as_str) + } + + #[test] + fn remote_component_gets_provenance_and_endpoint_stamped() { + use crate::compile::types::ImportEndpoint; + let mut consumer = ymap("name: consumer"); + let import = remote_resolved( + "safe-outputs:\n scripts:\n notify:\n run: node n.js\n", + Some(ImportEndpoint::GitHub { + name: "gh-conn".to_string(), + }), + ); + merge_resolved_imported_body(&mut consumer, &[import]).unwrap(); + + let notify = scripts_notify_tool(&consumer); + assert_eq!( + tool_str(notify, "component-source"), + Some("octo/repo/notify.md") + ); + assert_eq!(tool_str(notify, "component-sha"), Some(REMOTE_SHA)); + assert_eq!(tool_str(notify, "manifest-digest"), Some("digest123")); + assert_eq!(tool_str(notify, "component-repo-type"), Some("github")); + assert_eq!(tool_str(notify, "component-endpoint"), Some("gh-conn")); + } + + #[test] + fn remote_same_org_azure_component_stamps_git_without_endpoint() { + let mut consumer = ymap("name: consumer"); + // Endpoint-less remote import => same-org Azure Repos (`git`, no conn). + let import = remote_resolved( + "safe-outputs:\n jobs:\n notify:\n steps:\n - bash: echo hi\n", + None, + ); + merge_resolved_imported_body(&mut consumer, &[import]).unwrap(); + + let notify = consumer + .get(Value::String("safe-outputs".into())) + .and_then(Value::as_mapping) + .and_then(|so| so.get(Value::String("jobs".into()))) + .and_then(Value::as_mapping) + .and_then(|j| j.get(Value::String("notify".into()))) + .and_then(Value::as_mapping) + .expect("safe-outputs.jobs.notify present"); + assert_eq!(tool_str(notify, "component-repo-type"), Some("git")); + assert_eq!(tool_str(notify, "component-endpoint"), None); + assert_eq!(tool_str(notify, "component-sha"), Some(REMOTE_SHA)); + } + + #[test] + fn local_component_is_not_stamped() { + let mut consumer = ymap("name: consumer"); + let import = resolved( + "safe-outputs:\n scripts:\n notify:\n run: node n.js\n", + "", + ); + merge_resolved_imported_body(&mut consumer, &[import]).unwrap(); + + let notify = scripts_notify_tool(&consumer); + assert_eq!(tool_str(notify, "component-source"), None); + assert_eq!(tool_str(notify, "component-repo-type"), None); + } + + #[test] + fn component_cannot_spoof_provenance_keys() { + // A component authoring compiler-owned provenance keys into its own + // front matter must NOT influence the checkout. For a same-org + // (endpoint-less) remote import the compiler resolves no service + // connection, so a pre-set `component-endpoint` must be stripped + // (compiler fully owns the key). A spoofed `component-source` must be + // overwritten with the real source. + let mut consumer = ymap("name: consumer"); + let import = remote_resolved( + "safe-outputs:\n scripts:\n notify:\n run: node n.js\n \ + component-endpoint: attacker-conn\n component-source: evil/repo/x.md\n", + None, + ); + merge_resolved_imported_body(&mut consumer, &[import]).unwrap(); + + let notify = scripts_notify_tool(&consumer); + // Spoofed endpoint stripped (same-org => no connection). + assert_eq!(tool_str(notify, "component-endpoint"), None); + // Spoofed source overwritten with the compiler-resolved provenance. + assert_eq!( + tool_str(notify, "component-source"), + Some("octo/repo/notify.md") + ); + assert_eq!(tool_str(notify, "component-repo-type"), Some("git")); + } + + #[test] + fn remote_endpoint_overwrites_author_provided_component_endpoint() { + use crate::compile::types::ImportEndpoint; + // When the import DOES resolve a connection, the compiler value wins + // over any author-provided one. + let mut consumer = ymap("name: consumer"); + let import = remote_resolved( + "safe-outputs:\n scripts:\n notify:\n run: node n.js\n \ + component-endpoint: attacker-conn\n", + Some(ImportEndpoint::GitHub { + name: "real-conn".to_string(), + }), + ); + merge_resolved_imported_body(&mut consumer, &[import]).unwrap(); + + let notify = scripts_notify_tool(&consumer); + assert_eq!(tool_str(notify, "component-endpoint"), Some("real-conn")); + } + + #[test] + fn consumer_cannot_override_imported_custom_tool_provenance() { + for section in ["scripts", "jobs"] { + for key in COMPONENT_PROVENANCE_KEYS { + let mut consumer = ymap(&format!( + "safe-outputs:\n {section}:\n notify:\n {key}: attacker\n" + )); + let imported_tool = if section == "scripts" { + "safe-outputs:\n scripts:\n notify:\n run: node notify.js\n" + } else { + "safe-outputs:\n jobs:\n notify:\n steps:\n - bash: echo hi\n" + }; + let err = merge_resolved_imported_body( + &mut consumer, + &[remote_resolved(imported_tool, None)], + ) + .unwrap_err(); + let message = err.to_string(); + assert!(message.contains("compiler-owned"), "{message}"); + assert!(message.contains(key), "{message}"); + } + } + } + + #[test] + fn consumer_custom_tool_configuration_preserves_executor_and_provenance() { + let mut consumer = ymap("safe-outputs:\n scripts:\n notify:\n max: 1\n"); + let import = remote_resolved( + "safe-outputs:\n scripts:\n notify:\n run: node notify.js\n", + None, + ); + merge_resolved_imported_body(&mut consumer, &[import]).unwrap(); + + let notify = scripts_notify_tool(&consumer); + assert_eq!(tool_str(notify, "run"), Some("node notify.js")); + assert_eq!( + tool_str(notify, "component-source"), + Some("octo/repo/notify.md") + ); + assert_eq!(tool_str(notify, "component-sha"), Some(REMOTE_SHA)); + assert_eq!( + notify + .get(Value::String("max".into())) + .and_then(Value::as_u64), + Some(1) + ); + } + + #[test] + fn consumer_wins_for_scalars() { + let mut consumer = ymap("engine: copilot\nname: consumer"); + let imports = vec![resolved("engine: claude\ntarget: 1es", "")]; + merge_resolved(&mut consumer, "", &imports).unwrap(); + assert_eq!( + consumer[Value::String("engine".into())], + Value::String("copilot".into()) + ); + // Import-only scalar is adopted. + assert_eq!( + consumer[Value::String("target".into())], + Value::String("1es".into()) + ); + } + + #[test] + fn imported_body_and_combined_body_split_correctly() { + // merge_resolved_imported_body returns ONLY the imported bodies; the + // combined form (via merge_resolved) additionally appends the consumer + // body. Imports-first ordering. + let mut fm = ymap("name: consumer"); + let imports = vec![resolved("{}", "Import A."), resolved("{}", "Import B.")]; + let imported = merge_resolved_imported_body(&mut fm, &imports).unwrap(); + assert_eq!(imported, "Import A.\n\nImport B."); + + let mut fm2 = ymap("name: consumer"); + let combined = merge_resolved(&mut fm2, "Consumer body.", &imports).unwrap(); + assert_eq!(combined, "Import A.\n\nImport B.\n\nConsumer body."); + } + + #[test] + fn imported_body_empty_when_no_import_bodies() { + let mut fm = ymap("name: consumer"); + let imports = vec![resolved("tools:\n edit: {}", "")]; + let imported = merge_resolved_imported_body(&mut fm, &imports).unwrap(); + assert_eq!(imported, ""); + // Combined with a consumer body yields just the consumer body. + let mut fm2 = ymap("name: consumer"); + let combined = merge_resolved(&mut fm2, "Only consumer.", &imports).unwrap(); + assert_eq!(combined, "Only consumer."); + } + + #[test] + fn later_import_wins_over_earlier_for_scalars() { + let mut consumer = ymap("name: c"); + let imports = vec![resolved("engine: a", ""), resolved("engine: b", "")]; + merge_resolved(&mut consumer, "", &imports).unwrap(); + assert_eq!( + consumer[Value::String("engine".into())], + Value::String("b".into()) + ); + } + + #[test] + fn collections_union_additively() { + let mut consumer = ymap("tools:\n bash: {}"); + let imports = vec![resolved("tools:\n edit: {}", "")]; + merge_resolved(&mut consumer, "", &imports).unwrap(); + let tools = consumer[Value::String("tools".into())] + .as_mapping() + .unwrap(); + assert!(tools.contains_key(Value::String("bash".into()))); + assert!(tools.contains_key(Value::String("edit".into()))); + } + + #[test] + fn import_vs_import_collection_collision_errors() { + let mut consumer = ymap("name: c"); + let imports = vec![ + resolved("mcp-servers:\n x:\n url: a", ""), + resolved("mcp-servers:\n x:\n url: b", ""), + ]; + let err = merge_resolved(&mut consumer, "", &imports).unwrap_err(); + assert!(err.to_string().contains("more than one"), "{err}"); + } + + #[test] + fn imports_deep_merge_distinct_custom_safe_output_tools() { + let mut consumer = ymap("name: consumer"); + let imports = vec![ + resolved( + "safe-outputs:\n scripts:\n notify:\n run: notify.sh", + "", + ), + resolved( + "safe-outputs:\n scripts:\n archive:\n run: archive.sh\n jobs:\n deploy:\n steps: []", + "", + ), + ]; + merge_resolved(&mut consumer, "", &imports).unwrap(); + + let safe_outputs = consumer + .get(Value::String("safe-outputs".into())) + .and_then(Value::as_mapping) + .unwrap(); + let scripts = safe_outputs + .get(Value::String("scripts".into())) + .and_then(Value::as_mapping) + .unwrap(); + assert!(scripts.contains_key(Value::String("notify".into()))); + assert!(scripts.contains_key(Value::String("archive".into()))); + let jobs = safe_outputs + .get(Value::String("jobs".into())) + .and_then(Value::as_mapping) + .unwrap(); + assert!(jobs.contains_key(Value::String("deploy".into()))); + } + + #[test] + fn imports_reject_duplicate_custom_safe_output_tool_names() { + let mut consumer = ymap("name: consumer"); + let imports = vec![ + resolved( + "safe-outputs:\n scripts:\n notify:\n run: one.sh", + "", + ), + resolved( + "safe-outputs:\n scripts:\n notify:\n run: two.sh", + "", + ), + ]; + let err = merge_resolved(&mut consumer, "", &imports).unwrap_err(); + let message = err.to_string(); + assert!(message.contains("safe-outputs.scripts.notify"), "{message}"); + assert!(message.contains("more than one"), "{message}"); + } + + #[test] + fn consumer_redefining_imported_tool_errors() { + let mut consumer = ymap("tools:\n edit: {}"); + let imports = vec![resolved("tools:\n edit: {}", "")]; + let err = merge_resolved(&mut consumer, "", &imports).unwrap_err(); + assert!(err.to_string().contains("redefines"), "{err}"); + } + + #[test] + fn consumer_may_configure_imported_safe_output() { + let mut consumer = ymap("safe-outputs:\n notify:\n require-approval: true"); + let imports = vec![resolved( + "safe-outputs:\n notify:\n run: node notify.js\n max: 3", + "", + )]; + merge_resolved(&mut consumer, "", &imports).unwrap(); + let so = consumer[Value::String("safe-outputs".into())] + .as_mapping() + .unwrap(); + let notify = so[Value::String("notify".into())].as_mapping().unwrap(); + assert_eq!( + notify[Value::String("require-approval".into())], + Value::Bool(true) + ); + // Imported executor config is preserved. + assert_eq!( + notify[Value::String("run".into())], + Value::String("node notify.js".into()) + ); + } + + #[test] + fn consumer_redefining_safe_output_executor_errors() { + let mut consumer = ymap("safe-outputs:\n notify:\n run: evil.js"); + let imports = vec![resolved("safe-outputs:\n notify:\n run: notify.js", "")]; + let err = merge_resolved(&mut consumer, "", &imports).unwrap_err(); + assert!(err.to_string().contains("executor"), "{err}"); + } + + #[test] + fn consumer_scalar_config_of_imported_safe_output_errors() { + // A non-mapping consumer value (e.g. `notify: null` / `true` / a string) + // for an imported safe-output tool must be rejected: overlaying it would + // wipe the imported executor (`run`/`steps`/…) wholesale, which the + // executor-redefinition guard is meant to forbid. + for scalar in ["null", "true", "\"replace\""] { + let mut consumer = ymap(&format!("safe-outputs:\n notify: {scalar}")); + let imports = vec![resolved("safe-outputs:\n notify:\n run: notify.js", "")]; + let err = merge_resolved(&mut consumer, "", &imports).unwrap_err(); + assert!( + err.to_string().contains("non-mapping value"), + "scalar `{scalar}` should be rejected: {err}" + ); + } + } + + #[test] + fn body_concatenated_imports_then_consumer() { + let mut consumer = ymap("name: c"); + let imports = vec![resolved("name: i", "IMPORT BODY")]; + let body = merge_resolved(&mut consumer, "CONSUMER BODY", &imports).unwrap(); + assert_eq!(body, "IMPORT BODY\n\nCONSUMER BODY"); + } + + #[test] + fn imports_key_removed_after_merge() { + let mut consumer = ymap("imports:\n - local.md\nname: c"); + merge_resolved(&mut consumer, "", &[]).unwrap(); + assert!(!consumer.contains_key(Value::String("imports".into()))); + } + + #[test] + fn sequences_concatenated() { + let mut consumer = ymap("parameters:\n - name: p2"); + let imports = vec![resolved("parameters:\n - name: p1", "")]; + merge_resolved(&mut consumer, "", &imports).unwrap(); + let params = consumer[Value::String("parameters".into())] + .as_sequence() + .unwrap(); + assert_eq!(params.len(), 2); + } +} diff --git a/src/compile/imports/mod.rs b/src/compile/imports/mod.rs new file mode 100644 index 00000000..7a13c5d0 --- /dev/null +++ b/src/compile/imports/mod.rs @@ -0,0 +1,1280 @@ +//! Compile-time resolution for `imports:` entries. +//! +//! This module deliberately stops at resolution: it fetches/loads the imported +//! markdown manifest, parses its front matter and body, and records provenance. +//! Merging imported content into the consumer workflow is a later compile pass. + +pub mod alias; +#[cfg(test)] +mod integration_tests; +pub mod merge; +pub mod schema; + +use std::fs; +use std::path::{Path, PathBuf}; + +use anyhow::{Context, Result}; +use async_trait::async_trait; +use base64::{Engine as _, engine::general_purpose::STANDARD}; +use serde::Deserialize; + +use crate::compile::types::{ImportEndpoint, ImportEntry, ImportSource, ParsedImportSpec}; +use crate::hash::sha256_hex; + +const MAX_IMPORTS_PER_WORKFLOW: usize = 20; +const MAX_MANIFEST_BYTES: usize = 256 * 1024; +const IMPORT_GITATTRIBUTES: &str = "# Mark all cached import files as generated\n\ +* linguist-generated=true\n\ +# Keep local cached versions on merge\n\ +* merge=ours\n"; + +/// Fetches a single SHA-pinned component manifest. +/// +/// `Send + Sync` so a `&dyn ManifestFetcher` can be held across an await in a +/// `Send` future (e.g. `build_pipeline_ir`, which the `mcp-author` tool router +/// spawns on a multi-threaded runtime). +#[async_trait] +pub trait ManifestFetcher: Send + Sync { + async fn fetch(&self, spec: &ParsedImportSpec) -> Result>; +} + +/// GitHub Contents API-backed manifest fetcher using the author's `gh` auth. +/// +/// Handles both GitHub.com ([`ImportEndpoint::GitHub`]) and GitHub Enterprise +/// ([`ImportEndpoint::GitHubEnterprise`]) sources; for GHE the target server +/// host is passed to `gh` via the `GH_HOST` environment variable. +pub struct GhCliFetcher; + +fn github_contents_api_route(owner: &str, repo: &str, path: &str, reference: &str) -> String { + let encoded_path = path + .split('/') + .map(|segment| { + percent_encoding::utf8_percent_encode(segment, crate::ado::PATH_SEGMENT).to_string() + }) + .collect::>() + .join("/"); + + format!( + "repos/{}/{}/contents/{}?ref={}", + percent_encoding::utf8_percent_encode(owner, crate::ado::PATH_SEGMENT), + percent_encoding::utf8_percent_encode(repo, crate::ado::PATH_SEGMENT), + encoded_path, + percent_encoding::utf8_percent_encode(reference, crate::ado::QUERY_VALUE) + ) +} + +#[async_trait] +impl ManifestFetcher for GhCliFetcher { + async fn fetch(&self, spec: &ParsedImportSpec) -> Result> { + let route = + github_contents_api_route(&spec.owner, &spec.repo, &spec.path, spec.sha.as_str()); + + let mut command = tokio::process::Command::new("gh"); + command.args(["api", &route]); + // GitHub Enterprise: target the configured server host. `GH_HOST` makes + // `gh api` resolve the relative route against that instance. + if let Some(ImportEndpoint::GitHubEnterprise { host, .. }) = &spec.endpoint { + command.env("GH_HOST", host.as_str()); + } + + let output = command + .output() + .await + .with_context(|| format!("failed to run `gh api {route}` for import manifest"))?; + + if !output.status.success() { + let stderr = String::from_utf8_lossy(&output.stderr); + anyhow::bail!( + "`gh api {}` failed with status {}: {}", + route, + output.status, + stderr.trim() + ); + } + + #[derive(Deserialize)] + struct ContentsResponse { + content: String, + #[serde(default)] + encoding: Option, + } + + let response: ContentsResponse = serde_json::from_slice(&output.stdout) + .with_context(|| format!("failed to parse GitHub Contents API response for {route}"))?; + if response.encoding.as_deref().unwrap_or("base64") != "base64" { + anyhow::bail!( + "GitHub Contents API response for {} used unsupported encoding {:?}", + route, + response.encoding + ); + } + + let compact_content: String = response + .content + .chars() + .filter(|ch| !ch.is_whitespace()) + .collect(); + STANDARD + .decode(compact_content.as_bytes()) + .with_context(|| { + format!("failed to base64-decode GitHub Contents API response for {route}") + }) + } +} + +/// Azure Repos-backed manifest fetcher — the **primary** compile-time source. +/// +/// Fetches a SHA-pinned manifest from the ADO Git Items API. Endpoint-less +/// imports resolve against the consumer's own organization (from `repo_root`); +/// [`ImportEndpoint::AzureReposCrossOrg`] imports resolve against the +/// organization named in the endpoint. The import spec's `owner` maps to the +/// ADO **project** and `repo` to the repository name. +/// +/// The consumer org URL and non-interactive auth are resolved **lazily on the +/// first actual fetch** (and cached), so a fully-vendored committed cache — as +/// used by `ado-aw check` — performs no `git`/`az` subprocess or network work +/// (the cache is consulted before `fetch` is ever called). A resolution failure +/// is surfaced **fail-closed** at fetch time; an Azure-Repos-typed import never +/// silently falls back to GitHub. +pub struct AdoRepoFetcher { + client: reqwest::Client, + /// Repo root used to infer the consumer org for same-org imports. + repo_root: PathBuf, + /// Lazily-resolved consumer organization collection URL (same-org imports). + context_org_url: tokio::sync::OnceCell>, + /// Lazily-resolved non-interactive ADO auth. + auth: tokio::sync::OnceCell>, +} + +fn ado_org_url_from_env(mut get: impl FnMut(&str) -> Option) -> Option { + ["AZURE_DEVOPS_ORG_URL", "SYSTEM_COLLECTIONURI"] + .into_iter() + .find_map(|name| { + let value = get(name)?; + (!value.trim().is_empty()).then(|| crate::ado::normalize_org_url(&value)) + }) +} + +impl AdoRepoFetcher { + /// Construct a fetcher that resolves org/auth lazily on first fetch, using + /// `repo_root` to infer the consumer organization for same-org imports. + pub fn new(repo_root: PathBuf) -> Self { + Self { + client: reqwest::Client::new(), + repo_root, + context_org_url: tokio::sync::OnceCell::new(), + auth: tokio::sync::OnceCell::new(), + } + } + + /// Test constructor with the org URL + auth pre-resolved (no subprocess). + #[cfg(test)] + pub fn with_resolved( + context_org_url: std::result::Result, + auth: std::result::Result, + ) -> Self { + Self { + client: reqwest::Client::new(), + repo_root: PathBuf::new(), + context_org_url: tokio::sync::OnceCell::new_with(Some(context_org_url)), + auth: tokio::sync::OnceCell::new_with(Some(auth)), + } + } + + /// Consumer org URL for same-org imports, resolved + cached on first use. + async fn context_org_url(&self) -> &std::result::Result { + self.context_org_url + .get_or_init(|| async { + if let Some(org_url) = ado_org_url_from_env(|name| std::env::var(name).ok()) { + return Ok(org_url); + } + crate::ado::resolve_ado_context(&self.repo_root, None, None) + .await + .map(|ctx| ctx.org_url) + .map_err(|e| format!("{e:#}")) + }) + .await + } + + /// Non-interactive ADO auth, resolved + cached on first use. + async fn auth(&self) -> &std::result::Result { + self.auth + .get_or_init(|| async { + crate::ado::resolve_auth_non_interactive() + .await + .map_err(|e| format!("{e:#}")) + }) + .await + } +} + +#[async_trait] +impl ManifestFetcher for AdoRepoFetcher { + async fn fetch(&self, spec: &ParsedImportSpec) -> Result> { + // Fail-closed BEFORE any lazy org/auth resolution: a GitHub/GHE-typed + // import must never reach the Azure Repos fetcher. + if let Some( + other @ (ImportEndpoint::GitHub { .. } | ImportEndpoint::GitHubEnterprise { .. }), + ) = &spec.endpoint + { + anyhow::bail!( + "internal routing error: Azure Repos fetcher received a {:?} import", + other + ); + } + + let org_url = match &spec.endpoint { + None => self.context_org_url().await.as_deref().map_err(|reason| { + anyhow::anyhow!( + "cannot fetch same-org Azure Repos import `{}/{}/{}`: {}. \ + Set AZURE_DEVOPS_ORG_URL / SYSTEM_COLLECTIONURI or run from an \ + Azure Repos checkout; to import from GitHub, add an `endpoint:`.", + spec.owner, + spec.repo, + spec.path, + reason + ) + })?, + Some(ImportEndpoint::AzureReposCrossOrg { org, .. }) => org.as_str(), + Some(_) => unreachable!("github/ghe rejected above"), + }; + + let auth = self.auth().await.as_ref().map_err(|reason| { + anyhow::anyhow!( + "cannot authenticate to Azure Repos for import `{}/{}/{}`: {}", + spec.owner, + spec.repo, + spec.path, + reason + ) + })?; + + crate::ado::fetch_git_item( + &self.client, + org_url, + &spec.owner, + &spec.repo, + &spec.path, + spec.sha.as_str(), + auth, + ) + .await + .with_context(|| { + format!( + "failed to fetch Azure Repos import manifest `{}/{}/{}@{}`", + spec.owner, + spec.repo, + spec.path, + spec.sha.as_str() + ) + }) + } +} + +/// Routes each import to the correct fetcher based on its typed endpoint, +/// guaranteeing the compile-time fetch source matches the runtime checkout +/// source (see [`crate::compile::imports::alias`]). +/// +/// - endpoint-less / [`ImportEndpoint::AzureReposCrossOrg`] → [`AdoRepoFetcher`] +/// - [`ImportEndpoint::GitHub`] / [`ImportEndpoint::GitHubEnterprise`] → [`GhCliFetcher`] +/// +/// **Fail-closed:** an Azure-Repos-intended (endpoint-less) import can never be +/// silently served by GitHub, eliminating the source-confusion class of bug. +pub struct RoutingFetcher { + ado: AdoRepoFetcher, + github: GhCliFetcher, +} + +impl RoutingFetcher { + pub fn new(ado: AdoRepoFetcher) -> Self { + Self { + ado, + github: GhCliFetcher, + } + } +} + +/// Which fetcher a given endpoint routes to. Extracted as a pure function so +/// the source-confusion guard (an Azure-Repos-intended import must never be +/// served by GitHub, and vice-versa) can be unit-tested without any network. +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub enum FetcherKind { + /// Azure Repos (same-org endpoint-less, or cross-org). + AzureRepos, + /// GitHub.com or GitHub Enterprise. + GitHub, +} + +/// Classify an import endpoint to its fetcher. Endpoint-less imports are +/// same-org Azure Repos (primary); this is the single source of truth that +/// keeps compile-time fetch routing aligned with the runtime checkout in +/// [`crate::compile::imports::alias`]. +pub fn route_endpoint(endpoint: &Option) -> FetcherKind { + match endpoint { + None | Some(ImportEndpoint::AzureReposCrossOrg { .. }) => FetcherKind::AzureRepos, + Some(ImportEndpoint::GitHub { .. }) | Some(ImportEndpoint::GitHubEnterprise { .. }) => { + FetcherKind::GitHub + } + } +} + +#[async_trait] +impl ManifestFetcher for RoutingFetcher { + async fn fetch(&self, spec: &ParsedImportSpec) -> Result> { + match route_endpoint(&spec.endpoint) { + FetcherKind::AzureRepos => self.ado.fetch(spec).await, + FetcherKind::GitHub => self.github.fetch(spec).await, + } + } +} + +/// A resolved import manifest plus source provenance. +#[derive(Debug, Clone)] +pub struct ResolvedImport { + pub entry: ImportEntry, + pub source: ImportSource, + pub front_matter: serde_yaml::Value, + pub body: String, + pub provenance: ImportProvenance, +} + +/// Audit provenance for a resolved import manifest. +#[derive(Debug, Clone, PartialEq, Eq)] +pub struct ImportProvenance { + pub source: String, + pub sha: Option, + pub manifest_digest: String, +} + +/// Resolve top-level imports using `base_dir` for both local paths and the +/// cache root — a convenience over [`resolve_imports_with_repo_root`] for the +/// repo-root case. Currently only exercised by tests; the production path +/// always threads an explicit repo root. +#[cfg(test)] +pub async fn resolve_imports( + entries: &[ImportEntry], + base_dir: &Path, + fetcher: &dyn ManifestFetcher, +) -> Result> { + resolve_imports_with_repo_root(entries, base_dir, base_dir, fetcher).await +} + +/// Resolve top-level imports using an explicit repo root for the committed +/// `.ado-aw/imports` cache. +/// +/// TODO: nested imports (depth<=3). This pass intentionally resolves only the +/// workflow's declared top-level `imports:` list; transitive resolution will +/// layer on this entry point in a later merge pass. +pub async fn resolve_imports_with_repo_root( + entries: &[ImportEntry], + base_dir: &Path, + repo_root: &Path, + fetcher: &dyn ManifestFetcher, +) -> Result> { + if entries.len() > MAX_IMPORTS_PER_WORKFLOW { + anyhow::bail!( + "imports per workflow must be <= {}, got {}", + MAX_IMPORTS_PER_WORKFLOW, + entries.len() + ); + } + + let mut resolved = Vec::new(); + for entry in entries { + if let Some(import) = resolve_one(entry, base_dir, repo_root, fetcher) + .await + .with_context(|| format!("failed to resolve import `{}`", entry.uses))? + { + resolved.push(import); + } + } + Ok(resolved) +} + +async fn resolve_one( + entry: &ImportEntry, + base_dir: &Path, + repo_root: &Path, + fetcher: &dyn ManifestFetcher, +) -> Result> { + let source = entry.parse_source()?; + match &source { + ImportSource::Local { + path, + section, + optional, + } => { + let local_path = resolve_local_path(base_dir, path)?; + let bytes = match fs::read(&local_path) { + Ok(bytes) => bytes, + Err(err) if *optional && err.kind() == std::io::ErrorKind::NotFound => { + return Ok(None); + } + Err(err) => { + return Err(err).with_context(|| { + format!("failed to read local import {}", local_path.display()) + }); + } + }; + let digest = sha256_hex(&bytes); + let (front_matter, body) = parse_manifest_bytes(&bytes, section.as_deref())?; + Ok(Some(ResolvedImport { + entry: entry.clone(), + source: source.clone(), + front_matter, + body, + provenance: ImportProvenance { + source: path.clone(), + sha: None, + manifest_digest: digest, + }, + })) + } + ImportSource::Remote(spec) => { + let bytes = read_remote_manifest(repo_root, spec, fetcher).await?; + let digest = sha256_hex(&bytes); + let (front_matter, body) = parse_manifest_bytes(&bytes, spec.section.as_deref())?; + Ok(Some(ResolvedImport { + entry: entry.clone(), + source: source.clone(), + front_matter, + body, + provenance: ImportProvenance { + source: format!("{}/{}/{}", spec.owner, spec.repo, spec.path), + sha: Some(spec.sha.as_str().to_string()), + manifest_digest: digest, + }, + })) + } + } +} + +fn resolve_local_path(base_dir: &Path, import_path: &str) -> Result { + let normalized = import_path.strip_prefix("./").unwrap_or(import_path); + let path = Path::new(normalized); + if Path::new(import_path).is_absolute() || path.is_absolute() { + anyhow::bail!("local import path must be relative, got `{}`", import_path); + } + // Accept one documented leading `./`, then reject any remaining + // path-traversal or ambiguous segments before joining. + if normalized.contains('\\') + || normalized + .split('/') + .any(|segment| segment.is_empty() || segment == "." || segment == "..") + { + anyhow::bail!( + "local import path `{}` contains an invalid segment; `.`, `..`, empty \ + segments, and backslashes are not allowed", + import_path + ); + } + Ok(base_dir.join(path)) +} + +async fn read_remote_manifest( + repo_root: &Path, + spec: &ParsedImportSpec, + fetcher: &dyn ManifestFetcher, +) -> Result> { + let cache_path = cache_path(repo_root, spec)?; + if cache_path.exists() { + let bytes = fs::read(&cache_path) + .with_context(|| format!("failed to read cached import {}", cache_path.display()))?; + enforce_manifest_size(bytes.len(), &cache_path.display().to_string())?; + // Defense-in-depth: if a digest sidecar exists (written when ado-aw + // populated the cache), verify the cached bytes still hash to it. This + // detects tampering of the committed cache file — which GitHub collapses + // in diffs (`linguist-generated`) — before the manifest is trusted at + // compile time. A missing sidecar (older cache) is tolerated. + let sidecar = digest_sidecar_path(&cache_path); + match fs::read_to_string(&sidecar) { + Ok(expected) => { + let actual = sha256_hex(&bytes); + if actual != expected.trim() { + anyhow::bail!( + "cached import {} does not match its recorded digest (expected {}, got {}); \ + the committed cache may have been tampered with — delete it to re-fetch", + cache_path.display(), + expected.trim(), + actual + ); + } + } + Err(err) if err.kind() == std::io::ErrorKind::NotFound => {} + Err(err) => { + return Err(err).with_context(|| { + format!( + "failed to read import cache digest sidecar {}", + sidecar.display() + ) + }); + } + } + return Ok(bytes); + } + + let bytes = fetcher.fetch(spec).await?; + enforce_manifest_size( + bytes.len(), + &format!("{}/{}/{}", spec.owner, spec.repo, spec.path), + )?; + + let parent = cache_path + .parent() + .context("import cache path unexpectedly has no parent")?; + fs::create_dir_all(parent).with_context(|| { + format!( + "failed to create import cache directory {}", + parent.display() + ) + })?; + ensure_import_gitattributes(repo_root)?; + fs::write(&cache_path, &bytes) + .with_context(|| format!("failed to write cached import {}", cache_path.display()))?; + // Record the digest sidecar so a later read can detect cache tampering. + fs::write(digest_sidecar_path(&cache_path), sha256_hex(&bytes)).with_context(|| { + format!( + "failed to write import cache digest sidecar for {}", + cache_path.display() + ) + })?; + Ok(bytes) +} + +/// The `.sha256` sidecar path recording a cached manifest's content digest. +fn digest_sidecar_path(cache_path: &Path) -> PathBuf { + let mut os = cache_path.as_os_str().to_os_string(); + os.push(".sha256"); + PathBuf::from(os) +} + +fn enforce_manifest_size(size: usize, source: &str) -> Result<()> { + if size > MAX_MANIFEST_BYTES { + anyhow::bail!( + "import manifest {} is {} bytes, exceeding the {} byte limit", + source, + size, + MAX_MANIFEST_BYTES + ); + } + Ok(()) +} + +fn cache_path(repo_root: &Path, spec: &ParsedImportSpec) -> Result { + validate_cache_segment("owner", &spec.owner)?; + validate_cache_segment("repo", &spec.repo)?; + let mut path = repo_root + .join(".ado-aw") + .join("imports") + .join(&spec.owner) + .join(&spec.repo) + .join(spec.sha.as_str()); + // Preserve the component's directory structure under the SHA dir (mirrors + // the source repo) rather than flattening `/` -> `_`. Flattening is NOT + // injective (`a/b.md` and `a_b.md` collide onto the same cache file), which + // would silently serve one component's manifest for another from the same + // repo+SHA and bypass the digest sidecar. `validate_import_path_segments` + // enforces the same traversal guard so joining each segment is safe. + for segment in validate_import_path_segments(&spec.path)? { + path.push(segment); + } + Ok(path) +} + +fn validate_cache_segment(label: &str, value: &str) -> Result<()> { + if value.is_empty() + || value == "." + || value == ".." + || value.contains('\\') + || value.contains('/') + { + anyhow::bail!( + "remote import {} contains an invalid path segment: `{}`", + label, + value + ); + } + Ok(()) +} + +/// Validate a remote import path and return its `/`-separated segments. +/// +/// Rejects backslashes and any empty / `.` / `..` segment so the returned +/// segments can be joined onto the cache directory without escaping it. +fn validate_import_path_segments(path: &str) -> Result> { + if path.is_empty() || path.contains('\\') { + anyhow::bail!("remote import path contains an invalid segment: `{}`", path); + } + let segments: Vec<&str> = path.split('/').collect(); + if segments + .iter() + .any(|segment| segment.is_empty() || *segment == "." || *segment == "..") + { + anyhow::bail!("remote import path contains an invalid segment: `{}`", path); + } + Ok(segments) +} + +fn ensure_import_gitattributes(repo_root: &Path) -> Result<()> { + let imports_dir = repo_root.join(".ado-aw").join("imports"); + fs::create_dir_all(&imports_dir).with_context(|| { + format!( + "failed to create import cache attributes directory {}", + imports_dir.display() + ) + })?; + let attributes_path = imports_dir.join(".gitattributes"); + if !attributes_path.exists() { + fs::write(&attributes_path, IMPORT_GITATTRIBUTES).with_context(|| { + format!( + "failed to write import cache attributes {}", + attributes_path.display() + ) + })?; + } + Ok(()) +} + +fn parse_manifest_bytes( + bytes: &[u8], + section: Option<&str>, +) -> Result<(serde_yaml::Value, String)> { + enforce_manifest_size(bytes.len(), "resolved import manifest")?; + let content = + std::str::from_utf8(bytes).context("import manifest must be valid UTF-8 markdown")?; + let parts = super::common::split_markdown_front_matter(content, false)?; + let front_matter = match parts.yaml_raw { + Some(yaml) => { + let value: serde_yaml::Value = + serde_yaml::from_str(&yaml).context("failed to parse import YAML front matter")?; + match value { + serde_yaml::Value::Mapping(ref mapping) => { + // Transitive/nested imports are not yet resolved (only a + // workflow's own top-level `imports:` is). Rather than + // silently ignore a component's own `imports:` — which would + // drop tools/config it depends on with no diagnostic — fail + // loudly until nested resolution lands. + if mapping.contains_key(serde_yaml::Value::String("imports".to_string())) { + anyhow::bail!( + "imported component declares its own `imports:`, but nested \ + (transitive) imports are not yet supported; flatten the \ + component or inline the nested import" + ); + } + value + } + serde_yaml::Value::Null => value, + other => { + anyhow::bail!( + "import YAML front matter must be a mapping/object, got {}", + yaml_value_kind(&other) + ); + } + } + } + None => serde_yaml::Value::Null, + }; + + let body = match section { + Some(name) => extract_markdown_section(&parts.markdown_body, name)?, + None => parts.markdown_body, + }; + Ok((front_matter, body)) +} + +pub(super) fn yaml_value_kind(value: &serde_yaml::Value) -> &'static str { + match value { + serde_yaml::Value::Null => "null", + serde_yaml::Value::Bool(_) => "boolean", + serde_yaml::Value::Number(_) => "number", + serde_yaml::Value::String(_) => "string", + serde_yaml::Value::Sequence(_) => "sequence/array", + serde_yaml::Value::Mapping(_) => "mapping/object", + serde_yaml::Value::Tagged(_) => "tagged value", + } +} + +/// Extract a markdown `# Name` / `## Name` section, including its heading. +fn extract_markdown_section(body: &str, section: &str) -> Result { + let lines: Vec<&str> = body.lines().collect(); + let start = lines + .iter() + .position(|line| markdown_heading(line).is_some_and(|(_, name)| name == section)) + .ok_or_else(|| anyhow::anyhow!("import section `{}` was not found", section))?; + let start_level = markdown_heading(lines[start]) + .map(|(level, _)| level) + .ok_or_else(|| { + anyhow::anyhow!( + "import section `{}` heading could not be re-parsed", + section + ) + })?; + + let end = lines + .iter() + .enumerate() + .skip(start + 1) + .find_map(|(idx, line)| match markdown_heading(line) { + Some((level, _)) if level <= start_level => Some(idx), + _ => None, + }) + .unwrap_or(lines.len()); + + Ok(lines[start..end].join("\n").trim().to_string()) +} + +fn markdown_heading(line: &str) -> Option<(usize, &str)> { + let trimmed = line.trim_start(); + let level = trimmed.bytes().take_while(|byte| *byte == b'#').count(); + if !(level == 1 || level == 2) { + return None; + } + let rest = &trimmed[level..]; + if !rest.starts_with(char::is_whitespace) { + return None; + } + let name = rest + .trim() + .trim_end_matches('#') + .trim_end() + .trim() + .trim_end_matches('\r'); + if name.is_empty() { + return None; + } + Some((level, name)) +} + +#[cfg(test)] +mod tests { + use super::*; + + const SHA: &str = "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"; + + #[test] + fn ado_org_url_env_prefers_explicit_override() { + let value = ado_org_url_from_env(|name| match name { + "AZURE_DEVOPS_ORG_URL" => Some(" https://dev.azure.com/explicit/ ".to_string()), + "SYSTEM_COLLECTIONURI" => Some("https://dev.azure.com/pipeline/".to_string()), + _ => None, + }); + assert_eq!(value.as_deref(), Some("https://dev.azure.com/explicit")); + } + + #[test] + fn ado_org_url_env_falls_back_to_pipeline_collection_uri() { + let value = ado_org_url_from_env(|name| match name { + "SYSTEM_COLLECTIONURI" => Some("https://dev.azure.com/pipeline/".to_string()), + _ => None, + }); + assert_eq!(value.as_deref(), Some("https://dev.azure.com/pipeline")); + } + + #[test] + fn ado_org_url_env_ignores_empty_values() { + let value = ado_org_url_from_env(|_| Some(" ".to_string())); + assert!(value.is_none()); + } + + struct StaticFetcher { + bytes: Vec, + } + + #[async_trait] + impl ManifestFetcher for StaticFetcher { + async fn fetch(&self, _spec: &ParsedImportSpec) -> Result> { + Ok(self.bytes.clone()) + } + } + + struct PanicFetcher; + + #[async_trait] + impl ManifestFetcher for PanicFetcher { + async fn fetch(&self, _spec: &ParsedImportSpec) -> Result> { + panic!("fetcher must not be called on cache hit") + } + } + + fn import_entry(uses: &str) -> ImportEntry { + ImportEntry { + uses: uses.to_string(), + with: serde_json::Map::new(), + endpoint: None, + } + } + + fn manifest() -> &'static [u8] { + b"---\nimport-schema:\n region:\n type: string\n---\n# Imported\nBody\n" + } + + #[test] + fn github_contents_route_encodes_segments_and_ref_without_encoding_path_separators() { + let route = github_contents_api_route( + "octo space%+?:@!\u{e9}", + "repo name%+/#", + "dir one/100%+?:@!\u{e9}&=.md", + "feature/name % +&=?#\u{e9}", + ); + + assert_eq!( + route, + concat!( + "repos/octo%20space%25+%3F%3A%40%21%C3%A9/", + "repo%20name%25+%2F%23/contents/", + "dir%20one/100%25+%3F%3A%40%21%C3%A9&=.md", + "?ref=feature%2Fname%20%25%20%2B%26%3D%3F%23%C3%A9" + ) + ); + } + + #[tokio::test] + async fn local_import_resolves_front_matter_body_and_provenance() { + let repo = tempfile::tempdir().unwrap(); + let workflow_dir = repo.path().join("workflows"); + fs::create_dir_all(&workflow_dir).unwrap(); + let import_path = workflow_dir.join("component.md"); + fs::write(&import_path, manifest()).unwrap(); + + let resolved = resolve_imports_with_repo_root( + &[import_entry("component.md")], + &workflow_dir, + repo.path(), + &PanicFetcher, + ) + .await + .unwrap(); + + assert_eq!(resolved.len(), 1); + assert!(resolved[0].front_matter["import-schema"].is_mapping()); + assert_eq!(resolved[0].body, "# Imported\nBody"); + assert_eq!(resolved[0].provenance.source, "component.md"); + assert_eq!(resolved[0].provenance.sha, None); + assert_eq!( + resolved[0].provenance.manifest_digest, + sha256_hex(manifest()) + ); + } + + #[tokio::test] + async fn local_import_accepts_one_leading_dot_slash() { + let repo = tempfile::tempdir().unwrap(); + fs::write(repo.path().join("component.md"), manifest()).unwrap(); + + let resolved = resolve_imports_with_repo_root( + &[import_entry("./component.md")], + repo.path(), + repo.path(), + &PanicFetcher, + ) + .await + .unwrap(); + + assert_eq!(resolved[0].body, "# Imported\nBody"); + } + + #[tokio::test] + async fn remote_import_fetches_writes_cache_attributes_and_records_digest() { + let repo = tempfile::tempdir().unwrap(); + let entry = import_entry(&format!("acme/shared/components/deploy.md@{SHA}")); + let fetcher = StaticFetcher { + bytes: manifest().to_vec(), + }; + + let resolved = resolve_imports_with_repo_root(&[entry], repo.path(), repo.path(), &fetcher) + .await + .unwrap(); + + let cache_file = repo + .path() + .join(".ado-aw") + .join("imports") + .join("acme") + .join("shared") + .join(SHA) + .join("components") + .join("deploy.md"); + assert!(cache_file.exists()); + assert_eq!(fs::read(&cache_file).unwrap(), manifest()); + let attributes = fs::read_to_string( + repo.path() + .join(".ado-aw") + .join("imports") + .join(".gitattributes"), + ) + .unwrap(); + assert_eq!(attributes, IMPORT_GITATTRIBUTES); + assert_eq!( + resolved[0].provenance.source, + "acme/shared/components/deploy.md" + ); + assert_eq!(resolved[0].provenance.sha.as_deref(), Some(SHA)); + assert_eq!( + resolved[0].provenance.manifest_digest, + sha256_hex(manifest()) + ); + } + + #[tokio::test] + async fn remote_import_uses_cache_without_digest_sidecar_for_backward_compatibility() { + let repo = tempfile::tempdir().unwrap(); + let entry = import_entry(&format!("acme/shared/components/deploy.md@{SHA}")); + let cache_dir = repo + .path() + .join(".ado-aw") + .join("imports") + .join("acme") + .join("shared") + .join(SHA); + fs::create_dir_all(cache_dir.join("components")).unwrap(); + fs::write(cache_dir.join("components").join("deploy.md"), manifest()).unwrap(); + + let resolved = + resolve_imports_with_repo_root(&[entry], repo.path(), repo.path(), &PanicFetcher) + .await + .unwrap(); + + assert_eq!(resolved.len(), 1); + assert_eq!(resolved[0].body, "# Imported\nBody"); + } + + #[tokio::test] + async fn cached_import_rejects_unreadable_digest_sidecar() { + let repo = tempfile::tempdir().unwrap(); + let entry = import_entry(&format!("acme/shared/components/deploy.md@{SHA}")); + let cache_file = repo + .path() + .join(".ado-aw") + .join("imports") + .join("acme") + .join("shared") + .join(SHA) + .join("components") + .join("deploy.md"); + fs::create_dir_all(cache_file.parent().unwrap()).unwrap(); + fs::write(&cache_file, manifest()).unwrap(); + let sidecar = digest_sidecar_path(&cache_file); + fs::create_dir(&sidecar).unwrap(); + + let err = resolve_imports_with_repo_root(&[entry], repo.path(), repo.path(), &PanicFetcher) + .await + .unwrap_err(); + let msg = format!("{err:#}"); + assert!( + msg.contains("failed to read import cache digest sidecar"), + "got: {msg}" + ); + assert!(msg.contains(&sidecar.display().to_string()), "got: {msg}"); + } + + #[tokio::test] + async fn colliding_flattened_paths_get_distinct_cache_files() { + // Regression: `a/b.md` and `a_b.md` from the same repo+SHA previously + // flattened to the same cache file (`a_b.md`), silently serving one + // component's manifest for the other. The preserved directory structure + // gives them distinct cache paths. + let repo = tempfile::tempdir().unwrap(); + let slash = import_entry(&format!("acme/shared/a/b.md@{SHA}")); + let under = import_entry(&format!("acme/shared/a_b.md@{SHA}")); + let fetcher = StaticFetcher { + bytes: manifest().to_vec(), + }; + resolve_imports_with_repo_root( + std::slice::from_ref(&slash), + repo.path(), + repo.path(), + &fetcher, + ) + .await + .unwrap(); + resolve_imports_with_repo_root( + std::slice::from_ref(&under), + repo.path(), + repo.path(), + &fetcher, + ) + .await + .unwrap(); + + let base = repo + .path() + .join(".ado-aw") + .join("imports") + .join("acme") + .join("shared") + .join(SHA); + // `a/b.md` -> nested; `a_b.md` -> flat sibling. Distinct files. + assert!(base.join("a").join("b.md").exists()); + assert!(base.join("a_b.md").exists()); + } + + #[tokio::test] + async fn size_cap_rejects_large_manifest() { + let repo = tempfile::tempdir().unwrap(); + let entry = import_entry(&format!("acme/shared/component.md@{SHA}")); + let fetcher = StaticFetcher { + bytes: vec![b'x'; MAX_MANIFEST_BYTES + 1], + }; + + let err = resolve_imports_with_repo_root(&[entry], repo.path(), repo.path(), &fetcher) + .await + .unwrap_err(); + + assert!(format!("{err:#}").contains("exceeding the 262144 byte limit")); + } + + #[tokio::test] + async fn nested_imports_in_component_are_rejected() { + let repo = tempfile::tempdir().unwrap(); + fs::write( + repo.path().join("component.md"), + b"---\nimports:\n - other.md\n---\n# Body\n", + ) + .unwrap(); + + let err = resolve_imports_with_repo_root( + &[import_entry("component.md")], + repo.path(), + repo.path(), + &PanicFetcher, + ) + .await + .unwrap_err(); + assert!( + format!("{err:#}").contains("nested (transitive) imports are not yet supported"), + "got: {err:#}" + ); + } + + #[tokio::test] + async fn tampered_cache_is_rejected_via_digest_sidecar() { + let repo = tempfile::tempdir().unwrap(); + let entry = import_entry(&format!("acme/shared/components/deploy.md@{SHA}")); + let fetcher = StaticFetcher { + bytes: manifest().to_vec(), + }; + // First resolve populates the cache + digest sidecar. + resolve_imports_with_repo_root( + std::slice::from_ref(&entry), + repo.path(), + repo.path(), + &fetcher, + ) + .await + .unwrap(); + + // Tamper with the committed cache file (sidecar still records the + // original digest). + let cache_file = repo + .path() + .join(".ado-aw") + .join("imports") + .join("acme") + .join("shared") + .join(SHA) + .join("components") + .join("deploy.md"); + fs::write(&cache_file, b"---\n{}\n---\n# Tampered\nevil\n").unwrap(); + + let err = resolve_imports_with_repo_root(&[entry], repo.path(), repo.path(), &PanicFetcher) + .await + .unwrap_err(); + assert!( + format!("{err:#}").contains("does not match its recorded digest"), + "got: {err:#}" + ); + } + + #[tokio::test] + async fn section_selector_extracts_only_that_section() { + let repo = tempfile::tempdir().unwrap(); + let workflow_dir = repo.path(); + fs::write( + workflow_dir.join("component.md"), + b"---\n{}\n---\n# One\none\n## Two\ntwo\n### Detail\nkeep\n## Three\nthree\n", + ) + .unwrap(); + + let resolved = resolve_imports_with_repo_root( + &[import_entry("component.md#Two")], + workflow_dir, + repo.path(), + &PanicFetcher, + ) + .await + .unwrap(); + + assert_eq!(resolved[0].body, "## Two\ntwo\n### Detail\nkeep"); + } + + #[tokio::test] + async fn optional_missing_local_import_is_skipped_and_required_missing_errors() { + let repo = tempfile::tempdir().unwrap(); + + let optional = resolve_imports_with_repo_root( + &[import_entry("missing.md?")], + repo.path(), + repo.path(), + &PanicFetcher, + ) + .await + .unwrap(); + assert!(optional.is_empty()); + + let err = resolve_imports_with_repo_root( + &[import_entry("missing.md")], + repo.path(), + repo.path(), + &PanicFetcher, + ) + .await + .unwrap_err(); + assert!( + err.to_string() + .contains("failed to resolve import `missing.md`") + ); + } + + #[tokio::test] + async fn local_import_rejects_path_traversal() { + let repo = tempfile::tempdir().unwrap(); + for spec in [ + "././x", + "a/./b", + "../x", + "./../x", + "../../etc/passwd.md", + "a/../../b.md", + "a//b", + r"a\b", + ] { + let err = resolve_imports_with_repo_root( + &[import_entry(spec)], + repo.path(), + repo.path(), + &PanicFetcher, + ) + .await + .unwrap_err(); + assert!( + format!("{err:#}").contains("invalid segment"), + "spec `{spec}` should be rejected as traversal, got: {err:#}" + ); + } + } + + #[test] + fn local_import_rejects_absolute_path() { + let repo = tempfile::tempdir().unwrap(); + let absolute = repo.path().join("component.md"); + let err = resolve_local_path(repo.path(), &absolute.to_string_lossy()).unwrap_err(); + assert!( + err.to_string() + .contains("local import path must be relative"), + "got: {err:#}" + ); + } + + #[tokio::test] + async fn imports_per_workflow_limit_is_enforced() { + let repo = tempfile::tempdir().unwrap(); + let entries: Vec = (0..=MAX_IMPORTS_PER_WORKFLOW) + .map(|idx| import_entry(&format!("component-{idx}.md?"))) + .collect(); + + let err = resolve_imports_with_repo_root(&entries, repo.path(), repo.path(), &PanicFetcher) + .await + .unwrap_err(); + + assert!( + err.to_string() + .contains("imports per workflow must be <= 20") + ); + } + + use crate::compile::types::ImportEndpoint; + use crate::secure::{CommitSha, HostName}; + + fn remote_spec(endpoint: Option) -> ParsedImportSpec { + ParsedImportSpec { + owner: "proj".to_string(), + repo: "repo".to_string(), + path: "component.md".to_string(), + sha: CommitSha::parse(SHA).unwrap(), + section: None, + optional: false, + endpoint, + } + } + + #[test] + fn route_endpoint_maps_azure_repos_sources_to_azure_fetcher() { + // Endpoint-less => same-org Azure Repos (the primary, default source). + assert_eq!(route_endpoint(&None), FetcherKind::AzureRepos); + // Cross-org Azure Repos. + assert_eq!( + route_endpoint(&Some(ImportEndpoint::AzureReposCrossOrg { + name: "conn".to_string(), + org: crate::secure::AzureDevOpsOrgUrl::parse("https://dev.azure.com/other",) + .unwrap(), + })), + FetcherKind::AzureRepos + ); + } + + #[test] + fn route_endpoint_maps_github_sources_to_github_fetcher() { + assert_eq!( + route_endpoint(&Some(ImportEndpoint::GitHub { + name: "gh-conn".to_string(), + })), + FetcherKind::GitHub + ); + assert_eq!( + route_endpoint(&Some(ImportEndpoint::GitHubEnterprise { + name: "ghe-conn".to_string(), + host: HostName::parse("ghe.acme.com").unwrap(), + })), + FetcherKind::GitHub + ); + } + + /// Fail-closed guard: an endpoint-less (same-org Azure Repos) import whose + /// org/auth could not be resolved must hard-error — it must NEVER silently + /// fall through to GitHub. Regression guard for the source-confusion bug. + #[tokio::test] + async fn ado_fetcher_endpoint_less_without_org_fails_closed() { + let fetcher = AdoRepoFetcher::with_resolved( + Err("no ADO remote".to_string()), + Err("no creds".to_string()), + ); + let err = fetcher + .fetch(&remote_spec(None)) + .await + .expect_err("must fail closed without org"); + let msg = format!("{err:#}"); + assert!( + msg.contains("same-org Azure Repos") && msg.contains("no ADO remote"), + "unexpected error: {msg}" + ); + } + + /// A GitHub-typed spec must never be accepted by the Azure Repos fetcher. + #[tokio::test] + async fn ado_fetcher_rejects_github_typed_spec() { + let fetcher = AdoRepoFetcher::with_resolved( + Ok("https://dev.azure.com/org".to_string()), + Ok(crate::ado::AdoAuth::Pat("x".to_string())), + ); + let err = fetcher + .fetch(&remote_spec(Some(ImportEndpoint::GitHub { + name: "gh".to_string(), + }))) + .await + .expect_err("azure fetcher must reject a github-typed import"); + assert!( + format!("{err:#}").contains("internal routing error"), + "unexpected error: {err:#}" + ); + } +} diff --git a/src/compile/imports/schema.rs b/src/compile/imports/schema.rs new file mode 100644 index 00000000..89f5fe95 --- /dev/null +++ b/src/compile/imports/schema.rs @@ -0,0 +1,1020 @@ +//! `import-schema` modeling, consumer-`with` validation, and +//! `{{ inputs. }}` substitution. + +use std::collections::BTreeMap; + +use anyhow::{Context, Result}; +use serde_json::{Map as JsonMap, Value as JsonValue}; +use serde_yaml::{Mapping as YamlMapping, Value as YamlValue}; + +const IMPORT_SCHEMA_KEY: &str = "import-schema"; +const PLACEHOLDER_PREFIX: &str = "inputs."; + +#[derive(Debug, Clone, PartialEq)] +pub struct ImportSchema { + pub fields: BTreeMap, +} + +#[derive(Debug, Clone, PartialEq)] +pub struct SchemaField { + pub ty: SchemaType, + pub required: bool, + pub default: Option, + pub description: Option, +} + +#[derive(Debug, Clone, PartialEq)] +pub enum SchemaType { + String, + Number, + Boolean, + Choice(Vec), + Array(Box), + Object(BTreeMap), +} + +/// Parses a component's `import-schema:` front-matter block. +/// +/// Components without `import-schema:` return an empty schema. +pub fn parse_import_schema(front_matter: &YamlValue) -> Result { + let Some(schema_value) = mapping_get(front_matter, IMPORT_SCHEMA_KEY) else { + return Ok(ImportSchema { + fields: BTreeMap::new(), + }); + }; + + let schema_map = yaml_mapping(schema_value, IMPORT_SCHEMA_KEY)?; + Ok(ImportSchema { + fields: parse_fields(schema_map, IMPORT_SCHEMA_KEY, 0)?, + }) +} + +/// Validates consumer-provided `with:` values against an import schema. +/// +/// The returned map includes validated provided values plus defaults for absent +/// fields that define `default:`. +pub fn validate_with( + schema: &ImportSchema, + with: &JsonMap, +) -> Result> { + validate_fields(&schema.fields, with, "") +} + +/// Substitutes `{{ inputs. }}` placeholders in text. +/// +/// This is an ado-aw **compile-time** replacement in the `{{ ... }}` family +/// (like `{{ workspace }}`), deliberately NOT the ADO template-expression +/// delimiter `${{ ... }}`: our substituted output is embedded directly into +/// pipeline YAML, and ADO template-processes any `${{ ... }}` it sees there, so +/// reusing that delimiter would be a footgun. A `{{` immediately preceded by +/// `$` is therefore treated as an ADO `${{ ... }}` expression and passed +/// through verbatim. +/// +/// Whitespace around the expression inside `{{ ... }}` is allowed. Dotted paths +/// (for example `config.apiKey`) access object sub-fields. Missing keys are +/// intentionally left unchanged so [`apply_import_inputs`]'s leftover-placeholder +/// guard can flag them. +pub fn substitute_inputs(text: &str, inputs: &JsonMap) -> String { + let mut output = String::with_capacity(text.len()); + let mut cursor = 0; + + while let Some(relative_start) = text[cursor..].find("{{") { + let start = cursor + relative_start; + output.push_str(&text[cursor..start]); + + // A `{{` immediately preceded by `$` is an ADO `${{ ... }}` template + // expression, never one of our markers — leave it verbatim (the `$` + // was already pushed as part of `text[cursor..start]`). + let preceded_by_dollar = start > 0 && text.as_bytes()[start - 1] == b'$'; + + let expression_start = start + 2; + let Some(relative_end) = text[expression_start..].find("}}") else { + output.push_str(&text[start..]); + return output; + }; + let expression_end = expression_start + relative_end; + let original = &text[start..expression_end + 2]; + let expression = text[expression_start..expression_end].trim(); + + if preceded_by_dollar { + output.push_str(original); + cursor = expression_end + 2; + continue; + } + + match expression.strip_prefix(PLACEHOLDER_PREFIX) { + Some(path) if !path.is_empty() => match lookup_input_path(inputs, path) { + Some(value) => output.push_str(&render_json_value(value)), + None => output.push_str(original), + }, + _ => output.push_str(original), + } + + cursor = expression_end + 2; + } + + output.push_str(&text[cursor..]); + output +} + +#[derive(Debug, PartialEq, Eq)] +enum InputPlaceholderIssue { + Unresolved(String), + Unclosed, + Malformed, +} + +/// Scans `text` for an unresolved or unclosed `{{ inputs. }}` +/// placeholder remaining after substitution. +/// +/// Unlike [`substitute_inputs`], this does NOT skip a `$`-preceded `{{`: an +/// author who mistakenly wrote a `$`-delimited `${{ inputs.x }}` form must also +/// be flagged, because ADO would otherwise template-process it in the emitted +/// YAML. Any surviving marker of the `inputs.` namespace is a compile-time +/// error (a body reference to an input not supplied by the consumer `with:` / +/// absent from the component `import-schema:`). +fn find_input_placeholder_issue(text: &str) -> Option { + let mut cursor = 0; + while let Some(relative_start) = text[cursor..].find("{{") { + let start = cursor + relative_start; + let expression_start = start + 2; + let remainder = &text[expression_start..]; + let next_close = remainder.find("}}"); + let next_open = remainder.find("{{"); + let is_input_marker = remainder.trim_start().starts_with(PLACEHOLDER_PREFIX); + + match (next_close, next_open) { + (Some(relative_end), Some(relative_open)) if relative_open < relative_end => { + if is_input_marker { + return Some(InputPlaceholderIssue::Unclosed); + } + cursor = expression_start + relative_open; + } + (Some(relative_end), _) => { + let expression = remainder[..relative_end].trim(); + if let Some(path) = expression.strip_prefix(PLACEHOLDER_PREFIX) { + if path.is_empty() { + return Some(InputPlaceholderIssue::Malformed); + } + return Some(InputPlaceholderIssue::Unresolved(expression.to_string())); + } + cursor = expression_start + relative_end + 2; + } + (None, _) if is_input_marker => { + return Some(InputPlaceholderIssue::Unclosed); + } + (None, Some(relative_open)) => { + cursor = expression_start + relative_open; + } + (None, None) => break, + } + } + None +} + +/// Walks front-matter string scalars for an import-input placeholder issue. +fn find_front_matter_placeholder_issue(fm: &YamlValue) -> Option { + match fm { + YamlValue::String(s) => find_input_placeholder_issue(s), + YamlValue::Sequence(items) => items.iter().find_map(find_front_matter_placeholder_issue), + YamlValue::Mapping(mapping) => mapping.iter().find_map(|(key, value)| { + find_front_matter_placeholder_issue(key) + .or_else(|| find_front_matter_placeholder_issue(value)) + }), + _ => None, + } +} + +/// Walks front matter and substitutes import-input placeholders in every string +/// scalar. +pub fn substitute_front_matter(fm: &YamlValue, inputs: &JsonMap) -> YamlValue { + match fm { + YamlValue::String(s) => YamlValue::String(substitute_inputs(s, inputs)), + YamlValue::Sequence(items) => YamlValue::Sequence( + items + .iter() + .map(|item| substitute_front_matter(item, inputs)) + .collect(), + ), + YamlValue::Mapping(mapping) => { + let mut substituted = YamlMapping::new(); + for (key, value) in mapping { + substituted.insert( + substitute_front_matter(key, inputs), + substitute_front_matter(value, inputs), + ); + } + YamlValue::Mapping(substituted) + } + other => other.clone(), + } +} + +/// Parses, validates, defaults, substitutes, and consumes `import-schema:`. +/// +/// This is intentionally a pure transformation: it does not mutate a +/// `ResolvedImport` and does not merge the component into the consumer +/// workflow. +pub fn apply_import_inputs( + front_matter: &YamlValue, + body: &str, + with: &JsonMap, +) -> Result<(YamlValue, String)> { + let schema = parse_import_schema(front_matter)?; + let inputs = validate_with(&schema, with)?; + let stripped_front_matter = strip_import_schema(front_matter); + + let substituted_front_matter = substitute_front_matter(&stripped_front_matter, &inputs); + let substituted_body = substitute_inputs(body, &inputs); + + // Leftover-placeholder guard: any `{{ inputs. }}` still + // present after substitution is a reference to an input the consumer did + // not supply (via `with:`) and that the component `import-schema:` did not + // default. Fail closed — an unresolved marker embedded into the pipeline + // YAML or agent prompt is a footgun (ADO would template-process a stray + // `${{ ... }}`, and a leaked prompt marker is meaningless to the agent). + if let Some(issue) = find_front_matter_placeholder_issue(&substituted_front_matter) + .or_else(|| find_input_placeholder_issue(&substituted_body)) + { + match issue { + InputPlaceholderIssue::Unresolved(expr) => { + let key = expr.strip_prefix(PLACEHOLDER_PREFIX).unwrap_or(&expr); + anyhow::bail!( + "unresolved import input placeholder `{{{{ {expr} }}}}`: no input named `{key}` \ + was provided in the consumer `with:` or defaulted by the component \ + `import-schema:` (note: use the compile-time `{{{{ ... }}}}` delimiter, not \ + the ADO `${{{{ ... }}}}` template-expression delimiter)" + ); + } + InputPlaceholderIssue::Unclosed => { + anyhow::bail!( + "malformed import input placeholder: missing closing `}}}}` after \ + `{{{{ inputs.`" + ); + } + InputPlaceholderIssue::Malformed => { + anyhow::bail!( + "malformed import input placeholder: `{{{{ inputs. }}}}` must name an \ + input after `inputs.`" + ); + } + } + } + + Ok((substituted_front_matter, substituted_body)) +} + +fn parse_fields( + fields_map: &YamlMapping, + path: &str, + object_depth: usize, +) -> Result> { + let mut fields = BTreeMap::new(); + for (key, value) in fields_map { + let field_name = yaml_string(key, path)?; + let field_path = dotted_path(path, field_name); + if fields + .insert( + field_name.to_string(), + parse_schema_field(value, &field_path, object_depth)?, + ) + .is_some() + { + anyhow::bail!("duplicate import-schema field `{field_path}`"); + } + } + Ok(fields) +} + +fn parse_schema_field(value: &YamlValue, path: &str, object_depth: usize) -> Result { + let field_map = yaml_mapping(value, path)?; + let ty = parse_schema_type(field_map, path, object_depth)?; + let required = match mapping_get_in(field_map, "required") { + Some(YamlValue::Bool(required)) => *required, + Some(_) => anyhow::bail!("import-schema field `{path}.required` must be a boolean"), + None => false, + }; + let default = mapping_get_in(field_map, "default") + .map(|value| yaml_to_json(value, &dotted_path(path, "default"))) + .transpose()?; + let description = match mapping_get_in(field_map, "description") { + Some(YamlValue::String(description)) => Some(description.clone()), + Some(_) => anyhow::bail!("import-schema field `{path}.description` must be a string"), + None => None, + }; + + Ok(SchemaField { + ty, + required, + default, + description, + }) +} + +fn parse_schema_type( + field_map: &YamlMapping, + path: &str, + object_depth: usize, +) -> Result { + let ty_value = mapping_get_in(field_map, "type") + .ok_or_else(|| anyhow::anyhow!("import-schema field `{path}` is missing `type`"))?; + let ty = yaml_string(ty_value, &dotted_path(path, "type"))?; + + match ty { + "string" => Ok(SchemaType::String), + "number" => Ok(SchemaType::Number), + "boolean" => Ok(SchemaType::Boolean), + "choice" => parse_choice_type(field_map, path), + "array" => parse_array_type(field_map, path, object_depth), + "object" => parse_object_type(field_map, path, object_depth), + other => anyhow::bail!("import-schema field `{path}.type` has unsupported type `{other}`"), + } +} + +fn parse_choice_type(field_map: &YamlMapping, path: &str) -> Result { + let options_value = mapping_get_in(field_map, "options").ok_or_else(|| { + anyhow::anyhow!("choice import-schema field `{path}` is missing `options`") + })?; + let options_sequence = yaml_sequence(options_value, &dotted_path(path, "options"))?; + let mut options = Vec::with_capacity(options_sequence.len()); + for (index, option) in options_sequence.iter().enumerate() { + options.push(yaml_string(option, &format!("{}.options[{index}]", path))?.to_string()); + } + Ok(SchemaType::Choice(options)) +} + +fn parse_array_type( + field_map: &YamlMapping, + path: &str, + object_depth: usize, +) -> Result { + let items_value = mapping_get_in(field_map, "items") + .ok_or_else(|| anyhow::anyhow!("array import-schema field `{path}` is missing `items`"))?; + let items_map = yaml_mapping(items_value, &dotted_path(path, "items"))?; + Ok(SchemaType::Array(Box::new(parse_schema_type( + items_map, + &dotted_path(path, "items"), + object_depth, + )?))) +} + +fn parse_object_type( + field_map: &YamlMapping, + path: &str, + object_depth: usize, +) -> Result { + if object_depth > 0 { + anyhow::bail!( + "nested object import-schema field `{path}` is not supported; object properties are one level deep" + ); + } + let properties_value = mapping_get_in(field_map, "properties").ok_or_else(|| { + anyhow::anyhow!("object import-schema field `{path}` is missing `properties`") + })?; + let properties_map = yaml_mapping(properties_value, &dotted_path(path, "properties"))?; + Ok(SchemaType::Object(parse_fields( + properties_map, + &dotted_path(path, "properties"), + object_depth + 1, + )?)) +} + +fn validate_fields( + fields: &BTreeMap, + with: &JsonMap, + path_prefix: &str, +) -> Result> { + for key in with.keys() { + if !fields.contains_key(key) { + anyhow::bail!("unknown import input `{}`", dotted_path(path_prefix, key)); + } + } + + let mut effective = JsonMap::new(); + for (name, field) in fields { + let path = dotted_path(path_prefix, name); + match with.get(name) { + Some(value) => { + effective.insert(name.clone(), validate_value(&field.ty, value, &path)?); + } + None if field.default.is_some() => { + let default = field.default.as_ref().expect("checked is_some"); + effective.insert(name.clone(), validate_value(&field.ty, default, &path)?); + } + None if field.required => { + anyhow::bail!("missing required import input `{path}`"); + } + None => {} + } + } + Ok(effective) +} + +fn validate_value(ty: &SchemaType, value: &JsonValue, path: &str) -> Result { + match ty { + SchemaType::String => match value { + JsonValue::String(_) => Ok(value.clone()), + _ => type_error(path, "string", value), + }, + SchemaType::Number => { + if value.is_number() { + Ok(value.clone()) + } else { + type_error(path, "number", value) + } + } + SchemaType::Boolean => match value { + JsonValue::Bool(_) => Ok(value.clone()), + _ => type_error(path, "boolean", value), + }, + SchemaType::Choice(options) => match value { + JsonValue::String(value) if options.contains(value) => { + Ok(JsonValue::String(value.clone())) + } + JsonValue::String(value) => anyhow::bail!( + "import input `{path}` value `{value}` is not one of: {}", + options.join(", ") + ), + _ => type_error(path, "choice string", value), + }, + SchemaType::Array(item_ty) => match value { + JsonValue::Array(items) => { + let mut validated = Vec::with_capacity(items.len()); + for (index, item) in items.iter().enumerate() { + validated.push(validate_value(item_ty, item, &format!("{path}[{index}]"))?); + } + Ok(JsonValue::Array(validated)) + } + _ => type_error(path, "array", value), + }, + SchemaType::Object(properties) => match value { + JsonValue::Object(object) => Ok(JsonValue::Object(validate_fields( + properties, object, path, + )?)), + _ => type_error(path, "object", value), + }, + } +} + +fn type_error(path: &str, expected: &str, value: &JsonValue) -> Result { + anyhow::bail!( + "import input `{path}` must be {expected}, got {}", + json_value_kind(value) + ) +} + +fn strip_import_schema(front_matter: &YamlValue) -> YamlValue { + let YamlValue::Mapping(mapping) = front_matter else { + return front_matter.clone(); + }; + + let mut stripped = YamlMapping::new(); + for (key, value) in mapping { + if matches!(key, YamlValue::String(key) if key == IMPORT_SCHEMA_KEY) { + continue; + } + stripped.insert(key.clone(), value.clone()); + } + YamlValue::Mapping(stripped) +} + +fn mapping_get<'a>(value: &'a YamlValue, key: &str) -> Option<&'a YamlValue> { + let YamlValue::Mapping(mapping) = value else { + return None; + }; + mapping_get_in(mapping, key) +} + +fn mapping_get_in<'a>(mapping: &'a YamlMapping, key: &str) -> Option<&'a YamlValue> { + mapping.iter().find_map(|(mapping_key, value)| { + if matches!(mapping_key, YamlValue::String(mapping_key) if mapping_key == key) { + Some(value) + } else { + None + } + }) +} + +fn yaml_mapping<'a>(value: &'a YamlValue, path: &str) -> Result<&'a YamlMapping> { + match value { + YamlValue::Mapping(mapping) => Ok(mapping), + _ => anyhow::bail!( + "import-schema field `{path}` must be a mapping, got {}", + super::yaml_value_kind(value) + ), + } +} + +fn yaml_sequence<'a>(value: &'a YamlValue, path: &str) -> Result<&'a Vec> { + match value { + YamlValue::Sequence(sequence) => Ok(sequence), + _ => anyhow::bail!( + "import-schema field `{path}` must be a sequence, got {}", + super::yaml_value_kind(value) + ), + } +} + +fn yaml_string<'a>(value: &'a YamlValue, path: &str) -> Result<&'a str> { + match value { + YamlValue::String(value) => Ok(value), + _ => anyhow::bail!( + "import-schema field `{path}` must be a string, got {}", + super::yaml_value_kind(value) + ), + } +} + +fn yaml_to_json(value: &YamlValue, path: &str) -> Result { + serde_json::to_value(value) + .with_context(|| format!("import-schema field `{path}` default is not JSON-compatible")) +} + +fn lookup_input_path<'a>( + inputs: &'a JsonMap, + path: &str, +) -> Option<&'a JsonValue> { + let mut parts = path.split('.'); + let first = parts.next()?; + if first.is_empty() { + return None; + } + + let mut value = inputs.get(first)?; + for part in parts { + if part.is_empty() { + return None; + } + value = value.as_object()?.get(part)?; + } + Some(value) +} + +/// Render an import-input value for interpolation into a `{{ ... }}` +/// placeholder. +/// +/// **Trust boundary.** Import inputs are non-secret, compile-time author +/// choices: a component's schema defaults are pinned by commit SHA (reviewed at +/// import time) and a consumer's `with:` values are committed to the consumer +/// repo (reviewed at author time). Neither is agent- or runtime-controlled. +/// As defense-in-depth we still run string values through +/// [`crate::sanitize::sanitize_config`], which neutralizes Azure DevOps +/// pipeline logging commands (`##vso[` / `##[`) and strips control characters, +/// so an interpolated value can never smuggle a pipeline command into a +/// generated step. Non-string scalars need no neutralization; arrays and +/// objects are neutralized recursively at their string leaves before structural +/// serialization (an `import-schema` `array`/`object` input can carry a `##vso[` +/// sequence inside a nested string). +fn render_json_value(value: &JsonValue) -> String { + match value { + JsonValue::String(value) => crate::sanitize::sanitize_config(value), + JsonValue::Number(value) => value.to_string(), + JsonValue::Bool(value) => value.to_string(), + JsonValue::Array(_) | JsonValue::Object(_) => { + let sanitized = sanitize_json_strings(value); + serde_json::to_string(&sanitized).unwrap_or_else(|_| sanitized.to_string()) + } + JsonValue::Null => "null".to_string(), + } +} + +/// Deep-clone a JSON value, running every string leaf through +/// [`crate::sanitize::sanitize_config`]. Used before structurally serializing an +/// array/object import input so a nested string leaf cannot reach a generated +/// step with an un-neutralized `##vso[` / `##[` pipeline command. +fn sanitize_json_strings(value: &JsonValue) -> JsonValue { + match value { + JsonValue::String(s) => JsonValue::String(crate::sanitize::sanitize_config(s)), + JsonValue::Array(items) => { + JsonValue::Array(items.iter().map(sanitize_json_strings).collect()) + } + JsonValue::Object(map) => JsonValue::Object( + map.iter() + .map(|(k, v)| (k.clone(), sanitize_json_strings(v))) + .collect(), + ), + other => other.clone(), + } +} + +fn dotted_path(prefix: &str, key: &str) -> String { + if prefix.is_empty() { + key.to_string() + } else { + format!("{prefix}.{key}") + } +} + +fn json_value_kind(value: &JsonValue) -> &'static str { + match value { + JsonValue::Null => "null", + JsonValue::Bool(_) => "boolean", + JsonValue::Number(_) => "number", + JsonValue::String(_) => "string", + JsonValue::Array(_) => "array", + JsonValue::Object(_) => "object", + } +} + +#[cfg(test)] +mod tests { + use super::*; + use serde_json::json; + + fn yaml(input: &str) -> YamlValue { + serde_yaml::from_str(input).expect("valid yaml") + } + + fn schema_yaml() -> YamlValue { + yaml( + r#" +import-schema: + name: + type: string + required: true + description: Component name + count: + type: number + default: 3 + enabled: + type: boolean + default: true + mode: + type: choice + options: [fast, slow] + tags: + type: array + items: + type: string + config: + type: object + properties: + apiKey: + type: string + required: true + retries: + type: number + default: 2 +"#, + ) + } + + #[test] + fn parse_import_schema_supports_all_types_required_default_and_description() { + let schema = parse_import_schema(&schema_yaml()).unwrap(); + + assert!(matches!(schema.fields["name"].ty, SchemaType::String)); + assert!(schema.fields["name"].required); + assert_eq!( + schema.fields["name"].description.as_deref(), + Some("Component name") + ); + assert!(matches!(schema.fields["count"].ty, SchemaType::Number)); + assert_eq!(schema.fields["count"].default, Some(json!(3))); + assert!(matches!(schema.fields["enabled"].ty, SchemaType::Boolean)); + assert_eq!(schema.fields["enabled"].default, Some(json!(true))); + assert_eq!( + schema.fields["mode"].ty, + SchemaType::Choice(vec!["fast".to_string(), "slow".to_string()]) + ); + assert_eq!( + schema.fields["tags"].ty, + SchemaType::Array(Box::new(SchemaType::String)) + ); + match &schema.fields["config"].ty { + SchemaType::Object(properties) => { + assert!(matches!(properties["apiKey"].ty, SchemaType::String)); + assert!(properties["apiKey"].required); + assert_eq!(properties["retries"].default, Some(json!(2))); + } + other => panic!("expected object schema, got {other:?}"), + } + } + + #[test] + fn parse_import_schema_returns_empty_when_missing() { + let schema = parse_import_schema(&yaml("name: example\n")).unwrap(); + + assert!(schema.fields.is_empty()); + } + + #[test] + fn validate_with_fills_defaults_and_object_property_defaults() { + let schema = parse_import_schema(&schema_yaml()).unwrap(); + let with = json!({ + "name": "demo", + "mode": "fast", + "tags": ["a", "b"], + "config": { "apiKey": "secret" } + }); + let validated = validate_with(&schema, with.as_object().unwrap()).unwrap(); + + assert_eq!(validated["name"], json!("demo")); + assert_eq!(validated["count"], json!(3)); + assert_eq!(validated["enabled"], json!(true)); + assert_eq!(validated["tags"], json!(["a", "b"])); + assert_eq!( + validated["config"], + json!({ "apiKey": "secret", "retries": 2 }) + ); + } + + #[test] + fn validate_with_errors_for_missing_required() { + let schema = parse_import_schema(&schema_yaml()).unwrap(); + let err = validate_with(&schema, &JsonMap::new()).unwrap_err(); + + assert!( + err.to_string() + .contains("missing required import input `name`") + ); + } + + #[test] + fn validate_with_errors_for_unknown_key() { + let schema = parse_import_schema(&schema_yaml()).unwrap(); + let with = json!({ "name": "demo", "unknown": true }); + let err = validate_with(&schema, with.as_object().unwrap()).unwrap_err(); + + assert!(err.to_string().contains("unknown import input `unknown`")); + } + + #[test] + fn validate_with_errors_for_choice_not_in_options() { + let schema = parse_import_schema(&schema_yaml()).unwrap(); + let with = json!({ "name": "demo", "mode": "medium" }); + let err = validate_with(&schema, with.as_object().unwrap()).unwrap_err(); + + assert!(err.to_string().contains("mode")); + assert!(err.to_string().contains("fast, slow")); + } + + #[test] + fn validate_with_errors_for_array_element_type_mismatch() { + let schema = parse_import_schema(&schema_yaml()).unwrap(); + let with = json!({ "name": "demo", "tags": ["ok", 1] }); + let err = validate_with(&schema, with.as_object().unwrap()).unwrap_err(); + + assert!(err.to_string().contains("tags[1]")); + assert!(err.to_string().contains("string")); + } + + #[test] + fn substitute_inputs_supports_scalars_dotted_paths_json_values_and_missing_passthrough() { + let inputs = json!({ + "name": "demo", + "count": 7, + "enabled": false, + "tags": ["a", "b"], + "config": { "apiKey": "secret" } + }); + let text = concat!( + "name={{inputs.name}} ", + "key={{ inputs.config.apiKey }} ", + "count={{ inputs.count }} ", + "enabled={{ inputs.enabled }} ", + "tags={{ inputs.tags }} ", + "config={{ inputs.config }} ", + "missing={{ inputs.missing }}" + ); + + let substituted = substitute_inputs(text, inputs.as_object().unwrap()); + + assert_eq!( + substituted, + concat!( + "name=demo ", + "key=secret ", + "count=7 ", + "enabled=false ", + "tags=[\"a\",\"b\"] ", + "config={\"apiKey\":\"secret\"} ", + "missing={{ inputs.missing }}" + ) + ); + } + + #[test] + fn substitute_inputs_preserves_ado_template_expressions() { + // A `$`-preceded `{{ ... }}` is an ADO `${{ ... }}` template expression, + // NOT one of our markers — it must pass through untouched, even when it + // syntactically mentions our namespace (that leak is caught later by the + // guard, not silently substituted here). + let inputs = json!({ "name": "demo" }); + let text = "keep ${{ parameters.env }} and {{ inputs.name }} and ${{ inputs.name }}"; + let substituted = substitute_inputs(text, inputs.as_object().unwrap()); + assert_eq!( + substituted, + "keep ${{ parameters.env }} and demo and ${{ inputs.name }}" + ); + } + + #[test] + fn substitute_inputs_preserves_runtime_import_markers() { + let inputs = json!({ "name": "demo" }); + let text = "{{#runtime-import agents/x.md}} uses {{ inputs.name }}"; + let substituted = substitute_inputs(text, inputs.as_object().unwrap()); + assert_eq!(substituted, "{{#runtime-import agents/x.md}} uses demo"); + } + + #[test] + fn substitute_inputs_neutralizes_pipeline_commands_in_string_values() { + // A string import input containing an ADO logging command must be + // neutralized when interpolated (defense-in-depth), so it cannot smuggle + // a `##vso[` pipeline command into a generated step. + let inputs = json!({ "evil": "##vso[task.setvariable variable=x]y" }); + let substituted = substitute_inputs("val={{ inputs.evil }}", inputs.as_object().unwrap()); + // The neutralized form wraps the command in backticks so ADO renders it + // as inert text instead of executing it. + assert!( + substituted.contains("`##vso[`"), + "expected neutralized (backtick-wrapped) form: {substituted}" + ); + assert!( + !substituted.contains("##vso[task.setvariable"), + "the executable command tail must be broken up: {substituted}" + ); + } + + #[test] + fn substitute_inputs_neutralizes_pipeline_commands_in_nested_array_object_values() { + // Array/object import inputs (import-schema `array`/`object` types) are + // serialized structurally when interpolated into a string-scalar + // position. A `##vso[` pipeline command hidden in a nested string leaf + // must be neutralized just like a top-level string value — otherwise it + // could reach a generated step unsanitized. + let inputs = json!({ + "tags": ["##vso[task.setvariable variable=x]y"], + "obj": { "k": "##vso[task.setvariable variable=z]w" } + }); + let inputs = inputs.as_object().unwrap(); + + let arr = substitute_inputs("val={{ inputs.tags }}", inputs); + assert!( + arr.contains("`##vso[`"), + "expected neutralized form in array: {arr}" + ); + assert!( + !arr.contains("##vso[task.setvariable"), + "array leaf command tail must be broken up: {arr}" + ); + + let obj = substitute_inputs("val={{ inputs.obj }}", inputs); + assert!( + obj.contains("`##vso[`"), + "expected neutralized form in object: {obj}" + ); + assert!( + !obj.contains("##vso[task.setvariable"), + "object leaf command tail must be broken up: {obj}" + ); + } + + #[test] + fn substitute_front_matter_walks_nested_mappings_and_sequences() { + let fm = yaml( + r#" +name: "{{ inputs.name }}" +steps: + - bash: echo {{ inputs.config.apiKey }} +nested: + value: before {{inputs.name}} after +"#, + ); + let inputs = json!({ + "name": "demo", + "config": { "apiKey": "secret" } + }); + + let substituted = substitute_front_matter(&fm, inputs.as_object().unwrap()); + + assert_eq!(mapping_get(&substituted, "name"), Some(&yaml("demo"))); + let steps = mapping_get(&substituted, "steps") + .unwrap() + .as_sequence() + .unwrap(); + assert_eq!(mapping_get(&steps[0], "bash"), Some(&yaml("echo secret"))); + let nested = mapping_get(&substituted, "nested").unwrap(); + assert_eq!( + mapping_get(nested, "value"), + Some(&yaml("before demo after")) + ); + } + + #[test] + fn apply_import_inputs_strips_schema_and_substitutes_front_matter_and_body() { + let fm = yaml( + r#" +import-schema: + name: + type: string + required: true + count: + type: number + default: 2 +name: component-{{ inputs.name }} +variables: + count: "{{ inputs.count }}" +"#, + ); + let with = json!({ "name": "demo" }); + + let (front_matter, body) = apply_import_inputs( + &fm, + "Hello {{ inputs.name }} {{ inputs.count }}", + with.as_object().unwrap(), + ) + .unwrap(); + + assert!(mapping_get(&front_matter, IMPORT_SCHEMA_KEY).is_none()); + assert_eq!( + mapping_get(&front_matter, "name"), + Some(&yaml("component-demo")) + ); + let variables = mapping_get(&front_matter, "variables").unwrap(); + assert_eq!( + mapping_get(variables, "count"), + Some(&YamlValue::String("2".to_string())) + ); + assert_eq!(body, "Hello demo 2"); + } + + #[test] + fn apply_import_inputs_rejects_unresolved_body_placeholder() { + // `typo` is not in the schema and not supplied — `validate_with` won't + // catch a body-only reference, so the leftover guard must. + let fm = yaml("import-schema:\n name:\n type: string\n"); + let with = json!({ "name": "demo" }); + let err = apply_import_inputs(&fm, "Hello {{ inputs.typo }}", with.as_object().unwrap()) + .unwrap_err(); + let msg = format!("{err:#}"); + assert!(msg.contains("unresolved import input placeholder"), "{msg}"); + assert!(msg.contains("inputs.typo"), "{msg}"); + } + + #[test] + fn apply_import_inputs_rejects_leftover_dollar_delimited_marker() { + // An author who mistakenly used the old `${{ ... }}` form is flagged: + // it would otherwise be template-processed by ADO in the emitted YAML. + let fm = yaml("import-schema:\n name:\n type: string\n"); + let with = json!({ "name": "demo" }); + let err = apply_import_inputs(&fm, "Hello ${{ inputs.name }}", with.as_object().unwrap()) + .unwrap_err(); + assert!( + format!("{err:#}").contains("unresolved import input placeholder"), + "{err:#}" + ); + } + + #[test] + fn apply_import_inputs_rejects_unclosed_body_placeholder_after_valid_marker() { + let fm = yaml("import-schema:\n name:\n type: string\n"); + let with = json!({ "name": "demo" }); + let err = apply_import_inputs( + &fm, + "Hello {{ inputs.name }} then {{ inputs.name", + with.as_object().unwrap(), + ) + .unwrap_err(); + let msg = format!("{err:#}"); + assert!(msg.contains("malformed import input placeholder"), "{msg}"); + assert!(msg.contains("missing closing `}}`"), "{msg}"); + } + + #[test] + fn apply_import_inputs_rejects_unclosed_front_matter_placeholder() { + let fm = + yaml("import-schema:\n name:\n type: string\nname: \"component-{{ inputs.name\"\n"); + let with = json!({ "name": "demo" }); + let err = apply_import_inputs(&fm, "body", with.as_object().unwrap()).unwrap_err(); + let msg = format!("{err:#}"); + assert!(msg.contains("malformed import input placeholder"), "{msg}"); + assert!(msg.contains("missing closing `}}`"), "{msg}"); + } + + #[test] + fn apply_import_inputs_rejects_empty_input_path() { + let fm = yaml("import-schema: {}\n"); + let with = json!({}); + let err = + apply_import_inputs(&fm, "Hello {{ inputs. }}", with.as_object().unwrap()).unwrap_err(); + let msg = format!("{err:#}"); + assert!(msg.contains("malformed import input placeholder"), "{msg}"); + assert!(msg.contains("must name an input"), "{msg}"); + } + + #[test] + fn apply_import_inputs_leaves_ado_template_expression_untouched() { + // A genuine ADO `${{ parameters.x }}` (not our namespace) must survive + // substitution and NOT trip the guard. + let fm = yaml("name: keep\n"); + let with = json!({}); + let (_, body) = apply_import_inputs( + &fm, + "run in ${{ parameters.environment }}", + with.as_object().unwrap(), + ) + .unwrap(); + assert_eq!(body, "run in ${{ parameters.environment }}"); + } +} diff --git a/src/compile/ir/emit.rs b/src/compile/ir/emit.rs index f489f166..1b3a4356 100644 --- a/src/compile/ir/emit.rs +++ b/src/compile/ir/emit.rs @@ -61,6 +61,7 @@ mod tests { fetch_depth: None, fetch_tags: None, persist_credentials: None, + path: None, })); setup.push_step(Step::Bash(BashStep::new("Prep", "echo prep"))); diff --git a/src/compile/ir/job.rs b/src/compile/ir/job.rs index e78c8936..42d075b5 100644 --- a/src/compile/ir/job.rs +++ b/src/compile/ir/job.rs @@ -278,6 +278,7 @@ mod tests { fetch_depth: None, fetch_tags: None, persist_credentials: None, + path: None, })); assert_eq!(j.steps.len(), 1); } diff --git a/src/compile/ir/lower.rs b/src/compile/ir/lower.rs index 9af9870b..a0131f1e 100644 --- a/src/compile/ir/lower.rs +++ b/src/compile/ir/lower.rs @@ -355,6 +355,7 @@ fn lower_repository_resource(r: &RepositoryResource) -> Value { kind, name, r#ref, + endpoint, } => { m.insert(s("repository"), s(identifier)); m.insert(s("type"), s(kind)); @@ -362,6 +363,9 @@ fn lower_repository_resource(r: &RepositoryResource) -> Value { if let Some(r) = r#ref { m.insert(s("ref"), s(r)); } + if let Some(ep) = endpoint { + m.insert(s("endpoint"), s(ep)); + } } } Value::Mapping(m) @@ -1045,6 +1049,9 @@ fn lower_checkout(c: &CheckoutStep) -> Value { if let Some(pc) = c.persist_credentials { m.insert(s("persistCredentials"), Value::Bool(pc)); } + if let Some(path) = &c.path { + m.insert(s("path"), s(path)); + } Value::Mapping(m) } @@ -1422,6 +1429,33 @@ mod tests { } } + #[test] + fn lower_named_repository_emits_endpoint_when_present() { + let r = RepositoryResource::Named { + identifier: "shared".to_string(), + kind: "github".to_string(), + name: "acme/shared".to_string(), + r#ref: Some("refs/heads/main".to_string()), + endpoint: Some("shared-conn".to_string()), + }; + let v = lower_repository_resource(&r); + assert_eq!(v["endpoint"], Value::String("shared-conn".to_string())); + assert_eq!(v["type"], Value::String("github".to_string())); + } + + #[test] + fn lower_named_repository_omits_endpoint_when_absent() { + let r = RepositoryResource::Named { + identifier: "shared".to_string(), + kind: "git".to_string(), + name: "proj/shared".to_string(), + r#ref: Some("refs/heads/main".to_string()), + endpoint: None, + }; + let v = lower_repository_resource(&r); + assert!(v.get("endpoint").is_none()); + } + #[test] fn lower_env_value_runtime_expression_emits_hoisted_macro() { let g = Graph::default(); @@ -1612,7 +1646,10 @@ mod tests { .and_then(|v| v.as_mapping()) .expect("lowered job should contain pool mapping"); - assert_eq!(pool.get(s("name")).and_then(|v| v.as_str()), Some("CustomPool")); + assert_eq!( + pool.get(s("name")).and_then(|v| v.as_str()), + Some("CustomPool") + ); let demands: Vec<&str> = pool .get(s("demands")) .and_then(|v| v.as_sequence()) @@ -2885,12 +2922,14 @@ mod tests { fetch_depth: Some(1), fetch_tags: Some(false), persist_credentials: None, + path: Some("s/custom-checkout".to_string()), }; let v = lower_checkout(&c); let m = v.as_mapping().unwrap(); assert_eq!(m.get(s("checkout")).unwrap(), &s("self")); assert_eq!(m.get(s("fetchDepth")).unwrap(), &Value::from(1u32)); assert_eq!(m.get(s("fetchTags")).unwrap(), &Value::Bool(false)); + assert_eq!(m.get(s("path")).unwrap(), &s("s/custom-checkout")); } #[test] @@ -2902,6 +2941,7 @@ mod tests { fetch_depth: Some(0), fetch_tags: None, persist_credentials: None, + path: None, }; let v = lower_checkout(&c); let m = v.as_mapping().unwrap(); @@ -2917,6 +2957,7 @@ mod tests { fetch_depth: None, fetch_tags: None, persist_credentials: None, + path: None, }; let v = lower_checkout(&c); let m = v.as_mapping().unwrap(); @@ -2933,6 +2974,7 @@ mod tests { fetch_depth: None, fetch_tags: None, persist_credentials: None, + path: None, }; let v = lower_checkout(&c); let m = v.as_mapping().unwrap(); diff --git a/src/compile/ir/mod.rs b/src/compile/ir/mod.rs index 4c826e33..6bf1a298 100644 --- a/src/compile/ir/mod.rs +++ b/src/compile/ir/mod.rs @@ -251,6 +251,7 @@ pub enum RepositoryResource { kind: String, name: String, r#ref: Option, + endpoint: Option, }, } diff --git a/src/compile/ir/step.rs b/src/compile/ir/step.rs index ad485021..04e2a809 100644 --- a/src/compile/ir/step.rs +++ b/src/compile/ir/step.rs @@ -194,6 +194,8 @@ pub struct CheckoutStep { pub fetch_depth: Option, pub fetch_tags: Option, pub persist_credentials: Option, + /// Checkout location relative to `$(Agent.BuildDirectory)`. + pub path: Option, } /// Target of a [`CheckoutStep`]. @@ -266,6 +268,7 @@ mod tests { fetch_depth: None, fetch_tags: None, persist_credentials: None, + path: None, }); assert!(chk.id().is_none()); diff --git a/src/compile/job.rs b/src/compile/job.rs index d776ad2e..93664b62 100644 --- a/src/compile/job.rs +++ b/src/compile/job.rs @@ -31,13 +31,15 @@ impl Compiler for JobCompiler { output_path: &Path, front_matter: &FrontMatter, markdown_body: &str, + imported_prompt_body: &str, skip_integrity: bool, debug_pipeline: bool, ) -> Result { info!("Compiling for job target (typed IR)"); let extensions = super::extensions::collect_extensions(front_matter); - let ctx = super::extensions::CompileContext::new(front_matter, input_path).await?; + let mut ctx = super::extensions::CompileContext::new(front_matter, input_path).await?; + ctx.imported_prompt_body = imported_prompt_body.to_string(); let pipeline = super::job_ir::build_job_pipeline( front_matter, diff --git a/src/compile/mod.rs b/src/compile/mod.rs index bc522ee8..2491860b 100644 --- a/src/compile/mod.rs +++ b/src/compile/mod.rs @@ -13,16 +13,18 @@ pub(crate) mod agentic_pipeline; #[cfg(test)] mod codemod_integration_test; pub(crate) mod codemods; +pub mod custom_tools; pub mod extensions; pub(crate) mod filter_ir; mod gitattributes; +pub mod imports; pub(crate) mod ir; mod job; mod job_ir; mod onees; mod onees_ir; -pub(crate) mod pr_filters; mod path_layout_check; +pub(crate) mod pr_filters; pub mod source_path_guard; mod stage; mod stage_ir; @@ -55,12 +57,20 @@ pub use types::{CompileTarget, FrontMatter}; #[async_trait] pub trait Compiler: Send + Sync { /// Compile the front matter and markdown body into pipeline YAML. + /// + /// `imported_prompt_body` carries the substituted, joined bodies of any + /// imported components (empty when there are no imports). It is inlined + /// into the agent prompt at compile time — imported bodies cannot be + /// delivered by the default runtime-import path (which reads only the + /// consumer's own source file). + #[allow(clippy::too_many_arguments)] async fn compile( &self, input_path: &Path, output_path: &Path, front_matter: &FrontMatter, markdown_body: &str, + imported_prompt_body: &str, skip_integrity: bool, debug_pipeline: bool, ) -> Result; @@ -142,13 +152,33 @@ async fn compile_pipeline_inner( let parsed = common::parse_markdown_detailed_with_registry(&content, registry)?; let mut front_matter = parsed.front_matter; - let markdown_body = parsed.markdown_body; + let mut markdown_body = parsed.markdown_body; let codemod_report = parsed.codemods; let front_matter_mapping = parsed.front_matter_mapping; let leading_whitespace = parsed.leading_whitespace; let body_raw = parsed.body_raw; let source_sha256 = parsed.source_sha256; + // Resolve and merge cross-repository / local `imports:` (D8/D9). Runs only + // when the workflow declares imports, so import-free workflows are + // unaffected. The merge is applied to a CLONE of the front-matter mapping — + // the original `front_matter_mapping` is preserved untouched so that any + // codemod source-rewrite keeps the author's `imports:` in their file rather + // than writing the expanded/merged form back to disk. + // + // `imported_prompt_body` is the substituted, joined bodies of any imported + // components, inlined into the agent prompt at compile time (they cannot be + // delivered by the default runtime-import path, which reads the consumer's + // own source). Empty when the workflow declares no imports. + let (imported_prompt_body, merged_body) = resolve_and_merge_imports( + &mut front_matter, + &front_matter_mapping, + &markdown_body, + input_path, + ) + .await?; + markdown_body = merged_body; + // Sanitize all front matter text fields before any further processing. // This neutralizes pipeline command injection (##vso[), strips control // characters, and enforces content limits across all config values. @@ -173,9 +203,7 @@ async fn compile_pipeline_inner( // Checkout-aware path-layout advisories (warning-only): surface // hand-written paths that won't exist under the resolved checkout // layout, plus deprecated directory markers left in the agent body. - for warning in - path_layout_check::collect_path_layout_warnings(&front_matter, &markdown_body) - { + for warning in path_layout_check::collect_path_layout_warnings(&front_matter, &markdown_body) { eprintln!("Warning: {warning}"); } @@ -219,6 +247,7 @@ async fn compile_pipeline_inner( &yaml_output_path, &front_matter, &markdown_body, + &imported_prompt_body, skip_integrity, debug_pipeline, ) @@ -607,7 +636,18 @@ pub async fn check_pipeline(pipeline_path: &str) -> Result<()> { } let mut front_matter = parsed.front_matter; - let markdown_body = parsed.markdown_body; + + // Resolve + merge `imports:` so `check` validates the same fully-merged + // pipeline that `compile` produces. Reads the committed `.ado-aw/imports` + // cache (SHA-keyed), so this is offline when the cache is vendored. Uses the + // absolute `source_path` so the repo root (holding the cache) resolves. + let (imported_prompt_body, markdown_body) = resolve_and_merge_imports( + &mut front_matter, + &parsed.front_matter_mapping, + &parsed.markdown_body, + &source_path, + ) + .await?; use crate::sanitize::SanitizeConfig; front_matter.sanitize_config_fields(); @@ -631,6 +671,7 @@ pub async fn check_pipeline(pipeline_path: &str) -> Result<()> { pipeline_path, &front_matter, &markdown_body, + &imported_prompt_body, false, false, ) @@ -896,6 +937,61 @@ pub fn find_repo_root(start: &Path) -> Option { } } +/// Resolve and merge the workflow's `imports:` into `front_matter` and the body. +/// +/// Shared by [`compile_pipeline_inner`] and [`check_pipeline`] so that `check` +/// validates the *same* fully-merged pipeline that `compile` produces. Mutates +/// `front_matter` in place (imported front matter merged, `imports:` consumed) +/// and returns `(imported_prompt_body, merged_markdown_body)`: +/// - `imported_prompt_body` — the substituted, joined bodies of imported +/// components, inlined into the agent prompt at compile time. +/// - `merged_markdown_body` — imported bodies + the consumer body. +/// +/// A no-op returning `(String::new(), markdown_body)` when there are no imports. +/// +/// `source_path` is the absolute path of the agent source `.md`; its parent is +/// the base directory for local imports and the anchor for locating the repo +/// root (which holds the committed `.ado-aw/imports` cache). +async fn resolve_and_merge_imports( + front_matter: &mut FrontMatter, + front_matter_mapping: &serde_yaml::Mapping, + markdown_body: &str, + source_path: &Path, +) -> Result<(String, String)> { + custom_tools::reject_author_component_provenance(front_matter)?; + if front_matter.imports.is_empty() { + return Ok((String::new(), markdown_body.to_string())); + } + + let base_dir = source_path.parent().unwrap_or_else(|| Path::new(".")); + let repo_root = find_repo_root(base_dir).unwrap_or_else(|| base_dir.to_path_buf()); + + // Build the routing fetcher: endpoint-less / cross-org Azure Repos imports + // resolve via the ADO Git Items API (primary); GitHub/GHE imports resolve + // via `gh`. The Azure fetcher resolves its org URL + auth LAZILY on the + // first actual fetch (see `AdoRepoFetcher`), so a GitHub-only import set, a + // workflow with no ADO remote, or — crucially for `check`/`inspect` — a + // fully-vendored committed cache performs no `git`/`az` work at all (the + // cache is consulted before any fetch), and any resolution failure surfaces + // fail-closed only when an uncached Azure import is actually fetched. + let ado_fetcher = crate::compile::imports::AdoRepoFetcher::new(repo_root.clone()); + let fetcher = crate::compile::imports::RoutingFetcher::new(ado_fetcher); + + let mut merged_mapping = front_matter_mapping.clone(); + let (imported, combined) = crate::compile::imports::merge::merge_imports( + &mut merged_mapping, + markdown_body, + &front_matter.imports, + base_dir, + &repo_root, + &fetcher, + ) + .await?; + *front_matter = serde_yaml::from_value(serde_yaml::Value::Mapping(merged_mapping)) + .context("Failed to parse front matter after merging imports")?; + Ok((imported, combined)) +} + /// Public, read-only entry point that returns the typed [`ir::Pipeline`] /// for an agent source file **without** writing any YAML. /// @@ -918,7 +1014,18 @@ pub async fn build_pipeline_ir(input_path: &Path) -> Result<(FrontMatter, ir::Pi let parsed = common::parse_markdown_detailed(&content)?; let mut front_matter = parsed.front_matter; - let markdown_body = parsed.markdown_body; + + // Resolve + merge `imports:` so `inspect`/`graph`/`whatif`/`lint`/`trace` + // reason about the same fully-merged pipeline `compile` and `check` produce + // (imported tools, safe-outputs, custom jobs, and inlined bodies). Reads the + // vendored SHA-keyed cache, so it stays offline when the cache is present. + let (imported_prompt_body, markdown_body) = resolve_and_merge_imports( + &mut front_matter, + &parsed.front_matter_mapping, + &parsed.markdown_body, + input_path, + ) + .await?; use crate::sanitize::SanitizeConfig; front_matter.sanitize_config_fields(); @@ -936,7 +1043,8 @@ pub async fn build_pipeline_ir(input_path: &Path) -> Result<(FrontMatter, ir::Pi let output_path = input_path.with_extension("lock.yml"); let extensions = extensions::collect_extensions(&front_matter); - let ctx = extensions::CompileContext::new(&front_matter, input_path).await?; + let mut ctx = extensions::CompileContext::new(&front_matter, input_path).await?; + ctx.imported_prompt_body = imported_prompt_body; let pipeline = match front_matter.target { CompileTarget::Standalone => standalone_ir::build_standalone_pipeline( @@ -1023,6 +1131,33 @@ fn clean_generated_yaml(yaml: &str) -> String { mod tests { use super::*; + #[tokio::test] + async fn authored_component_provenance_is_rejected_before_merge() { + let content = r#"--- +name: Test +description: Test +safe-outputs: + scripts: + notify: + run: ./notify + component-source: attacker/repo/component.md + component-sha: 0123456789abcdef0123456789abcdef01234567 +--- +Body +"#; + let parsed = common::parse_markdown_detailed(content).unwrap(); + let mut front_matter = parsed.front_matter; + let error = resolve_and_merge_imports( + &mut front_matter, + &parsed.front_matter_mapping, + &parsed.markdown_body, + Path::new("agent.md"), + ) + .await + .unwrap_err(); + assert!(error.to_string().contains("compiler-owned"), "{error:#}"); + } + #[test] fn test_parse_minimal_front_matter() { let content = r#"--- diff --git a/src/compile/onees.rs b/src/compile/onees.rs index 424f9e9b..5df45067 100644 --- a/src/compile/onees.rs +++ b/src/compile/onees.rs @@ -35,13 +35,15 @@ impl Compiler for OneESCompiler { output_path: &Path, front_matter: &FrontMatter, markdown_body: &str, + imported_prompt_body: &str, skip_integrity: bool, debug_pipeline: bool, ) -> Result { info!("Compiling for 1ES target (typed IR)"); let extensions = super::extensions::collect_extensions(front_matter); - let ctx = super::extensions::CompileContext::new(front_matter, input_path).await?; + let mut ctx = super::extensions::CompileContext::new(front_matter, input_path).await?; + ctx.imported_prompt_body = imported_prompt_body.to_string(); let pipeline = super::onees_ir::build_onees_pipeline( front_matter, diff --git a/src/compile/onees_ir.rs b/src/compile/onees_ir.rs index d37b67a2..06f6ca2f 100644 --- a/src/compile/onees_ir.rs +++ b/src/compile/onees_ir.rs @@ -116,6 +116,7 @@ pub fn build_onees_pipeline( kind: ONEES_TEMPLATES_REPO_KIND.to_string(), name: ONEES_TEMPLATES_REPO_NAME.to_string(), r#ref: Some(ONEES_TEMPLATES_REPO_REF.to_string()), + endpoint: None, }, ); diff --git a/src/compile/stage.rs b/src/compile/stage.rs index e693e650..e5e41389 100644 --- a/src/compile/stage.rs +++ b/src/compile/stage.rs @@ -41,13 +41,15 @@ impl Compiler for StageCompiler { output_path: &Path, front_matter: &FrontMatter, markdown_body: &str, + imported_prompt_body: &str, skip_integrity: bool, debug_pipeline: bool, ) -> Result { info!("Compiling for stage target (typed IR)"); let extensions = super::extensions::collect_extensions(front_matter); - let ctx = super::extensions::CompileContext::new(front_matter, input_path).await?; + let mut ctx = super::extensions::CompileContext::new(front_matter, input_path).await?; + ctx.imported_prompt_body = imported_prompt_body.to_string(); let pipeline = super::stage_ir::build_stage_pipeline( front_matter, diff --git a/src/compile/standalone.rs b/src/compile/standalone.rs index 63cb40a6..104d67a2 100644 --- a/src/compile/standalone.rs +++ b/src/compile/standalone.rs @@ -30,13 +30,15 @@ impl Compiler for StandaloneCompiler { output_path: &Path, front_matter: &FrontMatter, markdown_body: &str, + imported_prompt_body: &str, skip_integrity: bool, debug_pipeline: bool, ) -> Result { info!("Compiling for standalone target (typed IR)"); let extensions = super::extensions::collect_extensions(front_matter); - let ctx = super::extensions::CompileContext::new(front_matter, input_path).await?; + let mut ctx = super::extensions::CompileContext::new(front_matter, input_path).await?; + ctx.imported_prompt_body = imported_prompt_body.to_string(); let pipeline = super::standalone_ir::build_standalone_pipeline( front_matter, diff --git a/src/compile/types.rs b/src/compile/types.rs index 1b38b280..536976aa 100644 --- a/src/compile/types.rs +++ b/src/compile/types.rs @@ -1091,6 +1091,10 @@ pub struct FrontMatter { /// MCP server configurations #[serde(default, rename = "mcp-servers")] pub mcp_servers: HashMap, + /// Reusable workflow imports. Entries may be local paths or SHA-pinned + /// cross-repository specs, with optional import-schema inputs. + #[serde(default)] + pub imports: Vec, /// Per-tool configuration for safe outputs #[serde(default, rename = "safe-outputs")] pub safe_outputs: HashMap, @@ -1190,11 +1194,333 @@ impl FrontMatter { } } +/// Typed cross-repository import endpoint. +/// +/// An **absent** endpoint (`None` on [`ImportEntry`]) denotes a +/// **same-organization Azure Repos** source — the primary, default case for +/// this ADO-native compiler. An **explicit** endpoint selects a different +/// source kind and names the ADO service connection the generated runtime +/// repository resource authenticates with. +/// +/// Deserializes from either a bare string (shorthand for a GitHub.com service +/// connection) or an object with `name`, an optional `type` +/// (`github` | `ghe` | `azure-repos`, defaulting to `github`), and the +/// type-specific `host` (GHE) or `org` (cross-org Azure Repos) field. +#[derive(Debug, Clone, PartialEq, Eq, Hash)] +pub enum ImportEndpoint { + /// GitHub.com, authenticated at runtime via ADO service connection `name`. + GitHub { + /// ADO service connection name. + name: String, + }, + /// GitHub Enterprise Server at server host `host`, via service connection `name`. + GitHubEnterprise { + /// ADO service connection name. + name: String, + /// GHES server host used by `GH_HOST` (e.g. `ghe.acme.com`). + host: crate::secure::HostName, + }, + /// Azure Repos in a *different* organization `org` (a full collection URL, + /// e.g. `https://dev.azure.com/otherorg`), via service connection `name`. + AzureReposCrossOrg { + /// ADO service connection name. + name: String, + /// Target organization collection URL. + org: crate::secure::AzureDevOpsOrgUrl, + }, +} + +impl<'de> Deserialize<'de> for ImportEndpoint { + fn deserialize(deserializer: D) -> std::result::Result + where + D: serde::Deserializer<'de>, + { + use serde::de; + + struct EndpointVisitor; + + impl<'de> de::Visitor<'de> for EndpointVisitor { + type Value = ImportEndpoint; + + fn expecting(&self, formatter: &mut std::fmt::Formatter) -> std::fmt::Result { + formatter.write_str( + "a service connection name string, or an object with `name`, optional \ + `type` (github|ghe|azure-repos), and a type-specific `host`/`org`", + ) + } + + fn visit_str( + self, + value: &str, + ) -> std::result::Result { + if value.trim().is_empty() { + return Err(E::custom("import endpoint name must not be empty")); + } + Ok(ImportEndpoint::GitHub { + name: value.to_string(), + }) + } + + fn visit_map(self, map: M) -> std::result::Result + where + M: de::MapAccess<'de>, + { + #[derive(Deserialize)] + #[serde(deny_unknown_fields)] + struct EndpointObject { + name: String, + #[serde(default, rename = "type")] + kind: Option, + #[serde(default)] + host: Option, + #[serde(default)] + org: Option, + } + + let obj = EndpointObject::deserialize(de::value::MapAccessDeserializer::new(map))?; + if obj.name.trim().is_empty() { + return Err(de::Error::custom( + "import endpoint `name` must not be empty", + )); + } + let kind = obj.kind.as_deref().unwrap_or("github"); + match kind { + "github" => { + if obj.host.is_some() || obj.org.is_some() { + return Err(de::Error::custom( + "`host`/`org` are not valid for import endpoint type `github`", + )); + } + Ok(ImportEndpoint::GitHub { name: obj.name }) + } + "ghe" => { + if obj.org.is_some() { + return Err(de::Error::custom( + "`org` is not valid for import endpoint type `ghe`", + )); + } + let host = obj.host.ok_or_else(|| { + de::Error::custom("import endpoint type `ghe` requires `host`") + })?; + let host = crate::secure::HostName::parse(host) + .map_err(|e| de::Error::custom(e.to_string()))?; + Ok(ImportEndpoint::GitHubEnterprise { + name: obj.name, + host, + }) + } + "azure-repos" => { + if obj.host.is_some() { + return Err(de::Error::custom( + "`host` is not valid for import endpoint type `azure-repos`", + )); + } + let org = obj.org.ok_or_else(|| { + de::Error::custom( + "import endpoint type `azure-repos` requires `org` \ + (the target organization collection URL)", + ) + })?; + let org = crate::secure::AzureDevOpsOrgUrl::parse(org) + .map_err(|e| de::Error::custom(e.to_string()))?; + Ok(ImportEndpoint::AzureReposCrossOrg { + name: obj.name, + org, + }) + } + other => Err(de::Error::custom(format!( + "unknown import endpoint type `{other}`; expected \ + `github`, `ghe`, or `azure-repos`" + ))), + } + } + } + + deserializer.deserialize_any(EndpointVisitor) + } +} + +/// A single `imports:` entry — either a bare spec string or an object form. +#[derive(Debug, Clone, PartialEq)] +pub struct ImportEntry { + /// The import spec string (`uses:` in object form). + pub uses: String, + /// Non-secret input values for the imported workflow schema. + pub with: serde_json::Map, + /// Typed remote source endpoint. `None` denotes same-org Azure Repos. + pub endpoint: Option, +} + +impl<'de> Deserialize<'de> for ImportEntry { + fn deserialize(deserializer: D) -> std::result::Result + where + D: serde::Deserializer<'de>, + { + use serde::de; + + #[derive(Deserialize)] + #[serde(deny_unknown_fields)] + struct ImportEntryObject { + uses: String, + #[serde(default)] + with: serde_json::Map, + #[serde(default)] + endpoint: Option, + } + + struct ImportEntryVisitor; + + impl<'de> de::Visitor<'de> for ImportEntryVisitor { + type Value = ImportEntry; + + fn expecting(&self, formatter: &mut std::fmt::Formatter) -> std::fmt::Result { + formatter.write_str( + "a string import spec or an object with `uses`, optional `with`, and optional `endpoint`", + ) + } + + fn visit_str(self, value: &str) -> std::result::Result { + Ok(ImportEntry { + uses: value.to_string(), + with: serde_json::Map::new(), + endpoint: None, + }) + } + + fn visit_map(self, map: M) -> std::result::Result + where + M: de::MapAccess<'de>, + { + let entry = + ImportEntryObject::deserialize(de::value::MapAccessDeserializer::new(map))?; + Ok(ImportEntry { + uses: entry.uses, + with: entry.with, + endpoint: entry.endpoint, + }) + } + } + + deserializer.deserialize_any(ImportEntryVisitor) + } +} + +/// Parsed import source. The simple local-vs-remote heuristic is: +/// after removing a trailing optional marker (`?`) and section (`#Section`), +/// any spec containing `@` is remote and must be `owner/repo/path@`; +/// all specs without `@` are treated as local paths for later resolution. +#[derive(Debug, Clone, PartialEq, Eq)] +#[allow(dead_code)] +pub enum ImportSource { + /// Local import path within the current repository. + Local { + /// Relative path to the imported markdown file. + path: String, + /// Optional markdown section selector. + section: Option, + /// Whether a missing import should be tolerated by later resolution. + optional: bool, + }, + /// Cross-repository import pinned to a full commit SHA. + Remote(ParsedImportSpec), +} + +/// Parsed cross-repository import spec. +#[derive(Debug, Clone, PartialEq, Eq)] +#[allow(dead_code)] +pub struct ParsedImportSpec { + /// Repository owner/organization. + pub owner: String, + /// Repository name. + pub repo: String, + /// Path within the repository. + pub path: String, + /// Full 40-character commit SHA pin. + pub sha: crate::secure::CommitSha, + /// Optional markdown section selector. + pub section: Option, + /// Whether a missing import should be tolerated by later resolution. + pub optional: bool, + /// Typed source endpoint carried from the [`ImportEntry`]. `None` denotes + /// same-org Azure Repos (the primary compile-time fetch source). + pub endpoint: Option, +} + +impl ImportEntry { + /// Parse and validate the import spec string. + #[allow(dead_code)] + pub fn parse_source(&self) -> anyhow::Result { + let raw = self.uses.trim(); + if raw.is_empty() { + anyhow::bail!("import spec must not be empty"); + } + + let (without_optional, optional) = match raw.strip_suffix('?') { + Some(value) => (value, true), + None => (raw, false), + }; + + let (base, section) = match without_optional.split_once('#') { + Some((base, section)) => { + if section.is_empty() { + anyhow::bail!("import section must not be empty"); + } + (base, Some(section.to_string())) + } + None => (without_optional, None), + }; + + if base.is_empty() { + anyhow::bail!("import path must not be empty"); + } + + if !base.contains('@') { + return Ok(ImportSource::Local { + path: base.to_string(), + section, + optional, + }); + } + + let (repo_and_path, ref_part) = base + .split_once('@') + .ok_or_else(|| anyhow::anyhow!("remote import spec must contain `@`"))?; + let sha = + crate::secure::CommitSha::parse(ref_part.to_string()).map_err(|_| { + anyhow::anyhow!( + "cross-repository imports must be pinned to a full 40-character commit SHA, got '{}'", + ref_part + ) + })?; + + let mut parts = repo_and_path.splitn(3, '/'); + let owner = parts.next().unwrap_or_default(); + let repo = parts.next().unwrap_or_default(); + let path = parts.next().unwrap_or_default(); + if owner.is_empty() || repo.is_empty() || path.is_empty() { + anyhow::bail!( + "remote import spec must be `owner/repo/path@<40-character-sha>`, got '{}'", + raw + ); + } + + Ok(ImportSource::Remote(ParsedImportSpec { + owner: owner.to_string(), + repo: repo.to_string(), + path: path.to_string(), + sha, + section, + optional, + endpoint: self.endpoint.clone(), + })) + } +} + /// Reserved keys inside the `safe-outputs:` map that configure the section /// itself rather than naming a safe-output tool. These must never be treated /// as tool names (e.g. in `--enabled-tools`, Stage-3 budgets, or unknown-key /// validation). -pub const SAFE_OUTPUT_RESERVED_KEYS: &[&str] = &["require-approval"]; +pub const SAFE_OUTPUT_RESERVED_KEYS: &[&str] = &["require-approval", "scripts", "jobs"]; /// Automatic action a manual-validation gate takes when its pending period /// elapses with no human response. Mirrors `ManualValidation@1`'s `onTimeout`. @@ -1272,6 +1598,38 @@ impl FrontMatter { .filter(|k| !SAFE_OUTPUT_RESERVED_KEYS.contains(&k.as_str())) } + /// Names of **custom** safe-output tools declared by imported components + /// under the `safe-outputs.scripts.` (entrypoint) and + /// `safe-outputs.jobs.` (arbitrary-ADO-steps) sections (decision + /// D16). These are agent-callable tools generated from imported manifests, + /// distinct from the built-in tools keyed at the top level. + pub fn custom_safe_output_tool_names(&self) -> Vec { + let mut names = Vec::new(); + for section in ["scripts", "jobs"] { + if let Some(map) = self.safe_outputs.get(section).and_then(|v| v.as_object()) { + names.extend(map.keys().cloned()); + } + } + names + } + + /// The full set of safe-output tool names for approval-partitioning and + /// emission: built-in tools (top-level keys) plus custom tools + /// (`scripts`/`jobs`). A top-level key that merely *configures* a custom + /// tool (same name) is folded into that custom tool rather than counted as a + /// separate built-in. + pub fn all_safe_output_tool_names(&self) -> Vec { + let custom: std::collections::HashSet = + self.custom_safe_output_tool_names().into_iter().collect(); + let mut names: Vec = self + .safe_output_tool_names() + .filter(|k| !custom.contains(*k)) + .cloned() + .collect(); + names.extend(custom); + names + } + /// Whether the workflow enables **any** safe-output tool. /// /// Single source of truth for the safe-outputs-summary feature gate: it @@ -1284,6 +1642,7 @@ impl FrontMatter { /// that was never downloaded. pub fn has_any_safe_output_tool(&self) -> bool { self.safe_output_tool_names().next().is_some() + || !self.custom_safe_output_tool_names().is_empty() } /// The parsed, sanitized `create-pull-request` config, or `None` when the @@ -1360,15 +1719,17 @@ impl FrontMatter { /// Partition enabled safe-output tool names into `(auto, reviewed)` where /// `reviewed` tools require manual approval and `auto` tools do not. Both - /// lists are sorted for deterministic emission. + /// lists are sorted for deterministic emission. Includes both built-in and + /// custom (`scripts`/`jobs`) tools; a custom tool's approval setting is read + /// from its top-level per-tool config key (or the section-level default). pub fn partition_safe_outputs_by_approval(&self) -> (Vec, Vec) { let mut auto = Vec::new(); let mut reviewed = Vec::new(); - for tool in self.safe_output_tool_names() { - if self.tool_requires_approval(tool).is_some() { - reviewed.push(tool.clone()); + for tool in self.all_safe_output_tool_names() { + if self.tool_requires_approval(&tool).is_some() { + reviewed.push(tool); } else { - auto.push(tool.clone()); + auto.push(tool); } } auto.sort(); @@ -1846,6 +2207,8 @@ pub struct Repository { #[serde(default = "default_ref")] #[serde(rename = "ref")] pub repo_ref: String, + #[serde(default)] + pub endpoint: Option, } fn default_ref() -> String { @@ -1871,6 +2234,9 @@ pub struct RepoEntry { /// Branch/tag ref. Defaults to `"refs/heads/main"`. #[serde(default = "default_ref", rename = "ref")] pub repo_ref: String, + /// Service connection name for GitHub/GitHub Enterprise repository resources. + #[serde(default)] + pub endpoint: Option, /// Whether the agent job checks out this repository. Defaults to `true`. #[serde(default = "default_checkout")] pub checkout: bool, @@ -2776,6 +3142,275 @@ impl SanitizeConfigTrait for LabelFilter { mod tests { use super::*; + const IMPORT_SHA: &str = "0123456789abcdef0123456789abcdef01234567"; + + fn bare_import(uses: impl Into) -> ImportEntry { + ImportEntry { + uses: uses.into(), + with: serde_json::Map::new(), + endpoint: None, + } + } + + // ─── imports field and spec parsing ───────────────────────────────────── + + #[test] + fn test_import_bare_remote_spec_parses_to_remote() { + let entry = bare_import(format!("owner/repo/path.md@{IMPORT_SHA}")); + + let source = entry.parse_source().unwrap(); + + match source { + ImportSource::Remote(spec) => { + assert_eq!(spec.owner, "owner"); + assert_eq!(spec.repo, "repo"); + assert_eq!(spec.path, "path.md"); + assert_eq!(spec.sha.as_str(), IMPORT_SHA); + assert_eq!(spec.section, None); + assert!(!spec.optional); + } + other => panic!("expected remote import, got {other:?}"), + } + } + + #[test] + fn test_import_object_form_captures_with_and_endpoint() { + let entry: ImportEntry = serde_yaml::from_str(&format!( + r#" +uses: acme/shared/deploy.md@{IMPORT_SHA} +with: + region: us-east-1 + retries: 3 +endpoint: shared-components-connection +"# + )) + .unwrap(); + + assert_eq!( + entry.endpoint, + Some(ImportEndpoint::GitHub { + name: "shared-components-connection".to_string() + }) + ); + assert_eq!( + entry.with.get("region").and_then(|v| v.as_str()), + Some("us-east-1") + ); + assert_eq!(entry.with.get("retries").and_then(|v| v.as_i64()), Some(3)); + + match entry.parse_source().unwrap() { + ImportSource::Remote(spec) => { + assert_eq!(spec.owner, "acme"); + assert_eq!(spec.repo, "shared"); + assert_eq!(spec.path, "deploy.md"); + assert_eq!(spec.sha.as_str(), IMPORT_SHA); + } + other => panic!("expected remote import, got {other:?}"), + } + } + + #[test] + fn test_import_endpoint_object_github_explicit() { + let entry: ImportEntry = serde_yaml::from_str(&format!( + "uses: acme/shared/deploy.md@{IMPORT_SHA}\nendpoint:\n name: gh-conn\n type: github\n" + )) + .unwrap(); + assert_eq!( + entry.endpoint, + Some(ImportEndpoint::GitHub { + name: "gh-conn".to_string() + }) + ); + // parse_source threads the endpoint into the spec. + match entry.parse_source().unwrap() { + ImportSource::Remote(spec) => assert_eq!(spec.endpoint, entry.endpoint), + other => panic!("expected remote import, got {other:?}"), + } + } + + #[test] + fn test_import_endpoint_object_ghe_requires_host() { + let entry: ImportEntry = serde_yaml::from_str(&format!( + "uses: acme/shared/deploy.md@{IMPORT_SHA}\n\ + endpoint:\n name: ghe-conn\n type: ghe\n host: ghe.acme.com\n" + )) + .unwrap(); + match entry.endpoint { + Some(ImportEndpoint::GitHubEnterprise { name, host }) => { + assert_eq!(name, "ghe-conn"); + assert_eq!(host.as_str(), "ghe.acme.com"); + } + other => panic!("expected GHE endpoint, got {other:?}"), + } + + // `ghe` without `host` must be rejected. + let err = serde_yaml::from_str::(&format!( + "uses: acme/shared/deploy.md@{IMPORT_SHA}\nendpoint:\n name: ghe-conn\n type: ghe\n" + )) + .unwrap_err(); + assert!(err.to_string().contains("requires `host`"), "{err}"); + } + + #[test] + fn test_import_endpoint_object_azure_repos_requires_org() { + let entry: ImportEntry = serde_yaml::from_str(&format!( + "uses: proj/repo/deploy.md@{IMPORT_SHA}\n\ + endpoint:\n name: xorg-conn\n type: azure-repos\n org: https://dev.azure.com/other\n" + )) + .unwrap(); + assert_eq!( + entry.endpoint, + Some(ImportEndpoint::AzureReposCrossOrg { + name: "xorg-conn".to_string(), + org: crate::secure::AzureDevOpsOrgUrl::parse("https://dev.azure.com/other",) + .unwrap(), + }) + ); + + let err = serde_yaml::from_str::(&format!( + "uses: acme/shared/deploy.md@{IMPORT_SHA}\n\ + endpoint:\n name: xorg-conn\n type: azure-repos\n org: https://attacker.example/steal\n" + )) + .unwrap_err(); + assert!(err.to_string().contains("dev.azure.com"), "{err}"); + + // `azure-repos` without `org` must be rejected. + let err = serde_yaml::from_str::(&format!( + "uses: proj/repo/deploy.md@{IMPORT_SHA}\n\ + endpoint:\n name: xorg-conn\n type: azure-repos\n" + )) + .unwrap_err(); + assert!(err.to_string().contains("requires `org`"), "{err}"); + } + + #[test] + fn test_import_endpoint_rejects_unknown_type_and_mismatched_fields() { + // Unknown type. + let err = serde_yaml::from_str::(&format!( + "uses: o/r/p.md@{IMPORT_SHA}\nendpoint:\n name: c\n type: bitbucket\n" + )) + .unwrap_err(); + assert!( + err.to_string().contains("unknown import endpoint type"), + "{err}" + ); + + // `org` on a github endpoint. + let err = serde_yaml::from_str::(&format!( + "uses: o/r/p.md@{IMPORT_SHA}\nendpoint:\n name: c\n type: github\n org: x\n" + )) + .unwrap_err(); + assert!( + err.to_string() + .contains("not valid for import endpoint type `github`"), + "{err}" + ); + } + + #[test] + fn test_import_endpoint_absent_is_same_org_azure_repos() { + let entry: ImportEntry = + serde_yaml::from_str(&format!("uses: proj/repo/deploy.md@{IMPORT_SHA}\n")).unwrap(); + assert_eq!(entry.endpoint, None); + match entry.parse_source().unwrap() { + ImportSource::Remote(spec) => assert_eq!(spec.endpoint, None), + other => panic!("expected remote import, got {other:?}"), + } + } + + #[test] + fn test_import_section_and_optional_marker_parse() { + let entry = bare_import(format!("owner/repo/path.md@{IMPORT_SHA}#Deploy?")); + + let source = entry.parse_source().unwrap(); + + match source { + ImportSource::Remote(spec) => { + assert_eq!(spec.section.as_deref(), Some("Deploy")); + assert!(spec.optional); + } + other => panic!("expected remote import, got {other:?}"), + } + } + + #[test] + fn test_import_non_sha_remote_refs_are_rejected() { + for ref_part in ["main", "v1.0.0", "abc123"] { + let entry = bare_import(format!("owner/repo/path.md@{ref_part}")); + + let err = entry.parse_source().unwrap_err().to_string(); + + assert!( + err.contains( + "cross-repository imports must be pinned to a full 40-character commit SHA" + ), + "unexpected error for {ref_part}: {err}" + ); + assert!( + err.contains(ref_part), + "error should include rejected ref {ref_part}: {err}" + ); + } + } + + #[test] + fn test_import_local_bare_path_parses_to_local() { + let entry = bare_import("shared/notify.md"); + + let source = entry.parse_source().unwrap(); + + match source { + ImportSource::Local { + path, + section, + optional, + } => { + assert_eq!(path, "shared/notify.md"); + assert_eq!(section, None); + assert!(!optional); + } + other => panic!("expected local import, got {other:?}"), + } + } + + #[test] + fn test_imports_field_deserializes_in_front_matter() { + let yaml = format!( + r#" +name: Test Agent +description: Test imports +imports: + - shared/notify.md + - uses: acme/shared/deploy.md@{IMPORT_SHA} + with: + region: us-east-1 + endpoint: shared-components-connection +"# + ); + + let fm: FrontMatter = serde_yaml::from_str(&yaml).unwrap(); + + assert_eq!(fm.imports.len(), 2); + assert!(matches!( + fm.imports[0].parse_source().unwrap(), + ImportSource::Local { .. } + )); + assert_eq!( + fm.imports[1].endpoint, + Some(ImportEndpoint::GitHub { + name: "shared-components-connection".to_string() + }) + ); + assert_eq!( + fm.imports[1].with.get("region").and_then(|v| v.as_str()), + Some("us-east-1") + ); + assert!(matches!( + fm.imports[1].parse_source().unwrap(), + ImportSource::Remote(_) + )); + } + // ─── SupplyChainConfig deserialization + resolution ────────────────────── fn parse_supply_chain(yaml: &str) -> SupplyChainConfig { @@ -3856,6 +4491,38 @@ Body assert_eq!(reviewed, vec!["add-pr-comment"]); } + #[test] + fn test_custom_tools_included_in_partition_and_names() { + let content = r#"--- +name: "Test" +description: "Test" +safe-outputs: + scripts: + send-notification: + run: node notify.js + jobs: + deploy-thing: + steps: [] + send-notification: + require-approval: true +--- + +Body +"#; + let (fm, _) = super::super::common::parse_markdown(content).unwrap(); + let mut custom = fm.custom_safe_output_tool_names(); + custom.sort(); + assert_eq!(custom, vec!["deploy-thing", "send-notification"]); + // The top-level `send-notification` key is config, not a separate tool. + let mut all = fm.all_safe_output_tool_names(); + all.sort(); + assert_eq!(all, vec!["deploy-thing", "send-notification"]); + // require-approval on the custom tool routes it to reviewed. + let (auto, reviewed) = fm.partition_safe_outputs_by_approval(); + assert_eq!(auto, vec!["deploy-thing"]); + assert_eq!(reviewed, vec!["send-notification"]); + } + #[test] fn test_require_approval_detailed_object() { let content = r#"--- diff --git a/src/engine.rs b/src/engine.rs index 02180e5b..91cbda68 100644 --- a/src/engine.rs +++ b/src/engine.rs @@ -29,6 +29,7 @@ pub const BLOCKED_ENV_KEYS: &[&str] = &[ "COPILOT_OTEL_ENABLED", "COPILOT_OTEL_EXPORTER_TYPE", "COPILOT_OTEL_FILE_EXPORTER_PATH", + "AZURE_DEVOPS_EXT_PAT", // Shell/system vars that could affect AWF or pipeline behavior "PATH", "HOME", @@ -1569,6 +1570,22 @@ mod tests { ); } + #[test] + fn engine_env_blocks_azure_devops_pat() { + let (fm, _) = parse_markdown( + "---\nname: test\ndescription: test\nengine:\n id: copilot\n env:\n AZURE_DEVOPS_EXT_PAT: evil\n---\n", + ) + .unwrap(); + let result = Engine::Copilot.env(&fm.engine); + assert!(result.is_err()); + assert!( + result + .unwrap_err() + .to_string() + .contains("compiler-controlled") + ); + } + #[test] fn engine_env_blocks_path() { let (fm, _) = parse_markdown( diff --git a/src/execute.rs b/src/execute.rs index 11f9dde4..dd10c450 100644 --- a/src/execute.rs +++ b/src/execute.rs @@ -6,12 +6,15 @@ use anyhow::{Context, Result}; use chrono::{SecondsFormat, Utc}; use log::{debug, error, info, warn}; -use serde::{Serialize, de::DeserializeOwned}; +use serde::{Deserialize, Serialize, de::DeserializeOwned}; use serde_json::Value; -use std::collections::HashMap; -use std::path::Path; +use std::collections::{HashMap, HashSet}; +use std::path::{Path, PathBuf}; +use std::process::Stdio; use tokio::fs::OpenOptions; -use tokio::io::AsyncWriteExt; +use tokio::io::{AsyncRead, AsyncReadExt, AsyncWriteExt}; +use tokio::process::Command; +use tokio::task::JoinHandle; use crate::ndjson::{self, EXECUTED_NDJSON_FILENAME, SAFE_OUTPUT_FILENAME}; use crate::safe_outputs::{ @@ -23,7 +26,7 @@ use crate::safe_outputs::{ UpdatePrResult, UpdateWikiPageResult, UpdateWorkItemResult, UploadBuildAttachmentResult, UploadPipelineArtifactResult, UploadWorkitemAttachmentResult, }; -use crate::sanitize::neutralize_pipeline_commands; +use crate::sanitize::{neutralize_pipeline_commands, sanitize, sanitize_config}; // Re-export memory types for use by main.rs pub use crate::tools::cache_memory::{MemoryConfig, process_agent_memory}; @@ -55,6 +58,1248 @@ impl ToolFilter { } } +/// Additional `ado-aw execute` custom safe-output modes. +#[derive(Debug, Default, Clone)] +pub struct CustomExecuteOptions { + /// Scripts-style native dispatcher config. + pub custom_config: Option, + /// Jobs-style wrapper phase: `pre` or `post`. + pub custom_phase: Option, + /// Jobs-style custom tool name. + pub tool: Option, + /// Compiler-resolved jobs-style proposal budget. + pub max: Option, + /// Jobs-style pre output path for selected proposals. + pub proposals_out: Option, + /// Jobs-style post input path for component result records. + pub results_in: Option, + /// Compiler-owned component provenance. + pub component_sha: Option, + pub component_source: Option, + pub manifest_digest: Option, + pub schema_digest: Option, +} + +impl CustomExecuteOptions { + /// Whether any custom-mode flag was supplied. + pub fn has_any_custom_flag(&self) -> bool { + self.custom_config.is_some() + || self.custom_phase.is_some() + || self.tool.is_some() + || self.max.is_some() + || self.proposals_out.is_some() + || self.results_in.is_some() + || self.component_sha.is_some() + || self.component_source.is_some() + || self.manifest_digest.is_some() + || self.schema_digest.is_some() + } +} + +const CUSTOM_SCHEMA_VERSION: u32 = 1; +const MAX_CUSTOM_PROCESS_OUTPUT_BYTES: usize = 1024 * 1024; +fn default_custom_max() -> usize { + crate::compile::custom_tools::DEFAULT_CUSTOM_MAX +} + +fn default_custom_cwd() -> PathBuf { + PathBuf::from(".") +} + +fn default_custom_timeout_minutes() -> u32 { + crate::compile::custom_tools::DEFAULT_CUSTOM_SCRIPT_TIMEOUT_MINUTES +} + +/// Scripts-style custom safe-output config emitted by the compiler. +#[derive(Debug, Deserialize)] +pub struct CustomScriptsConfig { + pub tools: HashMap, +} + +/// One scripts-style custom safe-output handler. +#[derive(Debug, Deserialize)] +pub struct CustomScriptToolConfig { + pub entrypoint: String, + #[serde(default = "default_custom_cwd")] + pub cwd: PathBuf, + #[serde(default = "default_custom_max")] + pub max: usize, + #[serde(default = "default_custom_timeout_minutes")] + pub timeout_minutes: u32, +} + +/// Compiler-owned custom component provenance attached to each final record. +#[derive(Debug, Clone, Serialize)] +pub struct CustomComponentProvenance { + pub source: Option, + pub sha: Option, + pub manifest_digest: Option, + pub schema_digest: Option, +} + +/// Attempt metadata attached to each custom execution record. +#[derive(Debug, Clone, Serialize)] +pub struct CustomAttemptMetadata { + pub number: u32, + pub staged: bool, + pub started_at: String, + pub ended_at: String, +} + +/// Final custom safe-output execution record. +/// +/// The top-level `name`, `status`, and `timestamp` fields intentionally mirror +/// the built-in `ExecutionRecord` so the existing audit reader can continue to +/// key off `name`/`status` while custom records carry richer provenance. +#[derive(Debug, Clone, Serialize)] +pub struct CustomExecutionRecord { + pub schema_version: u32, + pub tool: String, + pub proposal_id: String, + pub proposal_index: usize, + pub name: String, + pub status: String, + pub message: String, + #[serde(skip_serializing_if = "Option::is_none")] + pub data: Option, + pub component: CustomComponentProvenance, + pub attempt: CustomAttemptMetadata, + #[serde(skip_serializing_if = "Option::is_none")] + pub context: Option, + #[serde(skip_serializing_if = "Option::is_none")] + pub result: Option, + #[serde(skip_serializing_if = "Option::is_none")] + pub error: Option, + pub timestamp: String, +} + +#[derive(Debug)] +struct SelectedCustomProposal { + proposal_id: String, + proposal_index: usize, + entry: Value, + budget_result: Option, +} + +impl SelectedCustomProposal { + fn attempted(&self) -> bool { + self.budget_result.is_none() + } +} + +#[derive(Debug, Deserialize)] +struct ScriptResultLine { + status: String, + message: String, + data: Option, +} + +#[derive(Debug, Deserialize)] +struct ComponentResultLine { + #[serde(rename = "schema_version")] + _schema_version: u32, + proposal_id: String, + status: String, + message: String, + data: Option, +} + +struct CustomToolOutcome { + result: ExecutionResult, + record_status: &'static str, +} + +/// Execute a custom safe-output mode. Built-in execution is untouched unless +/// at least one custom flag is present. +pub async fn execute_custom_safe_outputs( + source: &Path, + safe_output_dir: &Path, + dry_run: bool, + options: CustomExecuteOptions, +) -> Result> { + match ( + options.custom_config.as_ref(), + options.custom_phase.as_deref(), + ) { + (Some(config_path), None) => { + execute_custom_scripts(config_path, safe_output_dir, dry_run, &options).await + } + (None, Some("pre")) => { + execute_custom_pre(source, safe_output_dir, dry_run, &options).await?; + Ok(Vec::new()) + } + (None, Some("post")) => { + execute_custom_post(source, safe_output_dir, dry_run, &options).await + } + (Some(_), Some(_)) => { + anyhow::bail!("--custom-config and --custom-phase are mutually exclusive") + } + (None, Some(other)) => { + anyhow::bail!("Unsupported --custom-phase '{other}' (expected 'pre' or 'post')") + } + (None, None) => { + anyhow::bail!("Custom execute mode requested without --custom-config or --custom-phase") + } + } +} + +async fn execute_custom_scripts( + config_path: &Path, + safe_output_dir: &Path, + dry_run: bool, + options: &CustomExecuteOptions, +) -> Result> { + if options.tool.is_some() + || options.max.is_some() + || options.proposals_out.is_some() + || options.results_in.is_some() + { + anyhow::bail!( + "--custom-config cannot be combined with --tool, --max, --proposals-out, or --results-in" + ); + } + + let config = load_custom_scripts_config(config_path).await?; + let safe_output_path = safe_output_dir.join(SAFE_OUTPUT_FILENAME); + let Some(entries) = load_safe_output_entries(&safe_output_path).await? else { + return Ok(Vec::new()); + }; + + let config_dir = config_path.parent().unwrap_or_else(|| Path::new(".")); + let provenance = provenance_from_options(options); + let mut budgets: HashMap = config + .tools + .iter() + .map(|(name, tool)| (name.clone(), (0, tool.max))) + .collect(); + + let mut results = Vec::new(); + for (i, entry) in entries.iter().enumerate() { + let Some(tool_name) = entry.get("name").and_then(|name| name.as_str()) else { + continue; + }; + let Some(tool_config) = config.tools.get(tool_name) else { + continue; + }; + + let proposal_context = entry.get("context").and_then(|value| value.as_str()); + let (executed, max) = budgets + .get_mut(tool_name) + .expect("budget map is initialized from config tools"); + let context_id = extract_entry_context(entry); + let proposal_id = proposal_id(tool_name, i); + if let Some(result) = + check_budget(entries.len(), i, tool_name, &context_id, *executed, *max) + { + append_custom_execution_record_for_result( + safe_output_dir, + tool_name, + &proposal_id, + i, + proposal_context, + &result, + provenance.clone(), + 0, + dry_run, + ) + .await; + results.push(result); + continue; + } + *executed += 1; + + let started_at = now_timestamp(); + let outcome = if dry_run { + let message = format!( + "Staged custom tool '{tool_name}' proposal '{proposal_id}'; would run '{}'", + sanitize_config(&tool_config.entrypoint) + ); + custom_status_to_outcome("staged", message, None) + } else { + let cwd = resolve_custom_cwd(config_dir, &tool_config.cwd); + run_custom_entrypoint( + tool_name, + &proposal_id, + &tool_config.entrypoint, + &cwd, + entry, + std::time::Duration::from_secs(u64::from(tool_config.timeout_minutes) * 60), + ) + .await + }; + let ended_at = now_timestamp(); + log_and_print_entry_result(i, entries.len(), tool_name, &outcome.result); + append_custom_execution_record_for_result_with_times( + safe_output_dir, + tool_name, + &proposal_id, + i, + proposal_context, + &outcome.result, + provenance.clone(), + CustomAttemptMetadata { + number: 1, + staged: dry_run, + started_at, + ended_at, + }, + Some(outcome.record_status), + ) + .await; + results.push(outcome.result); + } + + Ok(results) +} + +async fn execute_custom_pre( + _source: &Path, + safe_output_dir: &Path, + dry_run: bool, + options: &CustomExecuteOptions, +) -> Result<()> { + let tool = required_custom_tool(options)?; + let proposals_out = options + .proposals_out + .as_ref() + .context("--custom-phase pre requires --proposals-out")?; + if options.custom_config.is_some() || options.results_in.is_some() { + anyhow::bail!("--custom-phase pre cannot be combined with --custom-config or --results-in"); + } + + let max = required_custom_max(options)?; + let entries = load_entries_or_empty(safe_output_dir).await?; + let selected = select_custom_proposals(&entries, tool, max); + let attempted: Vec = selected + .iter() + .filter(|proposal| proposal.attempted()) + .map(proposal_with_id) + .collect(); + write_ndjson_values(proposals_out, &attempted).await?; + + if dry_run { + // ADO consumes this logging command and exposes the staged contract to + // downstream component steps as `ADO_AW_SAFE_OUTPUTS_STAGED=true`. + println!("##vso[task.setvariable variable=ADO_AW_SAFE_OUTPUTS_STAGED]true"); + } + println!( + "Wrote {} custom proposal(s) for '{}' to {}", + attempted.len(), + sanitize_config(tool), + proposals_out.display() + ); + Ok(()) +} + +async fn execute_custom_post( + _source: &Path, + safe_output_dir: &Path, + dry_run: bool, + options: &CustomExecuteOptions, +) -> Result> { + let tool = required_custom_tool(options)?; + let results_in = options + .results_in + .as_ref() + .context("--custom-phase post requires --results-in")?; + if options.custom_config.is_some() || options.proposals_out.is_some() { + anyhow::bail!( + "--custom-phase post cannot be combined with --custom-config or --proposals-out" + ); + } + + let max = required_custom_max(options)?; + let entries = load_entries_or_empty(safe_output_dir).await?; + let selected = select_custom_proposals(&entries, tool, max); + let attempted_ids: HashSet = selected + .iter() + .filter(|proposal| proposal.attempted()) + .map(|proposal| proposal.proposal_id.clone()) + .collect(); + let mut component_results = read_component_results(results_in, &attempted_ids).await?; + let provenance = provenance_from_options(options); + let mut results = Vec::new(); + + for proposal in selected { + let proposal_context = proposal + .entry + .get("context") + .and_then(|value| value.as_str()); + if let Some(result) = proposal.budget_result { + append_custom_execution_record_for_result( + safe_output_dir, + tool, + &proposal.proposal_id, + proposal.proposal_index, + proposal_context, + &result, + provenance.clone(), + 0, + dry_run, + ) + .await; + results.push(result); + continue; + } + + let started_at = now_timestamp(); + let outcome = if let Some(line) = component_results.remove(&proposal.proposal_id) { + custom_status_to_outcome(&line.status, line.message, line.data) + } else { + CustomToolOutcome { + result: ExecutionResult::failure(format!( + "Missing custom result for proposal_id '{}'", + sanitize(&proposal.proposal_id) + )), + record_status: "failed", + } + }; + let ended_at = now_timestamp(); + append_custom_execution_record_for_result_with_times( + safe_output_dir, + tool, + &proposal.proposal_id, + proposal.proposal_index, + proposal_context, + &outcome.result, + provenance.clone(), + CustomAttemptMetadata { + number: 1, + staged: dry_run || outcome.record_status == "staged", + started_at, + ended_at, + }, + Some(outcome.record_status), + ) + .await; + results.push(outcome.result); + } + + Ok(results) +} + +fn required_custom_tool(options: &CustomExecuteOptions) -> Result<&str> { + options + .tool + .as_deref() + .context("--custom-phase requires --tool") +} + +fn required_custom_max(options: &CustomExecuteOptions) -> Result { + let max = options.max.context("--custom-phase requires --max")?; + anyhow::ensure!(max > 0, "--custom-phase --max must be a positive integer"); + Ok(max) +} + +async fn load_custom_scripts_config(path: &Path) -> Result { + let contents = tokio::fs::read_to_string(path) + .await + .with_context(|| format!("Failed to read custom config: {}", path.display()))?; + let config: CustomScriptsConfig = serde_json::from_str(&contents) + .with_context(|| format!("Failed to parse custom config: {}", path.display()))?; + for (name, tool) in &config.tools { + anyhow::ensure!( + tool.max > 0, + "Custom tool '{}' has an invalid zero proposal budget", + sanitize_config(name) + ); + anyhow::ensure!( + (1..=crate::compile::custom_tools::MAX_CUSTOM_SCRIPT_TIMEOUT_MINUTES) + .contains(&tool.timeout_minutes), + "Custom tool '{}' timeout must be from 1 to {} minutes", + sanitize_config(name), + crate::compile::custom_tools::MAX_CUSTOM_SCRIPT_TIMEOUT_MINUTES + ); + } + Ok(config) +} + +async fn load_entries_or_empty(safe_output_dir: &Path) -> Result> { + let safe_output_path = safe_output_dir.join(SAFE_OUTPUT_FILENAME); + Ok(load_safe_output_entries(&safe_output_path) + .await? + .unwrap_or_default()) +} + +fn select_custom_proposals( + entries: &[Value], + tool: &str, + max: usize, +) -> Vec { + let mut selected = Vec::new(); + let mut executed = 0usize; + for (i, entry) in entries.iter().enumerate() { + if entry.get("name").and_then(|name| name.as_str()) != Some(tool) { + continue; + } + let context_id = extract_entry_context(entry); + let budget_result = check_budget(entries.len(), i, tool, &context_id, executed, max); + if budget_result.is_none() { + executed += 1; + } + selected.push(SelectedCustomProposal { + proposal_id: proposal_id(tool, i), + proposal_index: i, + entry: entry.clone(), + budget_result, + }); + } + selected +} + +fn proposal_id(tool: &str, index: usize) -> String { + format!("{}-{}", sanitize_config(tool), index) +} + +fn proposal_with_id(proposal: &SelectedCustomProposal) -> Value { + let mut value = proposal.entry.clone(); + if let Value::Object(ref mut map) = value { + map.insert( + "proposal_id".to_string(), + Value::String(proposal.proposal_id.clone()), + ); + map.insert( + "proposal_index".to_string(), + Value::Number(serde_json::Number::from(proposal.proposal_index)), + ); + } + value +} + +async fn write_ndjson_values(path: &Path, values: &[Value]) -> Result<()> { + if let Some(parent) = path.parent() + && !parent.as_os_str().is_empty() + { + tokio::fs::create_dir_all(parent) + .await + .with_context(|| format!("Failed to create directory: {}", parent.display()))?; + } + let mut contents = String::new(); + for value in values { + contents.push_str(&serde_json::to_string(value).context("Failed to serialize proposal")?); + contents.push('\n'); + } + tokio::fs::write(path, contents) + .await + .with_context(|| format!("Failed to write proposals file: {}", path.display())) +} + +fn resolve_custom_cwd(config_dir: &Path, cwd: &Path) -> PathBuf { + if cwd.is_absolute() { + cwd.to_path_buf() + } else { + config_dir.join(cwd) + } +} + +async fn run_custom_entrypoint( + tool: &str, + proposal_id: &str, + entrypoint: &str, + cwd: &Path, + proposal: &Value, + timeout: std::time::Duration, +) -> CustomToolOutcome { + let proposal_json = match serde_json::to_string(proposal) { + Ok(json) => json, + Err(err) => { + return CustomToolOutcome { + result: ExecutionResult::failure(format!( + "Failed to serialize custom proposal '{}': {}", + sanitize(proposal_id), + sanitize(&err.to_string()) + )), + record_status: "failed", + }; + } + }; + + let mut command = if cfg!(windows) { + let mut command = Command::new("cmd"); + command.arg("/C").arg(entrypoint); + command + } else { + let mut command = Command::new("sh"); + command.arg("-c").arg(entrypoint); + command + }; + #[cfg(unix)] + command.process_group(0); + #[cfg(windows)] + command.creation_flags(windows_sys::Win32::System::Threading::CREATE_SUSPENDED); + command + .current_dir(cwd) + .env("AW_PROPOSAL", &proposal_json) + .kill_on_drop(true) + .stdin(Stdio::piped()) + .stdout(Stdio::piped()) + .stderr(Stdio::piped()); + + let mut child = match command.spawn() { + Ok(child) => child, + Err(err) => { + return CustomToolOutcome { + result: ExecutionResult::failure(format!( + "Failed to start custom tool '{}': {}", + sanitize_config(tool), + sanitize(&err.to_string()) + )), + record_status: "failed", + }; + } + }; + let process_tree = match CustomProcessTree::attach(&child) { + Ok(process_tree) => process_tree, + Err(error) => { + let _ = child.start_kill(); + let _ = child.wait().await; + return CustomToolOutcome { + result: ExecutionResult::failure(format!( + "Failed to isolate custom tool '{}' process tree: {}", + sanitize_config(tool), + sanitize(&error.to_string()) + )), + record_status: "failed", + }; + } + }; + + // Write the proposal to the child's stdin CONCURRENTLY with draining its + // stdout/stderr. Writing the whole payload before reading any output would + // deadlock if the child emits more than a pipe buffer's worth before + // consuming stdin. The payload is ALSO available to the child via the + // `AW_PROPOSAL` env var; dropping the stdin handle when the write finishes + // signals EOF to a dispatcher that reads stdin. A stdin write error is + // non-fatal here (broken pipe if the child ignored stdin) — the child's own + // exit status is authoritative and handled below. + let stdin_payload = proposal_json.clone(); + if let Some(mut stdin) = child.stdin.take() { + tokio::spawn(async move { + let _ = stdin.write_all(stdin_payload.as_bytes()).await; + // `stdin` is dropped here, closing the pipe (EOF). + }); + } + + let (overflow_tx, mut overflow_rx) = tokio::sync::mpsc::unbounded_channel(); + let stdout_task = child + .stdout + .take() + .map(|stdout| spawn_pipe_reader(stdout, "stdout", overflow_tx.clone())); + let stderr_task = child + .stderr + .take() + .map(|stderr| spawn_pipe_reader(stderr, "stderr", overflow_tx)); + let started = tokio::time::Instant::now(); + let deadline = tokio::time::sleep(timeout); + tokio::pin!(deadline); + let status = tokio::select! { + result = child.wait() => match result { + Ok(status) => status, + Err(err) => { + let _ = process_tree.terminate(&mut child).await; + abort_pipe_reader(&stdout_task); + abort_pipe_reader(&stderr_task); + return CustomToolOutcome { + result: ExecutionResult::failure(format!( + "Failed to wait for custom tool '{}': {}", + sanitize_config(tool), + sanitize(&err.to_string()) + )), + record_status: "failed", + }; + } + }, + Some(stream) = overflow_rx.recv() => { + let _ = process_tree.terminate(&mut child).await; + abort_pipe_reader(&stdout_task); + abort_pipe_reader(&stderr_task); + return CustomToolOutcome { + result: ExecutionResult::failure(format!( + "Custom tool '{}' exceeded the {} byte {} capture limit", + sanitize_config(tool), + MAX_CUSTOM_PROCESS_OUTPUT_BYTES, + stream + )), + record_status: "failed", + }; + }, + () = &mut deadline => { + let termination = process_tree.terminate(&mut child).await; + abort_pipe_reader(&stdout_task); + abort_pipe_reader(&stderr_task); + return CustomToolOutcome { + result: ExecutionResult::failure(format!( + "Custom tool '{}' timed out after {} second(s){}", + sanitize_config(tool), + timeout.as_secs(), + termination + .err() + .map(|error| format!( + "; failed to terminate its process tree: {}", + sanitize(&error.to_string()) + )) + .unwrap_or_default() + )), + record_status: "failed", + }; + } + }; + let remaining = timeout.saturating_sub(started.elapsed()); + let (stdout, stderr) = + match tokio::time::timeout(remaining, collect_child_output(stdout_task, stderr_task)).await + { + Ok(Ok(output)) => output, + Ok(Err(error)) => { + return CustomToolOutcome { + result: ExecutionResult::failure(format!( + "Failed to read custom tool '{}' output: {}", + sanitize_config(tool), + sanitize(&error.to_string()) + )), + record_status: "failed", + }; + } + Err(_) => { + let termination = process_tree.terminate(&mut child).await; + return CustomToolOutcome { + result: ExecutionResult::failure(format!( + "Custom tool '{}' timed out after {} second(s) while draining output{}", + sanitize_config(tool), + timeout.as_secs(), + termination + .err() + .map(|error| format!( + "; failed to terminate its process tree: {}", + sanitize(&error.to_string()) + )) + .unwrap_or_default() + )), + record_status: "failed", + }; + } + }; + let output = std::process::Output { + status, + stdout, + stderr, + }; + + if !output.status.success() { + let stderr = String::from_utf8_lossy(&output.stderr); + let stdout = String::from_utf8_lossy(&output.stdout); + let detail = if stderr.trim().is_empty() { + stdout.trim() + } else { + stderr.trim() + }; + return CustomToolOutcome { + result: ExecutionResult::failure(format!( + "Custom tool '{}' exited with status {}{}", + sanitize_config(tool), + output.status, + if detail.is_empty() { + String::new() + } else { + format!(": {}", sanitize(detail)) + } + )), + record_status: "failed", + }; + } + + parse_script_result_stdout(tool, &output.stdout) +} + +fn spawn_pipe_reader( + mut reader: R, + stream: &'static str, + overflow: tokio::sync::mpsc::UnboundedSender<&'static str>, +) -> JoinHandle>> +where + R: AsyncRead + Unpin + Send + 'static, +{ + tokio::spawn(async move { + let mut output = Vec::new(); + let mut chunk = [0u8; 8192]; + loop { + let read = reader.read(&mut chunk).await?; + if read == 0 { + return Ok(output); + } + if output.len().saturating_add(read) > MAX_CUSTOM_PROCESS_OUTPUT_BYTES { + let _ = overflow.send(stream); + return Err(std::io::Error::other(format!( + "{stream} exceeded {MAX_CUSTOM_PROCESS_OUTPUT_BYTES} bytes" + ))); + } + output.extend_from_slice(&chunk[..read]); + } + }) +} + +fn abort_pipe_reader(reader: &Option>>>) { + if let Some(reader) = reader { + reader.abort(); + } +} + +async fn collect_child_output( + stdout: Option>>>, + stderr: Option>>>, +) -> std::io::Result<(Vec, Vec)> { + async fn collect( + reader: Option>>>, + ) -> std::io::Result> { + match reader { + Some(reader) => reader + .await + .map_err(|error| std::io::Error::other(error.to_string()))?, + None => Ok(Vec::new()), + } + } + + tokio::try_join!(collect(stdout), collect(stderr)) +} + +#[cfg(unix)] +struct CustomProcessTree { + process_group: i32, +} + +#[cfg(unix)] +impl CustomProcessTree { + fn attach(child: &tokio::process::Child) -> std::io::Result { + let pid = child + .id() + .ok_or_else(|| std::io::Error::other("custom tool exited before isolation"))?; + Ok(Self { + process_group: pid as i32, + }) + } + + async fn terminate(&self, child: &mut tokio::process::Child) -> std::io::Result<()> { + self.kill_group()?; + let _ = child.wait().await?; + Ok(()) + } + + fn kill_group(&self) -> std::io::Result<()> { + let result = unsafe { libc::kill(-self.process_group, libc::SIGKILL) }; + if result == 0 { + return Ok(()); + } + let error = std::io::Error::last_os_error(); + if error.raw_os_error() == Some(libc::ESRCH) { + Ok(()) + } else { + Err(error) + } + } +} + +#[cfg(unix)] +impl Drop for CustomProcessTree { + fn drop(&mut self) { + let _ = self.kill_group(); + } +} + +#[cfg(windows)] +struct CustomProcessTree { + job: usize, +} + +#[cfg(windows)] +impl CustomProcessTree { + fn attach(child: &tokio::process::Child) -> std::io::Result { + use windows_sys::Win32::Foundation::{CloseHandle, HANDLE}; + use windows_sys::Win32::System::JobObjects::{ + AssignProcessToJobObject, CreateJobObjectW, JOB_OBJECT_LIMIT_KILL_ON_JOB_CLOSE, + JOBOBJECT_EXTENDED_LIMIT_INFORMATION, JobObjectExtendedLimitInformation, + SetInformationJobObject, + }; + + let pid = child + .id() + .ok_or_else(|| std::io::Error::other("custom tool exited before isolation"))?; + let process = child + .raw_handle() + .ok_or_else(|| std::io::Error::other("custom tool exited before isolation"))? + as HANDLE; + let job = unsafe { CreateJobObjectW(std::ptr::null(), std::ptr::null()) }; + if job.is_null() { + return Err(std::io::Error::last_os_error()); + } + + let mut limits = JOBOBJECT_EXTENDED_LIMIT_INFORMATION::default(); + limits.BasicLimitInformation.LimitFlags = JOB_OBJECT_LIMIT_KILL_ON_JOB_CLOSE; + let configured = unsafe { + SetInformationJobObject( + job, + JobObjectExtendedLimitInformation, + (&raw const limits).cast(), + std::mem::size_of::() as u32, + ) + }; + if configured == 0 { + let error = std::io::Error::last_os_error(); + unsafe { + CloseHandle(job); + } + return Err(error); + } + if unsafe { AssignProcessToJobObject(job, process) } == 0 { + let error = std::io::Error::last_os_error(); + unsafe { + CloseHandle(job); + } + return Err(error); + } + if let Err(error) = Self::resume_windows_process(pid) { + unsafe { + CloseHandle(job); + } + return Err(error); + } + Ok(Self { job: job as usize }) + } + + fn resume_windows_process(pid: u32) -> std::io::Result<()> { + use windows_sys::Win32::Foundation::{CloseHandle, INVALID_HANDLE_VALUE}; + use windows_sys::Win32::System::Diagnostics::ToolHelp::{ + CreateToolhelp32Snapshot, TH32CS_SNAPTHREAD, THREADENTRY32, Thread32First, Thread32Next, + }; + use windows_sys::Win32::System::Threading::{ + OpenThread, ResumeThread, THREAD_SUSPEND_RESUME, + }; + + let snapshot = unsafe { CreateToolhelp32Snapshot(TH32CS_SNAPTHREAD, 0) }; + if snapshot == INVALID_HANDLE_VALUE { + return Err(std::io::Error::last_os_error()); + } + let mut entry = THREADENTRY32 { + dwSize: std::mem::size_of::() as u32, + ..THREADENTRY32::default() + }; + let mut found = unsafe { Thread32First(snapshot, &mut entry) } != 0; + while found { + if entry.th32OwnerProcessID == pid { + let thread = unsafe { OpenThread(THREAD_SUSPEND_RESUME, 0, entry.th32ThreadID) }; + if thread.is_null() { + let error = std::io::Error::last_os_error(); + unsafe { + CloseHandle(snapshot); + } + return Err(error); + } + let resumed = unsafe { ResumeThread(thread) }; + unsafe { + CloseHandle(thread); + CloseHandle(snapshot); + } + if resumed == u32::MAX { + return Err(std::io::Error::last_os_error()); + } + return Ok(()); + } + found = unsafe { Thread32Next(snapshot, &mut entry) } != 0; + } + unsafe { + CloseHandle(snapshot); + } + Err(std::io::Error::other( + "unable to find suspended custom tool thread", + )) + } + + async fn terminate(&self, child: &mut tokio::process::Child) -> std::io::Result<()> { + use windows_sys::Win32::System::JobObjects::TerminateJobObject; + + if unsafe { TerminateJobObject(self.job as _, 1) } == 0 { + return Err(std::io::Error::last_os_error()); + } + let _ = child.wait().await?; + Ok(()) + } +} + +#[cfg(windows)] +impl Drop for CustomProcessTree { + fn drop(&mut self) { + use windows_sys::Win32::Foundation::CloseHandle; + + unsafe { + CloseHandle(self.job as _); + } + } +} + +fn parse_script_result_stdout(tool: &str, stdout: &[u8]) -> CustomToolOutcome { + let stdout = String::from_utf8_lossy(stdout); + let lines: Vec<&str> = stdout + .lines() + .map(str::trim) + .filter(|line| !line.is_empty()) + .collect(); + if lines.len() != 1 { + // Distinguish "no output" from "extra output" and, when there is stray + // output, surface the first line so an author can locate a debug + // `console.log`/`print` that broke the one-JSON-line contract. + let detail = if lines.is_empty() { + " (no output — the tool must print exactly one JSON result line to stdout)".to_string() + } else { + let first: String = lines[0].chars().take(200).collect(); + format!( + " — is debug output going to stdout? first line: {}", + sanitize(&first) + ) + }; + return CustomToolOutcome { + result: ExecutionResult::failure(format!( + "Custom tool '{}' must print exactly one JSON line, got {}{}", + sanitize_config(tool), + lines.len(), + detail + )), + record_status: "failed", + }; + } + let parsed: ScriptResultLine = match serde_json::from_str(lines[0]) { + Ok(parsed) => parsed, + Err(err) => { + return CustomToolOutcome { + result: ExecutionResult::failure(format!( + "Custom tool '{}' printed malformed result JSON: {}", + sanitize_config(tool), + sanitize(&err.to_string()) + )), + record_status: "failed", + }; + } + }; + custom_status_to_outcome(&parsed.status, parsed.message, parsed.data) +} + +async fn read_component_results( + path: &Path, + attempted_ids: &HashSet, +) -> Result> { + let values = if path.exists() { + ndjson::read_ndjson_file(path).await? + } else { + Vec::new() + }; + let mut results = HashMap::new(); + for value in values { + let schema_version = value.get("schema_version").and_then(|v| v.as_u64()); + if schema_version != Some(CUSTOM_SCHEMA_VERSION as u64) { + anyhow::bail!( + "Custom result record has missing or unsupported schema_version: {}", + value + .get("schema_version") + .map(Value::to_string) + .unwrap_or_else(|| "".to_string()) + ); + } + let line: ComponentResultLine = + serde_json::from_value(value).context("Malformed custom result record")?; + if !attempted_ids.contains(&line.proposal_id) { + anyhow::bail!( + "Custom result references unknown proposal_id '{}'", + sanitize(&line.proposal_id) + ); + } + if results.insert(line.proposal_id.clone(), line).is_some() { + anyhow::bail!("Duplicate custom result record for proposal_id"); + } + } + Ok(results) +} + +fn custom_status_to_outcome( + status: &str, + message: String, + data: Option, +) -> CustomToolOutcome { + let message = sanitize(&message); + let data = data.map(sanitize_json_value); + match status { + "success" | "succeeded" => CustomToolOutcome { + result: match data { + Some(data) => ExecutionResult::success_with_data(message, data), + None => ExecutionResult::success(message), + }, + record_status: "succeeded", + }, + "failure" | "failed" => CustomToolOutcome { + result: match data { + Some(data) => ExecutionResult::failure_with_data(message, data), + None => ExecutionResult::failure(message), + }, + record_status: "failed", + }, + "staged" => CustomToolOutcome { + result: match data { + Some(data) => ExecutionResult::success_with_data(message, data), + None => ExecutionResult::success(message), + }, + record_status: "staged", + }, + other => CustomToolOutcome { + result: ExecutionResult::failure(format!( + "Custom result has unsupported status '{}'", + sanitize(other) + )), + record_status: "failed", + }, + } +} + +fn sanitize_json_value(value: Value) -> Value { + match value { + Value::String(s) => Value::String(sanitize(&s)), + Value::Array(values) => Value::Array(values.into_iter().map(sanitize_json_value).collect()), + Value::Object(map) => Value::Object( + map.into_iter() + .map(|(key, value)| (sanitize(&key), sanitize_json_value(value))) + .collect(), + ), + other => other, + } +} + +fn provenance_from_options(options: &CustomExecuteOptions) -> CustomComponentProvenance { + CustomComponentProvenance { + source: options.component_source.as_deref().map(sanitize_config), + sha: options.component_sha.as_deref().map(sanitize_config), + manifest_digest: options.manifest_digest.as_deref().map(sanitize_config), + schema_digest: options.schema_digest.as_deref().map(sanitize_config), + } +} + +fn now_timestamp() -> String { + Utc::now().to_rfc3339_opts(SecondsFormat::Secs, true) +} + +fn custom_record_status(result: &ExecutionResult, staged: bool) -> &'static str { + if result.is_budget_exhausted() { + "budget_exhausted" + } else if staged && result.success { + "staged" + } else { + execution_record_status(result) + } +} + +#[allow(clippy::too_many_arguments)] +async fn append_custom_execution_record_for_result( + safe_output_dir: &Path, + tool: &str, + proposal_id: &str, + proposal_index: usize, + proposal_context: Option<&str>, + result: &ExecutionResult, + provenance: CustomComponentProvenance, + attempt_number: u32, + staged: bool, +) { + let timestamp = now_timestamp(); + append_custom_execution_record_for_result_with_times( + safe_output_dir, + tool, + proposal_id, + proposal_index, + proposal_context, + result, + provenance, + CustomAttemptMetadata { + number: attempt_number, + staged, + started_at: timestamp.clone(), + ended_at: timestamp, + }, + None, + ) + .await; +} + +#[allow(clippy::too_many_arguments)] +async fn append_custom_execution_record_for_result_with_times( + safe_output_dir: &Path, + tool: &str, + proposal_id: &str, + proposal_index: usize, + proposal_context: Option<&str>, + result: &ExecutionResult, + provenance: CustomComponentProvenance, + attempt: CustomAttemptMetadata, + record_status_override: Option<&str>, +) { + let status = record_status_override + .map(str::to_string) + .unwrap_or_else(|| custom_record_status(result, attempt.staged).to_string()); + let data = result.data.clone().map(sanitize_json_value); + let message = sanitize(&result.message); + let record = CustomExecutionRecord { + schema_version: CUSTOM_SCHEMA_VERSION, + tool: sanitize_config(tool), + proposal_id: sanitize(proposal_id), + proposal_index, + name: sanitize_config(tool), + status: status.clone(), + message: message.clone(), + data: data.clone(), + component: provenance, + attempt, + context: proposal_context.map(sanitize), + result: if matches!(status.as_str(), "succeeded" | "staged") { + data + } else { + None + }, + error: if matches!(status.as_str(), "succeeded" | "staged") { + None + } else { + Some(message) + }, + timestamp: now_timestamp(), + }; + append_custom_execution_record(safe_output_dir, &record).await; +} + +async fn append_custom_execution_record(safe_output_dir: &Path, record: &CustomExecutionRecord) { + if let Err(err) = append_custom_execution_record_impl(safe_output_dir, record).await { + warn!( + "Failed to append custom execution record for {}: {}", + record.tool, + neutralize_pipeline_commands(&err.to_string()) + ); + } +} + +async fn append_custom_execution_record_impl( + safe_output_dir: &Path, + record: &CustomExecutionRecord, +) -> Result<()> { + let line = serde_json::to_string(record) + .context("Failed to serialize custom execution record")? + + "\n"; + let path = safe_output_dir.join(EXECUTED_NDJSON_FILENAME); + let mut file = OpenOptions::new() + .append(true) + .create(true) + .open(&path) + .await + .with_context(|| format!("Failed to open executed NDJSON file: {}", path.display()))?; + file.write_all(line.as_bytes()) + .await + .with_context(|| format!("Failed to append executed NDJSON file: {}", path.display()))?; + file.flush() + .await + .with_context(|| format!("Failed to flush executed NDJSON file: {}", path.display()))?; + Ok(()) +} + /// Execute all safe outputs from the NDJSON file in the specified directory pub async fn execute_safe_outputs( safe_output_dir: &Path, @@ -113,9 +1358,16 @@ pub async fn execute_safe_outputs( let mut results = Vec::new(); for (i, entry) in entries.iter().enumerate() { - if let Some(result) = - process_one_entry(i, entries.len(), entry, &mut budgets, filter, ctx, safe_output_dir) - .await + if let Some(result) = process_one_entry( + i, + entries.len(), + entry, + &mut budgets, + filter, + ctx, + safe_output_dir, + ) + .await { results.push(result); } @@ -198,8 +1450,13 @@ async fn process_one_entry( // Budget is consumed before execution so that failed attempts (target policy rejection, // network errors) still count — this prevents unbounded retries against a failing endpoint. if let Some(result) = enforce_budget(entry, budgets, total, i) { - append_execution_record(safe_output_dir, proposal_tool_name, &result, proposal_context) - .await; + append_execution_record( + safe_output_dir, + proposal_tool_name, + &result, + proposal_context, + ) + .await; return Some(result); } @@ -748,6 +2005,471 @@ mod tests { assert!(f.allows("add-pr-comment")); } + async fn write_success_script(dir: &Path) -> String { + tokio::fs::write( + dir.join("success.py"), + "import json\nprint(json.dumps({'status':'success','message':'ok'}))\n", + ) + .await + .unwrap(); + "python success.py".to_string() + } + + fn failing_entrypoint() -> &'static str { + if cfg!(windows) { "exit /B 1" } else { "exit 1" } + } + + async fn write_safe_outputs(dir: &Path, contents: &str) { + tokio::fs::write(dir.join(SAFE_OUTPUT_FILENAME), contents) + .await + .unwrap(); + } + + async fn write_custom_config(dir: &Path, entrypoint: &str, max: usize) -> PathBuf { + let path = dir.join("custom-config.json"); + let config = serde_json::json!({ + "tools": { + "send-notification": { + "entrypoint": entrypoint, + "cwd": ".", + "max": max + } + } + }); + tokio::fs::write(&path, serde_json::to_string(&config).unwrap()) + .await + .unwrap(); + path + } + + async fn write_custom_source(dir: &Path, max: usize) -> PathBuf { + let path = dir.join("agent.md"); + let content = format!( + r#"--- +name: Custom executor test +description: Test custom executor +safe-outputs: + jobs: + send-notification: + max: {max} +--- + +Test body. +"# + ); + tokio::fs::write(&path, content).await.unwrap(); + path + } + + async fn read_executed_records(dir: &Path) -> Vec { + ndjson::read_ndjson_file(&dir.join(EXECUTED_NDJSON_FILENAME)) + .await + .unwrap() + } + + #[tokio::test] + async fn test_custom_execute_scripts_native_dispatch_success_record() { + let temp_dir = tempfile::tempdir().unwrap(); + write_safe_outputs( + temp_dir.path(), + r#"{"name":"send-notification","context":"hello"}"#, + ) + .await; + let entrypoint = write_success_script(temp_dir.path()).await; + let config_path = write_custom_config(temp_dir.path(), &entrypoint, 3).await; + + let results = execute_custom_safe_outputs( + Path::new("unused.md"), + temp_dir.path(), + false, + CustomExecuteOptions { + custom_config: Some(config_path), + ..Default::default() + }, + ) + .await + .unwrap(); + + assert_eq!(results.len(), 1); + assert!(results[0].success); + let records = read_executed_records(temp_dir.path()).await; + assert_eq!(records[0]["name"], "send-notification"); + assert_eq!(records[0]["status"], "succeeded"); + assert_eq!(records[0]["message"], "ok"); + } + + #[tokio::test] + async fn test_custom_execute_scripts_native_dispatch_failure_record() { + let temp_dir = tempfile::tempdir().unwrap(); + write_safe_outputs(temp_dir.path(), r#"{"name":"send-notification"}"#).await; + let config_path = write_custom_config(temp_dir.path(), failing_entrypoint(), 3).await; + + let results = execute_custom_safe_outputs( + Path::new("unused.md"), + temp_dir.path(), + false, + CustomExecuteOptions { + custom_config: Some(config_path), + ..Default::default() + }, + ) + .await + .unwrap(); + + assert_eq!(results.len(), 1); + assert!(!results[0].success); + let records = read_executed_records(temp_dir.path()).await; + assert_eq!(records[0]["status"], "failed"); + } + + #[tokio::test] + async fn test_custom_script_timeout_returns_failed_outcome() { + let temp_dir = tempfile::tempdir().unwrap(); + let entrypoint = if cfg!(windows) { + "for /L %i in (0,0,1) do @rem" + } else { + "while true; do :; done" + }; + let outcome = run_custom_entrypoint( + "send-notification", + "send-notification-0", + entrypoint, + temp_dir.path(), + &serde_json::json!({"name": "send-notification"}), + std::time::Duration::from_millis(50), + ) + .await; + + assert!(!outcome.result.success); + assert_eq!(outcome.record_status, "failed"); + assert!(outcome.result.message.contains("timed out")); + } + + #[tokio::test] + async fn test_custom_script_output_is_bounded() { + let temp_dir = tempfile::tempdir().unwrap(); + tokio::fs::write( + temp_dir.path().join("flood.py"), + format!("print('x' * {})\n", MAX_CUSTOM_PROCESS_OUTPUT_BYTES + 1), + ) + .await + .unwrap(); + let outcome = run_custom_entrypoint( + "send-notification", + "send-notification-0", + if cfg!(windows) { + "python flood.py" + } else { + "python3 flood.py" + }, + temp_dir.path(), + &serde_json::json!({"name": "send-notification"}), + std::time::Duration::from_secs(10), + ) + .await; + + assert!(!outcome.result.success); + assert_eq!(outcome.record_status, "failed"); + assert!(outcome.result.message.contains("capture limit")); + } + + #[tokio::test] + async fn test_custom_script_timeout_terminates_descendants_after_parent_exit() { + let temp_dir = tempfile::tempdir().unwrap(); + tokio::fs::write( + temp_dir.path().join("spawn_child.py"), + "import subprocess, sys\nsubprocess.Popen([sys.executable, '-c', 'import time; time.sleep(30)'], stdout=sys.stdout, stderr=sys.stderr)\n", + ) + .await + .unwrap(); + let started = std::time::Instant::now(); + let outcome = run_custom_entrypoint( + "send-notification", + "send-notification-0", + if cfg!(windows) { + "python spawn_child.py" + } else { + "python3 spawn_child.py" + }, + temp_dir.path(), + &serde_json::json!({"name": "send-notification"}), + std::time::Duration::from_millis(200), + ) + .await; + + assert!(!outcome.result.success); + assert!(outcome.result.message.contains("timed out")); + assert!( + started.elapsed() < std::time::Duration::from_secs(5), + "descendant process tree was not terminated promptly" + ); + } + + #[tokio::test] + async fn test_custom_execute_scripts_budget_exhausted_record() { + let temp_dir = tempfile::tempdir().unwrap(); + write_safe_outputs( + temp_dir.path(), + r#"{"name":"send-notification","context":"first"} +{"name":"send-notification","context":"second"} +"#, + ) + .await; + let entrypoint = write_success_script(temp_dir.path()).await; + let config_path = write_custom_config(temp_dir.path(), &entrypoint, 1).await; + + let results = execute_custom_safe_outputs( + Path::new("unused.md"), + temp_dir.path(), + false, + CustomExecuteOptions { + custom_config: Some(config_path), + ..Default::default() + }, + ) + .await + .unwrap(); + + assert_eq!(results.len(), 2); + assert!(results[1].is_budget_exhausted()); + let records = read_executed_records(temp_dir.path()).await; + assert_eq!(records[1]["status"], "budget_exhausted"); + } + + #[tokio::test] + async fn test_custom_execute_scripts_dry_run_stages_without_spawn() { + let temp_dir = tempfile::tempdir().unwrap(); + write_safe_outputs(temp_dir.path(), r#"{"name":"send-notification"}"#).await; + let config_path = + write_custom_config(temp_dir.path(), "definitely-not-a-real-command-ado-aw", 3).await; + + let results = execute_custom_safe_outputs( + Path::new("unused.md"), + temp_dir.path(), + true, + CustomExecuteOptions { + custom_config: Some(config_path), + ..Default::default() + }, + ) + .await + .unwrap(); + + assert_eq!(results.len(), 1); + assert!(results[0].success); + let records = read_executed_records(temp_dir.path()).await; + assert_eq!(records[0]["status"], "staged"); + } + + #[tokio::test] + async fn test_custom_execute_jobs_pre_writes_filtered_proposals_with_ids() { + let temp_dir = tempfile::tempdir().unwrap(); + let source = temp_dir.path().join("source-is-not-read.md"); + let proposals_out = temp_dir.path().join("proposals.ndjson"); + write_safe_outputs( + temp_dir.path(), + r#"{"name":"send-notification","message":"first"} +{"name":"noop","context":"ignore"} +{"name":"send-notification","message":"second"} +"#, + ) + .await; + + execute_custom_safe_outputs( + &source, + temp_dir.path(), + false, + CustomExecuteOptions { + custom_phase: Some("pre".to_string()), + tool: Some("send-notification".to_string()), + max: Some(2), + proposals_out: Some(proposals_out.clone()), + ..Default::default() + }, + ) + .await + .unwrap(); + + let proposals = ndjson::read_ndjson_file(&proposals_out).await.unwrap(); + assert_eq!(proposals.len(), 2); + assert_eq!(proposals[0]["proposal_id"], "send-notification-0"); + assert_eq!(proposals[1]["proposal_id"], "send-notification-2"); + } + + #[tokio::test] + async fn test_custom_execute_jobs_post_enriches_component_results() { + let temp_dir = tempfile::tempdir().unwrap(); + let source = write_custom_source(temp_dir.path(), 2).await; + let results_in = temp_dir.path().join("results.ndjson"); + write_safe_outputs( + temp_dir.path(), + r#"{"name":"send-notification","context":"first"} +{"name":"send-notification","context":"second"} +"#, + ) + .await; + tokio::fs::write( + &results_in, + r#"{"schema_version":1,"proposal_id":"send-notification-0","status":"success","message":"ok0","data":{"url":"https://example.com"}} +{"schema_version":1,"proposal_id":"send-notification-1","status":"success","message":"ok1"} +"#, + ) + .await + .unwrap(); + + let results = execute_custom_safe_outputs( + &source, + temp_dir.path(), + false, + CustomExecuteOptions { + custom_phase: Some("post".to_string()), + tool: Some("send-notification".to_string()), + max: Some(2), + results_in: Some(results_in), + component_source: Some("repo/path".to_string()), + component_sha: Some("abc123".to_string()), + manifest_digest: Some("sha256:manifest".to_string()), + schema_digest: Some("sha256:schema".to_string()), + ..Default::default() + }, + ) + .await + .unwrap(); + + assert_eq!(results.len(), 2); + assert!(results.iter().all(|result| result.success)); + let records = read_executed_records(temp_dir.path()).await; + assert_eq!(records[0]["component"]["source"], "repo/path"); + assert_eq!(records[0]["component"]["sha"], "abc123"); + assert_eq!(records[0]["status"], "succeeded"); + } + + #[tokio::test] + async fn test_custom_execute_jobs_post_missing_result_becomes_failure() { + let temp_dir = tempfile::tempdir().unwrap(); + let source = write_custom_source(temp_dir.path(), 2).await; + let results_in = temp_dir.path().join("results.ndjson"); + write_safe_outputs( + temp_dir.path(), + r#"{"name":"send-notification"} +{"name":"send-notification"} +"#, + ) + .await; + tokio::fs::write( + &results_in, + r#"{"schema_version":1,"proposal_id":"send-notification-0","status":"success","message":"ok"}"#, + ) + .await + .unwrap(); + + let results = execute_custom_safe_outputs( + &source, + temp_dir.path(), + false, + CustomExecuteOptions { + custom_phase: Some("post".to_string()), + tool: Some("send-notification".to_string()), + max: Some(2), + results_in: Some(results_in), + ..Default::default() + }, + ) + .await + .unwrap(); + + assert_eq!(results.len(), 2); + assert!(!results[1].success); + assert!(results[1].message.contains("Missing custom result")); + let records = read_executed_records(temp_dir.path()).await; + assert_eq!(records[1]["status"], "failed"); + } + + #[tokio::test] + async fn test_custom_execute_jobs_post_unknown_schema_version_fails_closed() { + let temp_dir = tempfile::tempdir().unwrap(); + let source = write_custom_source(temp_dir.path(), 1).await; + let results_in = temp_dir.path().join("results.ndjson"); + write_safe_outputs(temp_dir.path(), r#"{"name":"send-notification"}"#).await; + tokio::fs::write( + &results_in, + r#"{"schema_version":99,"proposal_id":"send-notification-0","status":"success","message":"ok"}"#, + ) + .await + .unwrap(); + + let result = execute_custom_safe_outputs( + &source, + temp_dir.path(), + false, + CustomExecuteOptions { + custom_phase: Some("post".to_string()), + tool: Some("send-notification".to_string()), + max: Some(1), + results_in: Some(results_in), + ..Default::default() + }, + ) + .await; + + let err = result.unwrap_err(); + assert!( + err.to_string() + .contains("missing or unsupported schema_version: 99"), + "unexpected error: {err}" + ); + } + + #[test] + fn test_custom_execution_record_serializes_name_and_status() { + let record = CustomExecutionRecord { + schema_version: 1, + tool: "send-notification".to_string(), + proposal_id: "send-notification-0".to_string(), + proposal_index: 0, + name: "send-notification".to_string(), + status: "succeeded".to_string(), + message: "ok".to_string(), + data: None, + component: CustomComponentProvenance { + source: None, + sha: None, + manifest_digest: None, + schema_digest: None, + }, + attempt: CustomAttemptMetadata { + number: 1, + staged: false, + started_at: "2026-01-01T00:00:00Z".to_string(), + ended_at: "2026-01-01T00:00:01Z".to_string(), + }, + context: None, + result: None, + error: None, + timestamp: "2026-01-01T00:00:01Z".to_string(), + }; + + let value = serde_json::to_value(record).unwrap(); + assert_eq!(value["name"], "send-notification"); + assert_eq!(value["status"], "succeeded"); + } + + #[tokio::test] + async fn test_execute_no_custom_flags_normal_path_unaffected_smoke() { + let temp_dir = tempfile::tempdir().unwrap(); + assert!(!CustomExecuteOptions::default().has_any_custom_flag()); + + let results = execute_safe_outputs( + temp_dir.path(), + &ExecutionContext::default(), + &ToolFilter::default(), + ) + .await + .unwrap(); + + assert!(results.is_empty()); + } + #[test] fn test_stdout_print_neutralizes_result_message_pipeline_commands() { let message = "Uploaded '##vso[task.setvariable variable=X]y.txt'"; @@ -877,7 +2599,9 @@ mod tests { let temp_dir = tempfile::tempdir().unwrap(); let ctx = ExecutionContext::default(); - let results = execute_safe_outputs(temp_dir.path(), &ctx, &ToolFilter::default()).await.unwrap(); + let results = execute_safe_outputs(temp_dir.path(), &ctx, &ToolFilter::default()) + .await + .unwrap(); assert!(results.is_empty()); } @@ -893,7 +2617,9 @@ mod tests { tokio::fs::write(&safe_output_path, ndjson).await.unwrap(); let ctx = ExecutionContext::default(); - let results = execute_safe_outputs(temp_dir.path(), &ctx, &ToolFilter::default()).await.unwrap(); + let results = execute_safe_outputs(temp_dir.path(), &ctx, &ToolFilter::default()) + .await + .unwrap(); assert_eq!(results.len(), 2); assert!(results[0].success); @@ -913,7 +2639,9 @@ mod tests { tokio::fs::write(&safe_output_path, "").await.unwrap(); let ctx = ExecutionContext::default(); - let results = execute_safe_outputs(temp_dir.path(), &ctx, &ToolFilter::default()).await.unwrap(); + let results = execute_safe_outputs(temp_dir.path(), &ctx, &ToolFilter::default()) + .await + .unwrap(); assert!(results.is_empty()); } @@ -936,7 +2664,9 @@ mod tests { dry_run: true, ..Default::default() }; - let results = execute_safe_outputs(temp_dir.path(), &ctx, &ToolFilter::default()).await.unwrap(); + let results = execute_safe_outputs(temp_dir.path(), &ctx, &ToolFilter::default()) + .await + .unwrap(); assert_eq!(results.len(), 2); let executed_path = temp_dir.path().join(EXECUTED_NDJSON_FILENAME); @@ -967,7 +2697,9 @@ mod tests { dry_run: true, ..Default::default() }; - let results = execute_safe_outputs(temp_dir.path(), &ctx, &ToolFilter::default()).await.unwrap(); + let results = execute_safe_outputs(temp_dir.path(), &ctx, &ToolFilter::default()) + .await + .unwrap(); assert_eq!(results.len(), 2); let manifest = read_executed_manifest(&temp_dir).await; @@ -988,7 +2720,9 @@ mod tests { tokio::fs::write(&safe_output_path, "").await.unwrap(); let ctx = ExecutionContext::default(); - let results = execute_safe_outputs(temp_dir.path(), &ctx, &ToolFilter::default()).await.unwrap(); + let results = execute_safe_outputs(temp_dir.path(), &ctx, &ToolFilter::default()) + .await + .unwrap(); assert!(results.is_empty()); assert!(!temp_dir.path().join(EXECUTED_NDJSON_FILENAME).exists()); } @@ -1492,7 +3226,9 @@ mod tests { tokio::fs::write(&safe_output_path, ndjson).await.unwrap(); let ctx = ExecutionContext::default(); - let results = execute_safe_outputs(temp_dir.path(), &ctx, &ToolFilter::default()).await.unwrap(); + let results = execute_safe_outputs(temp_dir.path(), &ctx, &ToolFilter::default()) + .await + .unwrap(); // One entry processed (as a failure — unknown tool) assert_eq!(results.len(), 1); @@ -1659,7 +3395,9 @@ mod tests { ..Default::default() }; - let results = execute_safe_outputs(temp_dir.path(), &ctx, &ToolFilter::default()).await.unwrap(); + let results = execute_safe_outputs(temp_dir.path(), &ctx, &ToolFilter::default()) + .await + .unwrap(); assert_eq!(results.len(), 5); // Second create-work-item should be skipped @@ -1695,7 +3433,9 @@ mod tests { ..Default::default() }; - let results = execute_safe_outputs(temp_dir.path(), &ctx, &ToolFilter::default()).await.unwrap(); + let results = execute_safe_outputs(temp_dir.path(), &ctx, &ToolFilter::default()) + .await + .unwrap(); assert_eq!(results.len(), 1); assert!(results[0].success, "dry-run should succeed"); assert!( @@ -1727,7 +3467,9 @@ mod tests { ..Default::default() }; - let results = execute_safe_outputs(temp_dir.path(), &ctx, &ToolFilter::default()).await.unwrap(); + let results = execute_safe_outputs(temp_dir.path(), &ctx, &ToolFilter::default()) + .await + .unwrap(); assert_eq!(results.len(), 2); // create-work-item goes through Executor trait → dry-run intercepted assert!(results[0].message.contains("[DRY-RUN]")); diff --git a/src/main.rs b/src/main.rs index 13d16fd8..326e4d13 100644 --- a/src/main.rs +++ b/src/main.rs @@ -18,6 +18,7 @@ mod list; mod logging; mod mcp; mod mcp_author; +mod mcp_custom_tools; mod ndjson; mod remove; mod run; @@ -200,6 +201,7 @@ enum GraphCmd { } #[derive(Subcommand, Debug)] +#[allow(clippy::large_enum_variant)] enum Commands { /// Compile markdown to pipeline definition (or recompile all detected pipelines) Compile { @@ -242,6 +244,10 @@ enum Commands { /// Only expose these safe output tools (can be repeated). If omitted, all tools are exposed. #[arg(long = "enabled-tools")] enabled_tools: Vec, + /// Path to a compiler-generated JSON file of custom safe-output tool + /// definitions to register as dynamic MCP tools. + #[arg(long = "custom-tools")] + custom_tools: Option, }, /// Run the author-facing MCP server over stdio (IDE/Copilot Chat integration) McpAuthor {}, @@ -274,6 +280,36 @@ enum Commands { /// tools wait for approval. #[arg(long = "exclude")] exclude: Vec, + /// Compiler-generated scripts-style custom safe-output dispatcher config. + #[arg(long = "custom-config")] + custom_config: Option, + /// Jobs-style custom safe-output wrapper phase: pre or post. + #[arg(long = "custom-phase")] + custom_phase: Option, + /// Custom safe-output tool name for jobs-style pre/post phases. + #[arg(long = "tool")] + tool: Option, + /// Compiler-resolved proposal budget for jobs-style pre/post phases. + #[arg(long = "max")] + custom_max: Option, + /// Jobs-style pre phase output path for filtered proposals. + #[arg(long = "proposals-out")] + proposals_out: Option, + /// Jobs-style post phase input path for component result records. + #[arg(long = "results-in")] + results_in: Option, + /// Compiler-owned custom component source provenance. + #[arg(long = "component-source")] + component_source: Option, + /// Compiler-owned custom component SHA provenance. + #[arg(long = "component-sha")] + component_sha: Option, + /// Compiler-owned custom component manifest digest provenance. + #[arg(long = "manifest-digest")] + manifest_digest: Option, + /// Compiler-owned custom component schema digest provenance. + #[arg(long = "schema-digest")] + schema_digest: Option, }, /// Initialize a repository for AI-first agentic workflow authoring Init { @@ -820,9 +856,7 @@ async fn build_execution_context( ctx.tool_configs = front_matter .safe_outputs .iter() - .filter(|(k, _)| { - !crate::compile::types::SAFE_OUTPUT_RESERVED_KEYS.contains(&k.as_str()) - }) + .filter(|(k, _)| !crate::compile::types::SAFE_OUTPUT_RESERVED_KEYS.contains(&k.as_str())) .map(|(k, v)| (k.clone(), v.clone())) .collect(); // Merge ado-aw-debug.create-issue config under the same tool_configs map @@ -1019,9 +1053,7 @@ async fn main() -> Result<()> { // Also skipped in CI environments to avoid unnecessary outbound calls. let is_pipeline_internal = matches!( command, - Commands::Execute { .. } - | Commands::Mcp { .. } - | Commands::McpAuthor { .. } + Commands::Execute { .. } | Commands::Mcp { .. } | Commands::McpAuthor { .. } ); let update_handle = if !is_pipeline_internal && std::env::var_os("CI").is_none() { Some(tokio::spawn(update_check::check_for_update())) @@ -1059,13 +1091,20 @@ async fn main() -> Result<()> { output_directory, bounding_directory, enabled_tools, + custom_tools, } => { let filter = if enabled_tools.is_empty() { None } else { Some(enabled_tools) }; - mcp::run(&output_directory, &bounding_directory, filter.as_deref()).await?; + mcp::run( + &output_directory, + &bounding_directory, + filter.as_deref(), + custom_tools.as_deref(), + ) + .await?; } Commands::McpAuthor {} => { mcp_author::run_stdio().await?; @@ -1079,17 +1118,67 @@ async fn main() -> Result<()> { dry_run, only, exclude, + custom_config, + custom_phase, + tool, + custom_max, + proposals_out, + results_in, + component_source, + component_sha, + manifest_digest, + schema_digest, } => { - run_execute( - source, - safe_output_dir, - output_dir, - ado_org_url, - ado_project, - dry_run, - execute::ToolFilter { only, exclude }, - ) - .await?; + let custom_options = execute::CustomExecuteOptions { + custom_config, + custom_phase, + tool, + max: custom_max, + proposals_out, + results_in, + component_source, + component_sha, + manifest_digest, + schema_digest, + }; + if custom_options.has_any_custom_flag() { + if output_dir.is_some() + || ado_org_url.is_some() + || ado_project.is_some() + || !only.is_empty() + || !exclude.is_empty() + { + anyhow::bail!( + "Custom execute modes cannot be combined with --output-dir, --ado-org-url, --ado-project, --only, or --exclude" + ); + } + let results = execute::execute_custom_safe_outputs( + &source, + &safe_output_dir, + dry_run, + custom_options, + ) + .await?; + print_execution_summary(&results); + let failure_count = results.iter().filter(|r| !r.success).count(); + let warning_count = results.iter().filter(|r| r.is_warning()).count(); + if failure_count > 0 { + std::process::exit(1); + } else if warning_count > 0 { + std::process::exit(2); + } + } else { + run_execute( + source, + safe_output_dir, + output_dir, + ado_org_url, + ado_project, + dry_run, + execute::ToolFilter { only, exclude }, + ) + .await?; + } } Commands::Init { path, diff --git a/src/mcp.rs b/src/mcp.rs index 80d7befe..21c157f3 100644 --- a/src/mcp.rs +++ b/src/mcp.rs @@ -200,9 +200,8 @@ async fn try_root_commit_fallback(git_dir: &std::path::Path) -> Option { pub struct SafeOutputs { bounding_directory: PathBuf, output_directory: PathBuf, - /// ToolRouter is used by the rmcp framework's #[tool_handler] macro for - /// dispatching MCP tool calls. Clippy doesn't see this usage. - #[allow(dead_code)] + /// Runtime router used by the rmcp handler. This contains both the filtered + /// built-in routes and any dynamically registered custom routes. tool_router: ToolRouter, } @@ -239,9 +238,7 @@ fn resolve_git_dir_for_patch( } /// Check whether the working tree has uncommitted changes (staged or unstaged). -async fn check_uncommitted_changes( - git_dir: &std::path::Path, -) -> Result { +async fn check_uncommitted_changes(git_dir: &std::path::Path) -> Result { use tokio::process::Command; let status_output = Command::new("git") .args(["status", "--porcelain"]) @@ -395,10 +392,7 @@ fn should_keep_tool(tool_name: &str, enabled_tools: Option<&[String]>) -> bool { /// Apply the `enabled_tools` filter to `tool_router`, warn about unknown names, /// and log the before/after counts. -fn apply_tool_filter( - tool_router: &mut ToolRouter, - enabled_tools: Option<&[String]>, -) { +fn apply_tool_filter(tool_router: &mut ToolRouter, enabled_tools: Option<&[String]>) { let all_tools: Vec = tool_router .list_all() .iter() @@ -452,6 +446,12 @@ impl SafeOutputs { self.output_directory.join(SAFE_OUTPUT_FILENAME) } + /// Full path to the safe output NDJSON file, for dynamic custom-tool + /// handlers registered outside the static tool router. + pub(crate) fn custom_output_path(&self) -> PathBuf { + self.safe_output_path() + } + /// Read the current contents of the safe output file as NDJSON async fn read_safe_output_file(&self) -> Result> { ndjson::read_ndjson_file(&self.safe_output_path()).await @@ -488,6 +488,7 @@ impl SafeOutputs { bounding_directory: impl Into, output_directory: impl Into, enabled_tools: Option<&[String]>, + custom_tools: Option<&std::path::Path>, ) -> Result { let bounding_dir = bounding_directory.into(); let output_dir = output_directory.into(); @@ -522,6 +523,13 @@ impl SafeOutputs { // `None`; otherwise filtered against the explicit allowlist. apply_tool_filter(&mut tool_router, enabled_tools); + // Register config-driven custom safe-output tools (if any). These are + // added AFTER the built-in filter so a custom tool can never shadow a + // built-in (collisions are skipped with a warning). + if let Some(path) = custom_tools { + crate::mcp_custom_tools::apply_custom_tools(&mut tool_router, path)?; + } + Ok(Self { bounding_directory: bounding_dir, output_directory: output_dir, @@ -1501,7 +1509,7 @@ agent attempted work but couldn't finish (e.g., API timeouts, build failures, re } // Implement the server handler -#[tool_handler] +#[tool_handler(router = self.tool_router)] impl ServerHandler for SafeOutputs { fn get_info(&self) -> ServerInfo { ServerInfo::new(ServerCapabilities::builder().enable_tools().build()) @@ -1513,15 +1521,21 @@ pub async fn run( output_directory: &str, bounding_directory: &str, enabled_tools: Option<&[String]>, + custom_tools: Option<&std::path::Path>, ) -> Result<()> { // Create and run the server with STDIO transport - let service = SafeOutputs::new(bounding_directory, output_directory, enabled_tools) - .await? - .serve(stdio()) - .await - .inspect_err(|e| { - error!("Error starting MCP server: {}", e); - })?; + let service = SafeOutputs::new( + bounding_directory, + output_directory, + enabled_tools, + custom_tools, + ) + .await? + .serve(stdio()) + .await + .inspect_err(|e| { + error!("Error starting MCP server: {}", e); + })?; service .waiting() .await @@ -1536,7 +1550,7 @@ mod tests { async fn create_test_safe_outputs() -> (SafeOutputs, tempfile::TempDir) { let temp_dir = tempdir().unwrap(); - let safe_outputs = SafeOutputs::new(temp_dir.path(), temp_dir.path(), None) + let safe_outputs = SafeOutputs::new(temp_dir.path(), temp_dir.path(), None, None) .await .unwrap(); (safe_outputs, temp_dir) @@ -1640,7 +1654,9 @@ mod tests { git(&["add", "."]); git(&["commit", "-q", "-m", "agent change"]); - let base = SafeOutputs::find_merge_base(p).await.expect("resolves base"); + let base = SafeOutputs::find_merge_base(p) + .await + .expect("resolves base"); assert_eq!( base, base_sha, "merge-base must be the origin/main tip resolved via origin/HEAD" @@ -1650,7 +1666,7 @@ mod tests { #[tokio::test] async fn test_new_fails_with_invalid_bounding_directory() { let temp_dir = tempdir().unwrap(); - let result = SafeOutputs::new("/nonexistent/path", temp_dir.path(), None).await; + let result = SafeOutputs::new("/nonexistent/path", temp_dir.path(), None, None).await; assert!(result.is_err()); assert!( @@ -1664,7 +1680,7 @@ mod tests { #[tokio::test] async fn test_new_fails_with_invalid_output_directory() { let temp_dir = tempdir().unwrap(); - let result = SafeOutputs::new(temp_dir.path(), "/nonexistent/path", None).await; + let result = SafeOutputs::new(temp_dir.path(), "/nonexistent/path", None, None).await; assert!(result.is_err()); assert!(result.unwrap_err().to_string().contains("output_directory")); @@ -1839,12 +1855,128 @@ mod tests { assert_eq!(json["context"], "ctx"); } + #[tokio::test] + async fn test_safe_outputs_new_exposes_generated_custom_tools() { + let fm: crate::compile::types::FrontMatter = serde_yaml::from_str( + r#" +name: Test +description: Test +safe-outputs: + scripts: + send-notification: + description: Send a structured notification. + run: node notify.js + inputs: + title: { type: string, required: true, max-length: 120 } +"#, + ) + .unwrap(); + let schemas = crate::compile::custom_tools::generate_custom_tool_schemas(&fm).unwrap(); + let custom_tools_json = crate::compile::custom_tools::custom_tools_json(&schemas).unwrap(); + + let temp_dir = tempfile::tempdir().unwrap(); + let custom_tools_path = temp_dir.path().join("custom-tools.json"); + std::fs::write(&custom_tools_path, custom_tools_json).unwrap(); + + let so = SafeOutputs::new( + temp_dir.path(), + temp_dir.path(), + None, + Some(&custom_tools_path), + ) + .await + .unwrap(); + let tool_names: Vec = so + .tool_router + .list_all() + .iter() + .map(|t| t.name.to_string()) + .collect(); + + assert!(tool_names.contains(&"send-notification".to_string())); + assert!(tool_names.contains(&"noop".to_string())); + assert!(tool_names.contains(&"create-work-item".to_string())); + assert!( + ::get_tool(&so, "send-notification").is_some(), + "the MCP protocol handler must use the runtime router, not rebuild the static router" + ); + } + + #[tokio::test] + async fn test_server_handler_respects_runtime_tool_filter() { + let temp_dir = tempfile::tempdir().unwrap(); + let enabled = vec!["noop".to_string()]; + let so = SafeOutputs::new(temp_dir.path(), temp_dir.path(), Some(&enabled), None) + .await + .unwrap(); + + assert!(::get_tool(&so, "noop").is_some()); + assert!( + ::get_tool(&so, "add-build-tag").is_none(), + "the MCP protocol handler must honor the runtime enabled-tools filter" + ); + } + + #[tokio::test] + async fn test_custom_tools_do_not_shadow_builtin_routes() { + let baseline_dir = tempfile::tempdir().unwrap(); + let baseline = SafeOutputs::new(baseline_dir.path(), baseline_dir.path(), None, None) + .await + .unwrap(); + let baseline_count = baseline.tool_router.list_all().len(); + + let temp_dir = tempfile::tempdir().unwrap(); + let custom_tools_path = temp_dir.path().join("custom-tools.json"); + std::fs::write( + &custom_tools_path, + r#"[ + { + "name": "create-work-item", + "description": "custom replacement", + "inputSchema": { + "type": "object", + "additionalProperties": false, + "required": [], + "properties": {} + } + } + ]"#, + ) + .unwrap(); + + let so = SafeOutputs::new( + temp_dir.path(), + temp_dir.path(), + None, + Some(&custom_tools_path), + ) + .await + .unwrap(); + let tools = so.tool_router.list_all(); + assert_eq!(tools.len(), baseline_count); + assert_eq!( + tools + .iter() + .filter(|tool| tool.name.as_ref() == "create-work-item") + .count(), + 1 + ); + let create_work_item = tools + .iter() + .find(|tool| tool.name.as_ref() == "create-work-item") + .unwrap(); + assert_ne!( + create_work_item.description.as_deref(), + Some("custom replacement") + ); + } + // ─── Tool filtering tests ─────────────────────────────────────────── #[tokio::test] async fn test_tool_filtering_none_exposes_all() { let temp_dir = tempfile::tempdir().unwrap(); - let so = SafeOutputs::new(temp_dir.path(), temp_dir.path(), None) + let so = SafeOutputs::new(temp_dir.path(), temp_dir.path(), None, None) .await .unwrap(); let tools = so.tool_router.list_all(); @@ -1856,7 +1988,7 @@ mod tests { async fn test_tool_filtering_specific_tools() { let temp_dir = tempfile::tempdir().unwrap(); let enabled = vec!["create-pull-request".to_string()]; - let so = SafeOutputs::new(temp_dir.path(), temp_dir.path(), Some(&enabled)) + let so = SafeOutputs::new(temp_dir.path(), temp_dir.path(), Some(&enabled), None) .await .unwrap(); let tools = so.tool_router.list_all(); @@ -1879,7 +2011,7 @@ mod tests { let temp_dir = tempfile::tempdir().unwrap(); // Enable only a tool that doesn't exist — should still have always-on tools let enabled = vec!["nonexistent-tool".to_string()]; - let so = SafeOutputs::new(temp_dir.path(), temp_dir.path(), Some(&enabled)) + let so = SafeOutputs::new(temp_dir.path(), temp_dir.path(), Some(&enabled), None) .await .unwrap(); let tools = so.tool_router.list_all(); @@ -1902,7 +2034,7 @@ mod tests { "create-work-item".to_string(), "comment-on-work-item".to_string(), ]; - let so = SafeOutputs::new(temp_dir.path(), temp_dir.path(), Some(&enabled)) + let so = SafeOutputs::new(temp_dir.path(), temp_dir.path(), Some(&enabled), None) .await .unwrap(); let tools = so.tool_router.list_all(); @@ -1945,7 +2077,7 @@ mod tests { // Pass an enable list that includes every debug-only tool so they // remain in the router for this introspection check. let enabled: Vec = DEBUG_ONLY_TOOLS.iter().map(|s| s.to_string()).collect(); - let so = SafeOutputs::new(temp_dir.path(), temp_dir.path(), Some(&enabled)) + let so = SafeOutputs::new(temp_dir.path(), temp_dir.path(), Some(&enabled), None) .await .unwrap(); let router_tools: Vec = so @@ -1971,7 +2103,7 @@ mod tests { #[tokio::test] async fn test_filter_strips_debug_only_when_no_enabled_list() { let temp_dir = tempfile::tempdir().unwrap(); - let so = SafeOutputs::new(temp_dir.path(), temp_dir.path(), None) + let so = SafeOutputs::new(temp_dir.path(), temp_dir.path(), None, None) .await .unwrap(); let tool_names: Vec = so @@ -1995,7 +2127,7 @@ mod tests { async fn test_filter_keeps_debug_only_when_explicitly_enabled() { let temp_dir = tempfile::tempdir().unwrap(); let enabled = vec!["create-issue".to_string()]; - let so = SafeOutputs::new(temp_dir.path(), temp_dir.path(), Some(&enabled)) + let so = SafeOutputs::new(temp_dir.path(), temp_dir.path(), Some(&enabled), None) .await .unwrap(); let tool_names: Vec = so @@ -2015,7 +2147,7 @@ mod tests { async fn test_filter_strips_debug_only_when_other_tool_enabled() { let temp_dir = tempfile::tempdir().unwrap(); let enabled = vec!["create-work-item".to_string()]; - let so = SafeOutputs::new(temp_dir.path(), temp_dir.path(), Some(&enabled)) + let so = SafeOutputs::new(temp_dir.path(), temp_dir.path(), Some(&enabled), None) .await .unwrap(); let tool_names: Vec = so diff --git a/src/mcp_custom_tools.rs b/src/mcp_custom_tools.rs new file mode 100644 index 00000000..3a3bd433 --- /dev/null +++ b/src/mcp_custom_tools.rs @@ -0,0 +1,289 @@ +//! Dynamic (config-driven) custom safe-output MCP tools. +//! +//! Built-in safe-output tools are registered statically via the rmcp +//! `#[tool_router]` macro. Custom safe-output tools — declared by an imported +//! component's `safe-outputs.scripts.` / `safe-outputs.jobs.` +//! block (see the reusable-custom-safe-output-jobs feature) — are not known at +//! compile time, so they are surfaced from a **generated schema file** loaded +//! at server startup. +//! +//! The compiler emits a JSON array of tool definitions (name + description + +//! closed input JSON Schema, `additionalProperties: false`) and passes its path +//! via `--custom-tools`. Each entry is registered as a real MCP tool whose +//! generic handler appends a proposal NDJSON line tagged with the tool name — +//! the same `{ "name": , <...args> }` shape the built-in tools produce. +//! Budget, sanitization, and execution are enforced later by the Stage-3 +//! executor; the MCP server only records the proposal. + +use std::path::Path; +use std::sync::Arc; + +use anyhow::{Context, Result}; +use log::{info, warn}; +use rmcp::ErrorData as McpError; +use rmcp::handler::server::router::tool::{ToolRoute, ToolRouter}; +use rmcp::handler::server::tool::ToolCallContext; +use rmcp::model::{CallToolResult, Tool}; +use serde::Deserialize; +use serde_json::{Map, Value}; + +use crate::mcp::SafeOutputs; + +/// A single custom-tool definition as emitted by the compiler. +#[derive(Debug, Clone, Deserialize)] +pub struct CustomToolDef { + /// The MCP tool name (also the proposal `name` tag). + pub name: String, + /// Human-readable description shown to the agent. + #[serde(default)] + pub description: String, + /// Closed JSON Schema for the tool inputs (`additionalProperties: false`). + #[serde(rename = "inputSchema", default)] + pub input_schema: Map, +} + +/// Load custom-tool definitions from a compiler-generated JSON file. +/// +/// The file is a JSON array of [`CustomToolDef`]. A missing file is an error +/// (the caller only passes a path when custom tools were configured). +pub fn load_custom_tool_defs(path: &Path) -> Result> { + let contents = std::fs::read_to_string(path) + .with_context(|| format!("Failed to read custom-tools file: {}", path.display()))?; + let defs: Vec = serde_json::from_str(&contents) + .with_context(|| format!("Failed to parse custom-tools JSON: {}", path.display()))?; + Ok(defs) +} + +/// Register each custom-tool definition as a dynamic route on `tool_router`. +/// +/// A name that collides with an already-registered (built-in) tool is skipped +/// with a warning — built-ins are never shadowed by a custom tool. +pub fn register_custom_tools(tool_router: &mut ToolRouter, defs: Vec) { + for def in defs { + if tool_router.has_route(&def.name) { + warn!( + "Custom tool '{}' collides with an existing tool; skipping", + def.name + ); + continue; + } + tool_router.add_route(build_custom_route(def)); + } +} + +/// Build a dynamic [`ToolRoute`] whose handler records the agent's inputs as a +/// proposal NDJSON line. +fn build_custom_route(def: CustomToolDef) -> ToolRoute { + let tool_name = def.name.clone(); + let schema: Arc> = Arc::new(def.input_schema); + let tool = Tool::new(def.name.clone(), def.description.clone(), schema); + + ToolRoute::new_dyn(tool, move |ctx: ToolCallContext<'_, SafeOutputs>| { + let tool_name = tool_name.clone(); + let output_path = ctx.service.custom_output_path(); + let arguments = ctx.arguments.clone(); + Box::pin(async move { record_custom_proposal(&output_path, &tool_name, arguments).await }) + }) +} + +async fn record_custom_proposal( + output_path: &Path, + tool_name: &str, + arguments: Option>, +) -> std::result::Result { + append_custom_proposal(output_path, tool_name, arguments) + .await + .map_err(|error| { + warn!("Failed to record custom safe-output proposal '{tool_name}': {error:#}"); + McpError::internal_error( + format!( + "Failed to record custom safe-output proposal '{}'", + crate::sanitize::sanitize_config(tool_name) + ), + None, + ) + })?; + Ok(CallToolResult::success(vec![])) +} + +/// Append a `{ "name": , <...args> }` proposal line to the NDJSON file. +async fn append_custom_proposal( + output_path: &Path, + tool_name: &str, + arguments: Option>, +) -> Result<()> { + let mut entry = arguments.unwrap_or_default(); + // The `name` tag is compiler-owned and always wins over any agent input. + entry.insert("name".to_string(), Value::String(tool_name.to_string())); + let line = serde_json::to_string(&Value::Object(entry))? + "\n"; + + use tokio::io::AsyncWriteExt; + let mut file = tokio::fs::OpenOptions::new() + .create(true) + .append(true) + .open(output_path) + .await + .with_context(|| { + format!( + "Failed to open NDJSON for append: {}", + output_path.display() + ) + })?; + file.write_all(line.as_bytes()).await?; + file.flush().await?; + Ok(()) +} + +/// Load and register custom tools from `path`, logging a summary. +pub fn apply_custom_tools(tool_router: &mut ToolRouter, path: &Path) -> Result<()> { + let defs = load_custom_tool_defs(path)?; + let count = defs.len(); + register_custom_tools(tool_router, defs); + info!( + "Registered {count} custom safe-output tool(s) from {}", + path.display() + ); + Ok(()) +} + +#[cfg(test)] +mod tests { + use super::*; + use crate::compile::custom_tools::{custom_tools_json, generate_custom_tool_schemas}; + use crate::compile::types::FrontMatter; + + const SAMPLE: &str = r#"[ + { + "name": "send-notification", + "description": "Send a structured notification.", + "inputSchema": { + "type": "object", + "additionalProperties": false, + "required": ["title"], + "properties": { + "title": { "type": "string" }, + "severity": { "type": "string", "enum": ["info", "warning", "critical"] } + } + } + } + ]"#; + + fn parse_front_matter(yaml: &str) -> FrontMatter { + serde_yaml::from_str(yaml).unwrap() + } + + #[test] + fn test_load_custom_tool_defs() { + let dir = tempfile::tempdir().unwrap(); + let path = dir.path().join("custom-tools.json"); + std::fs::write(&path, SAMPLE).unwrap(); + + let defs = load_custom_tool_defs(&path).unwrap(); + assert_eq!(defs.len(), 1); + assert_eq!(defs[0].name, "send-notification"); + assert_eq!(defs[0].description, "Send a structured notification."); + assert_eq!( + defs[0].input_schema["additionalProperties"], + Value::Bool(false) + ); + } + + #[test] + fn test_register_custom_tools_adds_route() { + let defs = serde_json::from_str::>(SAMPLE).unwrap(); + let mut router: ToolRouter = ToolRouter::new(); + register_custom_tools(&mut router, defs); + assert!(router.has_route("send-notification")); + assert_eq!(router.list_all().len(), 1); + } + + #[test] + fn test_generated_custom_tool_json_registers_only_declared_tool() { + let fm = parse_front_matter( + r#" +name: Test +description: Test +safe-outputs: + scripts: + send-notification: + description: Send a structured notification. + run: node notify.js + inputs: + title: { type: string, required: true, max-length: 120 } +"#, + ); + let schemas = generate_custom_tool_schemas(&fm).unwrap(); + assert_eq!(schemas.len(), 1); + assert_eq!(schemas[0].name, "send-notification"); + + let empty_fm = parse_front_matter( + r#" +name: Test +description: Test +"#, + ); + assert!(generate_custom_tool_schemas(&empty_fm).unwrap().is_empty()); + + let dir = tempfile::tempdir().unwrap(); + let path = dir.path().join("custom-tools.json"); + std::fs::write(&path, custom_tools_json(&schemas).unwrap()).unwrap(); + let defs = load_custom_tool_defs(&path).unwrap(); + + let mut router: ToolRouter = ToolRouter::new(); + register_custom_tools(&mut router, defs); + assert!(router.has_route("send-notification")); + assert!(!router.has_route("deploy-thing")); + } + + #[test] + fn test_register_skips_duplicate_name() { + let mut router: ToolRouter = ToolRouter::new(); + let defs = serde_json::from_str::>(SAMPLE).unwrap(); + register_custom_tools(&mut router, defs); + // Registering the same name again is a no-op (existing route wins). + let dup = serde_json::from_str::>(SAMPLE).unwrap(); + register_custom_tools(&mut router, dup); + assert_eq!(router.list_all().len(), 1); + } + + #[tokio::test] + async fn test_append_custom_proposal_shape() { + let dir = tempfile::tempdir().unwrap(); + let path = dir.path().join("safe_outputs.ndjson"); + let mut args = Map::new(); + args.insert("title".to_string(), Value::String("Outage".to_string())); + // An agent-supplied "name" must be overridden by the compiler-owned tag. + args.insert("name".to_string(), Value::String("spoofed".to_string())); + + append_custom_proposal(&path, "send-notification", Some(args)) + .await + .unwrap(); + + let contents = std::fs::read_to_string(&path).unwrap(); + let v: Value = serde_json::from_str(contents.trim()).unwrap(); + assert_eq!(v["name"], "send-notification"); + assert_eq!(v["title"], "Outage"); + } + + #[tokio::test] + async fn test_record_custom_proposal_returns_mcp_error_on_persistence_failure() { + let dir = tempfile::tempdir().unwrap(); + let output_path = dir.path().join("output-is-a-directory"); + std::fs::create_dir(&output_path).unwrap(); + + let error = record_custom_proposal(&output_path, "send-notification", Some(Map::new())) + .await + .unwrap_err(); + + assert_eq!(error.code, rmcp::model::ErrorCode::INTERNAL_ERROR); + assert!( + error + .message + .contains("Failed to record custom safe-output proposal") + ); + assert!( + !error.message.contains(&output_path.display().to_string()), + "agent-visible MCP error must not expose the output path" + ); + } +} diff --git a/src/secure.rs b/src/secure.rs index 0490962c..5a595442 100644 --- a/src/secure.rs +++ b/src/secure.rs @@ -28,6 +28,7 @@ //! - [`GitRefName`] — a git ref obeying `git check-ref-format`. //! - [`BranchName`] — a git branch ref with extra length / leading-`-` / space rules. //! - [`CommitSha`] — a full 40-character hex commit SHA. +//! - [`AzureDevOpsOrgUrl`] — an HTTPS Azure DevOps organization collection URL. //! - [`ArtifactName`] — an ADO artifact / attachment name. //! - [`Identifier`] — an engine agent/model identifier. //! - [`HostName`] — a DNS-style hostname. @@ -227,6 +228,11 @@ validated_string! { CommitSha, "commit", validate::validate_commit_sha } +validated_string! { + /// An HTTPS Azure DevOps organization collection URL. + AzureDevOpsOrgUrl, "Azure DevOps organization URL", validate::validate_ado_org_url +} + validated_string! { /// An Azure DevOps artifact / attachment name. ArtifactName, "artifact_name", |value: &str, label: &str| { @@ -450,6 +456,13 @@ mod tests { assert!(CommitSha::parse("short").is_err()); } + #[test] + fn azure_devops_org_url_rules() { + assert!(AzureDevOpsOrgUrl::parse("https://dev.azure.com/myorg").is_ok()); + assert!(AzureDevOpsOrgUrl::parse("https://myorg.visualstudio.com").is_ok()); + assert!(AzureDevOpsOrgUrl::parse("https://attacker.example/myorg").is_err()); + } + #[test] fn artifact_name_rules() { assert!(ArtifactName::parse("build-logs_1.0").is_ok()); diff --git a/src/validate.rs b/src/validate.rs index 82b5f458..e04246e7 100644 --- a/src/validate.rs +++ b/src/validate.rs @@ -17,7 +17,7 @@ //! at deserialization time. New safe-output tools dealing with file paths or //! identifiers should type their fields with `secure::` newtypes. -use anyhow::Result; +use anyhow::{Context, Result}; use std::collections::HashMap; use std::path::{Path, PathBuf}; @@ -165,8 +165,9 @@ pub fn is_valid_provider_resource_url(s: &str) -> bool { !s.is_empty() && s.len() <= 256 && s.contains("://") - && s.chars() - .all(|c| c.is_ascii_alphanumeric() || matches!(c, '.' | '_' | '-' | ':' | '/' | '%' | '~')) + && s.chars().all(|c| { + c.is_ascii_alphanumeric() || matches!(c, '.' | '_' | '-' | ':' | '/' | '%' | '~') + }) } /// Validate an `engine.provider.base-url` value. @@ -740,12 +741,71 @@ pub fn validate_commit_sha(s: &str, label: &str) -> Result<()> { s.len() ); } + if !s.bytes().all(|b| b.is_ascii_hexdigit()) { anyhow::bail!("{label} must be a valid hex string: {s}"); } Ok(()) } +/// Validate a full Azure DevOps organization collection URL before any +/// compiler credential is sent to it. +pub fn validate_ado_org_url(s: &str, label: &str) -> Result<()> { + let url = url::Url::parse(s).with_context(|| format!("{label} must be a valid URL"))?; + anyhow::ensure!(url.scheme() == "https", "{label} must use https"); + anyhow::ensure!( + url.username().is_empty() && url.password().is_none(), + "{label} must not contain embedded credentials" + ); + anyhow::ensure!( + url.port().is_none(), + "{label} must not specify a non-default port" + ); + anyhow::ensure!( + url.query().is_none() && url.fragment().is_none(), + "{label} must not contain a query or fragment" + ); + let host = url + .host_str() + .map(str::to_ascii_lowercase) + .ok_or_else(|| anyhow::anyhow!("{label} must include a host"))?; + let segments: Vec = url + .path_segments() + .into_iter() + .flatten() + .filter(|segment| !segment.is_empty()) + .map(|segment| { + percent_encoding::percent_decode_str(segment) + .decode_utf8_lossy() + .into_owned() + }) + .collect(); + + if host == "dev.azure.com" { + anyhow::ensure!( + segments.len() == 1 && is_safe_path_segment(&segments[0]), + "{label} must be an Azure DevOps organization URL like \ + https://dev.azure.com/myorg" + ); + return Ok(()); + } + + if let Some(org) = host.strip_suffix(".visualstudio.com") { + anyhow::ensure!( + is_safe_path_segment(org), + "{label} has an invalid Azure DevOps organization name" + ); + anyhow::ensure!( + segments.is_empty() + || (segments.len() == 1 && segments[0].eq_ignore_ascii_case("DefaultCollection")), + "{label} legacy visualstudio.com URLs may only include /DefaultCollection" + ); + return Ok(()); + } + + anyhow::bail!("{label} must target dev.azure.com or an .visualstudio.com collection host") +} + /// Validate a string against `git check-ref-format` rules. /// /// Returns `Ok(())` if the name is valid, or an `Err` describing the violation. @@ -819,7 +879,9 @@ mod tests { fn test_is_valid_service_connection() { assert!(is_valid_service_connection("acr-conn")); assert!(is_valid_service_connection("My ACR Connection")); // spaces allowed - assert!(is_valid_service_connection("11112222-3333-4444-5555-666677778888")); + assert!(is_valid_service_connection( + "11112222-3333-4444-5555-666677778888" + )); assert!(!is_valid_service_connection("")); assert!(!is_valid_service_connection("with\nnewline")); assert!(!is_valid_service_connection("with'quote")); @@ -1244,6 +1306,28 @@ mod tests { assert!(validate_commit_sha("zzz3456789abcdef0123456789abcdef01234567", "c").is_err()); } + #[test] + fn test_validate_ado_org_url() { + for valid in [ + "https://dev.azure.com/my-org", + "https://my-org.visualstudio.com", + "https://my-org.visualstudio.com/DefaultCollection", + ] { + assert!(validate_ado_org_url(valid, "org").is_ok(), "{valid}"); + } + for invalid in [ + "http://dev.azure.com/my-org", + "https://evil.example.com/my-org", + "https://dev.azure.com.evil.example/my-org", + "https://dev.azure.com/my-org/project", + "https://user@dev.azure.com/my-org", + "https://dev.azure.com:444/my-org", + "https://dev.azure.com/my-org?token=x", + ] { + assert!(validate_ado_org_url(invalid, "org").is_err(), "{invalid}"); + } + } + #[test] fn test_validate_git_ref_name() { assert!(validate_git_ref_name("feature/my-analysis", "b").is_ok()); diff --git a/tests/ado-script-e2e/README.md b/tests/ado-script-e2e/README.md index 7432d02d..6a5a6482 100644 --- a/tests/ado-script-e2e/README.md +++ b/tests/ado-script-e2e/README.md @@ -16,6 +16,13 @@ Its normal test refs are immutable: Both pairs diverge from commit `52cfa195595a85eedc7ad405575bd92be93fb0e3`. +The same repository also hosts the immutable custom-safe-output component used +by candidate compiler smoke: + +| Ref | Commit | Purpose | +| --- | --- | --- | +| `e2e/custom-safe-output-v1` | `aa711dd17c4dfcde492b2bfad62e5fb1baad71f6` | Imported scripts/jobs component whose two routes tag the current child build. | + The registered definition is `\ado-script-e2e\ado-script e2e` (ID `2544`) in the `AgentPlayground` project. It uses Microsoft-hosted Ubuntu because the test needs only Node and Git; it does not depend on the internal executor E2E pool. diff --git a/tests/bash_lint_tests.rs b/tests/bash_lint_tests.rs index d962d1f4..5f6e5798 100644 --- a/tests/bash_lint_tests.rs +++ b/tests/bash_lint_tests.rs @@ -84,6 +84,7 @@ const FIXTURES: &[&str] = &[ "supply-chain-agent.md", "manual-review-agent.md", "github-app-token-agent.md", + "custom-safe-output-bash-coverage.md", ]; /// Step display names that the lint expects to find at least once across all @@ -137,6 +138,13 @@ const REQUIRED_STEP_DISPLAY_NAMES: &[&str] = &[ "Revoke GitHub App token", // src/compile/extensions/ado_script.rs github_app_token_revoke_step_typed() (activated by engine.github-app-token) "Prepare create-pull-request patch base", // src/compile/extensions/ado_script.rs prepare_pr_base_step_typed(PatchBase) (activated by safe-outputs.create-pull-request) "Prepare create-pull-request target worktree ref", // src/compile/extensions/ado_script.rs prepare_pr_base_step_typed(TargetWorktree) (activated by safe-outputs.create-pull-request) + "Detect custom proposals", // src/compile/agentic_pipeline.rs detect_custom_proposals_step + "Checkout pinned custom component", // src/compile/extensions/ado_script.rs checkout_component_step_typed + "Prepare custom safe-output executor", // src/compile/agentic_pipeline.rs prepare_custom_executor_binary_step + "Write custom safe-output config", // src/compile/agentic_pipeline.rs write_custom_scripts_config_step + "Execute custom safe output", // src/compile/agentic_pipeline.rs custom_scripts_execute_step + "Prepare custom safe-output proposals", // src/compile/agentic_pipeline.rs custom_jobs_pre_step + "Finalize custom safe-output results", // src/compile/agentic_pipeline.rs custom_jobs_post_step ]; fn ado_aw_binary() -> PathBuf { @@ -185,6 +193,30 @@ fn compile_fixture_with_flags( let src = fixtures_dir().join(fixture); let dest = workspace.join(fixture); std::fs::copy(&src, &dest).unwrap_or_else(|e| panic!("copy fixture {fixture}: {e}")); + if fixture == "custom-safe-output-bash-coverage.md" { + let cached_component = workspace + .join(".ado-aw") + .join("imports") + .join("AgentPlayground") + .join("ado-aw-e2e-fixture") + .join("aa711dd17c4dfcde492b2bfad62e5fb1baad71f6") + .join("components") + .join("custom-build-tags") + .join("component.md"); + std::fs::create_dir_all(cached_component.parent().unwrap()) + .expect("create custom component cache directory"); + std::fs::copy( + PathBuf::from(env!("CARGO_MANIFEST_DIR")) + .join("tests") + .join("compiler-smoke-e2e") + .join("component-fixture") + .join("components") + .join("custom-build-tags") + .join("component.md"), + &cached_component, + ) + .expect("copy custom component cache manifest"); + } let mut args = vec!["compile", dest.to_str().unwrap()]; args.extend_from_slice(extra_flags); diff --git a/tests/codemod_tests.rs b/tests/codemod_tests.rs index 813c0124..18e85dbd 100644 --- a/tests/codemod_tests.rs +++ b/tests/codemod_tests.rs @@ -300,6 +300,50 @@ fn test_integrity_check_inlined_imports_true_fails_on_body_edit() { ); } +#[test] +fn test_integrity_check_resolves_imports_and_passes() { + // Regression: `ado-aw check` must resolve `imports:` the same way `compile` + // does. Before the shared resolve-and-merge helper, `check` skipped import + // resolution entirely, so a freshly-compiled import-using workflow reported + // false "drift" (missing imported tools + body). A local import needs no + // cache/network, so this exercises the merge deterministically. + let dir = fresh_git_temp_dir(); + fs::write( + dir.path().join("component.md"), + "---\ntools:\n edit: true\n---\nImported guidance line.\n", + ) + .expect("write component"); + let source = write_source( + dir.path(), + "---\nname: check-imports-agent\ndescription: check resolves imports\nimports:\n - component.md\n---\nConsumer body.\n", + ); + + let compile_output = run_compile(&source); + assert!( + compile_output.status.success(), + "compile should succeed: {}", + String::from_utf8_lossy(&compile_output.stderr) + ); + let lock = source.with_extension("lock.yml"); + assert!(lock.exists(), "expected lock file at {}", lock.display()); + + // The imported tool + inlined imported body must be present in the lock. + let lock_content = fs::read_to_string(&lock).expect("read lock"); + assert!( + lock_content.contains("Imported guidance line."), + "compiled lock should inline the imported body" + ); + + // check must PASS on the freshly compiled, unedited import-using workflow. + let check_output = run_check(&lock); + assert!( + check_output.status.success(), + "check must resolve imports and pass on a fresh compile: stdout={:?} stderr={:?}", + String::from_utf8_lossy(&check_output.stdout), + String::from_utf8_lossy(&check_output.stderr) + ); +} + // ─── Non-mapping front matter ────────────────────────────────────────────── #[test] diff --git a/tests/compiler-smoke-e2e/README.md b/tests/compiler-smoke-e2e/README.md index 7585db42..e6fdd4f0 100644 --- a/tests/compiler-smoke-e2e/README.md +++ b/tests/compiler-smoke-e2e/README.md @@ -1,9 +1,12 @@ # Candidate compiler agentic smoke This suite validates compiler changes before release. It builds `ado-aw` and -`ado-script` from the exact pull-request or `main` commit, recompiles every -workflow under [`tests/safe-outputs/`](../safe-outputs/), and queues five fixed -AgentPlayground definitions against the regenerated YAML. +`ado-script` from the exact pull-request or `main` commit, recompiles four +selected release workflows under [`tests/safe-outputs/`](../safe-outputs/) +(canary, azure-cli, noop-target, and failure reporter) plus the candidate-only +custom-safe-output workflow in this directory, and queues five fixed +AgentPlayground definitions against the regenerated YAML. The weekly janitor is +intentionally outside the candidate lane. It complements rather than replaces the release-backed daily smoke: @@ -12,10 +15,10 @@ It complements rather than replaces the release-backed daily smoke: | Release smoke | Checked-in lock files and GitHub release assets | Customers can download and run the latest released compiler/runtime. | | Candidate smoke | Exact PR or nightly `main` checkout | Current compiler output compiles, passes integrity checks, and runs end to end before release. | -The candidate lane is deliberately ephemeral. It must not commit its generated -locks back to `tests/safe-outputs/`; those checked-in locks remain generated by -the latest released compiler so their runtime integrity check and downloaded -binary cannot drift. +The candidate lane is deliberately ephemeral. It must not commit any generated +lock back to the GitHub branch; the checked-in `tests/safe-outputs/` locks +remain generated by the latest released compiler so their runtime integrity +check and downloaded binary cannot drift. ## Flow @@ -28,8 +31,9 @@ binary cannot drift. `provenance.json`. 4. The test-only `compiler-smoke-e2e` TypeScript harness creates a detached worktree, adds an exact `supply-chain.pipeline-artifact` source to all five - smoke files, removes their schedules, recompiles them with the candidate - compiler, and runs `ado-aw check`. Both compiler subprocesses receive + smoke files, removes their schedules, resolves the SHA-pinned custom + component into the ephemeral import cache, recompiles them with the + candidate compiler, and runs `ado-aw check`. Both compiler subprocesses receive `ADO_AW_COMPILE_REMOTE_URL` set to the target `ado-aw-mirror` URL, so integrity-sensitive org/repository metadata matches the child checkout without changing the worktree's Git configuration. @@ -39,7 +43,11 @@ binary cannot drift. 6. Five fixed child definitions are queued concurrently with both that ref and its exact commit SHA. Each generated pipeline downloads and verifies the artifact from the still-running producer build. -7. The harness waits for all children, cancels non-terminal runs after a +7. The custom child imports both scripts-style and jobs-style tools from + `AgentPlayground/ado-aw-e2e-fixture` at an immutable commit. Each route adds + a distinct tag to its own child build, and the orchestrator verifies both + tags through the ADO Build Tags API. +8. The harness waits for all children, cancels non-terminal runs after a timeout, and deletes the per-run mirror ref only after every child is terminal. @@ -47,9 +55,18 @@ Candidate branches are never pushed to GitHub. ## Triggers -- Same-repository pull requests to `main`, path-filtered to compiler/runtime - code and these fixtures. -- Nightly at 01:00 UTC on `main`. +- Same-repository pull requests to `main` remain path-filtered to + compiler/runtime code and these fixtures, but definition `2559` requires a + collaborator comment before it queues. From the GitHub PR, use: + + ```text + /azp run ado-aw candidate compiler smoke + ``` + + Azure Pipelines posts the resulting optional check back to the PR. It is not + a required check in the `main` ruleset. +- Nightly at 01:00 UTC on `main`, with `always: true`, so the latest `main` + candidate is exercised daily even when no relevant files changed. - Manual runs for setup and diagnosis. The existing release smoke remains on its own schedule. @@ -65,6 +82,9 @@ forks.enabled = false forks.allowSecrets = false forks.allowFullAccessToken = false pipelineTriggerSettings.buildsEnabledForForks = false +isCommentRequiredForPullRequest = true +isCommentRequiredForInternalRepoPRs = true +commentOptionInternalRepos = all ``` The YAML also rejects `System.PullRequest.IsFork` as defense in depth, but that @@ -72,8 +92,8 @@ check is not the security boundary because a PR can modify its YAML. The live definition settings must be audited after registration and periodically from a trusted `main` run. Intentional PR definitions and scheduled/manual-only definitions are held in [`trigger-policy.json`](trigger-policy.json); the -orchestrator always audits its own `System.DefinitionId` as PR-only in addition -to that manifest. +orchestrator always audits its own `System.DefinitionId` as PR-only and +comment-gated in addition to that manifest. ## Fixed child definitions @@ -85,6 +105,12 @@ therefore runs only when the orchestrator explicitly supplies a candidate ref. The checked-in [`inert-child.yml`](inert-child.yml) is copied to those five paths when the base ref is created. +The candidate-only custom source is +[`custom-safe-output.md`](custom-safe-output.md). Its canonical component seed +is committed under [`component-fixture/`](component-fixture/) and published to +`ado-aw-e2e-fixture` on `refs/heads/e2e/custom-safe-output-v1`, pinned at +`aa711dd17c4dfcde492b2bfad62e5fb1baad71f6`. + See [`REGISTERED.md`](REGISTERED.md) for definition IDs and variables. ## Required permissions @@ -97,9 +123,10 @@ publication, needs: - Read builds and artifacts in AgentPlayground. Child build identities need Code Read on `ado-aw-mirror` and Build Read on the -producer definition. Authorize `agent-playground-read` and -`agent-playground-write` only on children whose generated fixture references -them. +producer definition. The custom child additionally needs Code Read on, and +pipeline-resource authorization for, `ado-aw-e2e-fixture`. Authorize +`agent-playground-read` and `agent-playground-write` only on children whose +generated fixture references them. Every child receives its own secret `GITHUB_TOKEN` for Copilot CLI authentication, using the same credential policy as the release-backed smoke @@ -117,8 +144,8 @@ Set these non-secret variables on the orchestrator: COMPILER_SMOKE_CANARY_DEFINITION_ID COMPILER_SMOKE_AZURE_CLI_DEFINITION_ID COMPILER_SMOKE_NOOP_TARGET_DEFINITION_ID -COMPILER_SMOKE_JANITOR_DEFINITION_ID COMPILER_SMOKE_REPORTER_DEFINITION_ID +COMPILER_SMOKE_CUSTOM_SAFE_OUTPUT_DEFINITION_ID ``` Optional overrides: @@ -150,8 +177,11 @@ The full live contract requires AgentPlayground and the fixed definitions: 1. the producer remains in progress after publishing its artifact; 2. every child downloads the exact producer `run-id`; 3. all five children succeed; -4. child provenance identifies the producer definition, build, and source SHA; -5. the per-run mirror ref is removed. +4. the custom child carries both + `ado-aw-custom-script-` and + `ado-aw-custom-job-`; +5. child provenance identifies the producer definition, build, and source SHA; +6. the per-run mirror ref is removed. Do not substitute `latest` artifact selection or transient feed versions if the exact-run contract fails. diff --git a/tests/compiler-smoke-e2e/REGISTERED.md b/tests/compiler-smoke-e2e/REGISTERED.md index 2182894c..7f9f8884 100644 --- a/tests/compiler-smoke-e2e/REGISTERED.md +++ b/tests/compiler-smoke-e2e/REGISTERED.md @@ -10,13 +10,21 @@ These definitions live in | `Candidate compiler smoke - canary` | `ado-aw-mirror` | `tests/safe-outputs/canary.lock.yml` | `2554` | | `Candidate compiler smoke - azure-cli` | `ado-aw-mirror` | `tests/safe-outputs/azure-cli.lock.yml` | `2555` | | `Candidate compiler smoke - noop-target` | `ado-aw-mirror` | `tests/safe-outputs/noop-target.lock.yml` | `2556` | -| `Candidate compiler smoke - janitor` | `ado-aw-mirror` | `tests/safe-outputs/janitor.lock.yml` | `2557` | | `Candidate compiler smoke - failure reporter` | `ado-aw-mirror` | `tests/safe-outputs/smoke-failure-reporter.lock.yml` | `2558` | +| `Candidate compiler smoke - custom safe outputs` | `ado-aw-mirror` | `tests/compiler-smoke-e2e/custom-safe-output.lock.yml` | `2564` | All five child definitions use `refs/heads/ado-aw-smoke-candidate-base` as their default branch. The ref is -permanent and inert; the harness never deletes it. Its seed commit is -`2b5fa7c336bd1f55a867cfc281e665472730b84c`. +permanent and inert; the harness never deletes it. + +Candidate janitor definition `2557` was retired. The release-backed janitor +definition `2548` remains scheduled weekly and is not part of this lane. + +The custom child imports +`AgentPlayground/ado-aw-e2e-fixture/components/custom-build-tags/component.md` +from immutable ref `refs/heads/e2e/custom-safe-output-v1`, pinned to +`aa711dd17c4dfcde492b2bfad62e5fb1baad71f6`. Definition `2564` is explicitly +authorized for that repository resource. ## Security record @@ -28,6 +36,9 @@ forks.enabled=false forks.allowSecrets=false forks.allowFullAccessToken=false pipelineTriggerSettings.buildsEnabledForForks=false +isCommentRequiredForPullRequest=true +isCommentRequiredForInternalRepoPRs=true +commentOptionInternalRepos=all ``` GitHub-backed AgentPlayground definitions that intentionally validate PRs were @@ -38,6 +49,16 @@ explicitly hardened on 2026-07-22: | `2544`, `2550` | `false` | `false` | `false` | `false` | | `2559` | `false` | `false` | `false` | `false` | +Definition `2559` is optional on pull requests. A collaborator with repository +write access queues it from the PR with: + +```text +/azp run ado-aw candidate compiler smoke +``` + +Its nightly `main` schedule remains independent and runs at 01:00 UTC with +`always: true`. + Release-smoke definitions `2545`-`2549` and scheduled trigger E2E definition `2551` have no CI or PR trigger metadata. Their schedules/manual queues remain independent of the candidate compiler PR lane. @@ -50,4 +71,7 @@ authentication. Definition `2558` additionally needs `ADO_AW_DEBUG_GITHUB_TOKEN`; server-side definition cloning does not copy secret values. +The custom child ID is configured on `2559` as +`COMPILER_SMOKE_CUSTOM_SAFE_OUTPUT_DEFINITION_ID=2564`. + No secret values belong in this file. diff --git a/tests/compiler-smoke-e2e/azure-pipelines.yml b/tests/compiler-smoke-e2e/azure-pipelines.yml index cabda4e8..7bcf75c0 100644 --- a/tests/compiler-smoke-e2e/azure-pipelines.yml +++ b/tests/compiler-smoke-e2e/azure-pipelines.yml @@ -1,9 +1,13 @@ # Candidate compiler agentic-smoke orchestrator. # # Builds ado-aw and ado-script from the exact checked-out PR/main commit, -# publishes an immutable candidate artifact, recompiles all five agentic smoke +# publishes an immutable candidate artifact, recompiles five agentic smoke # workflows, stages them on a short-lived ado-aw-mirror ref, and waits for the # five fixed child definitions to complete. +# +# PR eligibility remains path-filtered below, but definition 2559 requires a +# collaborator comment before queueing. Trigger it from GitHub with: +# /azp run ado-aw candidate compiler smoke trigger: none @@ -38,8 +42,8 @@ variables: EFFECTIVE_COMPILER_SMOKE_CANARY_DEFINITION_ID: $[ coalesce(variables['COMPILER_SMOKE_CANARY_DEFINITION_ID'], '') ] EFFECTIVE_COMPILER_SMOKE_AZURE_CLI_DEFINITION_ID: $[ coalesce(variables['COMPILER_SMOKE_AZURE_CLI_DEFINITION_ID'], '') ] EFFECTIVE_COMPILER_SMOKE_NOOP_TARGET_DEFINITION_ID: $[ coalesce(variables['COMPILER_SMOKE_NOOP_TARGET_DEFINITION_ID'], '') ] - EFFECTIVE_COMPILER_SMOKE_JANITOR_DEFINITION_ID: $[ coalesce(variables['COMPILER_SMOKE_JANITOR_DEFINITION_ID'], '') ] EFFECTIVE_COMPILER_SMOKE_REPORTER_DEFINITION_ID: $[ coalesce(variables['COMPILER_SMOKE_REPORTER_DEFINITION_ID'], '') ] + EFFECTIVE_COMPILER_SMOKE_CUSTOM_SAFE_OUTPUT_DEFINITION_ID: $[ coalesce(variables['COMPILER_SMOKE_CUSTOM_SAFE_OUTPUT_DEFINITION_ID'], '') ] EFFECTIVE_CRATES_IO_FEED: $[ coalesce(variables['CRATES_IO_FEED'], 'sparse+https://pkgs.dev.azure.com/msazuresphere/AgentPlayground/_packaging/cargo/Cargo/index/') ] jobs: @@ -53,6 +57,27 @@ jobs: persistCredentials: false displayName: Checkout ado-aw candidate + - script: | + set -euo pipefail + DIAGNOSTICS="$(Build.ArtifactStagingDirectory)/compiler-smoke-diagnostics" + mkdir -p "$DIAGNOSTICS" + jq -n \ + --arg schema "ado-aw/candidate-smoke-diagnostics/1" \ + --arg build_id "$(Build.BuildId)" \ + --arg definition_id "$(System.DefinitionId)" \ + --arg reason "$(Build.Reason)" \ + --arg source_branch "$(Build.SourceBranch)" \ + --arg source_version "$(Build.SourceVersion)" \ + '{ + schema: $schema, + build_id: $build_id, + definition_id: $definition_id, + reason: $reason, + source_branch: $source_branch, + source_version: $source_version + }' > "$DIAGNOSTICS/context.json" + displayName: Initialize candidate smoke diagnostics + - script: | set -euo pipefail if [ "${SYSTEM_PULLREQUEST_ISFORK:-false}" = "True" ] || \ @@ -231,12 +256,121 @@ jobs: set -euo pipefail BASE="$(System.CollectionUri)$(System.TeamProject)/_apis/build/definitions" POLICY="tests/compiler-smoke-e2e/trigger-policy.json" + DIAGNOSTICS="$(Build.ArtifactStagingDirectory)/compiler-smoke-diagnostics" + RAW_DIAGNOSTICS="$(Agent.TempDirectory)/compiler-smoke-policy" + SELF_ID="$(System.DefinitionId)" + mkdir -p "$RAW_DIAGNOSTICS" + + fetch_definition() { + local id="$1" + local attempt + local body + local curl_exit + local curl_metadata + local headers + local jq_error + local metadata + local raw_headers + local response_bytes + local sample + + for attempt in 1 2 3; do + body="$RAW_DIAGNOSTICS/definition-${id}-attempt-${attempt}.body" + raw_headers="$RAW_DIAGNOSTICS/definition-${id}-attempt-${attempt}.headers.raw" + headers="$DIAGNOSTICS/definition-${id}-attempt-${attempt}.headers" + metadata="$DIAGNOSTICS/definition-${id}-attempt-${attempt}.metadata" + jq_error="$DIAGNOSTICS/definition-${id}-attempt-${attempt}.jq-error" + sample="$DIAGNOSTICS/definition-${id}-attempt-${attempt}.body-sample" + : > "$body" + : > "$raw_headers" + : > "$jq_error" + curl_exit=0 + curl_metadata="" + + if curl_metadata="$(curl -sS --fail-with-body \ + --connect-timeout 15 \ + --max-time 60 \ + --dump-header "$raw_headers" \ + --output "$body" \ + --write-out 'http_code=%{http_code}\ncontent_type=%{content_type}\nnum_redirects=%{num_redirects}\nurl_effective=%{url_effective}\ntime_total=%{time_total}\n' \ + -H "Authorization: Bearer $SYSTEM_ACCESSTOKEN" \ + "$BASE/$id?api-version=7.1")"; then + : + else + curl_exit=$? + fi + + awk '{ + lower = tolower($0) + if ($0 ~ /^HTTP\// || + lower ~ /^(content-type|content-length|x-vss-e2eid|x-tfs-session|activityid|request-context):/) { + print + } + }' "$raw_headers" > "$headers" + rm -f "$raw_headers" + + response_bytes="$(wc -c < "$body" | tr -d ' ')" + { + printf 'definition_id=%s\n' "$id" + printf 'attempt=%s\n' "$attempt" + printf 'curl_exit=%s\n' "$curl_exit" + printf 'response_bytes=%s\n' "$response_bytes" + printf '%s\n' "$curl_metadata" + } > "$metadata" + + if [ "$curl_exit" -eq 0 ] && + jq -e 'type == "object"' "$body" >/dev/null 2>"$jq_error"; then + jq '{ + id, + name, + revision, + triggers, + repository: (.repository | {id, name, type, defaultBranch}), + variable_names: ((.variables // {}) | keys) + }' "$body" > "$DIAGNOSTICS/definition-${id}-snapshot.json" + rm -f "$jq_error" + cat "$body" + rm -f "$body" + return 0 + fi + + head -c 16384 "$body" > "$sample" + rm -f "$body" + { + echo "ADO definition response validation failed for definition $id (attempt $attempt/3)." + cat "$metadata" + if [ -s "$headers" ]; then + echo "response_headers_begin" + cat "$headers" + echo "response_headers_end" + fi + if [ -s "$jq_error" ]; then + echo "jq_error_begin" + cat "$jq_error" + echo "jq_error_end" + fi + if [ -s "$sample" ]; then + echo "response_sample_begin" + head -c 2048 "$sample" | LC_ALL=C tr -c '\11\12\15\40-\176' '?' + echo + echo "response_sample_end" + fi + } >&2 + + if [ "$attempt" -lt 3 ]; then + sleep "$((attempt * 2))" + fi + done + + return 1 + } + PR_IDS="$( jq -er ' select(.schema == "ado-aw/agentplayground-trigger-policy/1") | .pr_definition_ids[] ' "$POLICY" - ) $(System.DefinitionId)" + ) $SELF_ID" SCHEDULED_ONLY_IDS="$( jq -er ' select(.schema == "ado-aw/agentplayground-trigger-policy/1") @@ -244,10 +378,15 @@ jobs: ' "$POLICY" )" + SELF_JSON="" for id in $PR_IDS; do - JSON="$(curl -fsS \ - -H "Authorization: Bearer $SYSTEM_ACCESSTOKEN" \ - "$BASE/$id?api-version=7.1")" + if ! JSON="$(fetch_definition "$id")"; then + echo "Unable to audit PR definition $id after 3 attempts; see compiler-smoke-diagnostics." >&2 + exit 1 + fi + if [ "$id" = "$SELF_ID" ]; then + SELF_JSON="$JSON" + fi if ! printf '%s' "$JSON" | jq -e ' ([.triggers[]? | select(.triggerType == "continuousIntegration")] | length == 0) and any( @@ -260,14 +399,35 @@ jobs: )' >/dev/null; then NAME="$(printf '%s' "$JSON" | jq -r '.name // "unknown"')" echo "PR trigger policy drift detected for definition $id ($NAME)." >&2 + printf '%s' "$JSON" | jq '{id, name, revision, triggers}' >&2 exit 1 fi done + if [ -z "$SELF_JSON" ]; then + echo "Candidate definition $SELF_ID was not included in the PR policy audit." >&2 + exit 1 + fi + if ! printf '%s' "$SELF_JSON" | jq -e ' + any( + .triggers[]?; + .triggerType == "pullRequest" + and .isCommentRequiredForPullRequest == true + and .requireCommentsForNonTeamMembersOnly == false + and .requireCommentsForNonTeamMemberAndNonContributors == false + and .isCommentRequiredForInternalRepoPRs == true + and .commentOptionInternalRepos == "all" + )' >/dev/null; then + echo "Candidate compiler PR comment-gate policy drift detected for definition $SELF_ID." >&2 + printf '%s' "$SELF_JSON" | jq '{id, name, revision, triggers}' >&2 + exit 1 + fi + for id in $SCHEDULED_ONLY_IDS; do - JSON="$(curl -fsS \ - -H "Authorization: Bearer $SYSTEM_ACCESSTOKEN" \ - "$BASE/$id?api-version=7.1")" + if ! JSON="$(fetch_definition "$id")"; then + echo "Unable to audit scheduled-only definition $id after 3 attempts; see compiler-smoke-diagnostics." >&2 + exit 1 + fi if ! printf '%s' "$JSON" | jq -e ' [.triggers[]? | select( @@ -277,6 +437,7 @@ jobs: | length == 0' >/dev/null; then NAME="$(printf '%s' "$JSON" | jq -r '.name // "unknown"')" echo "Scheduled-only trigger policy drift detected for definition $id ($NAME)." >&2 + printf '%s' "$JSON" | jq '{id, name, revision, triggers}' >&2 exit 1 fi done @@ -305,8 +466,8 @@ jobs: COMPILER_SMOKE_CANARY_DEFINITION_ID: $(EFFECTIVE_COMPILER_SMOKE_CANARY_DEFINITION_ID) COMPILER_SMOKE_AZURE_CLI_DEFINITION_ID: $(EFFECTIVE_COMPILER_SMOKE_AZURE_CLI_DEFINITION_ID) COMPILER_SMOKE_NOOP_TARGET_DEFINITION_ID: $(EFFECTIVE_COMPILER_SMOKE_NOOP_TARGET_DEFINITION_ID) - COMPILER_SMOKE_JANITOR_DEFINITION_ID: $(EFFECTIVE_COMPILER_SMOKE_JANITOR_DEFINITION_ID) COMPILER_SMOKE_REPORTER_DEFINITION_ID: $(EFFECTIVE_COMPILER_SMOKE_REPORTER_DEFINITION_ID) + COMPILER_SMOKE_CUSTOM_SAFE_OUTPUT_DEFINITION_ID: $(EFFECTIVE_COMPILER_SMOKE_CUSTOM_SAFE_OUTPUT_DEFINITION_ID) - task: PublishPipelineArtifact@1 condition: always() diff --git a/tests/compiler-smoke-e2e/component-fixture/components/custom-build-tags/component.md b/tests/compiler-smoke-e2e/component-fixture/components/custom-build-tags/component.md new file mode 100644 index 00000000..e9fb308c --- /dev/null +++ b/tests/compiler-smoke-e2e/component-fixture/components/custom-build-tags/component.md @@ -0,0 +1,45 @@ +--- +safe-outputs: + scripts: + candidate-script-build-tag: + description: Add the candidate scripts-style proof tag to the current build. + max: 1 + run: node components/custom-build-tags/tag-build-script.js + inputs: + proof: + type: choice + options: [candidate-smoke] + required: true + env: + SYSTEM_ACCESSTOKEN: System.AccessToken + jobs: + candidate-job-build-tag: + description: Add the candidate jobs-style proof tag to the current build. + max: 1 + inputs: + proof: + type: choice + options: [candidate-smoke] + required: true + steps: + - bash: | + set -euo pipefail + : > "$ADO_AW_SAFE_OUTPUT_RESULTS" + while IFS= read -r proposal; do + proposal_id="$(printf '%s' "$proposal" | jq -er '.proposal_id')" + proof="$(printf '%s' "$proposal" | jq -er '.proof')" + test "$proof" = "candidate-smoke" + + tag="ado-aw-custom-job-$(Build.BuildId)" + printf '##vso[build.addbuildtag]%s\n' "$tag" + jq -cn \ + --arg proposal_id "$proposal_id" \ + --arg tag "$tag" \ + '{schema_version:1, proposal_id:$proposal_id, status:"success", message:("added jobs-style build tag " + $tag), data:{tag:$tag}}' \ + >> "$ADO_AW_SAFE_OUTPUT_RESULTS" + done < "$ADO_AW_SAFE_OUTPUT_PROPOSALS" + displayName: Add jobs-style candidate build tag +--- + +These tools are deterministic candidate-smoke probes. Call each only when the +consumer workflow explicitly requests `proof: candidate-smoke`. diff --git a/tests/compiler-smoke-e2e/component-fixture/components/custom-build-tags/tag-build-script.js b/tests/compiler-smoke-e2e/component-fixture/components/custom-build-tags/tag-build-script.js new file mode 100644 index 00000000..9ee5ba42 --- /dev/null +++ b/tests/compiler-smoke-e2e/component-fixture/components/custom-build-tags/tag-build-script.js @@ -0,0 +1,64 @@ +"use strict"; + +async function readStdin() { + const chunks = []; + for await (const chunk of process.stdin) { + chunks.push(Buffer.from(chunk)); + } + return Buffer.concat(chunks).toString("utf8"); +} + +function requireEnv(name) { + const value = process.env[name]?.trim(); + if (!value) { + throw new Error(`${name} is required`); + } + return value; +} + +async function main() { + const proposal = JSON.parse(await readStdin()); + if (proposal.proof !== "candidate-smoke") { + throw new Error("proposal proof must equal candidate-smoke"); + } + + const orgUrl = requireEnv("SYSTEM_COLLECTIONURI").replace(/\/+$/, ""); + const project = requireEnv("SYSTEM_TEAMPROJECT"); + const buildId = requireEnv("BUILD_BUILDID"); + const token = requireEnv("SYSTEM_ACCESSTOKEN"); + if (!/^[1-9][0-9]*$/.test(buildId)) { + throw new Error(`BUILD_BUILDID must be a positive integer, got '${buildId}'`); + } + + const tag = `ado-aw-custom-script-${buildId}`; + const url = + `${orgUrl}/${encodeURIComponent(project)}/_apis/build/builds/` + + `${buildId}/tags/${encodeURIComponent(tag)}?api-version=7.1`; + const authorization = Buffer.from(`:${token}`, "utf8").toString("base64"); + const response = await fetch(url, { + method: "PUT", + headers: { + Authorization: `Basic ${authorization}`, + "Content-Length": "0", + }, + }); + + if (!response.ok) { + const body = await response.text(); + throw new Error(`failed to add build tag (HTTP ${response.status}): ${body}`); + } + + process.stdout.write( + `${JSON.stringify({ + status: "success", + message: `added scripts-style build tag ${tag}`, + data: { tag }, + })}\n`, + ); +} + +main().catch((error) => { + const message = error instanceof Error ? error.message : String(error); + process.stderr.write(`[candidate-script-build-tag] ${message}\n`); + process.exitCode = 1; +}); diff --git a/tests/compiler-smoke-e2e/custom-safe-output.md b/tests/compiler-smoke-e2e/custom-safe-output.md new file mode 100644 index 00000000..d57dcdb6 --- /dev/null +++ b/tests/compiler-smoke-e2e/custom-safe-output.md @@ -0,0 +1,30 @@ +--- +name: "Candidate compiler smoke: custom safe outputs" +description: "Exercises imported scripts-style and jobs-style custom safe outputs" +target: standalone +pool: + name: AZS-1ES-L-Playground-ubuntu-22.04 +engine: + id: copilot + model: claude-sonnet-4.6 + timeout-minutes: 15 +imports: + - AgentPlayground/ado-aw-e2e-fixture/components/custom-build-tags/component.md@aa711dd17c4dfcde492b2bfad62e5fb1baad71f6 +safe-outputs: + noop: {} +--- + +## Candidate custom safe-output smoke + +You are a deterministic smoke test. Call exactly these two safe-output tools, +in this order: + +1. `candidate-script-build-tag` + - `proof`: `candidate-smoke` + +2. `candidate-job-build-tag` + - `proof`: `candidate-smoke` + +These are actual custom MCP tools, not labels or aliases for another tool. +Do **not** call `noop`, and do **not** substitute the built-in +`add-build-tag` tool. After both custom safe outputs are emitted, stop. diff --git a/tests/compiler_tests.rs b/tests/compiler_tests.rs index 89872480..26c8fca4 100644 --- a/tests/compiler_tests.rs +++ b/tests/compiler_tests.rs @@ -497,6 +497,10 @@ Do something. compiled.contains("my-read-sc"), "Compiled output should contain the read service connection name" ); + assert!( + compiled.contains("AZURE_DEVOPS_EXT_PAT: $(SC_READ_TOKEN)"), + "Compiled output should project the read-scoped token into the Agent sandbox" + ); // Should contain write token acquisition (SC_WRITE_TOKEN) assert!( @@ -771,6 +775,10 @@ Do something. compiled.contains("SC_READ_TOKEN"), "Compiled output should contain SC_READ_TOKEN" ); + assert!( + compiled.contains("AZURE_DEVOPS_EXT_PAT: $(SC_READ_TOKEN)"), + "Read-only permissions should project the token into the Agent sandbox" + ); assert!( !compiled.contains("SC_WRITE_TOKEN"), "Compiled output should not contain SC_WRITE_TOKEN when only read is configured" @@ -779,6 +787,83 @@ Do something. let _ = fs::remove_dir_all(&temp_dir); } +#[test] +fn test_permissions_read_token_contract_for_all_targets() { + for target in [None, Some("1es"), Some("job"), Some("stage")] { + let label = target.unwrap_or("standalone"); + let target_line = target + .map(|value| format!("target: {value}\n")) + .unwrap_or_default(); + let source = format!( + r#"--- +name: "Read Token Contract {label}" +description: "Ensures Stage 1 receives only the read-scoped ADO token" +{target_line}permissions: + read: agent-read + write: executor-write +safe-outputs: + noop: {{}} +--- + +## Test + +Call noop. +"# + ); + let (ok, compiled, stderr) = + compile_inline_source(&format!("read-token-contract-{label}"), &source); + assert!(ok, "{label}: compilation failed:\n{stderr}"); + + let document = parse_compiled_yaml(&compiled); + let agent = find_job_mapping_by_display_name(&document, "Agent") + .unwrap_or_else(|| panic!("{label}: missing Agent job")); + let run_agent = find_bash_step_containing(agent, "=== Running AI agent with AWF") + .unwrap_or_else(|| panic!("{label}: missing Agent execution step")); + let agent_env = run_agent + .get(yaml_key("env")) + .and_then(|value| value.as_mapping()) + .unwrap_or_else(|| panic!("{label}: Agent execution step must have an env mapping")); + + assert_eq!( + agent_env + .get(yaml_key("AZURE_DEVOPS_EXT_PAT")) + .and_then(|value| value.as_str()), + Some("$(SC_READ_TOKEN)"), + "{label}: permissions.read must project SC_READ_TOKEN into the Agent sandbox" + ); + for forbidden in ["SC_READ_TOKEN", "SC_WRITE_TOKEN", "SYSTEM_ACCESSTOKEN"] { + assert!( + !agent_env.contains_key(yaml_key(forbidden)), + "{label}: Agent execution env must not expose {forbidden}" + ); + } + + let detection = find_job_mapping_by_display_name(&document, "Detection") + .unwrap_or_else(|| panic!("{label}: missing Detection job")); + let run_detection = find_bash_step_containing( + detection, + "# Run threat analysis with AWF network isolation", + ) + .unwrap_or_else(|| panic!("{label}: missing Detection execution step")); + if let Some(detection_env) = run_detection + .get(yaml_key("env")) + .and_then(|value| value.as_mapping()) + { + for forbidden in [ + "AZURE_DEVOPS_EXT_PAT", + "SC_READ_TOKEN", + "SC_WRITE_TOKEN", + "SYSTEM_ACCESSTOKEN", + ] { + assert!( + !detection_env.contains_key(yaml_key(forbidden)), + "{label}: Detection execution env must not expose {forbidden}" + ); + } + } + } +} + /// Test that the 1ES fixture compiles correctly with no unreplaced markers /// and uses Copilot CLI (not Agency CLI) in custom jobs #[test] @@ -3433,7 +3518,9 @@ fn find_job_mapping_by_display_name<'a>( ) -> Option<&'a serde_yaml::Mapping> { match value { serde_yaml::Value::Mapping(map) => { - if map.get(yaml_key("displayName")).and_then(|v| v.as_str()) == Some(display_name) { + if map.contains_key(yaml_key("job")) + && map.get(yaml_key("displayName")).and_then(|v| v.as_str()) == Some(display_name) + { return Some(map); } map.values() @@ -3450,19 +3537,31 @@ fn find_bash_step_containing<'a>( job: &'a serde_yaml::Mapping, needle: &str, ) -> Option<&'a serde_yaml::Mapping> { - job.get(yaml_key("steps")) - .and_then(|v| v.as_sequence()) - .and_then(|steps| { - steps.iter().find_map(|step| { - let map = step.as_mapping()?; - let bash = map.get(yaml_key("bash")).and_then(|v| v.as_str())?; - if bash.contains(needle) { - Some(map) - } else { - None - } - }) - }) + job.values() + .find_map(|value| find_bash_step_containing_value(value, needle)) +} + +fn find_bash_step_containing_value<'a>( + value: &'a serde_yaml::Value, + needle: &str, +) -> Option<&'a serde_yaml::Mapping> { + match value { + serde_yaml::Value::Mapping(map) => { + if map + .get(yaml_key("bash")) + .and_then(|value| value.as_str()) + .is_some_and(|bash| bash.contains(needle)) + { + return Some(map); + } + map.values() + .find_map(|child| find_bash_step_containing_value(child, needle)) + } + serde_yaml::Value::Sequence(items) => items + .iter() + .find_map(|child| find_bash_step_containing_value(child, needle)), + _ => None, + } } fn assert_named_pool_demands(pool: &serde_yaml::Mapping, expected_os: Option<&str>) { @@ -8887,7 +8986,9 @@ fn test_smoke_failure_reporter_uses_registered_ado_names_and_staging_repo() { .join("tests") .join("safe-outputs") .join("smoke-failure-reporter.md"); - let reporter = fs::read_to_string(reporter_path).expect("read smoke-failure-reporter fixture"); + let reporter = fs::read_to_string(reporter_path) + .expect("read smoke-failure-reporter fixture") + .replace("\r\n", "\n"); for definition_name in [ "Daily safe-output smoke canary", @@ -8908,4 +9009,262 @@ fn test_smoke_failure_reporter_uses_registered_ado_names_and_staging_repo() { && reporter.contains("Search open issues on `jamesadevine/ado-aw-issues`"), "front matter and prompt must agree on the staging issue repository" ); + assert!( + reporter.contains("allowed-labels:\n - pipeline-failure\n - ado-aw-smoke"), + "reporter must allow only redundant copies of its two static labels" + ); + assert!( + reporter.contains("rejects every other agent-supplied label"), + "reporter prompt must explain the exact-label boundary" + ); +} + +#[test] +fn test_azure_cli_smoke_fails_closed_when_authentication_fails() { + let fixture_path = PathBuf::from(env!("CARGO_MANIFEST_DIR")) + .join("tests") + .join("safe-outputs") + .join("azure-cli.md"); + let fixture = fs::read_to_string(fixture_path) + .expect("read azure-cli smoke fixture") + .replace("\r\n", "\n"); + + for contract in [ + "If either command fails", + "`report-incomplete` from the `safeoutputs` server", + "Do not call `noop`", + "bash:\n - az\n - head", + "edit: false", + "Do not inspect MCP configuration", + "Do not invoke SafeOutputs through", + "Actually invoke the MCP tool", + ] { + assert!( + fixture.contains(contract), + "Azure CLI smoke must fail closed on an auth regression; missing contract: {contract}" + ); + } + + let (ok, compiled, stderr) = compile_inline_source("azure-cli-smoke-tool-policy", &fixture); + assert!(ok, "Azure CLI smoke should compile:\n{stderr}"); + let document = parse_compiled_yaml(&compiled); + let agent = find_job_mapping_by_display_name(&document, "Agent") + .expect("Azure CLI smoke should contain the Agent job"); + let run_agent = find_bash_step_containing(agent, "=== Running AI agent with AWF") + .expect("Azure CLI smoke should contain the Agent execution step"); + let command = run_agent + .get(yaml_key("bash")) + .and_then(|value| value.as_str()) + .expect("Agent execution step should have a bash body"); + for required in ["shell(az)", "shell(head)"] { + assert!( + command.contains(required), + "Azure CLI Agent command should allow only its required shell command {required}:\n{command}" + ); + } + for forbidden in ["--allow-all-tools", "--allow-all-paths"] { + assert!( + !command.contains(forbidden), + "Azure CLI Agent command must not contain {forbidden}:\n{command}" + ); + } +} + +// ─── Custom safe-output (imports/scripts/jobs) acceptance matrix (#1473) ───── + +/// Compile the custom scripts-style fixture with the front-matter `target:` +/// swapped to `target`, returning the compiled YAML. +fn compile_custom_for_target(target: Option<&str>) -> String { + let target_owned = target.map(str::to_string); + compile_fixture_tree_with_flags( + "custom-safe-output-scripts.md", + &[], + &["--skip-integrity"], + move |contents| match &target_owned { + Some(t) => contents.replacen( + "description: A workflow", + &format!("target: {t}\ndescription: A workflow"), + 1, + ), + None => contents, + }, + ) +} + +#[test] +fn custom_safe_output_emits_gated_executor_job_standalone() { + let compiled = compile_custom_for_target(None); + assert_valid_yaml(&compiled, "custom-safe-output-scripts.md"); + // A dedicated per-definition custom job is emitted. + assert!( + compiled.contains("Custom_send_notification"), + "expected a Custom_send_notification job:\n{compiled}" + ); + // It invokes the executor via the scripts-style --custom-config contract. + assert!( + compiled.contains("--custom-config"), + "expected the custom job to call `ado-aw execute --custom-config`:\n{compiled}" + ); + // The generated closed MCP tool schema is wired into the server launch. + assert!( + compiled.contains("--custom-tools"), + "expected the SafeOutputs MCP server to receive --custom-tools:\n{compiled}" + ); + assert!( + compiled.contains("\"additionalProperties\":false"), + "expected a closed (additionalProperties:false) generated schema:\n{compiled}" + ); + // require-approval on the custom tool routes it through ManualReview. + assert!( + compiled.contains("- job: ManualReview"), + "reviewed custom tool must emit a ManualReview gate:\n{compiled}" + ); + assert!( + compiled.contains("HasCustom_send_notification"), + "Detection must publish a per-tool proposal signal:\n{compiled}" + ); +} + +#[test] +fn custom_safe_output_compiles_for_all_targets() { + for target in [None, Some("1es"), Some("job"), Some("stage")] { + let compiled = compile_custom_for_target(target); + let label = target.unwrap_or("standalone"); + assert_valid_yaml( + &compiled, + &format!("custom-safe-output-scripts.md ({label})"), + ); + assert!( + compiled.contains("Custom_send_notification"), + "target {label}: expected a Custom_send_notification job:\n{compiled}" + ); + } +} + +#[test] +fn custom_safe_output_secret_scope_excludes_agent_and_detection() { + // The generated custom-tools schema (agent-facing) must NOT leak the + // executor contract into the Agent job beyond the closed input schema: + // the executor `--custom-config` / `--custom-phase` invocations belong only + // to the dedicated custom job, never the Agent or Detection jobs. + let compiled = compile_custom_for_target(None); + // Locate the Agent and Detection job bodies and assert they don't invoke + // the custom executor modes. + for marker in ["--custom-config", "--custom-phase"] { + // The only occurrences must be inside the Custom_ job. A crude but + // effective check: every line containing the marker must be part of a + // custom job region (which begins at `Custom_send_notification`). + let custom_start = compiled + .find("Custom_send_notification") + .expect("custom job present"); + for (idx, _) in compiled.match_indices(marker) { + assert!( + idx > custom_start, + "executor marker {marker} must only appear in the custom job region" + ); + } + } +} + +#[test] +fn candidate_custom_safe_output_fixture_compiles_from_vendored_cache() { + let repo = tempfile::tempdir().expect("create candidate fixture repo"); + let source_rel = PathBuf::from("tests") + .join("compiler-smoke-e2e") + .join("custom-safe-output.md"); + let source = repo.path().join(&source_rel); + fs::create_dir_all(source.parent().unwrap()).expect("create source directory"); + fs::copy( + PathBuf::from(env!("CARGO_MANIFEST_DIR")).join(&source_rel), + &source, + ) + .expect("copy candidate source"); + + let component_rel = PathBuf::from(".ado-aw") + .join("imports") + .join("AgentPlayground") + .join("ado-aw-e2e-fixture") + .join("aa711dd17c4dfcde492b2bfad62e5fb1baad71f6") + .join("components") + .join("custom-build-tags") + .join("component.md"); + let component = repo.path().join(component_rel); + fs::create_dir_all(component.parent().unwrap()).expect("create import cache directory"); + fs::copy( + PathBuf::from(env!("CARGO_MANIFEST_DIR")) + .join("tests") + .join("compiler-smoke-e2e") + .join("component-fixture") + .join("components") + .join("custom-build-tags") + .join("component.md"), + &component, + ) + .expect("copy cached component manifest"); + + let git = |args: &[&str]| { + let output = std::process::Command::new("git") + .args(args) + .current_dir(repo.path()) + .output() + .expect("run git"); + assert!( + output.status.success(), + "git {:?} failed: {}", + args, + String::from_utf8_lossy(&output.stderr) + ); + }; + git(&["init", "-q"]); + git(&["config", "user.name", "candidate-test"]); + git(&["config", "user.email", "candidate-test@example.com"]); + git(&["add", "."]); + git(&["commit", "-q", "-m", "candidate fixture"]); + + let output_path = repo.path().join("custom-safe-output.yml"); + let output = std::process::Command::new(env!("CARGO_BIN_EXE_ado-aw")) + .args([ + "compile", + source.to_str().unwrap(), + "--output", + output_path.to_str().unwrap(), + "--skip-integrity", + ]) + .output() + .expect("compile candidate custom fixture"); + assert!( + output.status.success(), + "candidate fixture compile failed: {}", + String::from_utf8_lossy(&output.stderr) + ); + + let compiled = fs::read_to_string(output_path).expect("read candidate output"); + assert_valid_yaml(&compiled, "custom-safe-output.md"); + for expected in [ + "Custom_candidate_script_build_tag", + "Custom_candidate_job_build_tag", + "checkout: self\n path: s", + "name: AgentPlayground/ado-aw-e2e-fixture", + "type: git", + "path: s/import_AgentPlayground_ado_aw_e2e_fixture_955c8702d066", + "aa711dd17c4dfcde492b2bfad62e5fb1baad71f6", + "node components/custom-build-tags/tag-build-script.js", + "--custom-config", + "--custom-phase pre", + "--custom-phase post", + "--max 1", + "\"timeout_minutes\": 10", + "SYSTEM_ACCESSTOKEN: $(System.AccessToken)", + "\"--enabled-tools\"", + "\"noop\"", + ] { + assert!( + compiled.contains(expected), + "candidate output missing {expected}:\n{compiled}" + ); + } + assert!( + !compiled.contains("\"--enabled-tools\",\n \"add-build-tag\""), + "candidate fixture must not expose built-in add-build-tag:\n{compiled}" + ); } diff --git a/tests/fixtures/custom-safe-output-bash-coverage.md b/tests/fixtures/custom-safe-output-bash-coverage.md new file mode 100644 index 00000000..28c18aae --- /dev/null +++ b/tests/fixtures/custom-safe-output-bash-coverage.md @@ -0,0 +1,8 @@ +--- +name: Custom Safe Output Bash Coverage +description: Exercises compiler-generated bash for scripts-style and jobs-style custom safe outputs. +imports: + - AgentPlayground/ado-aw-e2e-fixture/components/custom-build-tags/component.md@aa711dd17c4dfcde492b2bfad62e5fb1baad71f6 +--- + +Exercise both custom safe-output execution modes. diff --git a/tests/fixtures/custom-safe-output-scripts.md b/tests/fixtures/custom-safe-output-scripts.md new file mode 100644 index 00000000..01926ef1 --- /dev/null +++ b/tests/fixtures/custom-safe-output-scripts.md @@ -0,0 +1,23 @@ +--- +name: Custom Safe Output Acceptance +description: A workflow with a custom scripts-style safe-output tool for the acceptance matrix. +safe-outputs: + scripts: + send-notification: + description: Send a structured notification to the configured destination. + run: node send-notification.js + max: 3 + inputs: + title: + type: string + required: true + max-length: 120 + severity: + type: choice + options: [info, warning, critical] + required: true + send-notification: + require-approval: true +--- + +Analyze the run and call `send-notification` only when a person should act. diff --git a/tests/inspect_integration.rs b/tests/inspect_integration.rs index a211860e..9189fead 100644 --- a/tests/inspect_integration.rs +++ b/tests/inspect_integration.rs @@ -83,6 +83,42 @@ fn inspect_json_emits_schema_version_one() { ); } +#[test] +fn inspect_resolves_imports_and_shows_imported_custom_job() { + // Regression for Fix A: `build_pipeline_ir` (which powers `inspect`/`graph`) + // must resolve `imports:` so it reasons about the merged pipeline. A local + // import of a `safe-outputs.scripts` component contributes a `Custom_` + // job that only appears if imports are resolved. + let workspace = tempfile::tempdir().expect("create temp dir"); + std::fs::write( + workspace.path().join("component.md"), + "---\nsafe-outputs:\n scripts:\n notify-team:\n run: node notify.js\n---\nComponent body.\n", + ) + .expect("write component"); + let consumer = workspace.path().join("agent.md"); + std::fs::write( + &consumer, + "---\nname: importer\ndescription: imports a component\nimports:\n - component.md\n---\nConsumer body.\n", + ) + .expect("write consumer"); + + let out = Command::new(binary_path()) + .arg("inspect") + .arg(&consumer) + .output() + .expect("run ado-aw inspect"); + assert!( + out.status.success(), + "inspect exited non-zero. stderr:\n{}", + String::from_utf8_lossy(&out.stderr) + ); + let stdout = String::from_utf8_lossy(&out.stdout); + assert!( + stdout.contains("Custom_notify_team"), + "inspect must resolve imports and show the imported custom job, got:\n{stdout}" + ); +} + #[test] fn graph_dot_emits_digraph_with_known_edges() { let (_workspace, src) = fixture_copy("canary.md"); diff --git a/tests/safe-outputs/README.md b/tests/safe-outputs/README.md index c7bea030..4401fe13 100644 --- a/tests/safe-outputs/README.md +++ b/tests/safe-outputs/README.md @@ -38,11 +38,13 @@ five pipelines: These five registered definitions remain the **release-backed** contract: their checked-in YAML downloads the latest released compiler/runtime assets. A separate [`tests/compiler-smoke-e2e/`](../compiler-smoke-e2e/) orchestrator -builds the exact same five workflows with a compiler from the current PR or -nightly `main`, stages the regenerated sources/YAML on a short-lived -`ado-aw-mirror` ref, and runs five fixed candidate definitions. Keeping both -lanes distinguishes release packaging/download failures from regressions in -unreleased compiler output. +builds four selected release workflows (canary, azure-cli, noop-target, and +failure reporter) plus a candidate-only imported custom-safe-output workflow +with a compiler from the current PR or nightly `main`, stages the regenerated +sources/YAML on a short-lived `ado-aw-mirror` ref, and runs five fixed candidate +definitions. The weekly janitor remains release-only maintenance and is not +compiled or run by the candidate lane. Keeping both lanes distinguishes release +packaging/download failures from regressions in unreleased compiler output. Do **not** regenerate the checked-in `tests/safe-outputs/*.lock.yml` files with an unreleased development build. Their integrity step downloads the released diff --git a/tests/safe-outputs/azure-cli.md b/tests/safe-outputs/azure-cli.md index 82d326fb..e617a7fc 100644 --- a/tests/safe-outputs/azure-cli.md +++ b/tests/safe-outputs/azure-cli.md @@ -8,8 +8,13 @@ pool: name: AZS-1ES-L-Playground-ubuntu-22.04 engine: id: copilot - model: gpt-5-mini + model: claude-sonnet-4.6 timeout-minutes: 15 +tools: + bash: + - az + - head + edit: false permissions: read: agent-playground-read safe-outputs: @@ -40,13 +45,23 @@ Steps (run each in turn using your bash tool): -o tsv ``` - Capture the combined stdout/stderr (truncated to 400 characters if - longer) for the safe-output context below. + Do not suppress or overwrite the command's exit status. Capture the + combined stdout/stderr (truncated to 400 characters if longer) for the + safe-output context below. Treat a non-zero exit or empty project list as + a failure. -3. Call exactly one safe-output tool, `noop`, with: +3. If either command fails, invoke exactly one MCP tool: + `report-incomplete` from the `safeoutputs` server with the failing command + and its captured output, then stop. Do not call `noop`. + +4. Otherwise, invoke exactly one MCP tool: `noop` from the `safeoutputs` + server, with: - context: a brief one-line proof-of-life containing the az version string and the captured project-list output, prefixed with `ado-aw-smoke-$(Build.BuildId)-azure-cli:`. -Do not call any other tool. After the safe output is emitted, stop. +Use the native Copilot MCP tool interface. Do not inspect MCP configuration, +API keys, processes, files, or HTTP endpoints. Do not invoke SafeOutputs through +bash, `curl`, or raw HTTP. Do not print or describe a JSON tool request. +Actually invoke the MCP tool, then stop. diff --git a/tests/safe-outputs/canary.md b/tests/safe-outputs/canary.md index 31eb5d17..c772ac8e 100644 --- a/tests/safe-outputs/canary.md +++ b/tests/safe-outputs/canary.md @@ -8,7 +8,7 @@ pool: name: AZS-1ES-L-Playground-ubuntu-22.04 engine: id: copilot - model: gpt-5-mini + model: claude-sonnet-4.6 timeout-minutes: 20 permissions: read: agent-playground-read @@ -17,6 +17,7 @@ safe-outputs: noop: {} create-work-item: work-item-type: Task + assignee: devinejames@microsoft.com max: 1 include-stats: false add-build-tag: diff --git a/tests/safe-outputs/janitor.md b/tests/safe-outputs/janitor.md index bba905e9..73171bb0 100644 --- a/tests/safe-outputs/janitor.md +++ b/tests/safe-outputs/janitor.md @@ -8,7 +8,7 @@ pool: name: AZS-1ES-L-Playground-ubuntu-22.04 engine: id: copilot - model: gpt-5-mini + model: claude-sonnet-4.6 timeout-minutes: 30 permissions: read: agent-playground-read diff --git a/tests/safe-outputs/noop-target.lock.yml b/tests/safe-outputs/noop-target.lock.yml index e8682c1c..43628517 100644 --- a/tests/safe-outputs/noop-target.lock.yml +++ b/tests/safe-outputs/noop-target.lock.yml @@ -10,7 +10,7 @@ resources: jobs: - job: Agent displayName: Agent - timeoutInMinutes: 5 + timeoutInMinutes: 10 pool: name: AZS-1ES-L-Playground-ubuntu-22.04 steps: diff --git a/tests/safe-outputs/noop-target.md b/tests/safe-outputs/noop-target.md index 9034b734..0bb2318f 100644 --- a/tests/safe-outputs/noop-target.md +++ b/tests/safe-outputs/noop-target.md @@ -6,8 +6,8 @@ pool: name: AZS-1ES-L-Playground-ubuntu-22.04 engine: id: copilot - model: gpt-5-mini - timeout-minutes: 5 + model: claude-sonnet-4.6 + timeout-minutes: 10 permissions: read: agent-playground-read safe-outputs: @@ -16,9 +16,11 @@ safe-outputs: ## Noop target pipeline -You are the no-op target of the `queue-build` smoke. Call exactly one -safe-output tool: `noop`. Use these literal values (no improvisation): +You are the no-op target of the `queue-build` smoke. Invoke exactly one MCP +tool: `noop` from the `safeoutputs` server. Use these literal values (no +improvisation): - context: "ado-aw-smoke-noop-target build $(Build.BuildId) queued from queue-build smoke" +Do not print or describe a JSON tool request. Actually invoke the MCP tool. Do not call any other tool. After the safe output is emitted, stop. diff --git a/tests/safe-outputs/smoke-failure-reporter.md b/tests/safe-outputs/smoke-failure-reporter.md index b3a94f9c..b896ac26 100644 --- a/tests/safe-outputs/smoke-failure-reporter.md +++ b/tests/safe-outputs/smoke-failure-reporter.md @@ -8,7 +8,7 @@ pool: name: AZS-1ES-L-Playground-ubuntu-22.04 engine: id: copilot - model: gpt-5-mini + model: claude-sonnet-4.6 timeout-minutes: 20 permissions: read: agent-playground-read @@ -20,7 +20,9 @@ ado-aw-debug: labels: - pipeline-failure - ado-aw-smoke - allowed-labels: [] + allowed-labels: + - pipeline-failure + - ado-aw-smoke max: 5 --- @@ -69,9 +71,9 @@ front-matter `name:` values from their source Markdown. - finish time, - `result` and `status`, - the last 50 lines of the agent stage log if accessible. - - `labels`: `["pipeline-failure", "ado-aw-smoke"]` are added by - config; do **not** pass any agent-supplied labels — the fixture - sets `allowed-labels: []` (default-deny). + - `labels`: omit this field. `["pipeline-failure", "ado-aw-smoke"]` + are added by config. The executor permits only redundant copies of + those exact labels and rejects every other agent-supplied label. ### Hard limits