Skip to content

Harden extension registry CI: validate registry.dev.json and guard workflow path-list drift - #9338

Draft
JeffreyCA with Copilot wants to merge 2 commits into
mainfrom
copilot/harden-extension-registry-ci-validate-json
Draft

Harden extension registry CI: validate registry.dev.json and guard workflow path-list drift#9338
JeffreyCA with Copilot wants to merge 2 commits into
mainfrom
copilot/harden-extension-registry-ci-validate-json

Conversation

Copilot AI commented Jul 28, 2026

Copy link
Copy Markdown
Contributor

registry.dev.json carries the same approval policy as registry.json but had no validation CI, so TestDevRegistryFileIsValid never ran on a PR touching only the dev registry — ext-registry-ci triggered only on registry.json, and both cli-ci and the ADO release-cli pipeline exclude cli/azd/extensions/**. Separately, ext-registry-check.yml keeps its own inline copy of the registry path list, which can silently drift from REGISTRY_JSON_PATHS in the script.

Both failures are green checks, not red ones: a malformed dev registry merges clean and surfaces later in an unrelated cli/azd PR, and a registry path added to the script but missed in the workflow leaves the required check reporting success without the policy ever running.

Dev registry validation (ext-registry-ci.yml)

  • Added cli/azd/extensions/registry.dev.json to the paths: trigger.

  • Generalized the test selector to run both registry tests:

    go test ./pkg/extensions -run '^TestRegistryFileIsValid$|^TestDevRegistryFileIsValid$' -v
    
  • Generalized the step's ::error:: text and the comment block above it, which hardcoded registry.json.

Readable dev-registry failures (registry_files_test.go)

  • TestDevRegistryFileIsValid now uses collectValidationErrors + t.Fatalf instead of require.True(t, result.Valid, "%+v", result), matching the production test. With an unsatisfiable dependency injected:

    registry_files_test.go:53: registry.dev.json failed validation with 1 error(s):
          - [azure.ai.training] version 0.0.2-preview depends on azure.ai.training ">=999.0.0", but no published version satisfies it (available: 0.0.1-preview, 0.0.2-preview)
    

Workflow path-list drift guard (ext-registry-check)

  • Exported REGISTRY_JSON_PATHS on module.exports.forTests. The declaration moved above the export block, since referencing it from the object literal below would hit the TDZ at module load.
  • Added a vitest case that regex-extracts the inline registryPaths Set from ext-registry-check.yml and compares it against REGISTRY_JSON_PATHS. No YAML parser added to devDependencies. It asserts the match is non-null (so a renamed or restructured literal fails loudly rather than silently ceasing to guard) and that the extracted list is non-empty before comparing.
  • Added .github/workflows/ext-registry-check.yml to the scripts-ci trigger, so editing the workflow's list also runs the guard — otherwise one of the guard's two inputs could change without the test executing.

Mutation-checked against the three drift shapes: script gains a path, workflow gains a path, workflow Set literal renamed.

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
20 pipeline(s) were filtered out due to trigger conditions.
There may be pipelines that require an authorized user to comment /azp run to run.

…w path drift

Co-authored-by: JeffreyCA <9157833+JeffreyCA@users.noreply.github.com>
Copilot AI changed the title [WIP] Add validation CI for registry.dev.json in extension registry Harden extension registry CI: validate registry.dev.json and guard workflow path-list drift Jul 28, 2026
Copilot AI requested a review from JeffreyCA July 28, 2026 17:03
@JeffreyCA
JeffreyCA requested a review from Copilot July 28, 2026 21:56

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Hardens extension registry CI by validating both registries and detecting workflow/script path-list drift.

Changes:

  • Runs validation when registry.dev.json changes.
  • Improves development-registry validation errors.
  • Adds a test ensuring registry path lists remain synchronized.
Show a summary per file
File Description
cli/azd/pkg/extensions/registry_files_test.go Improves dev-registry failure output.
.github/workflows/scripts-ci.yml Triggers script tests for registry workflow changes.
.github/workflows/ext-registry-ci.yml Validates both registry files.
.github/scripts/test/ext-registry-check.test.js Tests workflow/script path-list consistency.
.github/scripts/src/ext-registry-check.js Exports registry paths for testing.

Review details

  • Files reviewed: 5/5 changed files
  • Comments generated: 0
  • Review effort level: Medium

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Harden extension registry CI: validate registry.dev.json and guard workflow path-list drift

3 participants