Add staged rollout to the pull request dashboard

[trask]

Closes #108.

Dashboard changes currently reach all 17 configured repositories the moment they merge, because every run resolves the reusable workflow and its scripts at the caller's commit.

This splits each entry path into a canary job and a stable job. Canary repositories (CANARY_REPOSITORIES, currently opentelemetry-java-instrumentation and shared-workflows) run the workflow and scripts from the triggering commit; every other repository runs them from the promoted rollout ref. pull-request-dashboard-repo.yml gains an optional code_ref input so a stable job pins the scripts to the same commit it calls the workflow at, and test_rollout.py rejects a half-applied promotion. Repository configuration stays live, so onboarding and settings changes still take effect immediately in both channels.

Both channels still call the local workflow, so this is behaviorally a no-op today. The first promotion is a follow-up PR that cuts a release and points the stable jobs at ...@<sha> # vX.Y.Z with a matching code_ref.

[opentelemetry-pr-dashboard]

Pull request dashboard status

Waiting on reviewers · refreshed 2026-08-01 17:59 UTC

Review the latest changes.

Status above doesn't look right?

• Just replied or pushed? Anything around or after the refresh time above may not be picked up yet — give it a few minutes.
• Anything look wrong? Report it with what you expected; it helps us improve the dashboard.

[Copilot]

Pull request overview

Adds staged canary/stable rollout channels for the pull request dashboard.

Changes:

• Splits all dashboard entry paths into canary and stable jobs.
• Pins workflow scripts while keeping repository configuration live.
• Adds rollout validation tests and operational documentation.

Reviewed changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 4 comments.

Show a summary per file

File	Description
pull-request-dashboard/README.md	Documents rollout and promotion procedures.
.github/workflows/pull-request-dashboard.yml	Implements canary/stable routing.
.github/workflows/pull-request-dashboard-repo.yml	Adds pinned code checkout with live configuration.
.github/scripts/pull-request-dashboard/test_rollout.py	Validates rollout wiring consistency.
.github/scripts/pull-request-dashboard/RATIONALE.md	Records rollout design rationale.

Suppressed comments (1)

.github/scripts/pull-request-dashboard/RATIONALE.md:54

• This is the second place in the rationale that describes code_ref as a tag, although promotion passes the commit SHA used by uses. Update it as well so the rationale does not continue to prescribe a ref that violates the repository's hash-pinning policy.

- The stable jobs pass the tag they are called at as `code_ref`, and the
  reusable workflow checks that ref out. Without it the workflow YAML would come
  from the tag while the scripts came from `main`, so any change to their
  interface would break the pinned repositories.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

Pull request overview

Copilot reviewed 5 out of 5 changed files in this pull request and generated no new comments.

Suppressed comments (2)

.github/scripts/pull-request-dashboard/test_rollout.py:69

• This global search does not verify that both targeted jobs still contain the duplicated canary condition. If one job drops its fromJSON(...) check, the literal in the other job keeps this test green; it will also start treating unrelated future fromJSON arrays as canary lists. Assert the expected literal in each targeted job block explicitly so this remains an effective drift guard.

    def test_inline_canary_lists_match_the_workflow_env(self) -> None:
        inline = re.findall(r"fromJSON\('(\[[^']*\])'\)", self.text)
        self.assertTrue(inline, "expected the targeted jobs to inline the canary list")
        for literal in inline:
            self.assertEqual(json.loads(literal), self.canary)

pull-request-dashboard/README.md:250

• The implementation checks this file out at github.sha, which is the commit that triggered the caller, not necessarily main (for example, a manual dispatch can select another ref). This also conflicts with the rationale's accurate “commit that triggered the run” wording. Document the actual source ref so operators do not assume configuration is always loaded from the latest default branch.

Repository configuration is never staged. `repositories.json` is always read
from `main`, so opting a repository in, or changing its settings, takes effect
on the next run in both channels.

[Copilot]

Pull request overview

Copilot reviewed 5 out of 5 changed files in this pull request and generated no new comments.

[Copilot]

Pull request overview

Copilot reviewed 5 out of 5 changed files in this pull request and generated no new comments.

Suppressed comments (3)

.github/scripts/pull-request-dashboard/RATIONALE.md:55

• Without code_ref, actions/checkout defaults to github.sha, so the scripts come from the triggering commit rather than unconditionally from main. Keep this explanation aligned with the actual fallback behavior.

  come from that commit while the scripts came from `main`, so any change to

.github/scripts/pull-request-dashboard/RATIONALE.md:41

• The canary checkout follows the triggering commit, not necessarily main (manual dispatches can target another ref). Describing it as main conflicts with the workflow and the README; use the triggering-commit wording here.

This issue also appears on line 55 of the same file.

  the workflow and its scripts from `main`; every other repository runs them

pull-request-dashboard/README.md:251

• The live-config checkout uses ${{ github.sha }}, which is the triggering commit and can be a non-main ref for a manual dispatch. Saying the file is always read from main overstates the behavior; document the triggering-commit snapshot instead.

Repository configuration is never staged. `repositories.json` is always read
from `main`, so opting a repository in, or changing its settings, takes effect
on the next run in both channels.

[Copilot]

Pull request overview

Copilot reviewed 5 out of 5 changed files in this pull request and generated no new comments.

Suppressed comments (1)

.github/scripts/pull-request-dashboard/test_rollout.py:114

• This substring check can pass when code_ref is commented out (for example, # code_ref: <sha>) or has extra suffix text, so the guard can miss the half-applied promotion it is intended to reject. Anchor the match to an active YAML line and require only the expected SHA plus an optional tag comment.

            self.assertIn(f"code_ref: {ref}", body, f"{job} runs scripts from a different ref")

[Copilot]

Pull request overview

Copilot reviewed 5 out of 5 changed files in this pull request and generated 1 comment.

[Copilot]

Pull request overview

Copilot reviewed 5 out of 5 changed files in this pull request and generated no new comments.

Suppressed comments (3)

.github/scripts/pull-request-dashboard/test_rollout.py:108

• The local-fallback check is also only a substring search, so a commented-out local call can hide an active unexpected workflow call. Require an active job-level uses key here as well.

                self.assertIn(LOCAL_USES, body, f"{job} calls an unexpected workflow")

.github/scripts/pull-request-dashboard/test_rollout.py:93

• This substring assertion can be satisfied by a commented-out local uses line, allowing a canary job to call a pinned/remote workflow without failing the rollout guard. Match an active four-space job-level key instead.

            self.assertIn(LOCAL_USES, body)

.github/scripts/pull-request-dashboard/test_rollout.py:27

• STABLE_USES.search() is not anchored to an active job-level key, so it also matches a commented-out # uses: ...@sha. A half-reverted stable job could therefore call the local workflow while retaining code_ref, yet this guard would pass. Anchor the pattern to the four-space job-level uses line so comments cannot satisfy it.

This issue also appears in the following locations of the same file:

• line 93
• line 108

STABLE_USES = re.compile(
    r"uses:\s*open-telemetry/shared-workflows/" + re.escape(REPO_WORKFLOW_PATH) + r"@(\S+)"
)

[Copilot]

Pull request overview

Copilot reviewed 5 out of 5 changed files in this pull request and generated no new comments.