Make Copilot ⏳ mean a review is actually in flight - #217
Merged
trask merged 3 commits intoAug 4, 2026
Conversation
The reviewers column showed `Copilot ⏳` on 23 of ~35 rows of the opentelemetry-java-instrumentation dashboard, so the icon carried almost no signal. Two causes: `copilot_review_status` treated a review as needing attention when its `finding_count` was non-zero. That count is the review's total inline comment count, which never shrinks, so it kept counting feedback the author had already resolved and held the pull request on finished work. Findings now come from unresolved, non-outdated review threads Copilot started. The renderer synthesized a pending Copilot row from `copilot_review_outstanding`, which is true whenever the gate applies and the review is missing or stale. A stale review is the ordinary state between a push and the next re-review, so the icon was close to permanent, and it conflated three situations with different owners: Copilot has never reviewed, a push made its review stale, and Copilot left findings. Only the last is anyone's work, and it already renders as an open review thread. The icon now means a review is genuinely in flight: a requested re-review, or the automatic first review on a pull request the gate is holding because Copilot has never reviewed it. Re-review requests are now keyed on staleness rather than on the gate. Findings on the current head sit on unchanged code, so re-reviewing it would reach the same verdict and be requested again on the next pass. Against live data for the 24 gated pull requests, the icon drops from 23 rows to the one pull request Copilot has never reviewed. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: f468856d-6c6f-4b97-a67c-3f87bc297a1c
Document the same open-thread predicate the gate uses, and state that copilot_review_stale presupposes an existing review. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: f468856d-6c6f-4b97-a67c-3f87bc297a1c
Pull request dashboard statusMerged · refreshed 2026-08-04 13:37 UTC Status above doesn't look right?
|
Contributor
There was a problem hiding this comment.
Pull request overview
Refines Copilot review gating so pending indicators represent active reviews and resolved/outdated findings no longer block routing.
Changes:
- Derives Copilot findings from unresolved, non-outdated threads.
- Requests re-reviews only after code changes.
- Updates pending-icon rendering, tests, and documentation.
Reviewed changes
Copilot reviewed 8 out of 8 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
pull-request-dashboard/README.md |
Documents revised gate and pending-icon behavior. |
.github/scripts/pull-request-dashboard/test_render.py |
Tests pending Copilot rendering states. |
.github/scripts/pull-request-dashboard/test_dashboard.py |
Tests thread-based findings and stale-review gating. |
.github/scripts/pull-request-dashboard/test_copilot_review.py |
Updates request-delivery expectations. |
.github/scripts/pull-request-dashboard/render.py |
Restricts pending icons to in-flight reviews. |
.github/scripts/pull-request-dashboard/RATIONALE.md |
Explains the revised gating design. |
.github/scripts/pull-request-dashboard/dashboard.py |
Adds stale-review and live-finding facts. |
.github/scripts/pull-request-dashboard/copilot_review.py |
Implements thread-based findings and push-only re-reviews. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
A pending review request is read for every PR, so returning early on it put the icon on ungated branches, where nothing holds the PR and a requested human reviewer is left off the row entirely. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: f468856d-6c6f-4b97-a67c-3f87bc297a1c
mx-psi
approved these changes
Aug 4, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Copilot ⏳shows on 23 of ~35 rows of the opentelemetry-java-instrumentation dashboard, so it tells you nothing. Most of those PRs have nothing for anyone to do — Copilot already reviewed them cleanly, and the only thing that changed since was a new commit.It now means what the legend says: a Copilot review is actually in flight. It appears when a re-review has been requested, or while a PR is held waiting for Copilot's automatic first review, and it clears when Copilot reports.
Two behaviour changes come with that:
Copilot's live findings still appear in the reviewers column as 💬, unchanged.
On the 24 gated PRs today, the icon goes from 23 rows to 1 — the one PR Copilot has never reviewed.
Scope: this only affects repos with
require_clean_copilot_review_branchesconfigured, which today is opentelemetry-java-instrumentation onmain. The other dashboards never showedCopilot ⏳and are unchanged.