Make Copilot ⏳ mean a review is actually in flight - #216
Closed
trask wants to merge 1 commit into
Closed
Conversation
The reviewers column showed `Copilot ⏳` on 23 of ~35 rows of the opentelemetry-java-instrumentation dashboard, so the icon carried almost no signal. Two causes: `copilot_review_status` treated a review as needing attention when its `finding_count` was non-zero. That count is the review's total inline comment count, which never shrinks, so it kept counting feedback the author had already resolved and held the pull request on finished work. Findings now come from unresolved, non-outdated review threads Copilot started. The renderer synthesized a pending Copilot row from `copilot_review_outstanding`, which is true whenever the gate applies and the review is missing or stale. A stale review is the ordinary state between a push and the next re-review, so the icon was close to permanent, and it conflated three situations with different owners: Copilot has never reviewed, a push made its review stale, and Copilot left findings. Only the last is anyone's work, and it already renders as an open review thread. The icon now means a review is genuinely in flight: a requested re-review, or the automatic first review on a pull request the gate is holding because Copilot has never reviewed it. Re-review requests are now keyed on staleness rather than on the gate. Findings on the current head sit on unchanged code, so re-reviewing it would reach the same verdict and be requested again on the next pass. Against live data for the 24 gated pull requests, the icon drops from 23 rows to the one pull request Copilot has never reviewed. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: f468856d-6c6f-4b97-a67c-3f87bc297a1c
Pull request dashboard statusClosed · refreshed 2026-08-01 17:27 UTC Status above doesn't look right?
|
Contributor
There was a problem hiding this comment.
Pull request overview
Refines Copilot review gating so ⏳ represents an active review rather than stale or resolved feedback.
Changes:
- Derives findings from unresolved, non-outdated Copilot threads.
- Re-requests reviews only for stale heads.
- Updates rendering, tests, and documentation.
Reviewed changes
Copilot reviewed 8 out of 8 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
pull-request-dashboard/README.md |
Updates gate and icon documentation. |
test_render.py |
Tests pending-icon rendering. |
test_dashboard.py |
Tests thread-based gate facts. |
test_copilot_review.py |
Updates request-delivery expectations. |
render.py |
Narrows pending Copilot display logic. |
RATIONALE.md |
Explains revised gate semantics. |
dashboard.py |
Adds stale and thread-finding facts. |
copilot_review.py |
Implements thread-based findings and stale-only requests. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| | `labels_to_display` | no | Case-sensitive shell-style label name patterns to display inline after PR titles. Exact names such as `breaking change` and wildcard patterns such as `size/*` are supported. Defaults to `[]`, which displays no labels. | | ||
| | `non_blocking_check_patterns` | no | Check-name globs for non-required checks whose failures should be identified in the live PR status comment. When the PR is waiting on the author, matching failures are reported only when at least one required check is failing and are noted alongside those failures. On other routes, matching failures are shown separately. Matching checks remain informational and do not affect routing or the dashboard CI column. | | ||
| | `require_clean_copilot_review_branches` | no | List of base branch names for which a Copilot review with no inline findings on the current head is required before routing a PR to reviewers or maintainers. The dashboard re-requests Copilot review when needed and does not duplicate a pending request. List only branches where automatic Copilot code review is enabled (typically `["main"]`); PRs targeting any other branch are never gated, so they cannot stall waiting for a review that never runs. Defaults to `[]` (no branches gated). | | ||
| | `require_clean_copilot_review_branches` | no | List of base branch names for which a Copilot review of the current head with no unresolved Copilot review threads is required before routing a PR to reviewers or maintainers. The dashboard re-requests Copilot review when a push has left the previous review stale, and does not duplicate a pending request. List only branches where automatic Copilot code review is enabled (typically `["main"]`); PRs targeting any other branch are never gated, so they cannot stall waiting for a review that never runs. Defaults to `[]` (no branches gated). | |
Comment on lines
+128
to
+132
| copilot_review_stale bool No Copilot review covers the | ||
| current head, so a re-review | ||
| would see unreviewed code. | ||
| Only a stale review is worth | ||
| re-requesting. |
Member
Author
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Copilot ⏳shows on 23 of ~35 rows of the opentelemetry-java-instrumentation dashboard, so it tells you nothing. Most of those PRs have nothing for anyone to do — Copilot already reviewed them cleanly, and the only thing that changed since was a new commit.It now means what the legend says: a Copilot review is actually in flight. It appears when a re-review has been requested, or while a PR is held waiting for Copilot's automatic first review, and it clears when Copilot reports.
Two behaviour changes come with that:
Copilot's live findings still appear in the reviewers column as 💬, unchanged.
On the 24 gated PRs today, the icon goes from 23 rows to 1 — the one PR Copilot has never reviewed.
Scope: this only affects repos with
require_clean_copilot_review_branchesconfigured, which today is opentelemetry-java-instrumentation onmain. The other dashboards never showedCopilot ⏳and are unchanged.