-
Notifications
You must be signed in to change notification settings - Fork 7
Add PR review and PR comment response templates #242
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Alan-Jowett
merged 6 commits into
microsoft:main
from
Alan-Jowett:add-pr-review-templates
Apr 14, 2026
Merged
Changes from all commits
Commits
Show all changes
6 commits
Select commit
Hold shift + click to select a range
6683142
Add PR review and PR comment response templates
4c0bc6f
Address PR review feedback: API correctness and consistency fixes
3df78c3
Address round 2 PR review feedback
1897ab3
Address round 3 PR review feedback
d0c263a
Address round 4 PR review feedback
440c4e3
Align contradiction report empty-state with global convention
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,108 @@ | ||
| <!-- SPDX-License-Identifier: MIT --> | ||
| <!-- Copyright (c) PromptKit Contributors --> | ||
|
|
||
| --- | ||
| name: pr-comment-responses | ||
| type: format | ||
| description: > | ||
| Output format for responding to pull request review comments. | ||
| Structures per-thread analysis, contradiction detection, and | ||
| response generation in either document or action mode. | ||
| produces: pr-comment-responses | ||
| --- | ||
|
|
||
| # Format: PR Comment Responses | ||
|
|
||
| The output MUST be a structured response plan for pull request review | ||
| threads. The format adapts based on `output_mode`: | ||
|
|
||
| - **Document mode**: produce the full report below. | ||
| - **Action mode**: use the same section structure below as an analysis | ||
| and planning artifact, rather than a prose report. | ||
|
|
||
| ## Output Structure | ||
|
|
||
| ### 1. Thread Summary | ||
|
|
||
| Summarize all review threads by state: | ||
|
|
||
| | State | Count | Description | | ||
| |-------|-------|-------------| | ||
| | **Pending** | N | Active threads requiring response | | ||
| | **Outdated** | N | Threads on code that has since changed | | ||
| | **Resolved** | N | Already resolved — skipped unless user requests | | ||
|
|
||
| - **Total threads**: count | ||
| - **Actionable threads**: count (pending only, unless user overrides) | ||
| - **Skipped threads**: count and reason (resolved, outdated) | ||
|
|
||
| ### 2. Contradiction Report | ||
|
|
||
| Identify conflicting feedback across different reviewers on the same | ||
| code area or design decision. For each contradiction: | ||
|
|
||
| ```markdown | ||
| #### Contradiction C-<NNN>: <Short Description> | ||
|
|
||
| - **Reviewer A** (@handle): <position summary> — <thread reference> | ||
| - **Reviewer B** (@handle): <position summary> — <thread reference> | ||
| - **Conflict**: <what specifically conflicts> | ||
| - **Resolution Options / Tradeoffs**: <neutral summary of the | ||
| available options, tradeoffs, and implications> | ||
| - **Decision Needed**: <what the user/team needs to decide or clarify | ||
| before proceeding> | ||
| ``` | ||
|
|
||
| If no contradictions are detected, state: "None identified." | ||
|
|
||
| ### 3. Per-Thread Responses | ||
|
|
||
| For each actionable thread, in file order: | ||
|
|
||
| ```markdown | ||
| #### Thread T-<NNN>: <File>:<Line> — <Short Description> | ||
|
|
||
| - **Reviewer**: @handle | ||
| - **Thread State**: Pending / Outdated | ||
| - **Comment Summary**: <1–2 sentence summary of the reviewer's point> | ||
| - **Response Type**: Fix / Explain / Both | ||
| - **Analysis**: <why this feedback is valid/invalid, what it implies> | ||
| - **Response**: | ||
| - *(If Fix)*: <specific code change with before/after> | ||
| - *(If Explain)*: <draft reply explaining the design decision> | ||
| - *(If Both)*: <code change + explanation> | ||
| - **Linked Contradiction**: C-<NNN> (if part of a detected contradiction) | ||
| ``` | ||
|
|
||
| ### 4. Action Summary | ||
|
|
||
| | Category | Count | Details | | ||
| |----------|-------|---------| | ||
| | **Code fixes applied** | N | Threads where code was changed | | ||
| | **Explanations provided** | N | Threads answered with rationale | | ||
| | **Skipped (resolved)** | N | Already resolved threads | | ||
| | **Skipped (outdated)** | N | Threads on changed code | | ||
| | **Needs discussion** | N | Contradictions or ambiguous feedback | | ||
|
|
||
| - **Files modified**: list of files changed by fixes | ||
| - **Commits**: list of commits created (action mode only) | ||
| - **Unresolved items**: threads that need human judgment | ||
|
|
||
| ## Formatting Rules | ||
|
|
||
| - Threads MUST be ordered by file path, then by line number within | ||
| each file. | ||
| - Every actionable thread MUST have a response — do not skip threads | ||
| without stating why. | ||
| - Code fixes MUST show before/after snippets with enough context | ||
| (at least 3 lines) to verify correctness. | ||
| - If a section has no content, state "None identified" — never omit | ||
| sections. | ||
| - In action mode, present the full analysis to the user and obtain | ||
| explicit confirmation before executing any mutation (code change, | ||
| comment post, thread resolution). | ||
|
|
||
| ## Response Type | ||
|
|
||
| - `response_type` MUST be one of: **Fix**, **Explain**, or **Both**. | ||
| - If a per-thread override is shown, it MUST use one of the same values. | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,239 @@ | ||
| <!-- SPDX-License-Identifier: MIT --> | ||
| <!-- Copyright (c) PromptKit Contributors --> | ||
|
|
||
| --- | ||
| name: respond-to-pr-comments | ||
| description: > | ||
| Process pull request review feedback and generate per-thread responses. | ||
| Supports document mode (structured response plan) or action mode | ||
| (make code fixes, post replies, and resolve threads via GitHub API). | ||
| Detects contradictory feedback across reviewers. | ||
| persona: systems-engineer | ||
| protocols: | ||
| - guardrails/anti-hallucination | ||
| - guardrails/self-verification | ||
|
Alan-Jowett marked this conversation as resolved.
|
||
| - guardrails/operational-constraints | ||
| format: pr-comment-responses | ||
| params: | ||
| pr_reference: "Pull request to respond to — URL or number (e.g., #42)" | ||
| review_threads: "Review feedback to address — 'all pending', specific thread URLs, or pasted comments" | ||
| codebase_context: "What this code does, relevant architecture, design decisions that inform responses" | ||
| response_mode: "How to respond per-thread — 'auto' (heuristic), 'fix' (code changes), or 'explain' (rationale)" | ||
| output_mode: "Output mode — 'document' (produce response plan) or 'action' (make changes and post replies via gh CLI)" | ||
| input_contract: null | ||
| output_contract: | ||
| type: pr-comment-responses | ||
| description: > | ||
| A structured per-thread response plan with code fixes and/or | ||
| explanations. In action mode, responses are executed as code | ||
| changes, reply comments, and thread resolutions. | ||
| --- | ||
|
|
||
| # Task: Respond to PR Review Comments | ||
|
|
||
| You are tasked with processing review feedback on a pull request and | ||
| generating responses for each review thread — either code fixes, | ||
| explanatory replies, or both. | ||
|
|
||
| ## Inputs | ||
|
|
||
| **Pull Request**: {{pr_reference}} | ||
|
|
||
| **Review Threads to Address**: {{review_threads}} | ||
|
|
||
| **Codebase Context**: {{codebase_context}} | ||
|
|
||
| **Response Mode**: {{response_mode}} | ||
|
|
||
| **Output Mode**: {{output_mode}} | ||
|
|
||
| ## Instructions | ||
|
|
||
| ### Phase 1: Gather Review Threads | ||
|
|
||
| 1. **Read all review threads** on the PR: | ||
| - Use `gh pr view {{pr_reference}} --comments` for a quick overview, but use | ||
| `gh api graphql` to fetch the authoritative review-thread data | ||
| needed for deterministic action mode execution | ||
| - For each review thread, record: | ||
| - `thread_id`: the GraphQL review thread ID (required for | ||
| `resolveReviewThread`) | ||
| - Reviewer handle | ||
| - File path and line number | ||
| - Thread state (pending, resolved, outdated) | ||
| - Full comment text and any replies | ||
| - Whether the thread is on code that still exists in the | ||
| current diff | ||
| - For each review comment within the thread, record: | ||
| - `comment_id`: the review comment database ID (required for | ||
| REST `in_reply_to` when posting a reply) | ||
| - Author handle | ||
| - Comment body | ||
| - Use a GraphQL query via `gh api graphql` that includes each | ||
| thread's ID, state, path, and line metadata, plus each | ||
| comment's database ID, author, and body | ||
| - Preserve these IDs in your working notes so later action | ||
| steps can post replies and resolve the correct threads | ||
|
|
||
| 2. **Filter threads** based on `review_threads` parameter: | ||
| - If `all pending` — include all threads with state `pending` | ||
| - If specific threads are listed — include only those | ||
| - Skip `resolved` threads unless the user explicitly requests them | ||
| - Flag `outdated` threads (code has changed since the comment) | ||
| and ask the user whether to address them | ||
|
|
||
| 3. **Read the current code** at each thread's location: | ||
| - Fetch the file content at the relevant lines | ||
| - Understand the surrounding context (function, class, module) | ||
| - Check if the code has changed since the review comment was posted | ||
|
|
||
| ### Phase 2: Detect Contradictions | ||
|
|
||
| Compare feedback across different reviewers on the same code area | ||
| or design decision: | ||
|
|
||
| 1. **Group threads by location**: threads on the same file within | ||
| 10 lines of each other, or threads referencing the same function | ||
| or design concept. | ||
|
|
||
| 2. **Compare positions**: for each group, check if reviewers disagree: | ||
| - Reviewer A says "add error handling" but Reviewer B says | ||
| "keep it simple, don't over-engineer" | ||
| - Reviewer A says "use approach X" but Reviewer B says | ||
| "use approach Y" | ||
| - Reviewer A approves a pattern but Reviewer B flags it | ||
|
|
||
| 3. **Report contradictions** with both positions and a recommended | ||
| resolution. Do NOT silently pick one side — flag for the user. | ||
|
|
||
| ### Phase 3: Analyze Each Thread | ||
|
|
||
| For each actionable thread, determine the response type: | ||
|
|
||
| 1. **If `response_mode` is `auto`**, apply these heuristics: | ||
|
|
||
| | Reviewer Feedback | Response Type | | ||
| |---|---| | ||
| | Points out a bug, missing check, or incorrect behavior | **Fix** | | ||
| | Asks "why" or questions a design choice | **Explain** | | ||
| | Suggests a refactor or alternative approach | **Both** | | ||
| | Requests documentation or comment changes | **Fix** | | ||
| | Flags a style or convention issue | **Fix** | | ||
| | Raises a concern without a specific ask | **Explain** | | ||
|
|
||
| 2. **If `response_mode` is `fix`** — generate a code fix for every | ||
| thread. If a fix is not applicable (e.g., the comment is a design | ||
| question), note this and fall back to an explanation. | ||
|
|
||
| 3. **If `response_mode` is `explain`** — generate an explanatory reply | ||
| for every thread. If the feedback clearly requires a code change | ||
| (e.g., a bug), note this and recommend the user switch to `auto`. | ||
|
|
||
| For each thread, produce: | ||
|
|
||
| - **Analysis**: Why the feedback is valid, partially valid, or based | ||
| on a misunderstanding. Be honest — if the reviewer is right, | ||
| acknowledge it. If they are wrong, explain why respectfully. | ||
| - **Fix** (if applicable): The specific code change, shown as | ||
| before/after with at least 3 lines of surrounding context. | ||
| - **Explanation** (if applicable): A draft reply to the reviewer | ||
| that explains the design decision, tradeoff, or rationale. Keep | ||
| it professional, concise, and technical. | ||
|
|
||
| ### Phase 4: Output | ||
|
|
||
| #### Document Mode (`output_mode: document`) | ||
|
|
||
| Produce the output following the `pr-comment-responses` format: | ||
| 1. Thread Summary (by state) | ||
| 2. Contradiction Report | ||
| 3. Per-Thread Responses (in file order) | ||
| 4. Action Summary | ||
|
|
||
| #### Action Mode (`output_mode: action`) | ||
|
|
||
| Execute responses with **mandatory user confirmation at every step**: | ||
|
|
||
| 1. **Present the full analysis** (thread summary, contradictions, | ||
| per-thread responses) to the user using the document structure. | ||
|
|
||
| 2. **For each thread with a code fix**: | ||
| a. Show the proposed diff to the user. | ||
| b. Ask: "Apply this fix? (yes / skip / edit)" | ||
| c. If confirmed, make the code change in the file. | ||
| d. Do NOT commit yet — batch all fixes first. | ||
|
|
||
| 3. **After all fixes are applied**: | ||
| a. Show the user a summary of all changes made. | ||
| b. Ask: "Commit and push these changes? (yes / no)" | ||
| c. If confirmed, commit with a descriptive message referencing | ||
| the review threads addressed, then push. | ||
|
|
||
| 4. **For each thread with an explanation**: | ||
| a. Show the draft reply to the user. | ||
| b. Ask: "Post this reply? (yes / skip / edit)" | ||
| c. If confirmed, write the reply payload to `reply.json` and post: | ||
|
Alan-Jowett marked this conversation as resolved.
|
||
| ```json | ||
| { | ||
| "body": "<reply text>", | ||
| "in_reply_to": <comment_id> | ||
| } | ||
| ``` | ||
| ``` | ||
| gh api repos/{owner}/{repo}/pulls/{pr_number}/comments \ | ||
| --method POST \ | ||
| --input reply.json | ||
| ``` | ||
|
|
||
|
Alan-Jowett marked this conversation as resolved.
|
||
| 5. **For threads that were fixed**: | ||
| a. Ask: "Resolve these threads? (yes / no)" | ||
| b. If confirmed, resolve each thread using: | ||
| ``` | ||
| gh api graphql \ | ||
| -f query='mutation($threadId: ID!) { | ||
| resolveReviewThread(input: {threadId: $threadId}) { | ||
| thread { isResolved } | ||
| } | ||
| }' \ | ||
| -F threadId="<thread_id>" | ||
| ``` | ||
|
|
||
| 6. **Never take any action without explicit user confirmation.** | ||
| If the user skips all items, produce a document-mode report instead. | ||
|
|
||
| ### Phase 5: Handle Edge Cases | ||
|
|
||
| - **No pending threads**: Report "No actionable review threads found" | ||
| and list any resolved/outdated threads for reference. | ||
| - **Large thread count (>20)**: Process in batches of 10. After each | ||
| batch, summarize progress and ask to continue. | ||
| - **Outdated threads**: Flag these separately. Ask the user whether | ||
| to address them — the code may have already changed to address | ||
| the feedback. | ||
| - **Threads on deleted files**: Skip with a note explaining the file | ||
| no longer exists. | ||
|
|
||
| ## Non-Goals | ||
|
|
||
| - Do NOT perform a new code review — focus only on addressing | ||
| existing feedback. | ||
| - Do NOT modify code beyond what is needed to address review comments. | ||
| - Do NOT resolve threads without user confirmation. | ||
| - Do NOT dismiss or ignore valid feedback — if a reviewer is correct, | ||
| acknowledge it and fix it. | ||
| - Do NOT take sides in contradictions — present both positions and | ||
| let the user decide. | ||
| - Do NOT push commits without explicit user confirmation. | ||
|
|
||
| ## Quality Checklist | ||
|
|
||
| Before finalizing, verify: | ||
|
|
||
| - [ ] Every pending thread has a response (fix, explanation, or both) | ||
| - [ ] Contradictions across reviewers are explicitly flagged | ||
| - [ ] Code fixes show before/after with sufficient context | ||
| - [ ] Draft replies are professional, concise, and technical | ||
| - [ ] Resolved and outdated threads are accounted for (skipped with reason) | ||
| - [ ] In action mode: user confirmation obtained before every mutation | ||
| - [ ] Thread states (pending/resolved/outdated) are accurately reported | ||
| - [ ] Files modified by fixes are listed in the action summary | ||
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.