Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
137 changes: 137 additions & 0 deletions .agents/skills/get-reviews/SKILL.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,137 @@
---
name: get-reviews
description: >-
This skill is utilized when requesting reviews from GitHub pull requests.
Get the reviews, organize, and resolve them by applying or dismissing.
argument-hint: "Provide the number of the pull request to get the reviews."
---

<!-- deno-fmt-ignore-file -->

Get reviews from GitHub pull requests
=====================================

This skill is utilized when requesting reviews from GitHub pull requests.

1. Get the reviews.
2. Organize the reviews.
3. Resolve the reviews by applying or dismissing them.


Get the reviews
---------------

To get the reviews from a GitHub pull request, you can use the GitHub API.
Check the [`gh` CLI tool][gh] is installed and authenticated.
`gh auth status` can be used to check the authentication status.
If `gh` isn't installed, try installing it by `apt install gh`.
If authentication is not set up, tell the contributor to run `gh auth login`
to authenticate with GitHub.

Use the GraphQL API to fetch the reviews for a specific pull request.
Check [fetch\_reviews.sh](./fetch_reviews.sh) to fetch the reviews
and save them in a JSON file:

- Replace `$VARIABLES` with the actual values or variables in the command.
- For incremental fetches, save `pageInfo.endCursor` from the previous
fetch and pass it as `after: $LAST_CURSOR` (a base64 cursor, not a
review thread node ID) so the query returns only new review threads.
- Use `jq` to filter the reviews and information if necessary.
Comment on lines +31 to +39
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick | 🔵 Trivial | ⚡ Quick win

Document the required environment variables for fetch_reviews.sh.

The documentation references fetch_reviews.sh and mentions replacing $VARIABLES, but doesn't explicitly list which environment variables the script requires. Based on the script (lines 4-8 of fetch_reviews.sh), users need to set:

  • PR_NUMBER (required)
  • NUMBER_OF_PR_COMMENTS (required)
  • NUMBER_OF_REVIEWS (required)
  • NUMBER_OF_THREADS (required)
  • NUMBER_OF_COMMENTS_PER_THREAD (required)
  • LAST_CURSOR (optional, for incremental fetches)

Consider adding a bullet list or example showing how to invoke the script with these variables.

📝 Suggested addition

After line 33, add:

Set the following environment variables before running the script:

 -  `PR_NUMBER`: The pull request number to fetch reviews for.
 -  `NUMBER_OF_PR_COMMENTS`: Maximum number of top-level PR comments to fetch.
 -  `NUMBER_OF_REVIEWS`: Maximum number of reviews to fetch.
 -  `NUMBER_OF_THREADS`: Maximum number of review threads to fetch per page.
 -  `NUMBER_OF_COMMENTS_PER_THREAD`: Maximum number of comments to fetch per thread.
 -  `LAST_CURSOR`: (Optional) Base64 cursor from `pageInfo.endCursor` of the previous fetch for pagination.

Example invocation:

    PR_NUMBER=765 NUMBER_OF_PR_COMMENTS=50 NUMBER_OF_REVIEWS=50 \
    NUMBER_OF_THREADS=50 NUMBER_OF_COMMENTS_PER_THREAD=50 \
    bash .agents/skills/get-reviews/fetch_reviews.sh
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In @.agents/skills/get-reviews/SKILL.md around lines 31 - 39, Add explicit
documentation to SKILL.md describing the environment variables required by
fetch_reviews.sh: list PR_NUMBER, NUMBER_OF_PR_COMMENTS, NUMBER_OF_REVIEWS,
NUMBER_OF_THREADS, NUMBER_OF_COMMENTS_PER_THREAD (all required) and LAST_CURSOR
(optional for incremental pagination), and show a one-line example of invoking
fetch_reviews.sh with these environment variables set; reference the script name
fetch_reviews.sh and the exact variable names so users can copy them directly.


The fetched JSON files in *plans/{PR\_NUMBER}/fetched/* contain the raw data
of PR comments, reviews, and review threads (with `pullRequestReview`
back-references on thread comments). When more context is needed later
(e.g., to resolve which review a thread belongs to, or to check the original
body of a comment), refer back to these files instead of re-fetching.

[gh]: https://cli.github.com/


Organize the reviews
--------------------

After fetching the PR and its reviews, organize the reviews.
[*plans* directory in the root](../../../plans/) is a good place to store them.
Comment thread
2chanhaeng marked this conversation as resolved.

- *plans/{PR\_NUMBER}/index.md*: The main file for the PR.
- The body of the PR
- *plans/{PR\_NUMBER}/reviews/{REVIEW\_ID}.md*: The file for each review
which is not resolved.
- After applying or dismissing the review, move the file to
*plans/{PR\_NUMBER}/reviews/resolved/{REVIEW\_ID}.md*.
- If the review file is too long, move the content to
*plans/{PR\_NUMBER}/reviews/{REVIEW\_ID}/index.md*, and separate the
content into multiple files in the same directory. In this case, after
resolving the review, move the whole directory to
*plans/{PR\_NUMBER}/reviews/resolved/{REVIEW\_ID}/*.
Comment thread
2chanhaeng marked this conversation as resolved.
- *plans/{PR\_NUMBER}/reviews/resolved/{REVIEW\_ID}.md*: The file for each
review which is resolved.

**Don't use the first comment of the review thread as the review ID.**
The ID of the review thread starts with “PRRT\_”.
Use the first comment ID of the review thread only on the link.

Review threads aren't the only place that asks for changes. A PR-level
comment (in `comments` of the fetched JSON, whose ID starts with “IC\_”)
or the body of a review (in `reviews` of the fetched JSON, whose ID starts
with “PRR\_”) can also point out things to fix. When such a comment or
review body requests modifications, organize it as its own review file
alongside the review-thread files, using the same directory layout and
naming rules:

- Use the node ID of the PR comment (“IC\_…”) or the review (“PRR\_…”)
as the `{REVIEW\_ID}` in the file path.
- If a PR comment or review body only contains approval, general
impressions, questions without a modification request, or other content
with nothing to fix, skip it and do not create a file for it.
- If only a part of a longer PR comment or review body requests changes,
create a file only for that request and quote the relevant excerpt in
the file so the context is preserved.

The format of review files should be as [review.md](./review.md).
Read the reviews and fill the format as a draft with the related information.
The files should be written in the contributor's language. But the title of
the item in the file (e.g., “Summary”, “Judgement”, “Plans”) should be in
English for consistency.

Empty the space between the “Title” and the “Summary” sections.

All related information with the review should be stored in
*plans/{PR\_NUMBER}/reviews/{REVIEW\_ID}.md* or the files in
*plans/{PR\_NUMBER}/reviews/{REVIEW\_ID}/*.

After organizing the reviews, show the links to the files to the contributor.


Resolve the reviews
-------------------

Let the contributor read the review files, decide the judgement and the plans
for each review, and let them update the review files if necessary.
After the contributor decides the judgement and the plans, apply or dismiss the
reviews based on the files.

Categorize the reviews and the plans, and apply them at once by category.
After applying the review, use [`/commit` skill](../commit/SKILL.md) to commit
the changes. The commit message should include the related review links.
`https://github.com/fedify-dev/fedify/pull/{PR_NUMBER}#discussion_r{REVIEW_THREAD.COMMENTS[0].DATABASE_ID}`

After committing the changes, update the review file to include the commit hash
and the comment section. If the review is dismissed, update the review file to
include the reason for dismissing and the comment section.

If the `Comments` are written only in the contributor's language, provide an
English translation and have the contributor review it. If they are written in
both languages, check for any discrepancies between the two. If differences
exist between the two versions, review them based on the facts and revise
the English version to match the content in the contributor's language.

Post all review comments in English, even if the file written in the
contributor's native language. The comments should be polite and constructive.

After resolving the reviews, pushing commits, posting comments, and updating
the reviews as resolved, move the review files to
*plans/{PR\_NUMBER}/reviews/resolved*. Before moving the files, check status and
comments of the PR from GitHub. Use [fetch\_reviews.sh](./fetch_reviews.sh),
but instead of fetching all reviews and attributes, fetch only the necessary
attributes to check the review status and comments.
Comment on lines +135 to +137
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick | 🔵 Trivial | ⚡ Quick win

Clarify how to fetch reduced attributes for verification.

Line 135-137 instructs "fetch only the necessary attributes to check the review status and comments" but doesn't explain the mechanism. Since fetch_reviews.sh has a fixed GraphQL query, would users need to modify the script, or is there another approach?

Consider either:

  1. Adding a separate lighter-weight verification script, or
  2. Clarifying that users should manually inspect the most recent fetched JSON file in plans/{PR_NUMBER}/fetched/ rather than re-running the full fetch.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In @.agents/skills/get-reviews/SKILL.md around lines 135 - 137, The doc
currently tells users to "fetch only the necessary attributes" but doesn't say
how; update .agents/skills/get-reviews/SKILL.md to either (A) point to a new
lightweight script (e.g., create fetch_reviews_minimal.sh alongside
fetch_reviews.sh) that runs a smaller GraphQL query selecting only review fields
needed for verification (review state, body/text, author/login, createdAt, and
id), or (B) explicitly state that users can avoid rerunning fetch by manually
inspecting the latest fetched JSON under plans/{PR_NUMBER}/fetched/ and list
which keys to look for (e.g., reviews.nodes[*].state, reviews.nodes[*].body,
reviews.nodes[*].author.login, reviews.nodes[*].createdAt); include a short
example sentence on which option to choose and where the minimal script or JSON
keys live so readers can locate the verification data easily.

88 changes: 88 additions & 0 deletions .agents/skills/get-reviews/fetch_reviews.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,88 @@
#!/usr/bin/env bash
set -euo pipefail

: "${PR_NUMBER:?PR_NUMBER is required}"
: "${NUMBER_OF_PR_COMMENTS:?NUMBER_OF_PR_COMMENTS is required}"
: "${NUMBER_OF_REVIEWS:?NUMBER_OF_REVIEWS is required}"
: "${NUMBER_OF_THREADS:?NUMBER_OF_THREADS is required}"
: "${NUMBER_OF_COMMENTS_PER_THREAD:?NUMBER_OF_COMMENTS_PER_THREAD is required}"

PR_PATH="plans/$PR_NUMBER"
FETCHED_PATH="$PR_PATH/fetched"
mkdir -p "$FETCHED_PATH"
TIMESTAMP=$(date +"%m%d%H%M")
FETCHED_FILE="$FETCHED_PATH/$TIMESTAMP.json"
Comment thread
coderabbitai[bot] marked this conversation as resolved.
gh api graphql -f query='query(
$owner: String!,
$repo: String!,
$number: Int!,
$prComments: Int!,
$reviews: Int!,
$threads: Int!,
$comments: Int!,
$after: String
) {
repository(owner: $owner, name: $repo) {
pullRequest(number: $number) {
comments(first: $prComments) {
nodes {
id
databaseId
author { login }
body
url
createdAt
}
}
reviews(first: $reviews) {
nodes {
id
databaseId
author { login }
body
state
url
createdAt
}
}
reviewThreads(first: $threads, after: $after) {
pageInfo {
hasNextPage
endCursor
}
nodes {
id
isResolved
isOutdated
path
line
comments(first: $comments) {
nodes {
id
databaseId
author { login }
body
url
createdAt
pullRequestReview {
id
databaseId
}
}
}
}
}
}
}
}' \
-F owner=fedify-dev \
-F repo=fedify \
-F number="$PR_NUMBER" \
-F prComments="$NUMBER_OF_PR_COMMENTS" \
-F reviews="$NUMBER_OF_REVIEWS" \
-F threads="$NUMBER_OF_THREADS" \
-F comments="$NUMBER_OF_COMMENTS_PER_THREAD" \
${LAST_CURSOR:+-F after="$LAST_CURSOR"} \
| jq . > "$FETCHED_FILE"

# cspell: ignore MMDDHHMM
128 changes: 128 additions & 0 deletions .agents/skills/get-reviews/review.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,128 @@
---
id: REVIEW_ID
description: summary of the review
link: The link to the review on GitHub
# If more than one URL is relevant — related PR comments or review-thread
# replies that provide additional context — use `links` instead of `link`.
# Use the `url` field of each comment from the fetched JSON.
# links:
# - https://github.com/.../pull/123#discussion_r4567
# - https://github.com/.../pull/123#discussion_r4568
commit: The hash of commit after applying the review to add the comment
# If the review was applied across multiple commits, use `commits` instead
# of `commit`. Both fields are optional and only set after the review is
# applied.
# commits:
# - abc1234
# - def5678
---

<!-- deno-fmt-ignore-file -->

Title
=====

<!-- One line summary of the review -->


Summary
-------

<!--
The summary of the review, but in little more detail
If the review is too short even enough with the title, omit this section.
-->


Judgement
---------

<!--
The judgement about the review.

The first line should be one of the following:
- **CORRECT**: If the review is correct and should be applied.
- **WRONG**: If the review is wrong and should be dismissed.
- **PARTIAL**: If the review is partially correct and should be applied with
some modifications.
- **NEEDS EVALUATION**: If the review requires testing or evaluation,
such as a review pointing out efficiency issues.
- **NEEDS DISCUSSION**: If the review needs further discussion,
such as about direction of the project or design choices.
Try to use these words to indicate judgement status whenever possible,
but if you feel they are truly insufficient, use an appropriate word
and then update this part of *SKILL.md*.

After the first line, explain the judgement in more detail.
- The reasons for the judgement.
- The key factors that influence the judgement.
-->


Plans
-----

<!--
If the review is judged as **WRONG**, omit this section or write the plans
to add comments explaining the code to prevent similar misunderstandings.

If the review is judged as **CORRECT** or **PARTIAL**,
write the plans to apply the review.
- How to apply the review?
- If the review is judged as **PARTIAL**, what are the modifications to apply?
- Why the plans can apply the review correctly?

If the review is judged as **NEEDS EVALUATION**,
write the plans to evaluate the review, such as testing plans.
- How to test or evaluate?
- What are the criteria for success or failure?
- Why the tests or evaluation can determine the correctness of the review?
And after the evaluation, write the plans to apply or dismiss the review
based on the evaluation results.

If the review is judged as **NEEDS DISCUSSION**,
write the plans to discuss the review, such as topics to discuss and
potential options.
- What are the topics to discuss?
- What are the potential options and their pros and cons?
- What is the most reasonable option and why?
-->


Comments
--------

<!--
Prepare the response comments in advance after applying the review or
judging the review as wrong. The comments should be polite and constructive.

If the contributor don't use English, separate this section into two parts,
- the language of the contributor using
- English, ready to be posted as a response

If the review is judged as **CORRECT**, write comments to apply the review
to explain how to apply the review and why it is correct. The comments to
applied reviews should be started with "Addressed in {COMMIT_HASH}.".

If the review is judged as **WRONG**, write comments to dismiss the review
to explain why the review is wrong and should be dismissed.

If the review is judged as **PARTIAL**, write comments to partially apply the
review to explain how to apply the review with modifications and why it is
partially correct. Referring to the above, write the incorrect parts based
on the **WRONG** part, and the correct parts based on the **CORRECT** part.

If the review is judged as **NEEDS EVALUATION**, write the comments to
evaluate the review to explain how to evaluate the review, the test results,
and the resulting application/rejection details.
Comment thread
2chanhaeng marked this conversation as resolved.
- If the results of the evaluation are the review is correct,
write comments referring to the **CORRECT** part.
- If the results of the evaluation are the review is wrong,
write comments referring to the **WRONG** part.

If the review is judged as **NEEDS DISCUSSION**,
write the comments to discuss the review.
- The comments should explain what to discuss
about the review and why it needs discussion.
- The comments should be polite and constructive.
-->