-
-
Notifications
You must be signed in to change notification settings - Fork 99
Add get-reviews agent skill
#765
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Changes from all commits
ee41d96
90449b6
eeb651c
56a604a
d02f865
66fe934
6fe2427
8149001
fdb80e4
090bc8a
69bb15e
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,137 @@ | ||
| --- | ||
| name: get-reviews | ||
| description: >- | ||
| This skill is utilized when requesting reviews from GitHub pull requests. | ||
| Get the reviews, organize, and resolve them by applying or dismissing. | ||
| argument-hint: "Provide the number of the pull request to get the reviews." | ||
| --- | ||
|
|
||
| <!-- deno-fmt-ignore-file --> | ||
|
|
||
| Get reviews from GitHub pull requests | ||
| ===================================== | ||
|
|
||
| This skill is utilized when requesting reviews from GitHub pull requests. | ||
|
|
||
| 1. Get the reviews. | ||
| 2. Organize the reviews. | ||
| 3. Resolve the reviews by applying or dismissing them. | ||
|
|
||
|
|
||
| Get the reviews | ||
| --------------- | ||
|
|
||
| To get the reviews from a GitHub pull request, you can use the GitHub API. | ||
| Check the [`gh` CLI tool][gh] is installed and authenticated. | ||
| `gh auth status` can be used to check the authentication status. | ||
| If `gh` isn't installed, try installing it by `apt install gh`. | ||
| If authentication is not set up, tell the contributor to run `gh auth login` | ||
| to authenticate with GitHub. | ||
|
|
||
| Use the GraphQL API to fetch the reviews for a specific pull request. | ||
| Check [fetch\_reviews.sh](./fetch_reviews.sh) to fetch the reviews | ||
| and save them in a JSON file: | ||
|
|
||
| - Replace `$VARIABLES` with the actual values or variables in the command. | ||
| - For incremental fetches, save `pageInfo.endCursor` from the previous | ||
| fetch and pass it as `after: $LAST_CURSOR` (a base64 cursor, not a | ||
| review thread node ID) so the query returns only new review threads. | ||
| - Use `jq` to filter the reviews and information if necessary. | ||
|
|
||
| The fetched JSON files in *plans/{PR\_NUMBER}/fetched/* contain the raw data | ||
| of PR comments, reviews, and review threads (with `pullRequestReview` | ||
| back-references on thread comments). When more context is needed later | ||
| (e.g., to resolve which review a thread belongs to, or to check the original | ||
| body of a comment), refer back to these files instead of re-fetching. | ||
|
|
||
| [gh]: https://cli.github.com/ | ||
|
|
||
|
|
||
| Organize the reviews | ||
| -------------------- | ||
|
|
||
| After fetching the PR and its reviews, organize the reviews. | ||
| [*plans* directory in the root](../../../plans/) is a good place to store them. | ||
|
2chanhaeng marked this conversation as resolved.
|
||
|
|
||
| - *plans/{PR\_NUMBER}/index.md*: The main file for the PR. | ||
| - The body of the PR | ||
| - *plans/{PR\_NUMBER}/reviews/{REVIEW\_ID}.md*: The file for each review | ||
| which is not resolved. | ||
| - After applying or dismissing the review, move the file to | ||
| *plans/{PR\_NUMBER}/reviews/resolved/{REVIEW\_ID}.md*. | ||
| - If the review file is too long, move the content to | ||
| *plans/{PR\_NUMBER}/reviews/{REVIEW\_ID}/index.md*, and separate the | ||
| content into multiple files in the same directory. In this case, after | ||
| resolving the review, move the whole directory to | ||
| *plans/{PR\_NUMBER}/reviews/resolved/{REVIEW\_ID}/*. | ||
|
2chanhaeng marked this conversation as resolved.
|
||
| - *plans/{PR\_NUMBER}/reviews/resolved/{REVIEW\_ID}.md*: The file for each | ||
| review which is resolved. | ||
|
|
||
| **Don't use the first comment of the review thread as the review ID.** | ||
| The ID of the review thread starts with “PRRT\_”. | ||
| Use the first comment ID of the review thread only on the link. | ||
|
|
||
| Review threads aren't the only place that asks for changes. A PR-level | ||
| comment (in `comments` of the fetched JSON, whose ID starts with “IC\_”) | ||
| or the body of a review (in `reviews` of the fetched JSON, whose ID starts | ||
| with “PRR\_”) can also point out things to fix. When such a comment or | ||
| review body requests modifications, organize it as its own review file | ||
| alongside the review-thread files, using the same directory layout and | ||
| naming rules: | ||
|
|
||
| - Use the node ID of the PR comment (“IC\_…”) or the review (“PRR\_…”) | ||
| as the `{REVIEW\_ID}` in the file path. | ||
| - If a PR comment or review body only contains approval, general | ||
| impressions, questions without a modification request, or other content | ||
| with nothing to fix, skip it and do not create a file for it. | ||
| - If only a part of a longer PR comment or review body requests changes, | ||
| create a file only for that request and quote the relevant excerpt in | ||
| the file so the context is preserved. | ||
|
|
||
| The format of review files should be as [review.md](./review.md). | ||
| Read the reviews and fill the format as a draft with the related information. | ||
| The files should be written in the contributor's language. But the title of | ||
| the item in the file (e.g., “Summary”, “Judgement”, “Plans”) should be in | ||
| English for consistency. | ||
|
|
||
| Empty the space between the “Title” and the “Summary” sections. | ||
|
|
||
| All related information with the review should be stored in | ||
| *plans/{PR\_NUMBER}/reviews/{REVIEW\_ID}.md* or the files in | ||
| *plans/{PR\_NUMBER}/reviews/{REVIEW\_ID}/*. | ||
|
|
||
| After organizing the reviews, show the links to the files to the contributor. | ||
|
|
||
|
|
||
| Resolve the reviews | ||
| ------------------- | ||
|
|
||
| Let the contributor read the review files, decide the judgement and the plans | ||
| for each review, and let them update the review files if necessary. | ||
| After the contributor decides the judgement and the plans, apply or dismiss the | ||
| reviews based on the files. | ||
|
|
||
| Categorize the reviews and the plans, and apply them at once by category. | ||
| After applying the review, use [`/commit` skill](../commit/SKILL.md) to commit | ||
| the changes. The commit message should include the related review links. | ||
| `https://github.com/fedify-dev/fedify/pull/{PR_NUMBER}#discussion_r{REVIEW_THREAD.COMMENTS[0].DATABASE_ID}` | ||
|
|
||
| After committing the changes, update the review file to include the commit hash | ||
| and the comment section. If the review is dismissed, update the review file to | ||
| include the reason for dismissing and the comment section. | ||
|
|
||
| If the `Comments` are written only in the contributor's language, provide an | ||
| English translation and have the contributor review it. If they are written in | ||
| both languages, check for any discrepancies between the two. If differences | ||
| exist between the two versions, review them based on the facts and revise | ||
| the English version to match the content in the contributor's language. | ||
|
|
||
| Post all review comments in English, even if the file written in the | ||
| contributor's native language. The comments should be polite and constructive. | ||
|
|
||
| After resolving the reviews, pushing commits, posting comments, and updating | ||
| the reviews as resolved, move the review files to | ||
| *plans/{PR\_NUMBER}/reviews/resolved*. Before moving the files, check status and | ||
| comments of the PR from GitHub. Use [fetch\_reviews.sh](./fetch_reviews.sh), | ||
| but instead of fetching all reviews and attributes, fetch only the necessary | ||
| attributes to check the review status and comments. | ||
|
Comment on lines
+135
to
+137
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🧹 Nitpick | 🔵 Trivial | ⚡ Quick win Clarify how to fetch reduced attributes for verification. Line 135-137 instructs "fetch only the necessary attributes to check the review status and comments" but doesn't explain the mechanism. Since Consider either:
🤖 Prompt for AI Agents |
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,88 @@ | ||
| #!/usr/bin/env bash | ||
| set -euo pipefail | ||
|
|
||
| : "${PR_NUMBER:?PR_NUMBER is required}" | ||
| : "${NUMBER_OF_PR_COMMENTS:?NUMBER_OF_PR_COMMENTS is required}" | ||
| : "${NUMBER_OF_REVIEWS:?NUMBER_OF_REVIEWS is required}" | ||
| : "${NUMBER_OF_THREADS:?NUMBER_OF_THREADS is required}" | ||
| : "${NUMBER_OF_COMMENTS_PER_THREAD:?NUMBER_OF_COMMENTS_PER_THREAD is required}" | ||
|
|
||
| PR_PATH="plans/$PR_NUMBER" | ||
| FETCHED_PATH="$PR_PATH/fetched" | ||
| mkdir -p "$FETCHED_PATH" | ||
| TIMESTAMP=$(date +"%m%d%H%M") | ||
| FETCHED_FILE="$FETCHED_PATH/$TIMESTAMP.json" | ||
|
coderabbitai[bot] marked this conversation as resolved.
|
||
| gh api graphql -f query='query( | ||
| $owner: String!, | ||
| $repo: String!, | ||
| $number: Int!, | ||
| $prComments: Int!, | ||
| $reviews: Int!, | ||
| $threads: Int!, | ||
| $comments: Int!, | ||
| $after: String | ||
| ) { | ||
| repository(owner: $owner, name: $repo) { | ||
| pullRequest(number: $number) { | ||
| comments(first: $prComments) { | ||
| nodes { | ||
| id | ||
| databaseId | ||
| author { login } | ||
| body | ||
| url | ||
| createdAt | ||
| } | ||
| } | ||
| reviews(first: $reviews) { | ||
| nodes { | ||
| id | ||
| databaseId | ||
| author { login } | ||
| body | ||
| state | ||
| url | ||
| createdAt | ||
| } | ||
| } | ||
| reviewThreads(first: $threads, after: $after) { | ||
| pageInfo { | ||
| hasNextPage | ||
| endCursor | ||
| } | ||
| nodes { | ||
| id | ||
| isResolved | ||
| isOutdated | ||
| path | ||
| line | ||
| comments(first: $comments) { | ||
| nodes { | ||
| id | ||
| databaseId | ||
| author { login } | ||
| body | ||
| url | ||
| createdAt | ||
| pullRequestReview { | ||
| id | ||
| databaseId | ||
| } | ||
| } | ||
| } | ||
| } | ||
| } | ||
| } | ||
| } | ||
| }' \ | ||
| -F owner=fedify-dev \ | ||
| -F repo=fedify \ | ||
| -F number="$PR_NUMBER" \ | ||
| -F prComments="$NUMBER_OF_PR_COMMENTS" \ | ||
| -F reviews="$NUMBER_OF_REVIEWS" \ | ||
| -F threads="$NUMBER_OF_THREADS" \ | ||
| -F comments="$NUMBER_OF_COMMENTS_PER_THREAD" \ | ||
| ${LAST_CURSOR:+-F after="$LAST_CURSOR"} \ | ||
| | jq . > "$FETCHED_FILE" | ||
|
|
||
| # cspell: ignore MMDDHHMM | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,128 @@ | ||
| --- | ||
| id: REVIEW_ID | ||
| description: summary of the review | ||
| link: The link to the review on GitHub | ||
| # If more than one URL is relevant — related PR comments or review-thread | ||
| # replies that provide additional context — use `links` instead of `link`. | ||
| # Use the `url` field of each comment from the fetched JSON. | ||
| # links: | ||
| # - https://github.com/.../pull/123#discussion_r4567 | ||
| # - https://github.com/.../pull/123#discussion_r4568 | ||
| commit: The hash of commit after applying the review to add the comment | ||
| # If the review was applied across multiple commits, use `commits` instead | ||
| # of `commit`. Both fields are optional and only set after the review is | ||
| # applied. | ||
| # commits: | ||
| # - abc1234 | ||
| # - def5678 | ||
| --- | ||
|
|
||
| <!-- deno-fmt-ignore-file --> | ||
|
|
||
| Title | ||
| ===== | ||
|
|
||
| <!-- One line summary of the review --> | ||
|
|
||
|
|
||
| Summary | ||
| ------- | ||
|
|
||
| <!-- | ||
| The summary of the review, but in little more detail | ||
| If the review is too short even enough with the title, omit this section. | ||
| --> | ||
|
|
||
|
|
||
| Judgement | ||
| --------- | ||
|
|
||
| <!-- | ||
| The judgement about the review. | ||
|
|
||
| The first line should be one of the following: | ||
| - **CORRECT**: If the review is correct and should be applied. | ||
| - **WRONG**: If the review is wrong and should be dismissed. | ||
| - **PARTIAL**: If the review is partially correct and should be applied with | ||
| some modifications. | ||
| - **NEEDS EVALUATION**: If the review requires testing or evaluation, | ||
| such as a review pointing out efficiency issues. | ||
| - **NEEDS DISCUSSION**: If the review needs further discussion, | ||
| such as about direction of the project or design choices. | ||
| Try to use these words to indicate judgement status whenever possible, | ||
| but if you feel they are truly insufficient, use an appropriate word | ||
| and then update this part of *SKILL.md*. | ||
|
|
||
| After the first line, explain the judgement in more detail. | ||
| - The reasons for the judgement. | ||
| - The key factors that influence the judgement. | ||
| --> | ||
|
|
||
|
|
||
| Plans | ||
| ----- | ||
|
|
||
| <!-- | ||
| If the review is judged as **WRONG**, omit this section or write the plans | ||
| to add comments explaining the code to prevent similar misunderstandings. | ||
|
|
||
| If the review is judged as **CORRECT** or **PARTIAL**, | ||
| write the plans to apply the review. | ||
| - How to apply the review? | ||
| - If the review is judged as **PARTIAL**, what are the modifications to apply? | ||
| - Why the plans can apply the review correctly? | ||
|
|
||
| If the review is judged as **NEEDS EVALUATION**, | ||
| write the plans to evaluate the review, such as testing plans. | ||
| - How to test or evaluate? | ||
| - What are the criteria for success or failure? | ||
| - Why the tests or evaluation can determine the correctness of the review? | ||
| And after the evaluation, write the plans to apply or dismiss the review | ||
| based on the evaluation results. | ||
|
|
||
| If the review is judged as **NEEDS DISCUSSION**, | ||
| write the plans to discuss the review, such as topics to discuss and | ||
| potential options. | ||
| - What are the topics to discuss? | ||
| - What are the potential options and their pros and cons? | ||
| - What is the most reasonable option and why? | ||
| --> | ||
|
|
||
|
|
||
| Comments | ||
| -------- | ||
|
|
||
| <!-- | ||
| Prepare the response comments in advance after applying the review or | ||
| judging the review as wrong. The comments should be polite and constructive. | ||
|
|
||
| If the contributor don't use English, separate this section into two parts, | ||
| - the language of the contributor using | ||
| - English, ready to be posted as a response | ||
|
|
||
| If the review is judged as **CORRECT**, write comments to apply the review | ||
| to explain how to apply the review and why it is correct. The comments to | ||
| applied reviews should be started with "Addressed in {COMMIT_HASH}.". | ||
|
|
||
| If the review is judged as **WRONG**, write comments to dismiss the review | ||
| to explain why the review is wrong and should be dismissed. | ||
|
|
||
| If the review is judged as **PARTIAL**, write comments to partially apply the | ||
| review to explain how to apply the review with modifications and why it is | ||
| partially correct. Referring to the above, write the incorrect parts based | ||
| on the **WRONG** part, and the correct parts based on the **CORRECT** part. | ||
|
|
||
| If the review is judged as **NEEDS EVALUATION**, write the comments to | ||
| evaluate the review to explain how to evaluate the review, the test results, | ||
| and the resulting application/rejection details. | ||
|
2chanhaeng marked this conversation as resolved.
|
||
| - If the results of the evaluation are the review is correct, | ||
| write comments referring to the **CORRECT** part. | ||
| - If the results of the evaluation are the review is wrong, | ||
| write comments referring to the **WRONG** part. | ||
|
|
||
| If the review is judged as **NEEDS DISCUSSION**, | ||
| write the comments to discuss the review. | ||
| - The comments should explain what to discuss | ||
| about the review and why it needs discussion. | ||
| - The comments should be polite and constructive. | ||
| --> | ||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
🧹 Nitpick | 🔵 Trivial | ⚡ Quick win
Document the required environment variables for fetch_reviews.sh.
The documentation references
fetch_reviews.shand mentions replacing$VARIABLES, but doesn't explicitly list which environment variables the script requires. Based on the script (lines 4-8 of fetch_reviews.sh), users need to set:PR_NUMBER(required)NUMBER_OF_PR_COMMENTS(required)NUMBER_OF_REVIEWS(required)NUMBER_OF_THREADS(required)NUMBER_OF_COMMENTS_PER_THREAD(required)LAST_CURSOR(optional, for incremental fetches)Consider adding a bullet list or example showing how to invoke the script with these variables.
📝 Suggested addition
After line 33, add:
🤖 Prompt for AI Agents