Feature/4 implement skill pr review#6
Open
miroslavpojer wants to merge 12 commits into
Open
Conversation
…DME and documentation for clarity
…nes and skill overview
…pts for PR review - Created `results-summary.md` to document evaluation results for the pr-review skill, detailing iterations, fixes, and overall trajectory. - Added `trigger-eval.json` containing a set of queries for evaluating trigger conditions, including both triggering and non-triggering examples. - Introduced `output-template.md` for standardized PR review output examples, outlining formatting rules and section guidelines. - Developed `security-antipatterns.md` as a reference for identifying security issues during PR reviews, including common patterns and severity guidance. - Implemented `classify_sections.py` to classify changed files into relevant PR review sections based on defined patterns. - Created `fetch_pr.sh` script to fetch PR details and changed files using the GitHub CLI, facilitating integration with the classification script.
miroslavpojer
requested review from
lsulak,
oto-macenauer-absa and
tmikula-dev
as code owners
miroslavpojer
added
the
work in progress
miroslavpojer
added
work in progress
…ew skill definition evals
miroslavpojer
removed
the
work in progress
lsulak
reviewed
May 26, 2026
| @@ -0,0 +1,36 @@ | |||
| # Sprint 23 — User Service Changelog | |||
There was a problem hiding this comment.
why also the token saving skill in the PR?
Contributor
Author
There was a problem hiding this comment.
This PR is build on top of previous as I am doing also folder moves.
Let's review.merge token-saving skill first, pls.
There was a problem hiding this comment.
Pull request overview
Adds a new pr-review skill to the toolkit, including structured review instructions, helper scripts to fetch/classify PR context, reference templates, and an eval suite; also updates documentation indexes and introduces a workflow to enforce release-notes presence in PR descriptions.
Changes:
- Introduces
skills/pr-review/with SKILL definition, references, scripts, and eval fixtures/results. - Adds end-user documentation for the new skill and updates README indexes/catalogs.
- Adds a GitHub Actions workflow to check for “Release Notes” in PR descriptions.
Reviewed changes
Copilot reviewed 27 out of 27 changed files in this pull request and generated 8 comments.
Show a summary per file
| File | Description |
|---|---|
.github/copilot-instructions.md |
Points PR review flow at the new pr-review skill (removes duplicated inline checks). |
.github/workflows/check_pr_release_notes.yml |
Adds a workflow to require release notes presence in PR descriptions. |
README.md |
Adds pr-review to the Skill Catalog. |
docs/README.md |
Adds pr-review to the docs index tables and normalizes table formatting. |
docs/getting-started.md |
Minor formatting/line-wrapping cleanup. |
docs/pr-review.md |
New user guide explaining purpose, sections, triggers, helpers, and installation. |
skills/pr-review/SKILL.md |
Core PR review skill definition and structured review checklist/format rules. |
skills/pr-review/evals/evals.json |
Adds eval suite definitions for the skill (fixtures + expectations). |
skills/pr-review/evals/fixture-map.md |
Maps eval IDs to fixture diffs and summarizes coverage. |
skills/pr-review/evals/results-summary.md |
Documents evaluation iterations and outcomes. |
skills/pr-review/evals/trigger-eval.json |
Adds trigger/negative-trigger queries to validate skill activation boundaries. |
skills/pr-review/evals/files/api-rename.diff |
Eval fixture for API contract rename scenario. |
skills/pr-review/evals/files/ci-gate-bypass.diff |
Eval fixture for CI gate bypass + secrets in workflow scenario. |
skills/pr-review/evals/files/db-migration-risks.diff |
Eval fixture for migration rollback/data-loss risk scenario. |
skills/pr-review/evals/files/dependency-bump-risk.diff |
Eval fixture for dependency bump risk scenario. |
skills/pr-review/evals/files/docs-release-notes.diff |
Eval fixture for “release notes” (non-review) negative case. |
skills/pr-review/evals/files/elevated-risk-auth-refactor.diff |
Eval fixture for elevated-risk auth change scenario. |
skills/pr-review/evals/files/iac-wildcard-iam.diff |
Eval fixture for IaC wildcard IAM scenario. |
skills/pr-review/evals/files/large-pr-and-vague-desc.diff |
Eval fixture for “large PR + vague description” scenario. |
skills/pr-review/evals/files/multi-section-risks.diff |
Eval fixture combining DB/API/CI risks in one PR. |
skills/pr-review/evals/files/skill-definition-clean.diff |
Eval fixture for a clean SKILL.md addition. |
skills/pr-review/evals/files/skill-definition-violations.diff |
Eval fixture for SKILL.md convention violations. |
skills/pr-review/evals/files/standard-clean-pr.diff |
Eval fixture for a clean “LGTM” utility PR. |
skills/pr-review/references/output-template.md |
Canonical output formatting examples for reviews. |
skills/pr-review/references/security-antipatterns.md |
Security anti-pattern reference for elevated-risk/security reviews. |
skills/pr-review/scripts/classify_sections.py |
Helper to infer which review sections apply based on changed file paths. |
skills/pr-review/scripts/fetch_pr.sh |
Helper to fetch PR description/files/diff via gh CLI. |
…ile paths in evaluation documentation and scripts for consistency.
…y and specificity
lsulak
reviewed
Jun 17, 2026
There was a problem hiding this comment.
I'd remove this file and put pychache dir into gitignore
lsulak
approved these changes
Jun 17, 2026
lsulak
left a comment
lsulak
left a comment
There was a problem hiding this comment.
I like it, let's try it :) very good for the first iteration
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Add
pr-reviewskillAdds a unified pull request review skill that applies structured, risk-aware code review based on the files a PR touches. Produces concise
Blocker / Important / Nitcomments grouped by severity.README.md— addedpr-reviewto the Skill Catalog tabledocs/README.md— addedpr-reviewto the Skill Guides tabledocs/pr-review.md— end-user guide covering sections, trigger phrases, helpers, and installRelease Notes
skills/pr-review/SKILL.md— full review instructions covering standard, API contracts, dependency bumps, CI/CD, infrastructure, DB migrations, and elevated-risk sectionsskills/pr-review/references/output-template.md— canonical output format examples for reviewsskills/pr-review/references/security-antipatterns.md— security patterns the skill actively scans forskills/pr-review/scripts/fetch_pr.sh— fetches PR diff and file list viaghskills/pr-review/scripts/classify_sections.py— determines which review sections apply from a file listskills/pr-review/evals/— trigger eval, fixture map, eval fixtures, and results summaryCloses #4