Skip to content

ci: combine AI issue triage with a maintainer rotation as the responsible reviewer - #1260

Draft
alexhancock wants to merge 1 commit into
mainfrom
alexhancock/ai-triage-with-human-backstop
Draft

ci: combine AI issue triage with a maintainer rotation as the responsible reviewer#1260
alexhancock wants to merge 1 commit into
mainfrom
alexhancock/ai-triage-with-human-backstop

Conversation

@alexhancock

Copy link
Copy Markdown
Contributor

Motivation and Context

Supersedes #1257 and #1254, which solved two halves of the same problem: #1257 added AI classification for new issues, #1254 added a maintainer rotation. Landing them separately would have meant two workflows racing to triage the same issue with no defined relationship between them.

This combines both into a single triage.yml with two stages:

  1. Codex classifies the issue and applies type / priority / component labels.
  2. The maintainer on rotation is assigned as the responsible human reviewer, @-mentioned with what the AI concluded so they're confirming a verdict rather than starting cold.

The key property is that stage 2 does not depend on stage 1. assign runs under always() gated only on preflight, so if Codex is unconfigured, times out, or returns garbage, the issue still lands on a human — the AI pass accelerates triage and never decides it. Missing secrets degrade a stage to a skip rather than a red X on the issue.

Codex classification is scoped to issues only. PRs already get their T-* labels from the diff via auto-label-pr.yml, so running the model over them would just be a second, worse source of the same labels.

Addressing @DaleSeo's feedback on #1254

  • read:org won't be enough to call addAssignees and createComment. Correct. ROTATION_TOKEN is now documented as needing read:org plus write on issues and pull requests, in the workflow header and in the docs table.
  • Can we use the latest version, v9? Done — actions/github-script is now v9.0.0, pinned to 373c709c.
  • I can't find any README notes in the PR. They didn't exist; the comment referenced notes I never wrote. Now written in docs/CONTRIBUTE.MD under "Triage" (the two stages, rotation mechanics, response-time expectations, secret table, dry-run instructions) and linked from the README's Development section. The workflow header points at the real file instead of a CONTRIBUTING.md that doesn't exist in this repo.

Two things I fixed along the way

  • The bot filter in governance/maintenance: add triage rotation workflow #1254 never worked. It used github.event.issue.user.type != 'Bot' || github.event.pull_request.user.type != 'Bot' — on any single event one of those contexts is empty, so that side is always true and the || always passed. Dependabot and release-plz PRs would have been assigned to the rotation. Now a single comparison on (issue.user.type || pull_request.user.type).
  • Model output no longer reaches a ${{ }} interpolation. The classification is passed into github-script through env rather than interpolated into the script body, and each field is validated against its allowed set before it reaches a comment body. Codex output is untrusted input, and ci: triage new issues with Codex #1257's prompt already treats issue text that way.

How Has This Been Tested?

actionlint clean. Both stages support workflow_dispatch with dry_run (default true) which logs the verdict and the rotation pick and changes nothing — I'll exercise that against a real issue before this comes out of draft, since neither stage can be fully tested until the secrets are configured on the repo.

Draft because it needs ROTATION_TOKEN created and @DaleSeo's read on folding his workflow into this one.

Breaking Changes

None. Deletes scripts/triage-new-issues.sh, the Bash triage script that has been dead since #714 disabled it with if: false — same removal #1257 made.

Types of changes

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to change)
  • Documentation update

Checklist

  • I have read the MCP Documentation
  • My code follows the repository's style guidelines
  • New and existing tests pass locally
  • I have added appropriate error handling
  • I have added or updated documentation as needed

…ner on weekly rotation as the responsible human reviewer
@github-actions github-actions Bot added T-documentation Documentation improvements T-CI Changes to CI/CD workflows and configuration T-config Configuration file changes labels Sep 11, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

T-CI Changes to CI/CD workflows and configuration T-config Configuration file changes T-documentation Documentation improvements

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant