Skip to content

GH-55259: [Docs] Add guidelines for reviewers on AI generated PRs #55260

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
raulcd wants to merge 4 commits into
base: main
Choose a base branch
from
+16 −0
Open

GH-55259: [Docs] Add guidelines for reviewers on AI generated PRs #55260

Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
424d84e
GH-55259: [Docs] Add guidelines for reviewers on AI generated PRs
raulcd Oct 26, 2026
005d8f0
Fix grammar
raulcd Oct 26, 2026
224480a
Improve formatting
raulcd Oct 26, 2026
30347ce
Update docs/source/developers/overview.rst
raulcd Oct 26, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 16 additions & 0 deletions docs/source/developers/overview.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -191,6 +191,22 @@ submitted contributions. See the `ASF's guidance on AI-generated code
<https://www.apache.org/legal/generative-tooling.html>`_ for further
information on licensing considerations.

We expect human contributors to actively engage on the review process. We
recognise the value of using AI tools to understand the changes being requested
but we expect a human-in-the-loop for all engagements. Reviewer time is limited
and reviewers are not expected to be talking to contributors' agents but to
contributors themselves. If a reviewer finds that a contributor seems to rely
excessively on AI tools instead of engaging with the review, the PR might be
closed and/or tagged with a label indicating excessive AI usage.

@thisisnic thisisnic Oct 26, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I appreciate we have a need for filtering out low-effort PRs, but I worry such a tag would create unnecessary drama, and think we might have an easier life by creating a positive tag and recommending reviewers filter on that if they want a low chance of AI PRs

For what could qualify someone for a positive tag, I quite like the prioritisation order listed at the bottom of this blog post by Andrew Lamb: https://andrew.nerdnetworks.org/blog/2026/10/19/my-plan-for-reviewing-prs-in-open-source-projects-in-the-age-of-ai/

@rok rok Oct 26, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually arrow-rs auto labels "Prolific Author" - https://github.com/apache/arrow-rs/pulls?q=is%3Apr+label%3A%22Prolific+Author%22

@thisisnic thisisnic Oct 26, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure, but that's something else entirely, just by the same person.

@thisisnic thisisnic Oct 26, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

FWIW I'm opposed to that as well as it's labelling the individual not the action. It's going to feel very personal to anyone who gets a false positive.

We could achieve the same thing by just closing those PRs with an explanatory message, if we wanted to limit PRs from people who have made lots of PRs to lots of repos.

@thisisnic thisisnic Oct 26, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(for clarity my original comment was referring only to "tagged with a label indicating excessive AI usage" and not the rest of the change)


.. dropdown:: Example of message for a PR with excessive AI usage
:animate: fade-in-slide-down
:class-container: sd-shadow-none

This PR doesn't appear to comply with our policy on tool-generated content.
Please see our AI generated code policy on AI-generated code:
https://arrow.apache.org/docs/developers/overview.html#ai-generated-code

Comment on lines +202 to +209

@thisisnic thisisnic Oct 26, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like that we are providing an example for folks to use. I wonder if we want a reviewer's guide to put this into perhaps?

.. Section on Experimental repositories:

.. include:: experimental_repos.rst
Expand Down
Loading