ReviewBot

What is ReviewBot

ReviewBot is a GitHub App that automatically performs AI-powered code review on pull requests. It fetches changed lines from each PR, asks Anthropic Claude to identify high-signal issues, and posts precise inline comments on the affected code.

How It Works

When a pull request is opened or updated, GitHub sends a signed webhook event to ReviewBot. ReviewBot verifies the webhook signature, fetches the pull request diff through the GitHub API, parses the diff into changed files and added line numbers, sends those added lines to Claude for review, validates Claude's JSON response, and posts each issue as an inline pull request review comment on the exact file and line.

Tech Stack

• Node.js
• TypeScript
• Express
• Octokit
• Anthropic Claude
• Railway

Prerequisites

• Node.js 18+
• A GitHub account
• An Anthropic API key

Step 1 — Clone and Install

git clone https://github.com/your-username/reviewbot.git
cd reviewbot
npm install

Step 2 — Create the GitHub App

1. Go to github.com/settings/apps/new
2. GitHub App name: ReviewBot (or any name)
3. Homepage URL: your deployed URL (or http://localhost:3000 for now)
4. Webhook URL: your deployed URL + /webhook
5. Webhook secret: generate a random string, save it as GITHUB_WEBHOOK_SECRET
6. Permissions needed — Repository permissions:
   • Pull requests: Read & Write
   • Contents: Read-only
7. Subscribe to events: Pull request
8. Where can this be installed: Only on this account
9. Click Create GitHub App
10. On the app page: note the App ID, save it as GITHUB_APP_ID
11. Generate a private key, download the .pem file
12. Convert it for the env var:

    awk 'NF {sub(/\r/, ""); printf "%s\\n",$0;}' your-key.pem

    Save the output as GITHUB_PRIVATE_KEY
13. Install the app on your own repositories

Step 3 — Configure Environment

cp .env.example .env

Fill in all values in .env.

Step 4 — Run Locally with a Tunnel

npm run dev

Use ngrok or smee.io to expose localhost:3000 to the internet for webhook delivery during development:

npx smee-client --url https://smee.io/your-channel --target http://localhost:3000/webhook

Step 5 — Deploy to Railway

1. Push to GitHub
2. Create new Railway project from repo
3. Add environment variables
4. Deploy — Railway auto-detects the Procfile

How to Test It

Open a PR on a repo where ReviewBot is installed. ReviewBot will automatically post inline comments within seconds.

Example Review Comment

[Security] eval() executes arbitrary code and is a critical security vulnerability. Replace with a safe alternative such as JSON.parse() for data or a proper expression parser library.

Environment Variables

Variable	Description	Where to find it
GITHUB_APP_ID	GitHub App ID used to authenticate as the app.	GitHub App settings page after app creation.
GITHUB_PRIVATE_KEY	GitHub App private key with newlines converted to \n.	Download from the GitHub App settings page, then convert with the README command.
GITHUB_WEBHOOK_SECRET	Shared secret used to verify webhook signatures.	The random webhook secret you set while creating the GitHub App.
ANTHROPIC_API_KEY	API key used to call Claude.	Anthropic Console.
PORT	HTTP port for the Express server.	Set by Railway in production or use 3000 locally.