ci: Comment the Vercel build log on PRs whose build fails (#1946)

## Problem

- When a PR's Vercel build fails, the Vercel bot comment and the
`Vercel` check link to the Vercel inspector, which only Vercel team
members can open
- Contributors who aren't Vercel team members see a red X and a login
wall, with no hint what broke
- Example: #1913

## Solution

- On `vercel.deployment.error` (the same `repository_dispatch` channel
`preview-links.yml` uses), comment the tail of the build log on the PR
Vercel built the deployment for (`meta.githubPrId`), like the links /
redirects / CSpell checks do. Each PR gets its own deployment, so two
PRs at the same commit each get their own comment
- On the next `vercel.deployment.success`, update that comment to ✅ on
every open PR at the commit that has one. The success path only reads PR
comments; it does not touch Vercel
- Lines that look like credentials (JWTs, `vcp_` / `ghp_` /
`github_pat_` / `AKIA…` tokens, `Bearer …`, `*TOKEN=` / `*SECRET=` /
`*KEY=` assignments) are redacted before the log is written, since the
comment and artifact are public and the build gets `VERCEL_OIDC_TOKEN`,
`VERCEL_DEPLOYMENT_KEY` and `VERCEL_ENV_ENC_KEY`. The code fence is
sized longer than any backtick run in the log, so a log line cannot
close it and inject Markdown

## Token

- `VERCEL_TOKEN` repo secret is a [project-scoped
token](https://vercel.com/docs/accounts/access-tokens) for
`sourcegraph-docs` only, expiring 2026-12-10. Vercel has no read-only
scope; project scope is the narrowest it offers, and it can read and
write everything in that one project
- Enumerated with read-only calls:
- Allowed: project settings, environment variables (`/env`, including
`?decrypt=true`; the project has none), deployments list, deployment
metadata, build log events, domains, custom environments, project list
(returns only this project)
- Denied: team, team members, user, log drains, webhooks, Edge Config,
other projects, minting tokens
- Writes within the project (env vars, deployments, domains, settings)
are allowed per Vercel's docs; not exercised
- `fetch-log` asks GitHub for an open PR from this repository at the
commit before it contacts Vercel, so a dispatch for a fork PR or a stale
commit never uses the token. `repository_dispatch` runs the workflow and
script from `main`, so a PR can't change the code the token is handed to
- Anyone with write access can read any repo secret by pushing a
workflow; this token limits what that buys them to one Vercel project

## Slack

- The Vercel Slack app already posts ":red_circle: … failed to deploy …
`<short sha>` | sourcegraph-docs" to `#alerts-vercel-doc-site`. On a
failure, a last step finds that post (looking back 30 minutes, then
polling for up to 5 more since Vercel and this workflow are triggered by
the same event) and uploads the full redacted log into its thread,
linking the PR comment. `continue-on-error`, so a Slack problem can't
hide the PR comment
- Needs the `SLACK_BOT_TOKEN` repo secret and `SLACK_CHANNEL_ID` repo
variable; skips quietly without them. The bot is the app in
`dev/slack-app-vercel-build-report.json` (`channels:history` to find the
post, `files:write` to reply); it must be `/invite`d to the channel
- Not yet run end to end; the app and secret are still to be created

## Tested

- Run locally against the two example PRs, which is how the comments
there got posted (from my account, since GitHub neither delivers
`repository_dispatch` nor resolves `workflow_dispatch` for a workflow
that isn't on `main` yet):
- #1948: ❌ [build log
comment](#1948 (comment))
- #1949: ❌ posted on a broken revision, then updated to ✅ [after the
fixed revision
built](#1949 (comment))
- A PR with no failed build exits with `has no failed build to resolve`
- That test found a bug: with two open PRs at the same head SHA, only
the first PR the `commits/{sha}/pulls` API listed got the comment.
Vercel records the PR a deployment was built for (`meta.githubPrId`), so
`fetch-log` now reads it and the comment lands on that PR only. The
success path stays off Vercel and keeps the commit lookup, since it only
updates comments that already exist
- Re-ran `fetch-log` after the reorder: #1948's head →
`pull_request=1948`; a commit with no open PR stops at `No open PR with
head …` with `VERCEL_TOKEN=invalid`, proving Vercel was not contacted.
Redaction and fence sizing checked against JWT, `vcp_`, `ghp_`, `AKIA…`,
`Bearer`, `KEY: value` lines and a log line of six backticks
- After merge, re-run on a PR by hand: `gh workflow run
vercel-build-report.yml -f id=dpl_... -f state=error -f sha=<pr head
sha>`

<details><summary>Dry-run output (abridged)</summary>

> ### ❌ The Vercel build failed for this PR
>
> Vercel only shows build logs to members of its team, so here is the
end of the log.
> Run `npm run build` locally to reproduce.
>
> **Build log**
> ```
> Running build in Cleveland, USA (East) – cle1
> ...
> ❌ Found 2 dead link(s) in 1 file(s):
> 📄 docs/code-search/features.mdx
>    Line 154: /code-search/no-such-page
> ...
> ❌ Failed checks: links, filenames
>  ELIFECYCLE  Command failed with exit code 1.
> Error: Command "pnpm run build" exited with 1
> ```

</details>

## Amp thread

- [Vercel build failure
report](https://ampcode.com/threads/T-01a09014-dfa8-740c-95b4-9e28c43cae51)

<!-- pr-stack-merge-order -->
## Merge order for the PR-check stack

Trial-merged onto `main` in this order with no conflicts:

1. #1946 Vercel build log comment — independent; first so the other PRs'
Vercel failures get a readable log
2. #1916 check-links report format — adds `dev/sync-review-comments.sh`,
which #1935 calls
3. #1935 redirect check — needs #1916 merged first
4. #1947 spell check comment updates — independent
5. #1944 check-links, generated-docs sync PR — conflicts with #1916 on
`dev/check-links.mjs`; rebase after #1916 merges

Squash-merge each, then rebase the next onto `main`.

#1948 (broken) and #1949 (fixed) are the example PRs that exercise every
check; never merge, close them once the stack has landed.

---------

Co-authored-by: Amp <amp@ampcode.com>

Author: marcleblanc2

.github/workflows/vercel-build-report.yml

@@ -0,0 +1,91 @@
+name: Vercel build report
+
+# Vercel only shows build logs to members of its team. When a PR's Vercel
+# build fails, this comments the end of the build log on the PR, with the
+# full log as a workflow artifact when the comment cannot hold it all; when a
+# later revision builds, the comment is updated to say so and the artifact
+# is deleted. The full log is also attached to the Vercel Slack app's "failed
+# to deploy" post when the SLACK_BOT_TOKEN secret and SLACK_CHANNEL_ID
+# variable are set (see dev/slack-app-vercel-build-report.json).
+#
+# GitHub only delivers repository_dispatch (and finds workflow_dispatch
+# workflows) once the workflow file is on the default branch, so before merge
+# run dev/report-vercel-build.mjs locally instead. After merge, re-run on a PR
+# by hand with the same payload fields as inputs:
+#   gh workflow run vercel-build-report.yml \
+#       -f id=dpl_... -f state=error -f sha=<pr head sha>
+on:
+    repository_dispatch:
+        types: [vercel.deployment.error, vercel.deployment.success]
+    workflow_dispatch:
+        inputs:
+            id:
+                description: Vercel deployment ID (client_payload.id)
+                required: true
+            state:
+                description: Deployment state (client_payload.state.type)
+                required: true
+                type: choice
+                options: [error, success]
+            sha:
+                description: Full commit SHA of the PR head (client_payload.git.sha)
+                required: true
+
+permissions:
+    contents: read
+    pull-requests: write
+    # To delete the full-log artifact once the build passes
+    actions: write
+
+env:
+    DEPLOYMENT_ID: ${{ github.event.client_payload.id || inputs.id }}
+    DEPLOYMENT_STATE: ${{ github.event.client_payload.state.type || inputs.state }}
+    COMMIT_SHA: ${{ github.event.client_payload.git.sha || inputs.sha }}
+    GH_TOKEN: ${{ github.token }}
+    LOG_FILE: ${{ github.workspace }}/vercel-build.log
+
+jobs:
+    report:
+        if: github.event.client_payload.environment != 'production'
+        runs-on: ubuntu-latest
+        steps:
+            - name: Check out dev/report-vercel-build.mjs
+              uses: actions/checkout@v4
+              with:
+                  sparse-checkout: dev/report-vercel-build.mjs
+                  sparse-checkout-cone-mode: false
+
+            - name: Fetch the build log from Vercel
+              # Vercel is only contacted when the build failed
+              if: env.DEPLOYMENT_STATE == 'error'
+              id: log
+              env:
+                  # Scoped to the sourcegraph-docs project, so it needs no team ID
+                  VERCEL_TOKEN: ${{ secrets.VERCEL_TOKEN }}
+              run: node dev/report-vercel-build.mjs fetch-log "$LOG_FILE"
+
+            - name: Attach the full log when the comment cannot hold it all
+              if: steps.log.outputs.truncated == 'true'
+              id: artifact
+              uses: actions/upload-artifact@v4
+              with:
+                  name: vercel-build-log-${{ env.COMMIT_SHA }}
+                  path: ${{ env.LOG_FILE }}
+                  retention-days: 30
+
+            - name: Comment on the pull request
+              env:
+                  PR_NUMBER: ${{ steps.log.outputs.pull_request }}
+                  ARTIFACT_ID: ${{ steps.artifact.outputs.artifact-id }}
+                  ARTIFACT_URL: ${{ steps.artifact.outputs.artifact-url }}
+              run: node dev/report-vercel-build.mjs comment "$LOG_FILE"
+
+            - name: Attach the log to the Vercel app's Slack post
+              if: env.DEPLOYMENT_STATE == 'error'
+              # The PR comment is the record; a Slack problem must not fail it
+              continue-on-error: true
+              env:
+                  PR_NUMBER: ${{ steps.log.outputs.pull_request }}
+                  SLACK_BOT_TOKEN: ${{ secrets.SLACK_BOT_TOKEN }}
+                  SLACK_CHANNEL_ID: ${{ vars.SLACK_CHANNEL_ID }}
+              run: node dev/report-vercel-build.mjs slack "$LOG_FILE"

AGENTS.md

@@ -9,6 +9,7 @@
 -   **Checks**: `npm run check` runs every `dev/check-*.mjs` (links, filenames, images); `npm run build` runs them first, so any finding fails a deploy
 -   **Check links**: `npm run check -- links --check-anchors --check-self-links` (CI comments on PRs that break links; see `dev/check-links.mjs`; the build runs it without flags, so only dead page links fail a deploy). When moving a page or renaming a heading, update every link to it; a redirect in `src/data/redirects.ts` does not satisfy the check. Link to this site with relative paths (`/admin/config/site-config`), never `https://sourcegraph.com/docs/…` or `https://docs.sourcegraph.com/…`. To also probe the external links you added: `npm run check -- links --check-anchors --check-self-links --check-external --diff <(git diff -U0 origin/main)`
 -   **Prove changed links resolve on a deploy**: `node dev/verify-links-live.mjs --site <vercel-preview-url>` prints a Markdown table for the PR description
+-   **Vercel build failures**: Vercel shows build logs only to its team members, so `.github/workflows/vercel-build-report.yml` comments the log tail on the PR (see `dev/report-vercel-build.mjs`). It reads Vercel with the `VERCEL_TOKEN` repo secret, a token scoped to the `sourcegraph-docs` project that expires 2026-12-10; mint a new one with `POST /v3/user/tokens?teamId=<team>` and `projectId` in the body. It also attaches the full log to the Vercel Slack app's "failed to deploy" post in `#alerts-vercel-doc-site`, using the `SLACK_BOT_TOKEN` repo secret and `SLACK_CHANNEL_ID` repo variable. The bot is the Slack app in `dev/slack-app-vercel-build-report.json`; to recreate it, paste that manifest at <https://api.slack.com/apps?new_app=1> (From a manifest), install it, copy its Bot User OAuth Token into the secret, and `/invite @Vercel build log` to the channel
 
 ## AI Chat Integration