Fix screenshot CI and refresh Substrate baselines

.github/workflows/playwright-screenshots.yaml

@@ -53,6 +53,17 @@ jobs:
   mock:
     if: github.event_name == 'push' || inputs.capture == 'mock' || inputs.capture == 'both'
     runs-on: ubuntu-latest
+
+    # Without this the job captures correctly and then throws the result away:
+    # create-pull-request gets a read-only GITHUB_TOKEN and the branch push dies with
+    # `Permission to kagent-dev/website.git denied to github-actions[bot]` (exit 128).
+    # Every run between 2026-09-16 and 2026-09-23 failed exactly there. Same pair as
+    # update-ref-docs.yaml, which has been opening PRs from this bot all along -- so the
+    # org-level "allow Actions to create pull requests" setting was never the problem.
+    permissions:
+      contents: write
+      pull-requests: write
+
     steps:
       - name: Check out the website
         uses: actions/checkout@v7
@@ -101,9 +112,13 @@ jobs:
 
       - name: Publish into the docs img tree
         working-directory: website/${{ env.PW_DIR }}
-        # sync-docs exits nonzero when the cluster-captured baselines are absent, which
-        # they legitimately are in this job. Publish what this job captured and move on.
-        run: npm run sync-docs || true
+        # No `|| true`. sync-docs publishes from `__screenshots__/`, which is committed,
+        # so every mapped image resolves here even though this job captures only the chat
+        # pair -- the cluster baselines it also copies are the ones already in the tree.
+        # A nonzero exit therefore means a baseline is genuinely missing, which is a real
+        # failure and should fail the run. The guard that used to sit here claimed the
+        # cluster baselines were absent in this job; they never were.
+        run: npm run sync-docs
 
       - uses: actions/upload-artifact@v7
         if: always()
@@ -141,23 +156,37 @@ jobs:
   # ---------------------------------------------------------------------------------
   # Live-cluster captures.
   #
-  # DISPATCH ONLY, for one remaining reason: this is unverified on a hosted runner.
-  # Agent Substrate runs workers under gVisor nested inside the worker pod, and its own
-  # reference environment is a GKE node pool, so kind-in-Docker-on-a-runner is an open
-  # question.
+  # RUNS ON PUSH as of 2026-09-23. It was dispatch-only, and that combination was a silent
+  # trap: `provisioners/**` and both version conrefs sat on the push filter while the only
+  # job that reads them never ran on a push. So bumping `versions/kagent.md` to
+  # 1.0.0-alpha2 started a run, executed the mock job (which reads no versions at all),
+  # and left the three cluster screenshots showing Agent Substrate v0.0.26 for eight days.
+  # Nothing failed; the trigger just fired a job that could not act on it.
+  #
+  # It installs the published chart on purpose: CI exists to capture what a reader
+  # installs, and a source build would quietly defeat that.
   #
-  # The chart this installs now exists — `versions/kagent.md` pins a published 1.x chart —
-  # so the job no longer fails at the provisioner's kagent step. It installs the published
-  # chart on purpose: CI exists to capture what a reader installs, and a source build would
-  # quietly defeat that.
+  # Both jobs deliberately share one path filter rather than splitting per job. Over-
+  # triggering costs a ~15m no-op run; under-triggering is the bug above, and it is
+  # invisible. The refreshed images land under `__screenshots__/` and `assets/img/`,
+  # neither of which is on the filter, so a merged refresh PR cannot re-trigger this.
   #
-  # Promote it to the `push` trigger once a dispatch run has gone green, and move the
-  # provisioner and version-conref paths onto that trigger at the same time — today they
-  # sit on the push filter but can only affect this job.
+  # Still unproven on a hosted runner: Agent Substrate runs workers under gVisor nested
+  # inside the worker pod, and its reference environment is a GKE node pool. The full
+  # path was verified on kind under Docker locally on 2026-09-23, which removes the
+  # likeliest reason it would fail here but not the question itself.
   # ---------------------------------------------------------------------------------
   cluster:
-    if: github.event_name == 'workflow_dispatch' && (inputs.capture == 'cluster' || inputs.capture == 'both')
+    if: >-
+      github.event_name == 'push' ||
+      (github.event_name == 'workflow_dispatch' &&
+      (inputs.capture == 'cluster' || inputs.capture == 'both'))
     runs-on: ubuntu-latest
+
+    permissions:
+      contents: write
+      pull-requests: write
+
     steps:
       - name: Check out the website
         uses: actions/checkout@v7
@@ -169,10 +198,17 @@ jobs:
           node-version: ${{ env.NODE_VERSION }}
 
       - name: Install cluster tooling
+        # Pinned, not @latest. This workflow's whole premise is that nothing changes
+        # without a git event -- the same reasoning that pins @playwright/test exactly.
+        # kind decides the node's container runtime and default image, so a floating
+        # version can move pixels (or break the install) with no commit to point at.
+        # v0.32.0 is the version the 2026-09-23 local verification ran on.
+        env:
+          KIND_VERSION: v0.32.0
         run: |
-          go install sigs.k8s.io/kind@latest || curl -fsSL -o /usr/local/bin/kind \
-            https://kind.sigs.k8s.io/dl/v0.32.0/kind-linux-amd64
-          chmod +x /usr/local/bin/kind || true
+          curl -fsSL -o /tmp/kind \
+            "https://kind.sigs.k8s.io/dl/${KIND_VERSION}/kind-linux-amd64"
+          sudo install -m 0755 /tmp/kind /usr/local/bin/kind
           kind --version
 
       - name: Install kubectl-ate
@@ -194,7 +230,13 @@ jobs:
       - name: Stand up the cluster at the docs-pinned versions
         working-directory: website/${{ env.PW_DIR }}
         env:
-          OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
+          # The provisioner only tests that this is non-empty before writing it into a
+          # ModelProviderConfig, and these captures (dashboard, agents list, substrate
+          # page on a fresh install) invoke no model. So a placeholder is correct here,
+          # not a workaround -- and it keeps a missing or rotated secret from failing a
+          # run that never needed one. (kagent's `make check-api-key` tests the same
+          # emptiness, but it is a Makefile target and nothing in this job runs it.)
+          OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY || 'sk-placeholder-for-screenshots' }}
         run: ./provisioners/kagent-kind.sh
 
       - name: Install the harness
@@ -218,7 +260,10 @@ jobs:
 
       - name: Publish into the docs img tree
         working-directory: website/${{ env.PW_DIR }}
-        run: npm run sync-docs || true
+        # No `|| true`, for the reason spelled out on the mock job's copy of this step:
+        # sync-docs publishes committed baselines, so every mapped image resolves in
+        # either job and a nonzero exit is a real failure.
+        run: npm run sync-docs
 
       - uses: actions/upload-artifact@v7
         if: always()

docs-site/content/kagent/1.x/observability/launch-ui.md

@@ -107,8 +107,8 @@ Two controls under the transcript branch the conversation rather than continue i
 
 The **Substrate** page shows whether there is capacity for an agent to run. It reads WorkerPools and ActorTemplates from Kubernetes, and live Actors and Worker assignments from the Agent Substrate API.
 
-{{< reuse-image-light src="img/kagent-ui-substrate.png" alt="The Substrate page, showing one worker pool and an idle worker" caption="Figure: The Substrate page" >}}
-{{< reuse-image-dark srcDark="img/kagent-ui-substrate-dark.png" alt="The Substrate page, showing one worker pool and an idle worker" caption="Figure: The Substrate page" >}}
+{{< reuse-image-light src="img/kagent-ui-substrate.png" alt="The Substrate page, showing one worker pool and no running actors" caption="Figure: The Substrate page" >}}
+{{< reuse-image-dark srcDark="img/kagent-ui-substrate-dark.png" alt="The Substrate page, showing one worker pool and no running actors" caption="Figure: The Substrate page" >}}
 
 Read the page from the top tiles down.
 
@@ -118,11 +118,12 @@ Read the page from the top tiles down.
 | **Templates ready** | {{< gloss "ActorTemplate" >}}ActorTemplates{{< /gloss >}} that have compiled and are ready to be instantiated. |
 | **Actors running** | Live Actors, each one an AgentInstance holding a conversation. |
 | **Workers busy** | {{< gloss "Worker" >}}Workers{{< /gloss >}} currently assigned to an Actor, against the total provisioned. |
+| **Scope** | The filters this page asked for, rather than what the API returned. Reads `all` until you set a filter, then `K8s: <namespace>; ATE: <atespace>`. |
 
-A fresh installation reports one worker pool, no actor templates, and no actors, because nothing has created an agent yet. The Workers table still lists the pool's worker as `idle`. That idle worker confirms that capacity is provisioned and waiting for the first agent.
+A fresh installation reports one worker pool, no actor templates, and no actors, because nothing has created an agent yet. The **Workers busy** tile reads `0/1`, and the Workers table lists the pool's single worker with the pod that backs it. That unassigned worker confirms that capacity is provisioned and waiting for the first agent. The Actors table reads `No actors on this page.`
 
 > [!NOTE]
-> An empty Actors table on a cluster that does have agents indicates an issue at the level of the Agent Substrate API rather than at kagent. The page reports `ate-api reported no actors in this scope` when it reaches the API and gets an empty answer, so check the scope selector before investigating further.
+> When the Substrate page cannot reach the Agent Substrate API, it reports the failure rather than an empty result. A warning above the tables carries the error, and the Actors table reads `This read could not reach ate-api, so there may be actors it did not see.` On a cluster that does have agents, `No actors on this page.` means that the read succeeded and matched nothing, so check the **Kubernetes namespace** and **ATE atespace** filters.
 
 ## Expose the UI outside the cluster
 

docs-site/playwright/README.md

@@ -229,19 +229,33 @@ Screenshots are pixel-compared, so captures must be stable across runs.
 
 | Capture | Source | Date |
 | --- | --- | --- |
-| `kagent-ui-chat` | kagent UI mock backend, `yarn dev` at kagent `42d3301d` | 2026-09-15 |
-| `kagent-ui-dashboard` / `-agents` / `-substrate` | **source build** at kagent `42d3301d`, chart `helm/kagent` from that checkout, on kind 1.37.0 | 2026-09-15 |
-
-**The three cluster captures did not come from the published chart.** They predate it: when
-they were taken, the chart the 1.x docs pin had no published tag, so they came from a source
-build through a `KAGENT_CHART_DIR` mode this harness no longer carries. They show a real,
-fully working install (Agent Substrate `0.0.26` published charts, one ready gVisor worker,
-189 discovered tools, zero agents), so they are honest about what a fresh install looks like.
-They are not honest about *which build* a reader gets.
-
-**Re-capture all three from the published chart.** `versions/kagent.md` now pins a 1.x
-chart that publishes and pulls, so the cluster job can install exactly what a reader installs.
-The job has not yet had a green dispatch run, so these captures still stand.
+| `kagent-ui-chat` | kagent UI mock backend, `vite` at kagent tag `v1.0.0-alpha2` (`373b56be`) | 2026-09-15, re-verified 2026-09-23 |
+| `kagent-ui-dashboard` / `-agents` | **published charts** kagent `1.0.0-alpha2` + Agent Substrate `0.2.0-beta5`, kind 1.37.0 | 2026-09-15, re-verified 2026-09-23 |
+| `kagent-ui-substrate` | **published charts** kagent `1.0.0-alpha2` + Agent Substrate `0.2.0-beta5`, kind 1.37.0 | 2026-09-23 |
+
+**The cluster captures now come from the published chart**, which closes the caveat these
+notes carried since 2026-09-15. A fresh `kagent-shots` cluster installed exactly what
+`versions/kagent.md` and `versions/agent-substrate.md` pin, and the capture ran against that.
+
+**`-dashboard` and `-agents` were re-verified, not rewritten.** They still match byte-for-byte
+against the published chart, so those two surfaces did not move between the 2026-09-15 source
+build and `1.0.0-alpha2`, and the old images were honest after all. Only `-substrate` changed
+(see below), so only that pair carries the new date.
+
+**What moved on the Substrate page at `1.0.0-alpha2`** — the page grew 1247px to 1308px, and
+these are the changes a reader sees:
+
+- the single **Scope** selector became two filters, **Kubernetes namespace** and **ATE atespace**;
+- the **Actor templates** table dropped its **Harness** column;
+- the **Workers** table replaced its **Actor** column (which read `idle`) with **IP**;
+- the Actors and Workers panels lost their per-panel search boxes, gained `N on this page`
+  counts and a `read just now` line, and the empty state changed from
+  `ate-api reported no actors in this scope` to `No actors on this page.`;
+- the page description dropped the internal name `ate-api` in favour of `Substrate`.
+
+Three of those falsified prose in `observability/launch-ui.md`, which was corrected in the
+same change. **`read just now` is a new volatile string** — it has held across re-captures so
+far, but it is the thing to suspect first if this capture ever starts flapping.
 
 The chat capture needs no such caveat going forward: it will always come from source,
 because the released image ships no mock service worker.
@@ -253,11 +267,15 @@ because the released image ships no mock service worker.
   theme-aware pair. Version is not a baseline axis because only 1.x uses these images (see
   "If a version line ever diverges"). Adding the version to a project name renames every
   committed baseline, so leave it until a second version is genuinely being captured.
-- **`@playwright/test` is pinned exactly (`1.49.0`, no caret).** CI triggers on input changes
-  rather than on a cron, and that reasoning only holds if nothing can change without a git
-  event. A caret would let `npm ci` pull a new minor with a different Chromium, shifting pixels
+- **`@playwright/test` is pinned exactly (no caret).** CI triggers on input changes rather
+  than on a cron, and that reasoning only holds if nothing can change without a git event. A
+  caret would let `npm ci` pull a new minor with a different Chromium, shifting pixels
   silently. The agentgateway harness gets away with `^1.49.0` only because it re-captures
-  nightly, which turns a browser bump into a reviewable PR.
+  nightly, which turns a browser bump into a reviewable PR. The pin works as designed:
+  Dependabot moved it 1.49.0 -> 1.63.0 on 2026-09-23 as a reviewable commit, and `package.json`
+  is on the workflow's path filter so the bump triggers a re-capture. That bump shifted no
+  pixels beyond tolerance — all eight baselines passed under the new Chromium before anything
+  was rewritten — so a browser bump is not automatically a re-baseline.
 - **`workers: 1`.** The specs drive a real cluster and captures can create resources, so
   concurrent runs would race each other.
 - **There is no `update:all`.** The two specs need different backends, so a single command that
@@ -267,17 +285,39 @@ because the released image ships no mock service worker.
 
 ## CI
 
-`.github/workflows/playwright-screenshots.yml` — `workflow_dispatch` plus a push to the docs
-branch touching any input that determines the screenshots (the version conrefs, the specs, the
-fixtures, the config, the provisioner).
+`.github/workflows/playwright-screenshots.yaml` — `workflow_dispatch` plus a push to `main`
+touching any input that determines the screenshots (the version conrefs, the specs, the
+fixtures, the config, the provisioner). **Both jobs run on a matching push** as of 2026-09-23.
 
 **Deliberately no cron.** The chart versions are pinned to exact docs releases and released
 charts are immutable, so a fixed version plus a fixed fixture renders identically every run.
 Nothing changes until an input file changes, and that is a git event — so the workflow triggers
 on the event rather than polling. A cron here would produce a nightly no-op, or a stream of
 empty PRs.
 
-Needs a model provider API key as an Actions secret, and a token that can open PRs.
+**One path filter serves both jobs, on purpose.** Over-triggering costs a ~15m no-op; under-
+triggering is invisible, and it already bit once — see below. Refreshed images land under
+`__screenshots__/` and `assets/img/`, neither of which is on the filter, so a merged refresh
+PR cannot re-trigger the workflow.
+
+Needs no secrets. A model provider key is read from `secrets.OPENAI_API_KEY` if one is set, but
+the job falls back to a placeholder, which is correct rather than a workaround: the chart's
+`check-api-key` only tests for non-emptiness and these captures invoke no model.
+
+### Two failures worth not re-introducing
+
+- **The cluster job was `workflow_dispatch`-only while `provisioners/**` and the version conrefs
+  sat on the `push` filter.** Only that job reads them, so a version bump started a run,
+  executed the *mock* job — which reads no versions at all — and finished green while the three
+  cluster screenshots stayed eight days stale. Nothing failed; the trigger fired a job that
+  could not act on it. **A path filter is only as good as the job it can reach.**
+- **Neither job declared `permissions:`,** so `create-pull-request` got a read-only
+  `GITHUB_TOKEN` and every run died at the branch push with `Permission to
+  kagent-dev/website.git denied to github-actions[bot]`. The capture itself succeeded every
+  time and the result was discarded, which is why it read as a screenshot problem. Both jobs now
+  declare `contents: write` + `pull-requests: write`, the same pair `update-ref-docs.yaml` has
+  always used — that workflow opens PRs from this bot routinely, so the org-level "allow Actions
+  to create pull requests" setting was never the cause.
 
 ## Files