From c5dbe85f0227752b472aebb3c20e32860c96a90e Mon Sep 17 00:00:00 2001 From: Buck Doyle Date: Thu, 10 Sep 2026 12:34:50 -0400 Subject: [PATCH] Baseline every Percy build against main MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Percy picks a build's baseline from git history when nothing tells it otherwise, so a branch stacked on another feature branch is compared against its parent rather than against main — and inherits whatever that parent's most recent build happened to contain. A parent build that lost snapshots is the case that hurts. The child compares its full set against a partial one, every snapshot with no counterpart in the baseline reads as new, and the build lands with most of the suite needing review. One such build compared 59 snapshots against a four-snapshot baseline on a sibling branch and reported 55 changes, none of them real. The reject step already keeps a partial build from becoming a baseline on `main`. It does not cover a partial build on a feature branch, which is not rejected and can still be inherited by anything stacked on it. Pinning the target branch closes that path without needing to. It also means a stacked PR's diff shows its cumulative effect against main, which is what an unstacked branch already gets. Co-Authored-By: Claude Opus 5 --- .github/workflows/ci-host.yaml | 17 +++++++++++++++++ 1 file changed, 17 insertions(+) diff --git a/.github/workflows/ci-host.yaml b/.github/workflows/ci-host.yaml index ae8bbd85507..15e6d102868 100644 --- a/.github/workflows/ci-host.yaml +++ b/.github/workflows/ci-host.yaml @@ -696,6 +696,23 @@ jobs: # failure) is visible in the shard log for each snapshot. PERCY_LOGLEVEL: debug PERCY_TOKEN: ${{ needs.check-percy.outputs.percy_needed == 'true' && secrets.PERCY_TOKEN_HOST || '' }} + # Compare against main, not against whatever branch this one was cut + # from. Left unset, Percy infers the baseline from git history, so a + # branch stacked on another feature branch baselines against its + # parent and inherits whatever that parent's last build contained. + # + # A parent build that lost snapshots is the bad case. The child + # compares its full set against a partial one, every snapshot with no + # counterpart reads as new, and most of the suite lands needing review + # for a reason nothing on the build explains. The reject step below + # keeps a partial build from becoming a baseline on `main`, but a + # partial build on a feature branch is not rejected and can still be + # inherited by anything stacked on it. + # + # Set here rather than on the finalize step: the baseline is chosen + # when the build is created by the first shard to upload, and finalize + # only seals it. + PERCY_TARGET_BRANCH: main PERCY_PARALLEL_NONCE: ${{ github.run_id }}-${{ github.run_attempt }} HOST_TEST_PARTITION: ${{ matrix.shardIndex }} HOST_TEST_PARTITION_COUNT: ${{ matrix.shardTotal }}