Automate multiversion documentation publishing to clawpack.github.com

[mandli]

Replaces the manual make versions → cp -r v5.7.x/* . → fix_links_top_level.py → rsync → git commit procedure with a GitHub Actions workflow, and fixes several latent bugs in that procedure along the way.

The published site (www.clawpack.org) keeps clawpack.github.com@master as its served source and stays on legacy branch-source Pages — this PR only automates the build and the push into it.

Why now

Chasing a deploy-pages failure (No artifacts named "github-pages" were found) turned up that it came from GitHub's auto-generated dynamic/pages/pages-build-deployment in the site repo, not from anything we wrote: someone had clicked "Re-run failed jobs" 27 days after the build, so the artifact had expired. A full re-run fixed it. But the underlying situation was that publishing is entirely manual, undocumented in places, and had silently broken in ways nobody noticed — including a live 404.

Bugs fixed

• make versions produced no site root. howto_doc.rst documented two mandatory post-build steps (cp -r <version>/* . plus a link fix) that existed nowhere in the Makefile. Now make versions-publish does the whole job.
• The promote step skipped dotfiles. cp -r <version>/* never matched .nojekyll; the site root has one only because nothing ever deleted it. Without it Pages stops serving _static/_sources and every page loses its CSS.
• Version links were broken below depth 1. fix_links_top_level.py only rewrote *.html, riemann/*.html and pyclaw/*.html. Confirmed live: www.clawpack.org/pyclaw/evolve/limiters.html serves ../../../dev/…, one ../ too many, which escapes the site root. The new build produces ../../dev/….
• v5.14.x silently vanished from local builds. sphinx-multiversion only reads refs/heads/* and refs/tags/*, ignoring remote-tracking refs, and git clone/actions/checkout create just one local branch. Since v5.14.x is smv_latest_version, a build without it has no top-level pages at all. make check-versions now catches this and prints the git branch command to fix it.
• smv_branch_whitelist was unanchored. re.match anchors only the start and . is a wildcard, so a branch named dev-experiment would have been built and published. Now ^(dev|v5\.14\.x)$.
• _build/ was not gitignored (only _build1 was), so ~400 MB of build output could be committed by a stray git add ..

Two constraints discovered while implementing

• sphinx-multiversion 0.2.4 cannot run on Sphinx 9. It calls Config.read(confdir, overrides) positionally; Sphinx 9.0 made those keyword-only. Boundary verified empirically: works through 8.2.3, breaks at 9.0.0. requirements-docs.txt now pins sphinx>=7.0,<9. Migrating off the (unmaintained) extension is listed as future work in howto_doc.rst.
• Only 9 of the 15 whitelisted refs are buildable. Tags v5.1.x–v5.6.x reference a plot_directive extension that no longer resolves, and sphinx-multiversion drops any ref whose conf.py fails to load with only a stderr line and a zero exit status. Those six are recorded in KNOWN_UNBUILDABLE so a new drop still fails the check, and they're in the publish preserve-list because their long-published HTML can no longer be regenerated.

The publish path is deliberately hard to make destructive

The site repo is not a pure build artifact — it holds ~20 top-level directories no build produces (gallery/, doxygen/, pdf/, notebooks/, v5.1.x–v5.6.x, …) plus hand-written root files, and pyclaw/ is only partly build-owned. So:

• The sync is additive; --delete is confined to version directories the build actually regenerated, and only on explicit opt-in.
• check_published_tree.sh asserts, before any commit, that nothing unmanaged changed, that CNAME/.nojekyll survived, and that the root is a promoted build.
• No force-push, no --orphan. peaceiris/actions-gh-pages and JamesIves/github-pages-deploy-action were rejected: their defaults delete everything absent from the publish dir and force-push.
• Publishing waits on the clawpack-org-website environment, so a human approves each write to the live site with the diff and the rollback SHA in the run summary.
• Pages stays on build_type: legacy: the published tree is ~1.6 GiB, over the 1 GB documented for Pages, and workflow mode enforces that cap at deploy time.

PRs build a single version only — sphinx-multiversion builds from committed refs via git archive, so make versions on a PR would show reviewers the site without the PR's changes.

Verification

• Clean make versions-publish builds 9 versions, promotes v5.14.x, rewrites 190 of 212 promoted files; check_built_site.sh passes. Order holds under make -j4.
• 19 tests: 5 in test_promote_latest.py (link rewriting at depths 0/1/2, dotfile carry, version dirs left alone, error paths) and 14 in test_check_published_tree.sh (the additive sync is accepted; root --delete is refused, including the case where it eats .git; --prune can't rescue it; modified unmanaged paths, bad CNAME, missing .nojekyll, unpromoted root and partial builds all refused).
• actionlint and shellcheck clean. 395 MB of _build stays out of git status.

Follow-up needed before this publishes anything

The workflow is inert until someone does two things: create a write-enabled deploy key on clawpack.github.com stored as CLAWPACK_SITE_DEPLOY_KEY, and create the clawpack-org-website environment with required reviewers. Then the rollout is scope: dry-run → target_branch: ci-preview (Pages doesn't serve it) → master.

Please run the ci-preview publish twice before pointing it at master — the second run must produce an empty diff. That's what proves the build is byte-reproducible, and it's what keeps a 1.5 GB repo from growing by ~177 MB on every push to dev.

Also note: make checkwarnings has never actually run in CI (this repo had no Actions history), and the baseline now needs regenerating under the sphinx<9 pin via the workflow's update_baseline dispatch.

Assisted-by: claude claude-opus-5