feat(templates): serve stale gallery cache immediately, revalidate in background (BE-3427)

[mattmillerai]

ELI-5

comfy templates ls/show/fetch reads its catalog from a small JSON file cached at ~/.cache/comfy-cli/gallery/index.json. PR #559 (BE-3393) gave that cache a 24h expiry, but past the expiry it re-fetches synchronously — so on an offline/firewalled machine every templates command hangs for the full 15s fetch timeout before falling back to the stale cache, once per invocation past the TTL.

This makes it stale-while-revalidate: a stale-but-present cache is served immediately, and the refresh happens in a detached background process that updates the cache for the next invocation. The current call never blocks on the network — online or offline.

Stacked on #559 (BE-3393)

This is a stacked PR: base = matt/be-3393-gallery-cache-ttl (the TTL + synchronous stale-fallback foundation, still open). It builds directly on that branch and changes its stale-path behavior from synchronous to SWR — explicitly the follow-up BE-3427 that #559 scoped out. GitHub will retarget this to main when #559 merges; review sees only the net SWR diff.

What changed

• _load_gallery — a TTL-expired but present cache is now returned right away, and _spawn_background_refresh() is fired to revalidate it. No inline fetch on this path.
• _spawn_background_refresh — launches a fully detached comfy templates _refresh-cache (start_new_session=True, stdio → /dev/null, spawn failure swallowed), mirroring the existing comfy run async job-watcher idiom (command/run/watcher.py). It outlives the parent without ever blocking it.
• _refresh-cache (hidden command) — fetch + atomic persist only; no output, no telemetry, never a non-zero exit. It validates the body before persisting and swallows every error, so a bad/garbage/offline response can never clobber the last-known-good cache.
• Staleness signal — a non-fatal renderer.warn on stderr ("serving the cached copy and refreshing in the background"), consistent with the foundation's existing fallback warning. The JSON envelope shape is unchanged (no per-payload stale field added).

Design decision (the ticket asked for one)

Detached subprocess, not a thread. CLI processes are short-lived, which rules out both thread options:

• A daemon thread is killed at interpreter exit before the ~1s online fetch+write completes → the cache would almost never refresh: a regression from the foundation's reliable auto-refresh.
• A non-daemon thread keeps the process alive until the fetch returns → on an offline machine it lingers up to the full 15s timeout after emitting output, every invocation. That just relocates the exact hang we're removing (and is just as bad for the agentic JSON callers this CLI targets, which wait on process exit).

A fully detached subprocess is the only option that returns the parent instantly in both the online and offline cases and reliably persists the refreshed cache for the next call. It's also the pattern already used in this repo for comfy run's async watcher. The "next invocation" refresh the ticket mentions is exactly what the detached child delivers — it writes the cache that the next ls/show/fetch reads.

Behavior preserved (Chesterton's fence)

• --refresh stays synchronous and still surfaces fetch errors (fatal gallery_load_failed) — an explicit user request is not deferred.
• No cache at all still fetches synchronously and is fatal on failure (nothing to serve).
• An unreadable/corrupt stale cache falls through to the synchronous path rather than serving garbage.
• The "don't clobber the last-good cache with a bad body / non-200 / offline" guarantees from feat(templates): add 24h TTL with stale-cache fallback to gallery cache (BE-3393) #559 are preserved — they now live in the _refresh-cache background entrypoint (which uses the same atomic _persist_cache + validate-before-write).

Tests

tests/comfy_cli/command/test_templates.py (updates the #559 TTL tests that asserted synchronous refetch, since the stale path is now SWR):

• expired cache → serves stale immediately, spawns background refresh, no inline fetch (_fetch_gallery monkeypatched to fail loudly), cache untouched by the foreground.
• future-mtime clock-skew → treated as stale: served + revalidated in background.
• _refresh-cache entrypoint → persists a fresh index; keeps the stale cache on fetch failure / garbage-200 / non-200; never exits non-zero.
• _spawn_background_refresh → asserts the detached spawn shape (argv ends templates _refresh-cache, start_new_session=True, stdio → DEVNULL); swallows OSError spawn failure.
• --refresh synchronous-path tests (fatal on failure, read-only-dir resilience) retained.
• Verified live end-to-end: hidden command routes via python -m comfy_cli; a stale cache serves in ~1s (no 15s block) with the staleness warning on stderr.

Full suite green locally (2587 passed, 37 skipped); ruff check + ruff format --check clean on the touched files.

Negative-claim falsification

Not applicable — this diff adds a capability (opportunistic background refresh) and makes stale-serving more available; it introduces no "unsupported"/deny/dead-end path. The staleness warning is a positive affordance, not a capability denial.

Judgment calls / known minors

• Rapid-fire invocations within the ~1s refresh window can each spawn a background refresher. They all os.replace atomically (last-writer-wins, no corruption); once the first completes, the cache is fresh again and subsequent calls don't spawn. Not worth a lockfile for this path.
• The ttl_auto_refresh synchronous fallback block below the SWR branch is now only reachable for the corrupt-stale-cache edge; left intact to minimize churn on the foundation's structure.
• comfy-cli uses (BE-####) PR/commit refs (per feat(templates): add 24h TTL with stale-cache fallback to gallery cache (BE-3393) #559 and repo history), so this follows that convention rather than the no-Linear-ref guardrail that applies to the ComfyUI/ComfyUI_frontend public repos.

[coderabbitai]

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: 8e7ec8a5-5685-4145-8c0f-55012ba2451b

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch matt/be-3427-templates-stale-while-revalidate

✨ Simplify code

• Create PR with simplified code
• Commit simplified code in branch matt/be-3427-templates-stale-while-revalidate

---

_{Comment @coderabbitai help to get the list of available commands.}

[github-actions]

🔍 Cursor Review — Consolidated panel

Triggered by @mattmillerai.

Found 8 finding(s).

Severity	Count
🟠 High	2
🟡 Medium	3
🟢 Low	2
⚪ Nit	1

Panel: 6/8 reviewers contributed findings.

Reviewers that did not contribute: kimi-k2.5:adversarial (empty), kimi-k2.5:edge-case (empty)