main-pinned-cowns

[matajoh]

Main-pinned cowns — a new PinnedCown subclass holds its value as a plain PyObject * on the main interpreter, never round-tripped through XIData. Behaviors whose request set contains any pinned cown are routed by the scheduler to a single-consumer main-thread queue and drained by the new pump entry point (or implicitly by wait, which auto-pumps when pinned cowns exist). Designed for objects that cannot survive cross-interpreter shipping — pyglet shapes, Tk widgets, GPU contexts, open file handles, ctypes pointers. The companion examples/boids.py rewrite demonstrates the coarse-grained pinned-dispatch pattern: per-cell physics stays on workers, and one @when(PinnedCown) per frame batches the write-back into main-thread matrices.

New Features

• PinnedCown(Cown[T]) — a cown whose value lives permanently on the main interpreter. Constructible only from the main interpreter (raises RuntimeError from workers); the value is never picklable, never reified twice, and never reconstructed in a worker. The capsule handle remains a first-class cross-interpreter shareable — workers may hold it, embed it in a regular Cown value graph, and place it in noticeboard entries, but only the main thread may acquire the value. See the new pinned_cowns page for the full contract and the coarse-grained-dispatch pattern.
• pump(deadline_ms=None, max_behaviors=None, raise_on_error=False) — drains the main-thread queue of behaviors whose request sets contain a PinnedCown. Call from your event loop's idle / on-tick hook (pyglet schedule_interval, Tk after, asyncio task, …); script-mode programs need not call it explicitly because wait pumps internally. Non-preemptive: deadline_ms gates starting the next behavior, not interrupting one already running. Body exceptions default to landing on the result cown's .exception; raise_on_error=True re-raises the first body exception after drain. Returns a new PumpResult NamedTuple (executed, deadline_reached, raised).
• set_pump_watchdog(warn_ms=1000, raise_ms=None, on_starve=None) — configure the pinned-queue starvation watchdog. Both thresholds gate on queue-non-empty time, not raw last-pump time, so programs running only unpinned work never trip them. Default is warn-only; users opt into fail-fast via an explicit raise_ms so interactive debugger sessions are not wedged by a breakpoint.
• set_wait_pump_poll(ms=50) — set the poll cadence for wait's auto-pump loop. Re-read every iteration so a concurrent call updates the active wait immediately.
• bocpy.PumpResult — three-field NamedTuple returned by pump. executed counts pinned behaviors whose lifecycle completed (including acquire-failure paths whose MCS chain still drained). deadline_reached is True only when the deadline_ms budget tripped before the queue drained. raised counts only body exceptions captured to a result cown (cleanup-path failures use PyErr_WriteUnraisable and do not count). Exported from bocpy.__all__.
• Coarse-grained pinned-dispatch examples/boids.py — the per-cell send("update") / main-thread receive("update") barrier is replaced by per-cell physics on workers plus one pinned @when per frame that captures every per-cell result cown together with the two main-thread PinnedCown matrices and performs the batched write-back. Same visual output, fully worker-parallel per-cell work, single main-thread touchpoint.

Public C ABI

• bocpy_main_interpid() — new static inline helper in <bocpy/bocpy.h> returning PyInterpreterState_GetID( PyInterpreterState_Main()) pre-typed as int_least64_t to match bocpy_interpid for owner-field equality checks. Safe to call from a worker sub-interpreter for diagnostic / assert use. Additive — existing consumers recompile unchanged; BOCPY_ABI is unchanged at 1. The templates/c_abi_consumer bocpy~= pin moves to ~=0.9 to signal the new ABI surface it was authored against.

Improvements

• @when loop-variable snapshot via default arg — the transpiler now accepts def b(c, i=i) as an explicit loop-snapshot idiom in addition to the existing implicit form (just reference the loop variable in the body). Trailing positional parameters beyond the cown count are also auto-captured by name (def b(c, factor) captures factor).
• @when alias decorators — the transpiler now recognises from bocpy import when as boc_when and import bocpy [as alias] followed by @bocpy.when(...) or @alias.when(...), provided the aliasing import is at module level. Previously only the bare @when form was detected.
• Behaviors.start() compiles the export module on main — the transpiler's rewritten module is now also instantiated as an in-memory types.ModuleType on the main thread (plus a linecache entry for traceback fidelity) so pump can resolve __behavior__N the same way workers do via their bootstrap.
• Scheduler-owned behavior pre-header — bq_node and the new pinned OR-fold byte moved out of the opaque BOCBehavior into a scheduler-owned boc_behavior_prehdr_t allocated immediately before each behavior (CPython _PyGC_Head style). boc_sched.c no longer needs any knowledge of BOCBehavior's internal layout; layout drift between the scheduler and its users is impossible by construction.
• terminator_wait_pumpable — new entry in boc_terminator.{c,h} lets the auto-pump loop wake on either count-zero or main-pinned-depth-becoming-non-zero, both wired through the existing single condition variable. Single-pumper enforcement on free-threaded builds (Py_GIL_DISABLED) lives alongside via a MAIN_PUMP_THREAD CAS that raises RuntimeError if a second thread tries to pump concurrently, cleared on every exit path including BaseException.

Bug Fixes

• Transpiler except ... as X mis-classification — ~ast.ExceptHandler binds X on the handler node itself rather than via ~ast.Name Store, so the transpiler's free-variable walker mis-classified any read of X inside the handler body as a free variable, appended it as a behavior parameter, and emitted a call site that referenced an out-of-scope name. Fixed by a new visit_ExceptHandler hook that registers X as a local before recursing into the handler. Regression locked by TestCapturedLocals::test_except_as_name_excluded.

Documentation

• New pinned_cowns page — concept and when to use, PinnedCown / pump / PumpResult / set_pump_watchdog / set_wait_pump_poll API, coarse-grained pinned-dispatch pattern, event-loop integration recipes (pyglet, Tk, asyncio), the queue-non-empty-time watchdog contract, free-threaded single-pumper rule, and free-threaded support trajectory. Linked from the root toctree.
• api expanded with the new PinnedCown / pump / PumpResult / set_pump_watchdog / set_wait_pump_poll entries.
• New "Talking to main-thread objects" subsection in the root README.md's "A taste of BOC" with a 10-line pyglet snippet illustrating the coarse-grained pattern; the public-API list picks up the five new symbols.
• examples/README.md calls out the rewritten boids.py and the new examples/benchmark.py --pinned-spinner flag.

Tests

• test/test_pinned_pump.py — new module covering the full PinnedCown / pump matrix: pure-pinned, mixed request sets, off-main construction rejection, locked error-string smoke tests, deadline_ms / max_behaviors bounding, body exceptions under default and raise_on_error=True, wait() auto-pump, shutdown drain via drop-exceptions, the watchdog warn-only and explicit-raise paths, the QUEUE_NONEMPTY_SINCE regression for unpinned-only workloads, hypothesis fuzz over mixed request sets, PinnedCown-handle round-trip through closure capture and through the noticeboard, Cown(PinnedCown) interop, and an acquire-failure fault-injection test that proves IN_PUMP_BODY / terminator_dec / MAIN_PUMP_THREAD cleanup runs on every exit path.
• test/test_transpiler.py — 192 new lines covering the def b(c, i=i) loop-snapshot form, @when alias decorators, and the except ... as X regression.

Internal

• examples/benchmark.py --pinned-spinner — high-rate pinned-dispatch overlay that adds one tail-recursing @when(PinnedCown) driven by pump(max_behaviors=1) on the main thread at a configurable rate while the existing chain-ring workload runs on workers. Used during development to verify worker-throughput regression under high-rate pinned dispatch; on CPython 3.14 at 4 workers / 10 s / 3 repeats the measured delta with the spinner active was −0.38%.
• Noticeboard read contract tightened — noticeboard now explicitly documents that calling noticeboard or notice_read from the main thread outside a behavior is undefined behavior; the supported main-thread read path is wait(noticeboard=True). Seeding the noticeboard with notice_write from the main thread before scheduling any behavior remains supported.
• test_matrix.TestVectorMethodsInCown migrated to the send("assert", ...) pattern — the in-cown Matrix vector tests previously asserted on result.value directly from the test thread, which violates the cown ownership contract. They now ship assertions out of each behavior via send("assert", ...) and collect on the test thread via a receive_asserts(count) helper, matching the project's BOC testing convention.

Closes #20