Conversation
…must pre-arm its next segment (red) Medina sala1, 2026-09-30: ctrl go -> ctrl PAUSE -> node go -> node pause. The load-time walk stops at the ctrl pause, GO 1 only advances the node's pointer, and GO 2 finds the node's first cue unarmed and fires it late (240 ms at sala1; 120 ms reproduced on test2 with the full 869f79ecc build). Also pins the shared go-chain walk helper, including skipping a local-but-disabled cue that the load walk used to stop at. (cherry picked from commit 7de2e0e)
…nter with nothing local Medina sala1, 2026-09-30: ctrl go -> ctrl PAUSE -> node go -> node pause. The node never pre-armed its first cue, so it fired late on every run (240 ms at sala1; 120 ms reproduced on test2 with the full 869f79ecc build, which does NOT cover this shape): - load/STOP pre-arm walks only the post_go='go' chain and stops at the controller's pause -> nothing armed; - go_script's no-local branch only advanced next_cue_pointer; - the controller never forwards setnextcue on an advance, and the UI and power-bridge GO are bare GOs, so 869f79ecc's PreArm never ran. Now go_script's no-local branch calls _prearm_after_advance(): it arms, synchronously, exactly what the NEXT GO will dispatch on this node, and runs the lookahead on the PreArm:<id> thread with set_next_cue's project-generation / selection-epoch guards. When the next GO is another node's too, nothing is armed yet -- no whole-show preload. The go-chain walk is shared as BaseEngine._first_local_enabled_in_go_chain (same predicate as go_script: _local AND enabled). The load walk uses it too, which also fixes it stopping at a local-but-disabled cue that arm() then refused (nothing pre-armed). (cherry picked from commit 354c7ec)
…_lock (red) A STOP or the next GO sent while a node pre-arms its next segment after a pointer advance must not wait for that arm: an audio target spawns a player and waits for its JACK ports (0.4 s measured, ~15 s on a failed port). Same arm-inside-the-lock shape 869f79ecc removed from set_next_cue's lookahead.
… under _command_lock _prearm_after_advance armed its target synchronously inside run_command's _command_lock, so an operator STOP (or a fast next GO) waited for the whole arm -- audioplayer spawn + JACK port wait. Move the target's arm onto the PreArm:<id> thread (_prearm_segment) ahead of the existing lookahead, with the same project-generation / selection-epoch guards; an arm that completes after a different project loaded is undone, as _prearm_lookahead undoes its own walk's. A GO that lands mid-arm waits on arm()'s _loading event rather than double-arming.
…d (red) STOP keeps the same script but bumps the project generation, kills every audio player and disarms all. A PreArm thread that finishes arming after it may leave a cue marked armed with a dead player; only a script change was checked. The target's arm() recursion also ran with no walk, so its post_go / action-target arms ignored the abort guards and went unrecorded. Covers the target arm and the existing lookahead cleanup.
…target's recursion - _prearm_segment arms the target with an _ArmWalk, so arm()'s own post_go / action-target recursion obeys the same generation/epoch guards and every cue it arms is recorded. - _undo_stale_prearm (shared by _prearm_segment and _prearm_lookahead) disarms what the thread armed when the script changed OR the project generation moved (STOP / load), not only on a script change. The next GO's safety-net re-arm is late; a dead player would be silent. rc_1 port: disarm() takes a reason here -- "project_changed" when the script changed, "stop" when only the generation moved; the two test asserts on disarm() carry it.
…e cue itself (red)
arm() excludes concurrent arms with a per-OBJECT Event (cue._loading), but
what an arm creates is keyed by cue ID: the tracked player, the JACK client
name Audio_Player-<uuid>, the video layer ids -- and Cue.__eq__/__hash__ are
by id. After reloading the same project the old and the new object of one
cue arm concurrently and step on each other.
A thread that waited on another thread's arm also just returns that arm's
result: when it did not load the cue, go() gives up ("cannot GO").
First step of the redesign of the stale pre-arm cleanup (869f9wqpn): PR #22's
_undo_stale_prearm could disarm a cue the next run was already playing.
Plan: cuems-RELATIONS Plans/2026-09-30-engine-prearm-publish-or-abandon.md
…lf if the arm it waited on failed - arm()'s in-flight marker moves from the cue object (cue._loading) to a handler registry keyed by cue id. What an arm creates is keyed by id (the tracked player, the JACK client name, the video layer ids) and Cue equality is by id, so the old and the new object of one cue must exclude each other. The claim records its holder and start time, and is released in the finally, before the post_go / action-target recursion. - An init caller that waited re-evaluates instead of returning the other arm's result: still unarmed -> it arms the cue itself. One total wait budget per call (_ARM_WAIT_TIMEOUT_S, the existing 5 s); once it is spent the caller does not arm. The timeout warning names the holder and how long it has held the cue. Tests that poked cue._loading now use the registry.
… publish or abandon (red) PR #22 build 3 decided an arm was stale AFTER arm() had published the cue and then disarmed it; the restarted run's GO could already be playing it (STOP -> GO -> GO inside one in-flight arm), or be playing it on a player the STOP had killed. Pins the redesign: - the epoch moves in stop_all_cues() (the STOP's first step, before the DMX/video reset and the player kill) and again in disarm_all(); - an arm whose epoch moved is refused at entry, and abandoned at the publish gate if the STOP lands mid-arm: never loaded, never in the armed list, its own resources released while it still holds the cue; - a stale waiter does not adopt a cue the new epoch armed; a current one arms it fresh, and nothing disarms it afterwards (the blocker); - a continuation dispatched before the STOP never reaches run_cue/reveal; - disarm_all no longer wipes a cue published during its loop; - go() carries its epoch into the fallback arm, the lookahead and the unrolled chain, and returns None instead of raising when a STOP refused the arm. Plan: cuems-RELATIONS Plans/2026-09-30-engine-prearm-publish-or-abandon.md
… is never published The decision 'is this arm still wanted?' moves inside CueHandler.arm(), before the cue becomes visible as armed, so no thread ever has to disarm a cue after the fact. - _disarm_epoch: bumped in stop_all_cues() (the first step of a STOP and of a load, before the DMX/video reset and the audio-player kill) and in disarm_all(), both under the handler lock. - Every arm carries the epoch of its REQUEST: arm(epoch=), _ArmWalk.epoch, _arm_ahead(arm_epoch=); read at entry otherwise, never re-read, and inherited by arm()'s own post_go / action-target recursion. - Epoch moved before the cue is claimed (entry, or waking from a wait) -> False, even if the cue is loaded by now: the new epoch loaded it. - Epoch moved while arm_cue() ran, or the cue was disabled meanwhile -> the arm is ABANDONED at the publish gate: not loaded, not in the armed list, no recursion; what it built is released while this thread still holds the cue's claim (_release_cue_resources, factored out of disarm()). A failed arm_cue() releases what it built the same way. - disarm_all(): epoch bump + snapshot + clearing the armed list are one lock section. Clearing after the loop wiped any cue published meanwhile (loaded but not in the list: unplayable until a reload). - go(): captures the epoch (a continuation inherits its chain's), passes it to the fallback arm, the lookahead and go_threaded, and refuses a dispatch whose epoch moved in its commit section -- returning None, not raising, when a STOP refused the re-arm. go_threaded uses it for its own-thread arm, the go_at_end lookahead and the continuation. Legacy test stubs with fixed arm()/go() signatures accept the new keyword.
…apture the arm epoch (red) Replaces the tests that pinned the after-the-fact undo: - build 3: test_a_stop_during_the_target_arm_undoes_everything_it_armed, test_the_lookahead_undoes_its_walk_after_a_stop_too - build 2: test_a_project_change_during_the_arm_undoes_it - 869f79ecc: test_disarms_newly_armed_cues_when_project_changed_underneath - ReArm: test_generation_change_mid_arm_disarms That undo ran after arm() had published the cue, so it could cut a cue the restarted run was already playing -- and, after reloading the same project, the new project's player, since Cue equality is by id. arm() abandons a stale arm itself now, so these threads must disarm nothing. Also pins: the PreArm walks and the ReArm thread carry the epoch captured by whoever spawned them (set_next_cue, _prearm_after_advance and _apply_cue_enabled_side_effects -- reached from the command thread and from an ActionCue's own thread), and the lookahead does not start once the STOP has begun.
…och their spawner captured - _undo_stale_prearm is gone. _prearm_segment and _prearm_lookahead disarm nothing: CueHandler.arm() refuses or abandons an arm whose STOP/load epoch moved, before the cue is ever published. Build 3's undo ran after the publish and could cut a cue the restarted run was already playing (STOP -> GO -> GO inside one in-flight arm), or leave it playing on a player the STOP had killed. - set_next_cue and _prearm_after_advance capture CUE_HANDLER.arm_epoch() on the command thread and hand it to the PreArm thread (_ArmWalk.epoch, _arm_ahead(arm_epoch=)). - _prearm_segment does not start the lookahead once the epoch has moved: stop_all_cues() bumps it before ready_script bumps the generation. - _arm_with_enabled_guard takes the epoch captured by _apply_cue_enabled_side_effects (command thread or an ActionCue's thread) and no longer disarms on a project change. After a reload of the same project that disarm acted on a stale object that compares equal to the new one and killed the new project's player. Same for the 869f79ecc lookahead's project_changed undo, also removed.
…dead (red) A STOP or a load kills every audio player. One still waiting for its JACK ports will never register them, but connect_player_to_outputs waits its full 30 x 0.5 s anyway -- holding that cue's arm for ~15 s, during which a GO for it waits 5 s and fails. A live process must keep the whole wait: the ceiling is not what changes.
…s has exited connect_player_to_outputs takes a should_abort predicate, checked on every attempt the port is still missing; new_audio_output passes 'this player's subprocess has exited'. A STOP or a load that kills a player mid-wait now ends that arm within one retry (0.5 s) instead of ~15 s, so the abandoned arm releases the cue before the restarted run asks for it. The wait for a LIVE process is unchanged: the 30 x 0.5 s ceiling stays as it is until fleet data says otherwise.
…nter and says so (red) go_script returned from 'Failed to re-arm ... cannot GO' without touching the pointer, so the node dispatched that cue on the NEXT GO and ran one GO behind the controller until STOP. Decision (Ion, 2026-09-30): skip the cue and annotate it; 5 s is plenty for a slow load. Also pins: the next segment is pre-armed on that path, and a skipped last segment does not make the next GO restart the node from contents[0].
…ter and logs it go_script's 'Failed to re-arm ... cannot GO' branch now calls _skip_unarmed_cue(): the pointer advances exactly as on the success path and is broadcast, ongoing_cue is set (so a pointer that becomes None reads as 'No more cues' on the next GO instead of restarting the node from the top), and the next segment is pre-armed as the no-local branch does. One ERROR line names the cue, why (its arm still in flight: held by which thread and for how long -- or the arm failed) and the new pointer. Decided 2026-09-30 (Ion): skip and annotate; 5 s is plenty for a slow load. Running status is not touched and nothing is stopped.
…red) - A continuation dispatched before a STOP still played when the STOP's own re-arm loaded its cue first: arm() refuses the stale request, but go_threaded only looked at cue.loaded. Same when the thread reaches its arm after the STOP is over. - The skip path did not stamp the disabled cues the GO walk passed, unlike the success path and the no-local branch. - The skip message said 'its arm failed' when the arm it waited on finished just after the wait gave up. - A ReArm whose arm was refused or failed went on to rejoin_chain, i.e. a second wait and a second arm.
- go_threaded checks the STOP/load epoch itself, not only cue.loaded: a continuation dispatched before a STOP no longer plays a cue the STOP's own re-arm loaded for the next run (arm() refused the stale request, but its result was ignored), nor one it finds loaded on reaching its arm after the STOP. stop_all_cues() cannot flag such a cue -- it was not armed when the STOP began. - _skip_unarmed_cue stamps the disabled cues the GO walk passed, as the success path and the no-local branch do, so enabling one mid-show can still rejoin. - The skip line tells a slow arm from a failed one: whether an arm was in flight is asked before arm() waits, not only after it gave up. - _arm_with_enabled_guard does not go on to rejoin_chain when the arm was refused or failed. - reset_armed_cues() removed: disarm_all() clears the list with its snapshot, and it had no other caller. Not taken: abandoning at the publish gate when only the selection epoch moved (a later setnextcue) -- that arm is consistent, merely early.
ibiltari
added a commit
that referenced
this pull request
Oct 1, 2026
…e cue itself (red)
arm() excludes concurrent arms with a per-OBJECT Event (cue._loading), but
what an arm creates is keyed by cue ID: the tracked player, the JACK client
name Audio_Player-<uuid>, the video layer ids -- and Cue.__eq__/__hash__ are
by id. After reloading the same project the old and the new object of one
cue arm concurrently and step on each other.
A thread that waited on another thread's arm also just returns that arm's
result: when it did not load the cue, go() gives up ("cannot GO").
First step of the redesign of the stale pre-arm cleanup (869f9wqpn): PR #22's
_undo_stale_prearm could disarm a cue the next run was already playing.
Plan: cuems-RELATIONS Plans/2026-09-30-engine-prearm-publish-or-abandon.md
(cherry picked from commit 827ddff)
ibiltari
added a commit
that referenced
this pull request
Oct 1, 2026
… publish or abandon (red) PR #22 build 3 decided an arm was stale AFTER arm() had published the cue and then disarmed it; the restarted run's GO could already be playing it (STOP -> GO -> GO inside one in-flight arm), or be playing it on a player the STOP had killed. Pins the redesign: - the epoch moves in stop_all_cues() (the STOP's first step, before the DMX/video reset and the player kill) and again in disarm_all(); - an arm whose epoch moved is refused at entry, and abandoned at the publish gate if the STOP lands mid-arm: never loaded, never in the armed list, its own resources released while it still holds the cue; - a stale waiter does not adopt a cue the new epoch armed; a current one arms it fresh, and nothing disarms it afterwards (the blocker); - a continuation dispatched before the STOP never reaches run_cue/reveal; - disarm_all no longer wipes a cue published during its loop; - go() carries its epoch into the fallback arm, the lookahead and the unrolled chain, and returns None instead of raising when a STOP refused the arm. Plan: cuems-RELATIONS Plans/2026-09-30-engine-prearm-publish-or-abandon.md rc5 backport (adapted from rc_1 0ea216a, not a plain cherry-pick): - a chain continuation is dispatched by the PREVIOUS cue's thread calling go(), whose fallback arm runs there, and this line has no chain-epoch STOP barrier: the no-play-after-STOP tests go through go() (waiting on another arm, arming its own cue, a STOP between the arm and the commit, the STOP's own re-arm loading the cue first); - go_threaded carries no epoch here (it has no own-thread arm); - disarm()/disarm_all() take no reason; - go_script must tolerate None from go().
…o-start, never played (red) Part A of cuems-RELATIONS Plans/2026-10-01-engine-prearm-past-pauses-and-inventory.md. Four INFO lines that say what a node holds armed and for how long; no behaviour change.
…easurement) Four INFO lines, no behaviour change: - "Armed <type> <id> in <t> s" at publish (engine side: audio spawn + JACK port; for video the OSC sends, the open is the videocomposer's own line); - "Armed inventory after <GO|PreArm:id>: ..." at the end of every GO (dispatched, nothing local, skipped) and of every PreArm thread, counting published cues by type, video layers, playing vs idle; - "Cue <id> started <t> s after it was armed" at the reveal commit, logged after reveal_cue() so it never delays it; - "Cue <id> disarmed after <t> s armed, never played (<reason>)" for a published cue no GO ever took (not on cue_end, not for abandoned arms). _armed_at / _ever_played are set at every publish, _ever_played again at go()'s commit. The PreArm threads now run through _prearm_thread(), which logs the inventory in a finally. Part A of cuems-RELATIONS Plans/2026-10-01-engine-prearm-past-pauses-and-inventory.md. Suite on test2: 950 passed.
…y after load and STOP (red) Found on the first test2 run: the project's root CueList was counted as 'other' and logged 'never played (stop)' at every STOP, and the load / STOP re-arm -- most of what a node holds -- was never sampled.
…ry after load and STOP A CueList holds nothing and go() never dispatches one, so it is neither inventory nor a wasted arm. ready_script() now logs the inventory under its reason (load / stop), so the analyzer's step function starts where the holding starts. Suite on test2: 952 passed.
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes ClickUp 869f9wqpn.
Problem
A node never pre-arms its first cue when another node's
pausebreaks the chain in front of it (controllergo, controllerpause, then the node's cues). The node arms the cue at GO and starts it late: 240 ms at Medina sala1. This is not the Badajoz shape (869f79ecc).The first version of this fix (builds 1–3) had a flaw found in review: after a STOP, a background
PreArmthread disarmed the cues it had armed. It did that afterarm()had published them, so after STOP → GO → GO inside one in-flight arm it could cut a cue the restarted run was already playing, or leave it playing on a player the STOP had killed. That version (0.1.0rc5-2) is withdrawn;0.1.0rc5-3replaces it.Changes
Pre-arm on pointer advance (unchanged from before)
BaseEngine._first_local_enabled_in_go_chain(start):go_script's own walk as a helper; the load walk uses it too.NodeEngine._prearm_after_advance(): when a GO advances the pointer with nothing local, arm exactly what the next GO will dispatch on this node, on thePreArm:<id>thread, never under_command_lock.Publish or abandon (
CueHandler.arm())stop_all_cues()(first step of a STOP/load) and indisarm_all()._undo_stale_prearmis gone).Cueequality is by id). A thread that waited on another's arm arms the cue itself if that arm was abandoned or failed, within the same 5 s wait. A stale waiter never adopts a cue the new epoch armed.go()carries its epoch into the fallback arm, the lookahead andgo_threaded, and refuses a dispatch whose epoch moved (returnsNone, no raise). A continuation dispatched before a STOP does not play after it.disarm_all()clears the armed list together with its snapshot (clearing it after the loop could leave a cue loaded but unplayable).Also
Pre-arm measurement (4 INFO log lines, no behaviour change;
1897104…9eaeae5)Armed <type> <id> in <t> sat publish.Armed inventory after <GO | PreArm:<id> | load | stop>: …: what the node holds (cues by type, video layers, audio players, playing vs idle; CueLists left out).Cue <id> started <t> s after it was armedat the reveal, logged afterreveal_cue()so it never delays it.Cue <id> disarmed after <t> s armed, never played (<reason>)for a published cue no GO took (not oncue_end, not for abandoned arms, not for CueLists).They exist to judge the pre-arm policy on data. Results (test2 node01): a 6-cue chain (two 4K, a 3.8 GB mp4, a 1.1 GB mov, two audio) arms in 0.70–0.78 s warm, 1.67–1.77 s cold, about 1 s of it a synchronous
ffprobeper video cue. A chain with prewaits of 5/35/65/95 s holds its chained cues 40–100 s before they play. Plan and results:cuems-RELATIONS/Plans/2026-10-01-engine-prearm-past-pauses-and-inventory.md§7. These commits can be dropped from this PR without touching the fix.Review
An independent Opus review of the diff found one real hole (a continuation could adopt the STOP's own re-arm), fixed in
ecafa23, plus three minors. Design and both review rounds:cuems-RELATIONS/Plans/2026-09-30-engine-prearm-publish-or-abandon.md.Testing
prova) give 0 not-armed / fallback / LATE / skip and equal anchors, as before.0.1.0rc5-3, 2026-09-30): node01 pre-arms its segment 0.34 s after GO 1; GO 2 starts on time with 0 not-armed / fallback / LATE / SKIPPED (240 ms late on rc5-1 that morning).The rc5 line carries the same change as
hotfix/rc5-2-prearm(tagv0.1.0rc5-3);go()there needed its own lock section (rc5 had no STOP barrier ingo()).