fix(api): persist strategy-expanded steps as the run workflow - #38
Open
raskevichai wants to merge 1 commit into
Open
fix(api): persist strategy-expanded steps as the run workflow#38raskevichai wants to merge 1 commit into
raskevichai wants to merge 1 commit into
Conversation
createRun expanded the strategy into effective_steps and then stored the raw request body as workflow_json. The expansion was used only to prepopulate the steps table. The engine rebuilds its graph from workflow_json at tick time, so it read a step array with no depends_on, treated every step as a graph root, and dispatched them all at once. That is why a sequential strategy did not enforce chain order: applyChain derived the edges correctly, they were just never written down. Store the expanded workflow instead, dropping strategy and reduce since expansion has consumed them into depends_on edges and a synthetic reduce step. Two idempotency checks compare the stored workflow_json against the incoming payload. Both had to move to the expanded form as well, or a plain retry of a strategy run would compare expanded-vs-raw and be rejected as a payload change. Expansion is now computed before the idempotency check so both sides use the same representation. Runs without a strategy keep storing the body verbatim, so their comparisons stay byte-stable against runs written before this change. Closes nullclaw#36.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #36.
Root cause
handleCreateRunexpands the strategy intoeffective_steps, then stores the raw request body asworkflow_json:The expanded array was used only to prepopulate the
stepstable. Since the engine rebuilds its graph fromworkflow_jsonat tick time (engine.zig:416), it read steps with nodepends_on, treated all of them as graph roots, and dispatched them concurrently.So
applyChainwas deriving the edges correctly the whole time - they were just never written down. That also explains #34 (sequentialnot enforcing chain order): same root cause, observed from the scheduling side.Changes
1. Store the expanded workflow. New
buildExpandedWorkflowJsoncopies the top-level fields, swaps ineffective_steps, and dropsstrategyandreduce- expansion has already consumed those intodepends_onedges and a synthetic reduce step. Leaving them in would let a later reader re-derive a graph that disagrees with the steps stored beside it.2. Idempotency comparisons moved to the same representation. This part is not in the issue but the fix is wrong without it.
workflow_jsonis compared against the incoming payload in two places (the early replay check, and the post-insert race path). With the expanded form stored and the raw body compared, an ordinary retry of a strategy run compares expanded-vs-raw, mismatches, and returns 409 conflict instead of a 200 replay. Expansion is therefore computed before the idempotency check so both sides use one representation.3. No-strategy path unchanged. When there is no strategy, the body is still stored verbatim. Runs written before this change stay byte-comparable, so existing idempotency keys keep replaying correctly.
Tests
Four regression tests:
sequential strategy stores expanded depends_on in workflow_json- the stored workflow hasdepends_onand no longer hasstrategy.workflow without strategy is stored verbatim- guards the backward-compatibility path above.strategy run replays instead of conflicting on identical body- guards the idempotency interaction; without change 2 this returns 409.strategy run still conflicts on a different body- conflict detection still works.Sanity check: restoring the original
insertRun(..., body, ...)line makes tests 1 and 3 fail, and restoring the fix produces a byte-identical tree.Verification
Scope
Not included, though they touch neighbouring code:
reducerejected as "unknown step type") is a separate defect:strategy.zigemitstype: "reduce"butStepTypehas no such variant. It needs either a new enum variant or a different synthetic type, which is a schema decision worth its own PR.engine.zig's dispatch loop and is unaffected by where the workflow is stored.I have not run the e2e suite (
tests/test_e2e.sh) since it needs Python mock workers and a live server; the change is covered at the API layer instead.