Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 9 additions & 4 deletions doc/rfc/stovepipe/steps/buildsignal.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Buildsignal stage

`buildsignal` polls the build-runner until a build reaches a terminal state, records the status, and — once the build is terminal — releases the queue's build slot, projects the outcome onto the `Request`, and publishes the **request id** to `record`. See [workflow.md](../workflow.md) for where it sits in the pipeline and [build.md](build.md) for how the build it polls was triggered.
`buildsignal` polls the build-runner until a build reaches a terminal state, records the status, and — once the build is terminal — releases the queue's build slot, projects the outcome onto the `Request`, advances a successful Request's last-green build baseline, and publishes the **request id** to `record`. See [workflow.md](../workflow.md) for where it sits in the pipeline and [build.md](build.md) for how the build it polls was triggered.

It handles only the poll loop: it does not decide build strategy, write greenness, or map targets to projects — those are `process`'s, `record`'s, and `analyze`'s jobs. It reports the *fact* (this request's build ended succeeded, failed, or cancelled); `record` derives what that fact *means* for greenness.

Expand Down Expand Up @@ -60,11 +60,14 @@ For a delivery carrying build id `B`:
- with the write-once rule, accepted -> running -> {succeeded|failed|cancelled} is monotonic
by mechanism, not by assumption about the backend.

7. If the stored status is terminal, and R does not already carry an outcome:
a. Release the queue's build slot: CAS-decrement Queue.in_flight_count, clamped at zero.
7. If the stored status is terminal:
a. If R does not already carry an outcome, release the queue's build slot:
CAS-decrement Queue.in_flight_count, clamped at zero.
- failure here aborts the step: R must not go terminal while still holding a slot.
b. CAS R from processing to the outcome the stored status projects onto it and persist B as R's terminal build id:
b. If R does not already carry an outcome, CAS R from processing to the outcome the stored status projects onto it and persist B as R's terminal build id:
succeeded -> succeeded, failed -> failed, cancelled -> cancelled. First writer wins.
c. If the winning durable R.State is succeeded, advance the last-green bookmark
in a CAS retry loop, even on a terminal redelivery. Preserve a newer bookmark.
Then publish R.ID to the record topic, partitioned by request id; ack, return.
No re-publish to buildsignal.
- record loads the Request directly by this key, derives greenness from its outcome, and
Expand All @@ -84,6 +87,8 @@ For a delivery carrying build id `B`:

**Why the slot is released before the outcome write, and why a failed release aborts it**: `Queue` and `Request` are separate entities with no cross-entity transaction, so the ordering picks which crash failure mode we accept. Both rules serve one invariant — *the request must not go terminal while still holding a slot* — because a terminal request is skipped by redelivery and by the DLQ reconciler alike, so nothing would ever decrement it. Failing this way leaves the request non-terminal: redelivery re-runs both steps and decrements again, transiently over-admitting by one slot until the zero clamp reconverges. Over-admission is the failure mode this pipeline already prefers, for the same reason and in the same words as the DLQ reconciler (see [process.md](process.md#in_flight_count-integrity)).

**Baseline timing.** After the winning Request outcome is durable, a successful Request advances `last_green_uri` and `last_green_request_id` in a CAS retry loop before publishing to `record`. Older candidates never regress the bookmark. A terminal redelivery retries the advance without releasing capacity again. Slot release and bookmark advancement are separate updates, so an admission between them can still select the previous baseline. Project resolution, facts, and Git promotion remain in `record`.

**Why `buildsignal` releases the slot rather than `record`**: the gate `process` claims is a *build* slot — it exists to bound concurrent builds per Queue — and once the build is terminal the build is over. Releasing here also keeps the invariant *a terminal `Request` has already released its slot*, which is what makes the DLQ reconciler's early-return on terminal requests safe.

**Why `record` hears only terminal signals**: `record` has no non-terminal work — by its own contract a non-terminal signal would be a pure no-op — and step 7 already branches on terminality to decide whether to keep polling, so gating the publish costs nothing and spares `record` a no-op delivery on every poll tick of every running build. Crash-safety is unaffected: a crash between the terminal `Update` and the publish redelivers the message; step 5 re-polls (the runner reports the same terminal status), step 6 no-ops, step 7 publishes. This is a deliberate divergence from SubmitQueue, whose buildsignal republishes to `speculate` on every tick — sound there because speculate is a state machine that may act on any signal; stovepipe has no such consumer.
Expand Down
8 changes: 4 additions & 4 deletions doc/rfc/stovepipe/steps/process.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ It handles everything *before* the first build is triggered: it does not run bui
Ingest publishes a `ProcessRequest` (request id only — producer and consumer share the store) to the process topic, **partitioned by Queue name**. Two consequences the design relies on:

1. **One partition per Queue**, and `BatchSize = 1` (strict serialization; see [sql-queue-rfc.md](../../sql-queue-rfc.md#strict-serialization-opt-in)) — so at most one `process` invocation runs per Queue at a time. The read-modify-write on the Queue row (gate check, count increment) is therefore race-free **without** a transaction, the same property SubmitQueue's `validate` gets from per-queue partition leasing.
2. **`process` is not the only Queue-row writer.** `ingest` stamps `latest_request_id` (the request id of the newest accepted head, not a URI; see [Backlog coalescing](#backlog-coalescing)) and `buildsignal` releases slots when a build goes terminal, and `record` advances the `last_green_uri` bookmark. They touch different fields under optimistic-locking CAS, so concurrent writes converge instead of clobbering.
2. **`process` is not the only Queue-row writer.** `ingest` stamps `latest_request_id` (the request id of the newest accepted head, not a URI; see [Backlog coalescing](#backlog-coalescing)) and `buildsignal` releases slots when a build goes terminal, then advances the `last_green_uri` bookmark for a successful Request. They touch different fields under optimistic-locking CAS, so concurrent writes converge instead of clobbering.

## Process algorithm

Expand Down Expand Up @@ -59,7 +59,7 @@ Validation is expensive and shares a baseline, so heads arriving while an earlie

| Source | Field | Meaning |
|---|---|---|
| Queue row | `last_green_uri` | Bookmark `record` advances on whole-repo green; empty until first green. |
| Queue row | `last_green_uri` | Bookmark `buildsignal` advances on a successful build; empty until first green. |
| Queue row | `in_flight_count` | Requests past `process` and not yet terminal. `process` increments on admit; `record` (or DLQ reconciliation) decrements on terminal. |
| Queue config | `max_concurrent` | Cap on concurrent in-flight validations. **Default 1** (global wiring default for MVP; per-queue override when a Stovepipe `queueconfig` extension lands). |

Expand All @@ -80,7 +80,7 @@ The scheme: each admit pins its baseline to last-green *at admit time*; a green
Correctness rests on four rules, all with MVP primitives already in place:

1. **Pin to last-green at admit** — a known-green baseline, which `process` already reads.
2. **Advance last-green by ingest order, monotonically** — `record` adopts the highest-counter green (per `CompareRequestID`) and never regresses. (Free at N=1, since verdicts land in order; explicit at N > 1.)
2. **Advance last-green by ingest order, monotonically** — `buildsignal` adopts the highest-counter green (per `CompareRequestID`) and never regresses. (Free at N=1, since verdicts land in order; explicit at N > 1.)
3. **Coalesce to the latest N** instead of the latest 1 — otherwise intermediates are superseded on sight and nothing overlaps.
4. **Fall back to serial on a rewrite** — the linear/superset property is what makes "adopt highest green" sound; rewrites already force a full build with no baseline.

Expand Down Expand Up @@ -191,7 +191,7 @@ Runtime coordination only — fields the pipeline writes under CAS:
| Field | Role | Written by |
|---|---|---|
| `name` | Stable logical id (`monorepo/main`); the string ingest accepts | ingest (create) |
| `last_green_uri` | Bookmark; empty until first green | record |
| `last_green_uri` | Bookmark; empty until first green | buildsignal |
| `in_flight_count` | Active Phase 1 validations | process (+1), record/DLQ (−1) |
| `latest_request_id` | Request id of the newest head ingest accepted | ingest |
| `version` | Optimistic-locking version | all writers |
Expand Down
Loading
Loading