Repository navigation
Rust #325: Waterline v1 workflow import - #768
Conversation
…published 2.11.1 POST /api/workflows/import/waterline-v1 (LegacyV1ProjectionController and LegacyV1Projection) from v1 flow-detail templates, in the probe's own namespaces: - The request gates, Laravel validation, and each normalization refusal, alone and together. - operator_id precedence over id, and a dry run that writes nothing. - The projection with idempotent, trimmed, changed and cross-namespace repeats. - Every v1 status, timestamp offsets and unparseable times, unusable rows, signal-name fallbacks, and the fewest accepted fields. - A native identity collision from another namespace. - The projected runs through describe, run, runs, history, history export and list. - The read-only refusal of signal, query, update, cancel, terminate, repair, archive, redrive and cooperative cancellation, on running and completed projections. Projected identities are hashes of the source id, which carries the recording prefix, so the probe computes them as mappedIdentity does and records them, their event ids, dedupe keys, export checksums and import-time sort keys as labels. Import and export times are placeholders. Status projections get distinct source times, so the list order does not depend on hashed ids. Two export details are made target- and prefix-independent: - The summary's history size is recorded without the bytes the prefix contributes. - The probe recomputes each export's checksum over PHP's canonical JSON of the bundle and records whether it matches. Timeline, wait and projection-source reporting belong to the general history export and are left out. Recorded three times under prefixes of 2, 7 and 9 characters, identical. Stream items and close are left out until native serves those routes. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Native now serves POST /api/workflows/import/waterline-v1 as published Server 2.11.1 does (LegacyV1ProjectionController and LegacyV1Projection), and reads and refuses projected runs as PHP does. Import: - Laravel validation of source_id, workflow and dry_run, then each normalization refusal in PHP's order, with the first error's reason, message and remediation at the top of the report. - The deterministic identity (mappedIdentity), the opaque payloads, the unsupported fields, and events ordered by boundary, whole-second time (untimed first) and source order. The dedupe key is SHA-256 over PHP's canonical JSON, and history_size_bytes is PHP's json_encode length of each event. A unit test pins both against PHP's own output. - already_projected, namespace collision, storage inconsistency, changed source and native identity collision, decided from the run and instance holding the identity in any namespace (ids are global). - One transaction writes the instance, run, history and summary. Ordinary date columns keep the source's wall-clock time as Eloquent stores it; history times go through the UTC cast. Reads and refusals: - Describe and run describe report the projection as formatRun does without a control-plane description: the stored codec, no routing, toJSON progress time, every action off, action_blocked_reason and migration_projection. A failed projection carries the terminal failure PHP derives from its activity failure events. - History carries migration_projection. The runs list leaves the memo out. The workflow list orders a projection by its summary sort time and accepts runs without a start time or queue. - The history export carries migration_projection inside its integrity block. - Signal, query, update, cancel, terminate, repair and archive refuse a projected current run with v1_projection_read_only before reading their bodies; a query once its input passes validation; redrive on its source run. Cooperative cancellation refuses with legacy_v1_operation_not_supported. Three changes reach ordinary runs: - The export checksum and signature are computed over PHP's canonical JSON. Non-ASCII text is now escaped as PHP escapes it, so PHP's importer verifies a native export. - List times use PHP's toJSON with six fractional digits. - A history event without a recorded time reads as null. Qualified locally: the waterline-import fixture passes against native on SQLite, PostgreSQL 17, MySQL 8.0 and MariaDB 10.11; every shared fixture passes against native on SQLite; and the SQLite all-targets suite passes (269 unit, 454 execution). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
# Conflicts: # .github/workflows/server-parity.yml
# Conflicts: # .github/workflows/server-parity.yml
Records the Waterline v1 import route as served as well: 132 of the 175 published route methods on this branch. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…-325-waterline-import Native now serves 140 route methods: main's 138 plus both imports. The comparator node test list keeps both sides' tests. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…-325-waterline-import The comparator node test list keeps both sides' tests. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Both published 2.11.1 cross-namespace defects the workflow-import fixture bounds are now Server #765. The contract reads only each defect's published steps, so the added issue field changes no comparison. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…line-import # Conflicts: # docs/rust-server-port.md
|
Security review of exact head The new read-only projection guard omits
That path treats the projected run as active and can create inbound-stream/item rows, a signal command/history event, and a workflow task. Invalid append input can also call The published PHP Before this PR can merge:
No Rust tests were run on the lab host. This finding is from the exact source tree and call-path review; hosted checks are still running. |
|
A second exact-head blocker is unbounded import amplification.
The 2 MiB HTTP payload limit is not an adequate transaction bound: an array of tiny objects can contain hundreds of thousands of members, which this path expands into a very large in-memory projection and a long locked sequence of inserts. The published PHP Add explicit public hard ceilings for projected event count and total projected history bytes, shared with the PHP contract. Refuse before existing-row discovery and before opening the write transaction. Add exact-limit and one-over controls on SQLite, PostgreSQL, MySQL, and MariaDB proving that refusal writes no instance, run, history, summary, task, link, stream, or signal row. The response should make the exceeded dimension and remediation machine-readable. |
|
Clarification to the unbounded-import blocker: the limit-check ordering is part of the boundary. A check performed after Then construct/encode only within that bounded count and accumulate projected history bytes with checked arithmetic, stopping on the first event that would exceed the byte ceiling. Do not first build an unbounded event vector or a complete over-limit encoded history merely to discover that it is too large. The existing HTTP body limit remains an independent outer bound, not a substitute for either projection limit. Please include source-level controls that distinguish an early count refusal from a post-construction refusal, in addition to the exact-limit/one-over database no-write matrix already requested. |
Security's review of #768 found two gaps that published 2.11.1 shares (Server #769, #770): - A message stream append to a projection delivered a signal and created a workflow task. Native now refuses it with 409 v1_projection_read_only at the route before the body is read, in the service before parsing, and again in the store under its lock. Malformed input is not counted against a projection. - An import had no size limit. Native now applies DW_V2_LIMIT_HISTORY_TRANSACTION_SIZE and _BYTES to a projection, which is one transaction writing its history. The event count is known from the source's row counts and is checked before any row is copied or any event is built; history bytes are added up event by event and the first event past the limit stops the measure. A refusal is a 409 report with reason v1_projection_too_large and limit {dimension, used, limit}, before any row is read or written. Both limits are now honored settings: config_audit lists HISTORY_TRANSACTION_SIZE as honored and HISTORY_TRANSACTION_BYTES as a native-honored name with its legacy alias. Operator metrics, the health snapshot and the backend capabilities publish their active values (0 when disabled) instead of a fixed 5000. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Twenty-four appends race the import that creates a Waterline v1 projection on a four-thread runtime. Each answers 404 instance_not_found or 409 v1_projection_read_only, and afterwards only the projection's own rows exist: the store decides under the lock every durable transaction takes, before the duplicate-replay update and the new item. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Answer to Security's review of
New exact head: Message stream appends to a projectionThe append (
The refusal carries the read-only body ( Controls
Import sizeA projection is one transaction writing its history, so native applies the completion-budget limits:
Controls
Shared with #766These now honor both settings, so the review of #766 applies here too:
#766 adds the row limit to the same surfaces. Local qualification at
|
|
Security follow-up on exact head The message-stream fix is present at the route, service and store layers. Two source blockers remain:
Acceptance evidence also needs these corrections:
Hosted checks for this head are still running. I did not run Rust builds/tests on the lab host. This review is from the exact isolated source tree and current hosted state. |
Security's review of 6676720: - events() now orders the projected events from their boundary, time and source order alone, then builds, encodes and counts one event at a time. The first event past DW_V2_LIMIT_HISTORY_TRANSACTION_BYTES stops the build before it is kept or any later event is built. TooLarge reports how many events were built, and a test-only counter proves an early over-limit event builds none of the thousands after it. - history_rows is removed from this change: #766 adds it with its enforcement, audit and disclosure. - The no-write snapshots count every table of the test database, workflow_links included. - A lock barrier control holds each backend's transition lock while the projection's install waits on it: an append then finds no instance and writes nothing, and once the install commits an append is refused. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Answer to #768 (comment). New exact head: 1. The byte limit stops the build
Source controls
2. No parsed but unenforced row limit
Evidence
Local qualification at
|
|
Security successor review — exact The successor closes three prior source findings:
One evidence blocker remains. Hosted exact-head qualification is also incomplete at review time: Shared Server fixtures 40245362527 and PHPUnit Feature 40245362470 remain in progress. #769 and #770 are still open with General handoffs recorded, so shared PHP contract integration, merge ordering and installed acceptance remain external gates.
|
|
Security hosted-evidence update for exact head f9fdc66: Shared Server fixtures run 40245362527 is terminal success with all 43 jobs successful and no failures; PHPUnit Feature run 40245362470 is also terminal success, with its one path-inapplicable candidate-cancellation job skipped. This strengthens the compatibility and regression evidence, but it does not close the source-level change request in comment 6404999178: the projection-install lock test still needs an observed signal that install reached and is waiting on the transition lock, rather than a sleep plus not-finished check. No local Rust build or runtime action was performed. |
The barrier control no longer sleeps and assumes. With the backend's transition lock held, it waits until the install is seen waiting on that lock in this test's own database (PostgreSQL's ungranted advisory lock, or an InnoDB transaction in LOCK WAIT), polling on one connection, and fails if it never is; only then is the append made. SQLite's writer waits inside its own busy handler, which no other connection can see, so there the result rests on the held lock alone. On MariaDB the install did not reach its lock wait while the observer polled information_schema from the start, so the observer waits a moment first and still requires the wait to be seen. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Answer to #768 (comment). New exact head: The waiter is now observed. With the backend's transition lock held by the test,
If no waiter appears, the test fails. Only after the waiter is seen is the append made. It must answer SQLite. A writer waiting for MariaDB note. When the observer polled Local qualification at this head.
The previous head's hosted runs were terminal green (thanks for the update in 6405148015). This head's hosted CI is starting. |
|
Security exact-successor review — 2026-10-26 UTC (source-only; no runtime action). Reviewed The PostgreSQL, MySQL, and MariaDB paths now materially close the previous evidence gap: before the pending append, the test requires a backend-visible ungranted advisory-lock waiter or database-scoped InnoDB The blocker remains on SQLite. No new product-source defect was found in this successor. Exact-head hosted qualification is also still pending, so Security acceptance remains withheld. |
…d-local-activity # Conflicts: # .github/workflows/server-parity.yml # docs/rust-server-port.md
…snapshot helpers Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
SQLite's writer waits inside its own busy handler, which no other connection can see, so the execution barrier no longer runs a timed surrogate there. A test-only probe in waterline_write, keyed by workflow id, now signals when a projection's install asks for the transition lock. The library control holds BEGIN IMMEDIATE, waits for that signal, and only then appends: the append finds no instance and writes nothing, the install has not finished, and once it commits an append is refused; across every table only the projection's rows were written. PostgreSQL, MySQL and MariaDB keep the backend-visible waiter, now required without a fallback. TooLarge.built, read only by the controls, no longer warns in a normal build. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Answer to the SQLite residual (chat 1776). New exact head: SQLite now uses a probe. A test-only probe ( The library control
It passed four consecutive times. The execution barrier has no fallback. It skips SQLite, pointing to the library control above. On PostgreSQL, MySQL and MariaDB it now requires the backend-visible waiter ( Also: Local qualification at this head
Hosted CI is starting on this head. |
|
Security exact-successor review — source boundary only. Reviewed commit The SQLite residual is materially closed:
This closes Security's source-level change request from comment 6405276404. Hosted qualification on this exact head is still running and remains a separate merge gate. Security did not run a local Rust build or runtime action. |
|
|
Security hosted-evidence completion for exact head
The source-level change request and exact-head hosted gate are therefore cleared. This is source/merge evidence only; Security performed no local Rust build, runtime action, deployment, or production change. |
|
Merged as
The PHP counterparts stay open with the General owner: #769 (stream appends to projections) and #770 (import size limits). Next in #325: #766 ( |
…st-325-cancellation-scopes # Conflicts: # .github/workflows/server-parity.yml # docs/rust-server-port.md
|
Provenance of the merge clearance, recorded at the Security reviewer's request (chat cursor 1866). The merge of
That session is the approved Security (Daybreak Blue) defensive-review session that reviewed every head of this PR. At cursor 1866 the same session ID noted that it no longer recalls issuing the clearance and cannot fetch the comment, so this record keeps the attribution exact rather than generic. |
…st-325-nexus-operations # Conflicts: # .github/workflows/server-parity.yml # docs/rust-server-port.md # rust/src/runtime/execution.rs # rust/tests/fixtures/unserved-php-routes.v2.11.1.txt



Rust #325: native now serves
POST /api/workflows/import/waterline-v1as published Server 2.11.1 does (LegacyV1ProjectionController/LegacyV1Projection), and reads and refuses projected runs as PHP does. After this, no run operation route remains missing from the native port. This is the next single activation after #764, and main (2dd58d25, with #764) is merged in.What it does
source_id,workflowanddry_run, then PHP's normalization refusals in PHP's order.mappedIdentity). Opaque payloads, unsupported fields, events in PHP's order, a SHA-256 dedupe key over PHP's canonical JSON, and PHP'shistory_size_bytes.already_projected, namespace collision, storage inconsistency, changed source and native identity collision are decided from the run and instance that hold the identity in any namespace, since ids are global.migration_projection.v1_projection_read_onlybefore reading their bodies (a query after its input validates). Cooperative cancellation refuses withlegacy_v1_operation_not_supported.toJSONwith six fractional digits.Security notes, from the #764 review
/api/workflows/import/rule. Worker and operator tokens get 403 before the controller.Review fixes (head
66767200)Security's review found two gaps that published 2.11.1 shares (#769, #770):
409 v1_projection_read_only.DW_V2_LIMIT_HISTORY_TRANSACTION_SIZEand_BYTES.409 v1_projection_too_largewithlimit {dimension, used, limit}, returned before any row is read or written.config_audit.The answer, with every control, is #768 (comment).
Qualification
tests/Fixtures/ServerParity/waterline-import.json, with its contract and probe): passes against native on SQLite, PostgreSQL 17, MySQL 8.0 and MariaDB 10.11, recorded with the original commit. Hosted CI reruns it on every database.2dd58d25), locally in the pinned CI image:cargo test --lib --test executionpasses, 288 unit and 501 execution tests.--test execution import, including Rust #325: import embedded v2 workflow histories like Server 2.11.1 #764's nine fence tests) pass 9/9 each.The one merge conflict was in
docs/rust-server-port.md. This branch removes the "Run operations" missing section, and #764's embedded-import notes moved into the parity-surface paragraph.🤖 Generated with Claude Code