Skip to content

Waterline v1 projection import has no event or history byte limit #770

Open

Description

@rmcdaniel

Published Server 2.11.1's Waterline v1 projection import has no limit on how much history one import writes.

Behavior. POST /api/workflows/import/waterline-v1 (LegacyV1ProjectionController, LegacyV1Projection::normalize) turns every logs, signals and exceptions row into one projected history event. It sorts them, then writes each event with its own INSERT in a single transaction. Nothing bounds the row count, the event count or the projected history bytes. Within the 2 MiB request cap, one admin request can carry hundreds of thousands of tiny rows. That one request then causes a large memory, database-lock and write burst.

Expected. An explicit, public hard limit on projected events and on total projected history bytes. It should be checked before any existing row is read and before the write transaction, with zero rows written on refusal.

Native. Server #768 (Rust #325) reuses the completion-budget limits from Workflow #930 (DW_V2_LIMIT_HISTORY_TRANSACTION_SIZE, default 5,000 events; DW_V2_LIMIT_HISTORY_TRANSACTION_BYTES, default 16 MiB), since one projection import is one transaction writing that history.

  • A source is refused from a cheap row count before it is copied, then by its exact event count and history bytes after normalization. Both checks come before any read or write.
  • The refusal is a 409 report with reason: v1_projection_too_large and limit: {dimension, used, limit}.

Please confirm or change these names and defaults for PHP, so both servers enforce the same contract.

Found by Security's review of #768. Handed to the General owner for the PHP contract and fix.

Activity

  1. added a commit that references this issue on Oct 25, 2026
  2. rmcdaniel commented on Oct 25, 2026

    @rmcdaniel
    MemberAuthor

    Handoff record. The General owner accepted the PHP contract and fix in the native agent chat (server-325, 2026-10-25 22:58 UTC). The proposed shared contract:

    • DW_V2_LIMIT_HISTORY_TRANSACTION_SIZE, default 5,000 events, and DW_V2_LIMIT_HISTORY_TRANSACTION_BYTES, default 16 MiB. 0 disables a limit.
    • The refusal is 409 with status: rejected, reason: v1_projection_too_large and limit: {dimension, used, limit}, before any row is read or written.
    • The count comes from the source's row counts (start, rows, terminal) and is checked before any row is copied, hashed or built into an event. Then events are built one at a time, and the first event past the byte limit stops the build.

    General will verify that PHP's exact count and encoding match before parity is claimed.

    Native counterpart. Server #768 implements this.

    • Its source controls show that a count refusal builds no event, and that an early over-limit event builds no later one.
    • Its database tests show the exact limit is admitted and one over is refused, with no row written.
    • Both settings are honored in its config audit and published in operator metrics, health and backend capabilities.

    Merge ordering. Native #768 can merge first and is documented as ahead of published 2.11.1. This issue stays open until the PHP Server enforces the same contract.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions