Skip to content

[ATT-03] Synchronize bounded attention snapshots and live deltas #5

Closed
Closed
[ATT-03] Synchronize bounded attention snapshots and live deltas#5
Labels
fork-backlogMaintainer backlog for the multi-machine forkpriority:P0Stable reconciliation and attention core

Description

@jdbencardinop
jdbencardinop
opened on Sep 14, 2026

ID: ATT-03 (F-002). Priority: P0. Dependencies: #3, #4 (complete). Backlog: #1.

Implemented locally on phase2/attention-sync: runtime landing
490cf029268513fd2917469b74055aaaf8270d13, final Foundation attestation
820e37c508569381ef5a61bd5b6c932fdb5c152d, clean verified tip
5d5c9380b411848940057f31610c105f10d30392. No push or PR.

As explicitly approved, one owned attention.subscribe stream emits paginated
bootstrap, catch-up, fence and live data; page tokens chain within it rather
than exposing independently callable snapshot pages. Fork migration48 adds a
bounded derived delivery journal/view, not another authoritative event log.
The shared reducer stages until the fence and clears incomplete bootstrap
cursors on disconnect. Details, bounds and evidence are in the result comment.

Problem and upstream seam

Independent snapshot and hot-stream calls can lose a resolution between them.
Bounded pagination must not cap an environment at 100 attention items or claim
an incomplete badge is complete.

Audited upstream already exposes acquire-before-consume
OrchestrationEngineService.subscribeDomainEvents, sequence-bounded replay and
projection receipts. Reuse the stable equivalents; no new global event log.

Protocol decisions

  1. Delivery cursor and snapshot page token are different opaque types. Bind
    them to environment, projection/delivery generation, canonical filter and
    authorized scope. Item revision is not a stream cursor.
  2. Register the post-commit attention consumer before capturing a committed
    projection watermark W. Capture registration/watermark through the serialized
    projector boundary so an in-flight commit cannot escape both paths.
  3. Enumerate current rows with immutable-ID keyset pagination, not mutable
    updatedAt ordering. Stage pages client-side while retaining all compact
    changes after W. Pages are a bootstrap scan, not an immutable snapshot at W.
  4. After the final page, fence at F after relevant projection/publication
    receipts drain. Apply all retained changes in (W,F], then emit sync-complete
    at F and atomically expose the synchronized cache. Continue live above F.
    Revisions make overlap idempotent and prevent older deltas overwriting newer
    scanned rows. Removals carry tombstone revisions during this handoff.
  5. Filter exit, resolution, deletion or scope removal emits a remove/invalidate,
    not just absence. Summary updates obey the same ordering. An empty filtered
    batch can still advance the stream watermark.
  6. A reconnect cursor is reusable only if its entire replay interval is
    available. Otherwise send explicit reset-required and restart bootstrap.
    Server restart/projection rebuild may rotate the delivery generation;
    clients must not assume an old sequence is comparable.

Bounds and recovery

  • Default 50 rows, maximum 100 rows per page/message, not total dataset size.
    Paginate both items and summaries; byte bounds apply as well as row counts.
  • Reuse stable live-stream byte/queue/time budgets. Overflow, replay truncation
    or bootstrap expiry produces typed reset/backpressure, never a dropped event
    followed by sync-complete.
  • Do not keep a SQLite read transaction open while waiting for remote pages.
    Use bounded bootstrap buffers; a slow client may have to restart. Do not
    replace this with full-history transfer or unbounded per-client snapshots.
  • Resume/new bootstrap uses existing reconnect backoff. No timer-based polling
    or tight automatic reset loop; surface persistent inability to synchronize.
  • Malformed input returns validation error; expired/generation-stale cursors
    return reset; unauthorized scope returns authorization failure and discards
    inaccessible cache. Filter changes begin a new generation of client state.
  • Notification delivery is suppressed throughout bootstrap and resumed replay.
    Only events above a completed live fence are notification candidates.

Acceptance criteria

Exercise commits immediately before/after registration, during every page,
between last page/fence, and during transition to live mode. Include inserts
before the keyset, filter entry/exit, remove/recreate, changing sort fields,
more than 100 items, duplicate/out-of-order observations, stale cursors,
restart/rebuild, disconnect, scope change and slow-consumer overflow.

At sync-complete(F), the client cache must equal the authorized filtered
projection at F, with no missing/duplicated material change. Use barriers and
receipts rather than sleeps. Validate this against the actual stable primitives
before choosing lower-level implementation details.

Coverage and non-goals

Typed server/contracts plus shared protocol reducer; all clients/providers and
local/direct/SSH/relay/tunnel transports. No UI, notification preferences,
global acknowledgement, mandatory relay, raw thread payloads or polling.

Activity

  1. jdbencardinop
    added
    fork-backlogMaintainer backlog for the multi-machine fork
    status:blockedRelease or dependency gate remains; no implementation yet
    priority:P0Stable reconciliation and attention core
    on Sep 14, 2026
  2. jdbencardinop
    added a parent issue on Sep 14, 2026
  3. jdbencardinop
    marked this as blocked by 2 issues on Sep 14, 2026
  4. jdbencardinop
    marked this as blocking on Sep 14, 2026
  5. jdbencardinop
    added
    status:readyScope ready and all implementation gates satisfied
    status:in-progressActively implementing the scoped issue
    and removed
    status:blockedRelease or dependency gate remains; no implementation yet
    status:readyScope ready and all implementation gates satisfied
    on Sep 28, 2026

  6. jdbencardinop commented on Sep 29, 2026

    @jdbencardinop
    jdbencardinop
    on Sep 29, 2026
    Author

    ATT-03 implemented locally

    • Branch: phase2/attention-sync
    • Runtime source landing: 490cf029268513fd2917469b74055aaaf8270d13
    • Final fence-regression landing: fe5ad8f8a4130fd9ff389e8953dc57134be0b261
    • Complete Foundation attestation: 820e37c508569381ef5a61bd5b6c932fdb5c152d
    • Clean verified tip: 5d5c9380b411848940057f31610c105f10d30392
    • Base: ATT-02 d701ac195d8f7eb35f750e47747bebc073dac545

    The maintainer approved one owned backpressured stream and a bounded derived
    journal rather than independently callable snapshot/stream operations or a
    second authoritative event history. The complete 64-path Foundation intent
    preserves earlier functionality and the composed generated workspace lock.
    All five maintained roots verify; the dependency DAG is valid.

    Delivered protocol

    attention.subscribe is in the common WebSocket RPC group under
    orchestration:read, advertised by optional capabilities.attentionSync.
    The owning server supplies environment identity; client input cannot select
    another authority. Delivery cursors are HMAC-signed and bound to environment,
    authenticated session/scopes, canonical filters and delivery generation.
    Page tokens use a different signed kind and only chain pages within that
    owned subscription.

    The stream emits:

    1. begin with committed watermark W and snapshot/resume mode;
    2. immutable-key page messages for mutable current items and summaries;
    3. retained deltas through a committed post-page fence F;
    4. sync-complete(F);
    5. strictly subsequent live deltas.

    Registration precedes capture of W. Short SQL transactions serialize
    authorization, watermark/page/fence reads with the projection writer. No
    transaction remains open while waiting on a remote ACK. Material SQL triggers
    write the compact derived journal in the same transaction as projection
    changes, so no asynchronous attention publisher can lag behind the fence.
    Existing post-commit domain events are wakeups, not raw-thread payloads.

    Filters use immutable IDs, not mutable update timestamps. Removals and filter
    exits carry transport versions; empty filtered batches still advance delivery.
    The shared client-runtime reducer stages pages/replay, preserves newer scanned
    values against older overlapping changes and exposes cache state only at F.
    Delete/recreate is ordered by delivery version rather than a domain revision
    that can restart at 1.

    Bounds and recovery

    • Default 50, maximum 100 combined entries per page/message; no dataset cap.
    • 64 KiB encoded-message limit and existing ACK-aware live-stream budgeting.
    • At most 1000 compact changes / 8 MiB journal retention.
    • 30-second bootstrap/replay deadline; one sliding wake signal, not an
      unbounded per-client copy of snapshots or raw domain events.
    • Delivery tokens expire after 24 hours; reconnect also requires the complete
      contiguous retained interval.
    • Generation change, rebuild, missing replay, overflow, oversized rows or
      bootstrap expiry resets explicitly. Malformed or wrong-scope cursors fail.
    • Authority is rechecked during reads and on idle authorization-change/expiry
      wakeups. Unauthorized clients must discard cache.
    • Startup/rebuild rotates generation and gates reads until projection replay
      completes; historical replay is never advertised as live notification work.
    • Mid-bootstrap disconnect discards its incomplete cursor. The reducer does
      not automatically retry: [ATT-04] Aggregate attention across connected environments #6 must reuse normal reconnect backoff and surface
      persistent synchronization failure, not create a reset loop.
    • Only live frames above a completed fence produce notification candidates;
      no notification delivery or preferences are implemented.

    Persistence and compatibility

    Fork migration 48 adds a rebuildable delivery view, bounded journal,
    generation state and capture/retention triggers. Existing projections, prior
    migration source and immutable audit remain unchanged. Migration tests prove
    ATT-02 backfill, rollback, retry, idempotence and fail-closed missing-trigger
    detection.

    The relevance predicate is shared with the current projector: ordinary content
    deltas do not generate attention journal entries or delivery queries. The
    server/client integration test uses a dev-only workspace dependency on
    @t3tools/client-runtime; runtime/provider pins did not change.

    The shared contracts serve web, desktop and mobile over the existing
    local/direct/SSH/relay/tunnel RPC path. Older servers omit the capability and
    must not be assumed to support the subscription.

    Evidence

    The focused 10-file regression pass completed 136 cases before the final
    test-only enhancement. The final synchronization module then passed all
    nine cases, including commits during every page and during catch-up above
    the already-captured fence. These counts overlap; they are not summed as
    independent coverage.

    Engine-backed tests compare all entity values and delivery versions at
    the fence, not only keys. They cover >100 items and >100 summaries,
    registration barriers, in-flight rollback, insertion behind a keyset,
    resolution/filter exit, delete/recreate, exact post-page fencing and later
    live changes, resumed replay, malformed/scope/page tokens, journal holes,
    disk-backed server restart, rebuild gating, UTF-8 byte limits, retention
    overflow and idle revocation. Reducer tests cover overlap/tombstones,
    out-of-order gaps, bootstrap disconnect and inaccessible-cache clearing.

    Two selected real authenticated WebSocket RPC cases passed, including denial
    without read scope. Server/contracts/client-runtime typechecks and new-code
    lint passed; the server bundle built. Existing unrelated warnings in the
    large server test file were not rewritten.

    Boundaries and next slice

    No UI, cross-environment aggregation, mandatory relay, notification sink,
    global acknowledgements, polling or raw conversation transfer. No live
    database was opened/migrated, no browser was launched and no new packaged
    platform qualification is claimed. Existing qualified release/handoff branches
    are unchanged. No push or PR was created.

    ATT-04 / #6 can now aggregate these independently authoritative streams in
    the shared client runtime. Scope each cache/subscription correctly, observe
    capability/authorization/reset states, and retain the protocol's atomic
    visibility and notification suppression rules.

    Model/harness: GPT-6 Astra (gpt-6-astra), GitHub Copilot CLI. No subagents
    or tpatch provider inference were used.

  7. jdbencardinop
    removed
    status:in-progressActively implementing the scoped issue
    on Sep 29, 2026
  8. jdbencardinop
    closed this as completedon Sep 29, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    fork-backlogMaintainer backlog for the multi-machine forkpriority:P0Stable reconciliation and attention core

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions