Skip to content

Rust #325: Nexus catalog administration, service-call execution, admission limits and call reads - #793

Merged
rmcdaniel merged 7 commits into
mainfrom
rust-325-nexus
Oct 26, 2026
Merged

rmcdaniel merged 7 commits into
mainfrom
rust-325-nexus

Conversation

@rmcdaniel

@rmcdaniel rmcdaniel commented Oct 26, 2026 •

Copy link
Copy Markdown
Member

Part of #325: Nexus plan steps 1, 2a, 2b and 2c (plan, step 2 design). Native service catalog administration and Nexus service-call execution, matching published Server 2.11.1, fixture first. The steps are batched to save CI cycles; their commits are separate.

Review tier: advisory (pre-RC native). It changes the native server, a native-owned schema relation, the shared parity harness and the port log. It touches no published PHP surface, credentials, production data or CI trust. Policy: #325 comment 6423232503.

Step 1: catalog administration and reads

Fixture first. service-endpoints was recorded against published 2.11.1 on its SQLite reference: 72 requests in order, in a fresh namespace.

  • It creates, reads, updates and deletes endpoints, services and operations through all 15 admin routes.
  • Lookups and creates: lowercased names; 409 with the existing id for a taken name; 404 for a missing parent first (endpoint_not_found carries accepted: false).
  • Validation refusals for names, descriptions, documents, operation mode and kind, and the binding check.
  • Deadline policies: wrong keys, a bad timeout_seconds, a malformed RFC3339, an impossible calendar date or hour, and years outside 1970–9999.
  • Updates: an operation rename is prohibited, an empty rename is allowed, and the binding check runs again against stored values.
  • Deletes refuse an endpoint with services and a service with operations.
  • Catalog reads over the live rows, by page, filter and identity, including another namespace's identity (404).
  • Generated ids and whole-second times are placeholders that must recur consistently. Everything else is exact.
  • Its contract test rejects 15 mutations, among them a 501, an uppercase id, a moved updated_at on an unchanged update, a list read back as an object, services listed by name, and an allowed rename.

Native. The new module rust/src/runtime/service_endpoints.rs serves ServiceCatalogController's endpoint, service and operation routes:

  • Names are lowercased with PHP 8's strtolower, in the path and the body. The database's own comparison finds them, as PHP's query does.
  • A create checks the name, then stores a lowercase ULID (HasUlids) and microsecond times. An update writes only when a field changes, as Eloquent's dirty check does.
  • JSON documents read back as PHP's array cast reads them: {} is [], and {"0":..,"1":..} is a list.
  • Service and operation lists come in creation order: PHP's relations order by created_at, id before the controller's name order. Endpoint lists are by name.
  • Every write takes native's serialized write transaction.
  • Admin role only. The named-namespace allow-list admits exactly these route shapes. The deeper execute and service-call routes stay operator routes, answering 501.

rust/src/runtime/service_catalog.rs now renders the namespace's rows: PHP's paging (count, then the window), filters by endpoint and service id, related lists capped at 100 with related_has_more, and an operation's endpoint and service when the namespace holds them. Native records no service calls yet, so a calls list is still PHP's empty page.

Shared validation rules (laravel_rules.rs):

  • Rule::Regex, Laravel's regex: as validateRegex applies it: to a string or to a number's PHP string.
  • Rule::Prohibited, an implicit rule.
  • size measures a number without a numeric rule by its PHP string, as Laravel's getSize does. Before, max:191 measured 500 by its value. The existing Rule::Pattern users are unchanged.

Three documented differences, also in the module doc and the port log:

  1. Native JSON documents don't keep object key order. An update that only reorders a document's keys writes nothing, where PHP's strict comparison writes it and moves updated_at.
  2. Concurrent creates of one name: PHP can hit its unique index (500), and a delete can race a child's create. Native serializes the writes, so the second create gets 409.
  3. Laravel also reads fields from the query string. Native reads a missing field from it only under its exact name, without PHP's metadata[a]=1 array syntax.

Takeover no longer refuses a PHP database that holds endpoints, services or operations; a source with service calls is still refused. Tests write the rows as published PHP does (lowercase ULIDs, microsecond times, json_encode documents with escaped slashes), take the database over, and check that native renders them, keeps updated_at on an unchanged update, lists a new native service after PHP's, and refuses deleting the endpoint.

Not in this PR: service execution, service-call show and cancel, and the start_service_operation command (Nexus steps 2–4).

The 15 routes leave unserved-php-routes.v2.11.1.txt.

Step 2a: service-call execution, show and cancel

Fixture first. service-calls was recorded against published 2.11.1: 82 requests in a fresh namespace (71 for step 2a, 11 more for step 2b). It passes against published PHP on its SQLite reference and as an app on MySQL 8, MariaDB 10.11 and PostgreSQL 17. It covers:

  • Refusals before admission: catalog 404s, validation, the payload codec, Avro arguments and envelopes, search attributes, and the runtime transport's refusal of a provider reference before routing.
  • Accepted calls: activity_execution and invocable_http, with stamped handlers (an uppercase ULID, or the binding's own ids), caller options, sync and async modes; show by trimmed id, and 404 under another operation.
  • Handler failures: a binding without a target is a 409, retried in the request up to retry_policy.max_attempts unless its type is non-retryable, with service_call_attempts recorded.
  • Deadlines: timeout_seconds sets call_deadline_at; a deadline_at already past is admitted and timed out in the same request (200, accepted: true).
  • Cancel: with a reason; a repeat is 409 service_call_terminal; mode: none is 409 cancellation_not_allowed; the reason is validated after the lookup.
  • The boundary: required roles and caller allow and deny lists give 403 with the target redacted, while show on the audit row is unredacted. An open circuit is 503 with retry 30; an invalid shared limit is 503 service_boundary_admission_unavailable.
  • Idempotent replays: the describe shape for a first or a cancelled call, a new call for another caller namespace, and a replay refused after the operation gains a required role (403 with a new audit row).
  • Shared admission limits (step 2b): a rate of one per minute refuses the second call 429 rate_limit_exceeded with the configured retry of 7, while another caller namespace has its own window; an async call's concurrency lease stays taken, so the next is refused 429 concurrency_limit_exceeded; a sync call returns its lease, so the next is admitted; show reads the throttled audit row. The probe starts the rate window at least 15 seconds before a minute ends so the window can't roll over mid-case.

The contract matches exact answers with placeholders for generated values: lowercase and uppercase ULIDs, whole-second and microsecond times, also inside messages. Its test rejects 11 mutations.

Native. The new module rust/src/runtime/service_calls.rs ports executeOperation, serviceCallShow and serviceCallCancel over Workflow 2.10.2's DefaultServiceControlPlane, from a source-level spec of the published code:

  • the boundary's authorization and circuit checks, the shared limits, and the audit row written for every decision, with the caller principal;
  • the dispatch loop with retry bookkeeping and backoff;
  • lazy deadline expiry on execute, replay, show and cancel, with show's controller half built from the row as read before the expiry, as PHP does;
  • the merged array_replace responses.

Step 2b, shared admission limits (SharedServiceBoundaryPolicy::admitWithLock): the per-minute rate and the concurrency limit per caller boundary (caller|target|endpoint|service|operation), checked in PHP's order (rate, then concurrency) and counted only when admitted. PHP keeps these counters in its cache; native keeps them in dw_service_admission_counters, read and written under the transition lock in the call's own transaction. The keys and lives are PHP's: rate:<sha256>:<minute> for 120 s and concurrency:<sha256> for a day. A sync call returns its lease when the request ends, even when dispatch fails, as PHP's finally does; an async call's lease stays until it expires. Allowed calls record PHP's admission metadata (admission_scope: shared_cache and the four limits), and refusals record observed_window_count or observed_in_flight.

Schema version 7 (file 0007_service_calls.sql) adds two native-owned relations on SQLite, PostgreSQL, MySQL and MariaDB: dw_service_call_deadlines for each call's deadline and dw_service_admission_counters for the limits. Catalogs, takeover relations, parity schema scripts and namespace cleanup are updated. The SQLite parity schema check now also ignores the deadline index, which step 2a missed.

Not in this PR, each answered 501 rust_capability_not_implemented before anything is written:

  • workflow-bound handler kinds (step 3);
  • an external_payload reference in arguments;
  • cancelling into a linked workflow.

Takeover still refuses a source that holds service calls (step 5).

Step 2c, catalog call reads. Recording calls made the /api/service-catalog call reads wrong: they still answered PHP's empty page and not-found. GET /api/service-catalog/calls and GET /api/service-catalog/calls/{id} now read the recorded calls as ServiceCatalogReadController::callsIndex and callShow do:

  • ServiceCatalog's scopes: by default the calls the namespace made or that target it; also owned, caller and target.
  • Status and outcome filters, and their buckets.
  • Newest accepted first, as orderByDesc reads it on each database. A rejected call has no accepted_at, so where it sorts follows the database's NULL order, for PHP and native alike.
  • ServiceCallView's list item and detail, with the caller and link references, the result envelope and the failure type. Payload references and the Nexus result show only to the namespace that owns the call.

The service-calls fixture gains nine catalog reads recorded against published 2.11.1 (91 cases in all). Each list holds only accepted or only rejected calls, so the recorded order is the same on every database.

Decided differences, also in the module doc and the port log:

  1. Deadlines live in dw_service_call_deadlines, not in 2.11.1's workflow_service_calls.call_deadline_at column. Native keeps the frozen PHP tables plus its own relations, so takeover layouts stay recognisable; step 5 will copy deadlines when takeover accepts service calls. This deviates from the plan's "add the column".
  2. Native commits a call's admission and dispatch in one serialized transaction, where PHP commits each step on its own; the rows end the same. A non-zero retry backoff is waited out with the transaction committed.
  3. The admission counters live in a table under the transition lock, not in a cache under a cache lock: native never answers 503 because the lock was busy, and a call whose audit row fails to write takes back its rate count too. The namespace-wide limits and hard ceilings run only at their default of none.

The execute, show and cancel routes leave unserved-php-routes.v2.11.1.txt too. With this PR native answers 174 of the 175 published route methods; only the cancellation-scope checkpoint is left.

Also:

Qualification

Head 355aa395e, seven commits on main 65b11820 (#790). The first six replay the qualified working branches onto that base; the only hand edits are the route count in the port log and the CI test list. The seventh is step 2c.

  • Full native suite on SQLite at 0529d87f1 (cargo test --all-targets): lib 328, execution 555, upgrade_inspect 2, all passing. At 355aa395e: the seven service_calls tests, including catalog_reads_list_the_recorded_calls.
  • Execution tests on SQLite, PostgreSQL 17, MySQL 8 and MariaDB 10.11: the seven service_calls tests (including shared_admission_limits_count_per_boundary and catalog_reads_list_the_recorded_calls), the three service_endpoints tests, and the namespaces, route_coverage and workflow_stream_bounds tests that the schema and route changes touch.
  • Takeover qualification with schema version 7 (--ignored): PostgreSQL 13, MySQL 19 and MariaDB 19 tests, including qualification_takeover_keeps_php_catalog_rows_that_native_then_serves. On SQLite, takeover_keeps_php_catalog_rows_that_native_then_serves and the upgrade tests run in the full suite.
  • SQLite parity schema check (schema.php, as the hosted job runs it) against the frozen 2.5.14 layout: pass, 49 tables and 333 indexes.
  • Shared fixtures recorded over HTTP:
    • published 2.11.1 passes service-calls (91 cases, including step 2c's catalog reads) on its SQLite reference and as an app on MySQL 8, MariaDB 10.11 and PostgreSQL 17; service-endpoints and service-catalog passed the same way for step 1;
    • a native debug server passes service-calls (91), service-endpoints and service-catalog on all four backends.
  • Contract tests: service calls, service endpoints and service catalog all pass, with each mutation rejected (53 tests before step 2c; step 2c adds five call-read mutations).
  • No PHP app/ changes. The only PHP is parity harness: the probes and the schema scripts.

🤖 Generated with Claude Code

rmcdaniel and others added 7 commits October 26, 2026 21:27
Native now answers the 15 admin routes under /api/service-endpoints as
published Server 2.11.1's ServiceCatalogController does: create, list, show,
update and delete endpoints, services and operations by lowercased name,
with PHP's validation, 404 and 409 bodies, Eloquent's dirty check and the
deletion refusals. The catalog reads under /api/service-catalog now render
the namespace's endpoint, service and operation rows with PHP's paging and
related limits; calls stay empty until native records service calls.

The shared rules gain Laravel's regex (strings and numbers) and prohibited,
and size a number without a numeric rule by its PHP string, as getSize does.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The service-endpoints fixture drives the 15 catalog administration routes
and the catalog reads over their rows in order, 72 requests in a fresh
namespace, and was recorded against published Server 2.11.1 on its SQLite
reference. Generated ids and whole-second times are placeholders that must
recur consistently; everything else is exact. The port log counts 161 of
175 served route methods and lists the three documented differences.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Native now serves the endpoint, service and operation tables, so takeover
no longer refuses a source that holds their rows; service calls are still
refused. Tests write the rows as published PHP does (lowercase ULIDs,
microsecond times, json_encode documents with escaped slashes) on SQLite,
PostgreSQL, MySQL and MariaDB, take the database over, and check that
native renders, compares and extends them as its own.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
POST .../operations/{operation}/execute, GET .../service-calls/{call}
and POST .../service-calls/{call}/cancel now answer as published Server
2.11.1's ServiceCatalogController does over Workflow 2.10.2's
DefaultServiceControlPlane, for the handler kinds that stay accepted
(activity_execution and invocable_http): Laravel validation, the payload
codec and Avro arguments, admission at the service boundary with an
audit row for every decision (required roles, caller allow and deny
lists, the circuit breaker, and the 503 for an invalid shared limit),
the stamped handler, handler failures retried in the request, deadlines
expired lazily on every read, idempotent replays that re-check
authorization, and cancellation.

A workflow-bound handler kind, an operation with a valid shared limit,
an external_payload argument and cancelling into a linked workflow
answer 501 before anything is written.

Schema version 7 adds the native-owned dw_service_call_deadlines, so
the frozen PHP tables stay as they are and takeover layouts stay
recognisable. The service-calls fixture records 71 requests against
published 2.11.1, with shared placeholder matching for generated ids
and times.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sonar flags the closure as redundant.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The service boundary's per-minute rate and concurrency limits now admit
or refuse calls instead of answering 501. PHP keeps these counters in its
cache under a cache lock (SharedServiceBoundaryPolicy::admitWithLock);
native keeps them in the native-owned dw_service_admission_counters,
read and written under the transition lock in the call's transaction,
with PHP's keys and lives: a rate count per boundary and minute for
120 s, and a concurrency lease per boundary for a day. A sync call
returns its lease when the request ends, even after a failed dispatch;
an async call's lease stays until it expires, as PHP's does.

Schema version 7's migration is renamed 0007_service_calls.sql and adds
the counters table on SQLite, PostgreSQL, MySQL and MariaDB, with the
catalogs, takeover relations and parity schema scripts. The SQLite
parity schema check now also ignores the deadline index, which the
previous step missed.

The service-calls fixture gains 11 cases recorded against published
2.11.1: a rate refusal with its configured retry, another caller's own
window, an async lease kept, a sync lease returned, and the throttled
audit row. The probe opens the rate window at least 15 seconds before a
minute ends so it cannot roll over mid-case.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
GET /api/service-catalog/calls and GET /api/service-catalog/calls/{id}
now read the calls native records, as published Server 2.11.1's
ServiceCatalogReadController does: ServiceCatalog's scopes (by default
the calls the namespace made or that target it), the status and outcome
filters and buckets, newest accepted first as orderByDesc reads them on
each database, and ServiceCallView's list item and detail with the
caller and link references, the result envelope and the failure type.
Payload references and the Nexus result show only to the namespace that
owns the call. Before this, both answered PHP's empty page and
not-found even when calls existed.

The service-calls fixture gains nine catalog reads recorded against
published 2.11.1. Each list holds only accepted or only rejected calls:
a rejected call has no accepted_at, and where NULL sorts differs between
databases, for PHP as for native.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@rmcdaniel rmcdaniel changed the title Rust #325: Nexus catalog administration, service-call execution and shared admission limits Rust #325: Nexus catalog administration, service-call execution, admission limits and call reads Oct 26, 2026
@sonarqubecloud

Copy link
Copy Markdown

// Endpoints: lowercased names, PHP's array documents, 409 with the id.
let (status, billing) = request(&app, "POST", "/api/service-endpoints",
json!({"endpoint_name": " Billing ", "description": "Payments", "metadata": {"team": "core", "empty": {}}})).await;
assert_eq!(status, StatusCode::CREATED, "{billing}");
let (status, billing) = request(&app, "POST", "/api/service-endpoints",
json!({"endpoint_name": " Billing ", "description": "Payments", "metadata": {"team": "core", "empty": {}}})).await;
assert_eq!(status, StatusCode::CREATED, "{billing}");
assert!(is_ulid(&billing["id"]), "{billing}");
@rmcdaniel

Copy link
Copy Markdown
Member Author

CodeQL (not a required check) flags two high-severity "cleartext logging" alerts in rust/tests/execution/service_endpoints.rs. Both are assertion messages that format a test variable named billing, which holds the JSON of a test endpoint called "Billing". CodeQL's heuristic reads the name as financial data; no secret or real data is involved. I didn't restart this PR's CI for a variable rename. The fix renames the variable to first_endpoint and lands in the next #325 PR (2.11.1 takeover), which closes both alerts on main.

@rmcdaniel
rmcdaniel added this pull request to the merge queue Oct 26, 2026
@github-merge-queue
github-merge-queue Bot removed this pull request from the merge queue due to failed status checks Oct 26, 2026
@rmcdaniel
rmcdaniel added this pull request to the merge queue Oct 26, 2026
rmcdaniel added a commit that referenced this pull request Oct 26, 2026
#793's merge group failed one PostgreSQL corpus cell on worker-sessions
reacquire_terminal: ttl_span 1799 where the recording has 1800. PHP
computes a new session's ttl_expires_at from its request clock, then
Eloquent stamps created_at when it inserts the row; when the insert
lands in the next second, the probe's ttl_span (ttl_expires_at minus
created_at) reads one second short. Native stamps both from one clock
reading, so the race is the published reference's.

The contract now reads a ttl_span exactly one second short of the
recorded one as the recorded one, and nothing else. Its test checks
that one second short passes and that two seconds short or one second
long still fail.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@rmcdaniel

Copy link
Copy Markdown
Member Author

Requeued at 22:38Z, pinned to the same head 355aa395e. The first group (38fc87c02) failed one cell, postgres-storage (postgres-17, corpus), on the worker-sessions reacquire_terminal step: ttl_span 1799 against the recorded 1800. This PR doesn't touch worker sessions. The race is in the published PHP reference: it computes ttl_expires_at from its request clock, then Eloquent stamps created_at at insert, so a second boundary in between shortens the span by one. Native stamps both from one clock reading. The contract allowance and its tests ship in the next #325 PR (2.11.1 takeover).

Merged via the queue into main with commit eec4251 Oct 26, 2026
56 of 57 checks passed
@rmcdaniel

Copy link
Copy Markdown
Member Author

Merged through the merge queue at 23:02Z as eec4251a9. Main is that exact group commit: main + #791 + this PR's head 355aa395e, the tree the queue tested.

  • First enqueue at 22:10Z; that group (38fc87c02) failed one worker-sessions cell on the PHP reference's one-second race, explained above.
  • Requeued pinned at 22:38Z; the second group passed: Shared Server fixtures, PHPUnit Feature, Public Boundary, and the automatic queue Sonar run.

Native now answers 174 of 175 published route methods. Next: #795 (takeover of published 2.11.1 databases), already open and stacked on this.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants