Skip to content

Select complete bound workloads with strict cost and capability evidence - #522

Merged
zzylol merged 6 commits into
mainfrom
feat/complete-workload-cost-selection
Sep 8, 2026
Merged

zzylol merged 6 commits into
mainfrom
feat/complete-workload-cost-selection

Conversation

@zzylol

@zzylol zzylol commented Sep 8, 2026

Copy link
Copy Markdown
Contributor

Before

The migration stack reported joint shared-producer lifecycle components, but did not compare complete bound workloads before deployment. Missing source, state, transport and query-operator costs could not be distinguished from a fully priced decision. Exact fallback compilation unnecessarily required unused state/window evidence.

After

  • Add a strict, provider-priced coverage manifest over the existing physical projection (not a second semantic DAG), with exact implementation identities and workload/capability/version/horizon scope.
  • Compare the Planner-selected continuously maintained workload and its whole-workload exact alternative before commitment. Charge each shared source/state/location once and every reachable consumer/result operation at its recurrence.
  • Require complete, finite, nonnegative, fresh and executable quotes; retain unavailable reasons, and fail closed if no completely priced candidate remains. Exact-service access must be attested, not inferred from a cheap number.
  • Wire evidence into version-2 canonical startup snapshots and live compile-and-publish. Preserve version-1/no-quote compatibility explicitly without claiming complete costing.
  • Expose a read-only cost-manifests endpoint, an offline manifest example, live decision reports, startup report logging and provider instructions.
  • Skip unused state/window evidence for native exact fallback (regression reproduced before fixing).

Validation

Six focused cost tests cover selection reversal under expensive retention, one producer for two consumers, missing/duplicate/stale/mismatched/infeasible/negative evidence, strict v2 replay, and no unused-state evidence for exact fallback.

Production-process acceptance exercises quoted v2 shared exact dashboard startup and live distributed manifest/quote/publish/query flow, including actual Collector Rust updates and concurrent generation cutover. Synthetic fixture prices test selection; they are not production calibration.

Local combined-stack results: 654 control-plane library tests, 28 binary tests, one control-plane integration test, 977 data-plane library tests, and three production-process tests passed. cargo fmt -p control_plane -p data_plane -- --check and git diff --check passed. Existing compiler warnings were not expanded into unrelated cleanup.

Scope and dependency

Stacked on #521 (test/successful-generation-cutover), following #513#520 and merged Planner #356/#357. Do not merge independently or automatically.

This implements provider-priced complete selection for the supported runtime profile, not exhaustive search across all Planner algorithms/lifecycles. Additional Planner-authorized bindable forests can use the same comparison boundary. The deployment must supply and refresh real cost/capability evidence; no production telemetry is invented. The generic Planner physical-formula provider is not reimplemented using guessed source statistics.

No independent reviewer was used; this PR has implementation tests and self-review.

Planner #357 compatibility update

Pins all Planner crates and the emitted PLANNER_REVISION to merged commit c3410d14865497758d212e4265ad25c782187de1. The existing provider-priced workload comparison remains compatible; this update does not enable the separate offline empirical execution prototype. Regenerate manifests and quotes for the new Planner revision.

Includes the reviewed exact-fallback fix: enumerate and deduplicate every named input source, and reject incomplete or unresolved source coverage. On this branch, 657 control-plane library tests, 29 binary tests and one integration test pass with the merged Planner revision.

@zzylol
zzylol changed the base branch from test/successful-generation-cutover to main September 8, 2026 20:18
@zzylol
zzylol merged commit 5ad30be into main Sep 8, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant