Skip to content

feat(planner): bind offline frequency evidence and preserve o11y fallbacks - #524

Merged
zzylol merged 5 commits into
mainfrom
codex/empirical-o11y-322
Sep 8, 2026
Merged

zzylol merged 5 commits into
mainfrom
codex/empirical-o11y-322

Conversation

@zzylol

@zzylol zzylol commented Sep 8, 2026

Copy link
Copy Markdown
Contributor

Before this PR

The control plane could not use query-matched offline frequency measurements to choose a configuration. Bare selectors, sort roots, and comparison/filter roots could also fail binding instead of preserving their exact query semantics.

After this PR

  • Accept validated offline evidence for candidate ordering and fixed-snapshot frequency comparison. Require compatible parameters and formal accuracy minima; exact winners or unavailable evidence preserve exact execution.
  • Preserve unsupported o11y root shapes as explicit exact fallback. Binding success remains separate from installed execution and query acceleration.
  • Integrate current main, align all Planner dependencies and executable example snapshots with abb2f20e27091ac0c60715dc42b9b59c75b3447c, and adapt the typed physical-resource API while retaining the archived evidence JSON schema.
  • Reject legacy keyed/non-column summary updates that the one-column runtime adapter cannot represent. In particular, constant unit weight must not be interpreted as sample-value SUM.
  • Include offline replay/export examples and tests. Historical offline reports remain historical observations; they were not regenerated by this integration.

This PR supplies the offline evidence and binding layer. It does not establish end-to-end o11ybench execution, warm coverage, or performance/resource benefits versus Prometheus; those require the execution and matched-comparison follow-ups.

Validation

  • Fresh-source compilation: 658 control-plane library tests, 6 offline-evidence tests, 1 exact-fallback integration test, and 14 data-plane legacy-summary tests passed.
  • Both offline examples built against the pinned dependency. The pinned upstream corpus replay bound 27/27 queries in each of exact/default modes; this is binding evidence, not a serving or speedup result.
  • Five Python export tests, package formatting, and diff checks passed.
  • Shared-target source roots were explicitly refreshed before compiling to avoid reusing another worktree's test artifact.

@zzylol

zzylol commented Sep 8, 2026

Copy link
Copy Markdown
Contributor Author

Documentation migration completed in c2a632e: docs/design_docs/empirical-o11y-execution-plan.md, docs/offline-o11y-final-2026-09-08.md and docs/user_guide/o11y-replay.md now live here, together with tools/empirical-bench. Cross-repository links were repaired; historical results remain explicitly historical. Planner PR ProjectASAP/ASAPPlanner#358 is closed in favor of this repository-owned documentation. Backend-only runtime work is separate and still under validation.

@zzylol

zzylol commented Sep 8, 2026

Copy link
Copy Markdown
Contributor Author

Created the two stacked implementation follow-ups: #525 (cost-selected production execution replay) → #526 (matched exact comparison and scoped resource reporting). Both are Draft: conformance/regression tests pass, but the complete real-corpus run, matching provider calibration, isolated exact baseline and full lifecycle/resource accounting remain unverified. #525 also integrates the existing deployment foundation through #505; see its Stack section before merging. No performance-benefit or five-criteria-completion claim is made.

@zzylol
zzylol merged commit a20d56d into main Sep 8, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant