Conversation
This was referenced Sep 8, 2026
Contributor
Author
|
Closing at the author’s request: evaluation documentation belongs in ASAPQuery-backend. All three documents have been moved, link-adjusted, committed and pushed in ProjectASAP/ASAPQuery-backend#524 (commit c2a632e). The backend also now owns the sketch-bench tools. No documentation from this PR needs to merge into ASAPPlanner. Core shared resource/evidence work remains in #357. This closure does not claim completion or performance benefits for the backend-only runtime prototype. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
Refs #322 and depends on #357. Document the corrected repository boundary and backend-driven o11y evaluation scope.
Before this PR
The proposed evaluation mixed planner-only replay, benchmark production and synthetic fixed-snapshot timing. Those tools did not establish the requested backend execution benefits.
After this PR
This PR now contains only three English documentation files:
All sketch-bench producers/probes and the small offline recommendation diagnostic belong in ProjectASAP/ASAPQuery-backend#524. The standalone planner-only
o11y_replayis removed. The synthetico11y_exact_benchis removed from this PR and is not part of the prototype. No Rust binaries, Cargo dependency changes, benchmark scripts, or generated result files remain in this diff.Current prototype target — backend work, not completed by this PR
Use the supplied
/mydata/metrics.txtOpenMetrics data and upstream o11y queries. Data and queries enter ASAPQuery-backend, which calls ASAPPlanner and performs real data-plane execution without ASAPCollector or Prometheus runtime dependencies.Compare exact and ASAP query results on the same data, labels and evaluation times before reporting latency/CPU/memory/storage benefits. Report build/ingestion costs separately. Unsupported queries and exact fallbacks must not count as accelerated successes. The earlier 27-query binding fixture is not the current upstream corpus (28 query occurrences, 24 unique queries).
The runtime implementation is still in progress in the backend. This documentation PR does not claim end-to-end speedups, lower resource use, full query support, or completion of that prototype.
Review and validation
Stacked on the #357 branch; merge #357 first, then retarget to main. Documentation-only diff; local whitespace checks pass. Historical artifacts remain available at https://github.com/ProjectASAP/ASAPPlanner/releases/tag/offline-evidence-2026-09-08.