feat(insights): retain business changes and improve capability discovery - #759
Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
|
Important Review skippedAuto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: Repository UI Review profile: ASSERTIVE Plan: Advanced Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
158e14f to
362791a
Compare
|
The latest updates on your projects. Learn more about Unkey Deploy
|
Greptile SummaryThis PR updates Insights investigation guidance to retain independent business comparisons and distinguish unavailable diagnostics from findings. It also introduces compact cross-category query discovery, clarifies saved versus cohort-filtered goal and funnel definitions, and expands synthetic evaluation coverage.
Confidence Score: 4/5The behavioral changes appear sound, but the explicit repository typing requirement must be satisfied before merging. No correctness or security failure remains; the only accepted issue is the newly introduced use of prohibited Files Needing Attention: apps/insights/src/evals/quality.ts Important Files Changed
Flowchart%%{init: {'theme': 'neutral'}}%%
flowchart TD
A[Unknown analytics capability] --> B[Request compact full catalog]
B --> C[Select relevant builder name]
C --> D[Targeted discovery lookup]
D --> E[Receive complete query contract]
E --> F[Execute get_data query]
F --> G[Retain independent measured comparisons]
F --> H[Record unavailable diagnostics as investigation limits]
Reviews (1): Last reviewed commit: "fix(ai): distinguish saved definitions f..." | Re-trigger Greptile |
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
| function depthRevenueTool(reordered: boolean, available = true) { | ||
| return { | ||
| ...analyticsTools.get_data, | ||
| execute: (value: unknown) => { |
There was a problem hiding this comment.
The new evaluation code uses unknown for the synthetic revenue input. The same added pattern appears in the discovery executor and accepted-finish state. This violates the repository directive to use proper explicit types instead of any, unknown, or never, so it must be corrected before merging.
Context Used: Basic guidelines for the project so vibe coders do... (source)
Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!
1 participant
Investigations could drop a measured attribution decline after seeing stable sales and higher refunds. Preserve independent changes and controls from the same subject, and distinguish those comparisons from unrelated sibling findings. Keep unavailable diagnostics as investigation limits.
Discovery accepts an explicit all-category search and returns a compact full catalog; targeted lookups still return complete query contracts. Goal and funnel descriptions distinguish saved configuration from cohort-filtered measurements. Four synthetic holdouts cover competing revenue facts, field order, and capability discovery outside an initially guessed category.
Validation: repository lint, all 33 typecheck/build tasks, 510 Insights tests (6 live tests skipped), and 9 discovery/cohort tests passed. Fresh native-model comparisons retained all three material revenue comparisons in 2/2 corrected core runs where baseline omitted attribution in 2/2. Unfiltered catalog output is approximately 77% smaller. The final combined Terra batch passed 13/13 automated checks with no rejected finishes. Separate semantic review still found uneven wording and unnecessary reads when a capability is absent; automated checks are not a complete quality score.
Independent code review found and resolved two fixture-contract issues: compact discovery parity and source-window bounds. A follow-up review of the rebased changes found no blockers. The existing Terra model remains unchanged: neither a broader prompt rewrite nor a more expensive default demonstrated a sufficient overall benefit.
Scope: investigation instructions, discovery output, tool descriptions, and synthetic evaluation coverage. The separately reviewed recovery-verification fix is already in staging via #760; this branch is rebased onto it. No schema changes or dependency on #751.