fix(insights): preserve business priorities through investigation selection

[izadoesdev]

Business context could load without affecting the final investigations: more than eight scheduled signals bypassed selection, long briefs displaced current corrections, and manual family refill restored general work the selector excluded.

Remove the count bypass within the existing serialized candidate-size bound, preserve the complete saved brief plus newest relevant correction within a capped source budget, omit bibliography only from ranking input, and refill only specialist families after manual selection. Due work, critical reliability, specialist coverage, investigation limits, the 15-second timeout and zero retries remain enforced. Investigation snapshots retain full sources. No prompt changes.

Fresh frozen synthetic comparisons using the production model: the priority ranked first in 4/10 baseline with-context paths versus 10/10 revised paths, then 6/6 reversed-order checks. In the busy nine-signal case, four irrelevant investigations became two useful findings across two repetitions. Tokens rose from 21,409 to 52,875 because the revised run examined evidence; this is a relevance improvement, not a cost or production-latency claim. Paired and reversed runs retain 52 selection paths, 32 native investigation traces and all visible requests/results/rejected drafts. Six existing live selection controls passed. Four no-context automatic publication checks fail equally across the two implementations; these arms deliberately remove the owner-purpose assertion, and require semantic interpretation rather than a general pass-rate claim.

Validation: integrated root lint; all 33 workspace typecheck tasks; 99 focused tests/517 assertions; pre-push suite 27/27 tasks (26 cached). Independent code review found no actionable regression. Rebased onto the merged canonical-reply fix #771; range-diff confirms the reviewed patch is unchanged. Independent semantic review covers all 84 full traces plus six summary-only controls. Parent mapping confirms the corrected selection defects are baseline-only. Residual unchanged-agent issues remain: one reversed-order finding broadens Google referrals to search, and team-described delivery semantics sometimes lose attribution. All measured funnel cohorts and controls survive; this is not a general grounding pass. The review also identifies seven avoidable serialized read turns and two corrected citation retries across the full sample, with no identical rereads.

Scope: investigation selection and its native paired evaluator. No dependency on held draft #751, schema change, or production-data evaluation. AI-assisted maintainer contribution.

Automated review disposition: the synthetic data is isolated evaluation fixtures in the existing native quality-eval mechanism, never product placeholder data; the suite explicitly requests synthetic-only measurements and preserves this boundary in its README. unknown is confined to opaque trace/usage serialization and a call counter, never accessed as trusted business data or used as a type constraint; current Bun fullstack guidance explicitly permits unknown at untrusted boundaries and the production SDK output remains validated. The checked-in Biome config explicitly disables noConsole, and this CLI follows the existing quality runner's JSON progress reporting. These three guideline-only comments do not identify runtime regressions or require changing the frozen experiment.

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated
dashboard	Ready	Preview	Sep 9, 2026 9:27am UTC
databuddy-status	Ready	Preview	Sep 9, 2026 9:27am UTC
documentation	Ready	Preview	Sep 9, 2026 9:27am UTC

[unkey-deploy]

The latest updates on your projects. Learn more about Unkey Deploy

Name	Status	Preview	Inspect	Updated (UTC)
links (preview)	Ready	Visit Preview	Inspect	Sep 9, 2026 9:26am

[coderabbitai]

Important

Review skipped

Auto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Repository UI

Review profile: ASSERTIVE

Plan: Advanced

Run ID: be7541cb-fcde-41e8-8840-f6d27b5868a0

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[greptile-apps]

Greptile Summary

This PR changes business-aware investigation planning so bounded candidate sets continue through model selection, preserves the complete saved business brief and newest relevant correction within a capped source budget, and prevents manual refill from restoring excluded general-family work.

• Removes the candidate-count bypass while retaining serialized-size, timeout, retry, and portfolio limits.
• Dynamically budgets ranking context for profile chunks and the newest relevant team reply while omitting bibliography fields from ranking input.
• Restricts manual post-selection refill to specialist families and preserves due and critical-reliability handling.
• Adds focused selection and coverage tests plus a paired synthetic evaluation harness.
• The implementation behavior appears consistent with the intended selection contract, but the new evaluator violates explicit repository rules governing mock data, concrete typing, and console output.

Confidence Score: 4/5

The selection behavior appears sound, but the explicit repository requirements violated by the new evaluation harness must be satisfied before merging.

No behavioral regression remains after tracing source ordering, fallback behavior, portfolio limits, and planned signal types; the remaining findings are concrete repository-rule violations in the evaluator.

Files Needing Attention: apps/insights/src/evals/context-selection.ts

Important Files Changed

Filename	Overview
apps/insights/src/business-aware-selection.ts	Removes the candidate-count bypass and revises source budgeting to retain all profile chunks plus the newest relevant correction under a hard cap.
apps/insights/src/coverage-planner.ts	Prevents manual selection refill from reintroducing unselected general-family candidates while retaining due and reliability safeguards.
apps/insights/src/business-aware-selection.test.ts	Adds coverage for large bounded candidate sets, maximum-sized saved context, current corrections, and conservative over-budget fallback.
apps/insights/src/coverage-planner.test.ts	Updates manual portfolio expectations and verifies that explicit general-traffic exclusions survive planning.
apps/insights/src/evals/context-selection.ts	Adds the paired selection evaluator, but its synthetic data, unknown types, and console output violate explicit repository instructions.
apps/insights/src/evals/quality.ts	Exports the existing quality evaluator for reuse by the new context-selection harness.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart LR
  A[Detected eligible signals] --> B{Serialized candidates within 48k?}
  B -- No --> F[Conservative coverage portfolio]
  B -- Yes --> C[Load complete saved brief]
  C --> D[Add newest relevant correction]
  D --> E[Business-aware model selection]
  E -- Failure or timeout --> F
  E -- Selection --> G[Preserve selected signals]
  G --> H[Add due and critical reliability work]
  H --> I{Manual scan?}
  I -- Yes --> J[Refill unused specialist families only]
  I -- No --> K[Keep scheduled portfolio bounded]
  J --> L[Freeze investigation plan]
  K --> L
  F --> L

Loading

_{Reviews (1): Last reviewed commit: "fix(insights): preserve business priorit..." | Re-trigger Greptile}