feat(insights): add grounded activation-date comparisons - #781
Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub.
2 Skipped Deployments
|
|
Important Review skippedAuto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: Repository UI Review profile: ASSERTIVE Plan: Advanced Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
|
The latest updates on your projects. Learn more about Unkey Deploy
|
bac9ee3 to
85ebde6
Compare
|
@greptileai review the final integrated head Please check the optional durable daily counts, date/timezone/count validation, bounded pooled comparison, source-signal selector, and preservation of the public/private native retention checks. No extra detector query or agent phase is added. A known same-population daily conflict keeps the whole run private even if weekly totals agree; prose or omission of the selector cannot bypass that gate. Legacy snapshots without daily data remain supported. Root lint, all 33 package typechecks, 704 insights tests (six live-only skipped), and 49 shared schema tests pass. The final integrated model comparison is running next. Please provide an explicit verdict on this exact head, including any correctness or security findings; a skipped-review status is not approval. |
Greptile SummaryAdds grounded, precomputed activation-date comparisons to retention investigations while preserving aggregate-only results for sparse or uniform cohorts.
Confidence Score: 5/5The PR appears safe to merge with no outstanding findings. No new changes were made after the previous review SHA. The prior type-boundary finding was correctly withdrawn, the forbidden iteration was replaced with a for-of loop, and both previous threads are resolved. Important Files Changed
Flowchart%%{init: {'theme': 'neutral'}}%%
flowchart TD
A[Run two weekly retention queries] --> B[Validate overall and daily rows]
B --> C[Store aggregate and daily measurements]
C --> D{Aggregate retention declined?}
D -- No --> E[No activation-date detail]
D -- Yes --> F[Evaluate bounded contiguous date groups]
F --> G{Population and materiality thresholds met?}
G -- No --> H[Publish aggregate-only finding]
G -- Yes --> I[Offer precomputed retentionDetail]
I --> J[Model may cite frozen signal detail]
J --> K{Contradictory daily read observed?}
K -- Yes --> L[Keep investigation private]
K -- No --> M[Publish validated finding]
Reviews (4): Last reviewed commit: "fix(insights): align retention copy with..." | Re-trigger Greptile |
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
|
@greptileai review the final pushed head The concrete forEach finding is fixed and its thread resolved. Please reassess the untrusted-input test-helper finding using the task-specific boundary-validation guidance and the resolved #780 precedent linked in its reply. Fresh model runs exposed one valid aggregate being withheld because its cause/purpose was treated as unknown. The native prompt now explicitly preserves an unexplained measured return finding when optional date detail is unavailable, and identifies the saved team definition as the supplied event purpose. It still distinguishes that meaning from verified emitter code and leaves contradictory measurements private. This is the only production change since your reviewed 85ebde6 head; all validation, pooled statistics and conflict logic are unchanged. 704 insights tests (six live-only skipped), 49 shared tests, root lint, all 33 package typechecks and all 27 pre-push test tasks pass. Independent integration review passed 275 native insights and 49 shared checks; the fresh final-head paired model comparison is being prepared. Please provide an explicit verdict on this exact head with all findings addressed. |
|
Want your agent to iterate on Greptile's feedback? Try greploops. |
|
@greptileai review final head All review threads are resolved. Fresh evaluations now preserve all six valid candidate findings, including the previously suppressed sparse aggregate, but exposed a conflicting instruction: the shared schema requests an 8–10-word summary plus a 4–8-word headline, while a detailed native comparison leaves 13 words. Two resulting briefs exceeded the 60-word budget. This update replaces those field descriptions only for native retention with concise qualitative 4–6-word guidance for each field. It adds no validator, repair phase or query, and leaves publication guards, structured evidence and the previously reviewed purpose instruction unchanged. Root lint, 33 package typechecks, 704 insights tests (six live-only skipped) and all 27 pre-push test tasks pass. A new unchanged-fixture paired comparison will qualify this exact head. Please provide an explicit final-head verdict. |
1 participant
Retention investigations discard validated daily activation cohorts after calculating the weekly comparison. They can report an overall drop, but cannot explain whether it is larger among profiles activating on particular dates without querying again and changing the cohort definition.
This change retains those daily counts in the existing signal and offers one optional, code-rendered activation-date comparison alongside the weekly result. It compares selected dates with corresponding prior-week dates and the remaining cohorts. All four pooled populations require at least 50 eligible profiles and conservative materiality thresholds. This is exploratory scope detail, not onset, cause or post-selection statistical significance. Sparse and uniform cases keep the aggregate alone.
The model selects
{retentionDetail: true}in its existing finish call. Raw daily rows remain durable but are omitted from the prompt. Native title/summary guidance shares the short copy budget; the saved team definition supplies event purpose, so unavailable date detail or unknown cause alone does not suppress a useful complete aggregate. No extra detector query, agent phase, UI control or table is added. A contradictory daily read keeps the whole run private even when weekly totals agree; dropping the selector or rewriting it as prose cannot bypass that gate. Legacy aggregate-only signals remain supported.Final paired evaluation: 12 fresh unscripted runs, three synthetic cases × two repetitions × both heads, through each head's real detector, preparation and agent. Baseline aee85ea published four of six valid findings; final head 31e9b24 published all six. The candidate selected the exact supported date contrast twice and no spurious sparse/uniform detail. Every candidate passed the unchanged evaluator: one model turn, zero agent reads and 40–56-word briefs. Both heads used the existing two detector queries per run. Total model turns fell 11→6, rejected finishes 5→0, median duration 7.8s→4.3s and tokens 70,290→37,292. These are bounded synthetic results, not production prevalence or a general latency guarantee. SQL compilation and native tool behavior are retained; database execution is replaced with frozen rows.
Validation: 704 insights tests passed (six live-only skipped), 49 shared tests, root lint, all 33 package typechecks and 27 pre-push test tasks. Independent integration review passed 275 native insights and 49 shared checks; later prompt/schema-description changes were separately reviewed and the converted native/non-native schemas checked. Earlier model attempts are retained: they exposed a withheld valid aggregate and competing copy-length instructions that the final head fixes. Every observable step is audited; no private reasoning claim. Review threads are resolved and an explicit final-head Greptile review is requested.
Scope: retention evidence persistence, rendering, native prompt and regression tests. No dependencies. Known overlap: draft #751 edits agent.ts and SPEC. This branch is rebased onto staging including merged #780 and preserves its public/private publication protections.
AI-assisted implementation and review using Codex.