Repository navigation
fix(report): estimate a cloud candidate's violations by searching, not by default - #524
Conversation
…t by default A cloud candidate's estimatedViolations of zero usually meant the agent never searched, while the recipe called that zero honest. A new 'Estimating violations' section asks for a fast, good-faith estimate of whether the pattern is happening now: a handful of searches, starting from the real shape of a past fix, searching for the problem rather than the protection, separating syntactic from data-flow patterns, spot-checking hits, and writing the low end of the range. Zero stays a valid answer once the agent has looked.
|
@claude /review |
|
Claude finished @theCodeDrift's task in 1m 50s —— View job Review completeReview mode: incremental — read 0 prior review thread(s) before reviewing.
Findings[New] Missing topic-version bump — posted inline on Minor, not inlined: no test pins the new Otherwise the change reads well: the new section's guidance (search for the problem not the protection, treat syntactic vs. data-flow patterns differently, spot-check and scale, write the low end) is internally consistent with the updated |
The recipe body changed, so a cached v4 copy needs a signal that its guidance on estimating cloud violations is superseded.
An agent once searched for the escape helper, found where escaping already happened, and published zero. Pin the section that asks for a search for the problem itself, and that the old 'saw no instance' zero is gone.
Added in 8dea6da: a test pins the section's key instructions (look before answering zero, search for the problem not the protection, start from the real shape, write the low end), and asserts the old "saw no instance" zero is gone from step 2. The topic-version finding was fixed in 807a88e. — AI Coding Agent |
A cloud candidate's
estimatedViolationsof zero usually meant the agent never searched. The recipe allowed that: "Zero is honest if you proposed the candidate from review alone and saw no instance in the code." In one onboarding run, an agent picked an HTML-escaping candidate whose fix had landed four separate times. It grepped once for the escape helpers, which shows where escaping already happens and nothing about where it is missing, and published zero.This keeps zero as a valid answer and asks the agent to look first.
## Estimating violationssection in thereportrecipe, alongside## Counting surfaced. It frames the estimate as "how likely is this happening now" and keeps it fast, at a handful of searches with no whole-file reads or subagents. It asks the agent to:historyzeros are honest only for a source the agent read and found empty.report-facts.tsmatch, so the embedded facts schema says the same thing.There is no changeset: the
reportcommand and its topic are not inv0.12.0, so the existing report changesets cover this.