Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion docs/SPEC.md
Original file line number Diff line number Diff line change
Expand Up @@ -341,7 +341,7 @@ Current design documents:
keep the template library in shape so it doesn't dilute the
agent's attention, with implications for code review, smoke
tests, structure audit, and 360
- `efficacy-benchmark.md` — the pre-registered experiment that
- `efficacy-benchmark.md` — the experiment, its rules fixed in advance, that
measures whether a generated context file improves an agent's
output, and the harness a template change is measured with
- `testing-ai-assets.md` — the generic method the benchmark applies:
Expand Down
4 changes: 2 additions & 2 deletions docs/design/design-notes.md
Original file line number Diff line number Diff line change
Expand Up @@ -1210,8 +1210,8 @@ work outside accepted scope, not every possible improvement.

## 16. Outcome evaluation

The concrete, pre-registered form of this section is
`docs/design/efficacy-benchmark.md`: one application, three arms, paired
The concrete form of this section is
`docs/design/efficacy-benchmark.md`: one application, five arms, paired
trials, a layered judge and a verdict rule fixed before the first run. The
sketch below is the direction it was derived from.

Expand Down
2 changes: 1 addition & 1 deletion docs/design/testing-ai-assets.md
Original file line number Diff line number Diff line change
Expand Up @@ -309,7 +309,7 @@ checklist above and the calibration rules that keep it honest.
## Hand-off - the solid-ai-templates measurement

The concrete application of this method to the template library itself is
`efficacy-benchmark.md` in this directory, the pre-registered experiment
`efficacy-benchmark.md` in this directory, the experiment
that was blocked until this generic method concluded. This document hands
it the design, and the benchmark supplies the parts this one leaves
generic.
Expand Down
Loading
Loading