From d1b0223db131358709f5c18a4066e36c4a905f5b Mon Sep 17 00:00:00 2001 From: Branimir Georgiev Date: Fri, 25 Sep 2026 12:07:38 +0300 Subject: [PATCH] docs(efficacy): the control read at three judgings per tree The gpt-6-astra top-up completed three judgings on each of the six trees. The control README records the means: every primary falls on the tree built to damage it and rises on one built to improve it. Part of #1767. Co-Authored-By: Claude Opus 5.5 (1M context) --- tests/efficacy/control/README.md | 22 ++++++++++++++++++++-- 1 file changed, 20 insertions(+), 2 deletions(-) diff --git a/tests/efficacy/control/README.md b/tests/efficacy/control/README.md index f28a4aaa..c8ae8b68 100644 --- a/tests/efficacy/control/README.md +++ b/tests/efficacy/control/README.md @@ -132,8 +132,7 @@ counted. A bold cell differs from `base-1` in the same run. | test_quality | 3 | 3 | **4** | **4** | **4** | **4** | `improved-2` holds two judgings, meaned. The top-up to three per tree stopped -on the judge plan's usage limit after one call; a later `--repeat 3` on the -same root attempts only what is still owed. +on the judge plan's usage limit after one call; the next section completes it. #### Opus 5 through the claude backend, a cross-check on the rubric @@ -180,6 +179,25 @@ trusted. The repair is tracked in #1827. +### Three judgings per tree, 2026-09-25 + +gpt-6-astra at effort `high`, the same root as the single judgings above, +topped up with `--repeat 3`. Evidence found 100 % on every judging. Each +cell is the mean, followed by the three scores; a bold cell differs from +`base-1`. + +| Dimension | base-1 | degraded-1 | obscured-1 | improved-1 | improved-2 | improved-3 | +|---|---|---|---|---|---|---| +| **design** | 3.00 (333) | **2.00** (222) | 3.00 (333) | **3.67** (344) | **3.67** (434) | **4.00** (444) | +| **readability** | 3.00 (333) | 3.00 (333) | **1.00** (111) | 3.00 (333) | 3.00 (333) | **3.33** (343) | +| **maintainability** | 3.00 (333) | **2.33** (223) | 3.00 (333) | 3.00 (333) | 3.00 (333) | **4.00** (444) | + +Averaged over three, every primary falls on the tree built to damage it and +rises on a tree built to improve it. `design` and `maintainability` fall on +`degraded-1`, `readability` falls on `obscured-1`, and all three rise on +`improved-3`. The one-point flips of single judgings remain inside each +triple, which is what the mean is there to absorb. + ## Why the base is a pinned archive `base.zip` holds one trial's output: the application every tree is built