Skip to content

feat(efficacy): the control gains a second application and the security and data-protection trees - #1855

Merged
braboj merged 3 commits into
mainfrom
feat/efficacy-control-security
Sep 25, 2026
Merged

braboj merged 3 commits into
mainfrom
feat/efficacy-control-security

Conversation

@braboj

@braboj braboj commented Sep 25, 2026

Copy link
Copy Markdown
Collaborator

Part of #1767. This is the control half of "each judge row passes the control fixture" (design §5.8). The re-run itself comes after merge.

The second application. base-2.zip is git archive of the uncounted calibration trial (claude-sonnet-5, arm none, commit f74c0cc, 1542 s). Two builds reproduce the same sha256, and the builder pins it the way it pins base.zip.

Four trees built from it, each described in the control README:

Tree Row it moves Change
insecure-1 security, down secret key a literal, password stored and compared as typed, user lookup built from a string, next followed anywhere
secured-1 security, up secret key from the environment first, cookie flags, three security headers, next checked by a named function
leaky-1 data protection, down customer name, email and address logged; erasure only raises the flag
protected-1 data protection, up one PERSONAL_FIELDS read by erasure and export, one lookup for a live customer, secure_delete

Checks:

  • Twelve new landing rows: each edit changed the tree it names and no other. The self-test (in CI) builds both applications, and all 23 rows landed.
  • Each tree's own suite: 89/89 on all five.
  • The hidden suite at 65bfb9a: every tree fails exactly the same 240 checks as base-2 and as score.py's run of the trial, and passes the same 293.
  • Limit: the trial seeds only through its command, not when the application opens an empty database as the spec asks. So 240 checks behind sign-in fail on every tree alike, and the comparison covers the 293 that pass. The README records this as a dated measurement.

The app already scores high on both rows. base-2 already has one guard, CSRF on every form, a salted hash and a local-only next, so a rise in security can only be about 4 → 5. If gpt-6-astra already gives base-2 a 5, the rise cannot be shown, and that is a finding about the row.

Existing code changed. pinned() takes the archive and digest, tree extraction moved into extract(), and landings() reads each table against its own application. The first application's six trees and their 11 landing rows are unchanged.

No CHANGELOG entry: the benchmark is this repository's own.

Checks run: smoke 33/33; control self-test 23/23; conformance 0 failed, its readings read (four files, the refactor named above).

🤖 Generated with Claude Code

braboj and others added 3 commits September 25, 2026 13:13
…ty and data-protection trees

The round-1 application has no sign-in or customers, so the security and
data_protection rows could not move on it. base-2.zip pins the uncounted
calibration trial (git archive of f74c0cc) by sha256, and the builder makes
four trees from it: insecure-1 and secured-1 for security, leaky-1 and
protected-1 for data protection. Twelve landing checks confirm each edit
changed the tree it names and no other; the self-test in CI builds both
applications.

Part of #1767.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…its limit

Every tree of the second application fails the same 240 hidden-suite checks
as its base, and passes the same 293: the calibration trial does not seed an
empty database, so checks behind sign-in fail alike on all five. Also wraps
a line past the declared width.

Part of #1767.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Part of #1767.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@braboj braboj added P1 High — must fix before next milestone task Atomic implementable work labels Sep 25, 2026
@braboj
braboj merged commit 79774b4 into main Sep 25, 2026
1 check passed
@braboj
braboj deleted the feat/efficacy-control-security branch September 25, 2026 10:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

P1 High — must fix before next milestone task Atomic implementable work

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant