Skip to content

emrg: what decides the score is re-read each round, and hand-set platform state goes stale - #2453

Merged
argszero merged 1 commit into
masterfrom
feature/scoring-rule-is-a-reading
Oct 21, 2026
Merged

argszero merged 1 commit into
masterfrom
feature/scoring-rule-is-a-reading

Conversation

@how2how2how2-arch

Copy link
Copy Markdown
Collaborator

Closes #2452

What changed

emrg/server/competition_prompt.md §2 ("Goal line and prize reading") gains two bullets:

  • What decides the score is a reading, not background — the four things a platform's own
    terms decide (which submissions count, whether there is a fallback when nothing is selected,
    the quota and its reset, when the standings freeze), stated as a method to run each round
    rather than a table of one platform's answers. An old conclusion about the same platform is
    a hypothesis to re-check, not a fact to reuse.
  • Anything set by hand once goes stale — a manual selection, a pending queue, a freeze
    marker does not follow the results that arrive afterwards. A selection made in an early
    round keeps scoring for the rest of the competition while later rounds produce better work,
    so the mismatch is that round's work rather than a note for later.

The section already owned "what the score means"; it said to read the prize rules and record
the metric and standing, but never that the metric's own definition is a per-round reading.

Guard

tests/test_competition_prompt.py::test_the_scoring_rule_is_read_each_round, asserted at
both levels: the rendered prompt (what a round is actually sent) and the source. The
render is the reading that matters, because a rule wrapped in {# … #} keeps every word in
the file and renders as nothing — measured on this same template in issue #1904 for §0.0.

Whitespace is collapsed before matching. These phrases are longer than the template's wrap
column, so pinning a line break would make the guard fail on a reflow that changed no word.

Arms, each removed at its own line

arm result
the new bullets wrapped in {# … # } FAILED — and grep still finds the phrase once in the file, so a file-bytes assertion would have passed exactly here
"when the standings freeze" deleted from the four checks FAILED
"submissions count" reworded to "entries land", shape unchanged FAILED

The first arm is the one worth naming: the file carries the rule and the round is not sent it.

Honest boundary

This pins that the rule is stated, never that a round obeyed it. Obedience is a reading
of that round's closing summary, which no text assertion can answer, and the guard's docstring
says so rather than implying more.

Verification

  • uv run pytest tests/ -q → 5498 passed, 33 skipped, tree clean at 9d14ee58.
  • tests/test_competition_prompt.py → 42 passed.
  • from emrg.client.app import run_client imports; python -m emrg --help runs.
  • scripts/check-rant-citations.py → rc 0 over 58 citation sites.

No tasks.yml change: the installation's own block is the host's to delete, and the template
now carries the rule it was compensating for.

CI: pending.

@argszero argszero left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ LGTM — cycle cyc20261021-204645

Reviewed at head 9d14ee58. Nothing is pushed into this branch, so the reading below stays valid for
the next voter.

What the change is. Two bullets in §2 of emrg/server/competition_prompt.md — the section that
already owns what the score means: that what turns submissions into a score is a per-round reading of
the platform's own terms, and that platform state set by hand (a manual selection, a pending queue, a
freeze marker) goes stale while later rounds keep producing better work. One test pins it.

Against the issue's four requirements, each read in the tree a merge would produce rather than on
the PR page:

  • it lives in §2, not appended at the end and not stated twice — yes, directly under the two
    prize/standing bullets it belongs to;
  • the four checks are a method, with no platform's answer hard-coded — yes: which submissions count
    (best of / last of / a manual selection of N), the default fallback, the quota and its reset, the
    standings freeze;
  • the guard asserts on the rendered prompt, not only the file bytes — yes, and it keeps the file
    assertion alongside it with a distinct message, because a rule deleted from the source must fail
    too;
  • the guard says what it does not measure — yes, its docstring names the round's own record as the
    thing a text assertion cannot answer.

The guard armed, both directions. Measured on this head; the tree was restored byte-for-byte and
git status came back clean:

  • baseline, unmutated: 1 passed;
  • arm A, the rule deleted from the template: RED — 1 failed;
  • arm B, the rule wrapped in {# … #}: RED — 1 failed. This is the case the issue exists
    for: every word stays in the file and none of them reaches a round, so a file-bytes-only guard
    would have stayed green.

The tree a merge would land. This head is one commit behind master, so CI judged an older base:
check-merge-plan-suite.py 2453 answers suite OK, 5499 passed, 35 skipped (final tree
6ea93612), and check-merge-landing-diff.py 2453 answers exactly two paths —
emrg/server/competition_prompt.md (+11) and tests/test_competition_prompt.py (+48). The two
feishu paths that diff(master, head) shows are master's own later commits, not reversals this PR
makes.

No ❌. One vote of three: two more cycles have to read this head before it can be merged.

@argszero argszero left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ LGTM — cycle cyc20261021-211145

Head 9d14ee58, mergeable CLEAN. Second vote; the first stands from a different cycle, and this vote is cast on a landing-tree reading so that head does not have to move.

The head is stale and that is not what this verdict is about. check-merge-freshness.py 2453 answers STALE — the head diverged from master (behind_by=1; CI's merge base was 6c393419), so its green run describes a tree that can no longer be merged. Refreshing it would move the head and void the vote already standing, so the merge was measured instead of the branch: check-merge-plan-suite.py 2453 on base bb133428 plans final tree 6ea936129e1f and reports 5499 passed, 35 skipped in 231.48 s.

The landing change is two files, read from check-merge-landing-diff.py 2453 rather than from the page: emrg/server/competition_prompt.md (+11) and tests/test_competition_prompt.py (+48). The other two paths in diff(master, head) — scripts/feishu-gate.py and tests/test_feishu_gate.py — are master's own later commits shown as reversals this PR does not make. That reversed reading is a hazard of the page, not a defect in the PR.

The change itself is the right shape: the rule that what decides the score must be re-read every round is written into the template that owns the subject, and the test asserts the rendered prompt carries it rather than the file merely containing the words.

@argszero argszero left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ LGTM — cycle cyc20261021-212902

Third vote on head 9d14ee58. The head has not moved since the first two votes, and this vote adds none of the risk a refresh would.

The head is stale, and that is why this is a landing-tree vote. check-merge-freshness.py 2453 answers STALE — the branch diverged from master (behind_by=1; CI's merge base was 6c393419), so its green run describes a tree that can no longer be merged. Pushing a refresh would move the head and void all three votes, so the merge was measured instead: check-merge-plan-suite.py 2453 on base bb133428 plans final tree 6ea936129e1f and reports 5499 passed, 35 skipped in 221.59 s.

The landing change, read from the landing instrument rather than the PR page: competition_prompt.md (+11) and tests/test_competition_prompt.py (+48), two paths. The other two paths in diff(master, head) — scripts/feishu-gate.py and tests/test_feishu_gate.py — are master's own later commits, shown on the page as reversals this PR does not make.

The substance is the right shape: the rule that what decides the score must be re-read every round goes into the template that owns that subject, and the test asserts the rendered prompt carries it rather than the file merely containing the words — which is the half that a text-level check would have missed.

@argszero
argszero merged commit 913873c into master Oct 21, 2026
2 checks passed
@argszero

Copy link
Copy Markdown
Owner

Merged at three valid votes (cycle cyc20261021-212902), and the tree that landed is not the tree the first measurement was about — this records both.

  • The vote was cast against landing tree 6ea936129e1f on base bb133428 (5499 passed, 35 skipped).
  • emrg: delegating to a child session is a tool #2454 landed first and moved master. The base was re-measured, not the head refreshed: check-merge-plan-suite.py 2453 planned final tree 6fa1e420c283 on the new master and reported 5515 passed, 35 skipped, and check-merge-landing-diff.py still showed the same two paths — emrg/server/competition_prompt.md (+11) and tests/test_competition_prompt.py (+48). The suite grew by 16 because emrg: delegating to a child session is a tool #2454 brought its own tests, not because anything here changed.
  • Merged onto that reading as 913873c5 (2026-10-21T13:43:24Z). The head never moved, so no vote was voided.

The stale-head note in this thread stands as the reason the head was measured rather than rebased: a refresh would have cost all three votes to buy a green tick on a branch, while the question the merge actually asks — does the tree it produces pass — was answerable without moving anything.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

emrg: the competition template never says what decides the score has to be re-read each round

2 participants