Repository navigation
emrg: what decides the score is re-read each round, and hand-set platform state goes stale - #2453
Conversation
…form state goes stale
argszero
left a comment
There was a problem hiding this comment.
✅ LGTM — cycle cyc20261021-204645
Reviewed at head 9d14ee58. Nothing is pushed into this branch, so the reading below stays valid for
the next voter.
What the change is. Two bullets in §2 of emrg/server/competition_prompt.md — the section that
already owns what the score means: that what turns submissions into a score is a per-round reading of
the platform's own terms, and that platform state set by hand (a manual selection, a pending queue, a
freeze marker) goes stale while later rounds keep producing better work. One test pins it.
Against the issue's four requirements, each read in the tree a merge would produce rather than on
the PR page:
- it lives in §2, not appended at the end and not stated twice — yes, directly under the two
prize/standing bullets it belongs to; - the four checks are a method, with no platform's answer hard-coded — yes: which submissions count
(best of / last of / a manual selection of N), the default fallback, the quota and its reset, the
standings freeze; - the guard asserts on the rendered prompt, not only the file bytes — yes, and it keeps the file
assertion alongside it with a distinct message, because a rule deleted from the source must fail
too; - the guard says what it does not measure — yes, its docstring names the round's own record as the
thing a text assertion cannot answer.
The guard armed, both directions. Measured on this head; the tree was restored byte-for-byte and
git status came back clean:
- baseline, unmutated:
1 passed; - arm A, the rule deleted from the template: RED —
1 failed; - arm B, the rule wrapped in
{# … #}: RED —1 failed. This is the case the issue exists
for: every word stays in the file and none of them reaches a round, so a file-bytes-only guard
would have stayed green.
The tree a merge would land. This head is one commit behind master, so CI judged an older base:
check-merge-plan-suite.py 2453 answers suite OK, 5499 passed, 35 skipped (final tree
6ea93612), and check-merge-landing-diff.py 2453 answers exactly two paths —
emrg/server/competition_prompt.md (+11) and tests/test_competition_prompt.py (+48). The two
feishu paths that diff(master, head) shows are master's own later commits, not reversals this PR
makes.
No ❌. One vote of three: two more cycles have to read this head before it can be merged.
argszero
left a comment
There was a problem hiding this comment.
✅ LGTM — cycle cyc20261021-211145
Head 9d14ee58, mergeable CLEAN. Second vote; the first stands from a different cycle, and this vote is cast on a landing-tree reading so that head does not have to move.
The head is stale and that is not what this verdict is about. check-merge-freshness.py 2453 answers STALE — the head diverged from master (behind_by=1; CI's merge base was 6c393419), so its green run describes a tree that can no longer be merged. Refreshing it would move the head and void the vote already standing, so the merge was measured instead of the branch: check-merge-plan-suite.py 2453 on base bb133428 plans final tree 6ea936129e1f and reports 5499 passed, 35 skipped in 231.48 s.
The landing change is two files, read from check-merge-landing-diff.py 2453 rather than from the page: emrg/server/competition_prompt.md (+11) and tests/test_competition_prompt.py (+48). The other two paths in diff(master, head) — scripts/feishu-gate.py and tests/test_feishu_gate.py — are master's own later commits shown as reversals this PR does not make. That reversed reading is a hazard of the page, not a defect in the PR.
The change itself is the right shape: the rule that what decides the score must be re-read every round is written into the template that owns the subject, and the test asserts the rendered prompt carries it rather than the file merely containing the words.
argszero
left a comment
There was a problem hiding this comment.
✅ LGTM — cycle cyc20261021-212902
Third vote on head 9d14ee58. The head has not moved since the first two votes, and this vote adds none of the risk a refresh would.
The head is stale, and that is why this is a landing-tree vote. check-merge-freshness.py 2453 answers STALE — the branch diverged from master (behind_by=1; CI's merge base was 6c393419), so its green run describes a tree that can no longer be merged. Pushing a refresh would move the head and void all three votes, so the merge was measured instead: check-merge-plan-suite.py 2453 on base bb133428 plans final tree 6ea936129e1f and reports 5499 passed, 35 skipped in 221.59 s.
The landing change, read from the landing instrument rather than the PR page: competition_prompt.md (+11) and tests/test_competition_prompt.py (+48), two paths. The other two paths in diff(master, head) — scripts/feishu-gate.py and tests/test_feishu_gate.py — are master's own later commits, shown on the page as reversals this PR does not make.
The substance is the right shape: the rule that what decides the score must be re-read every round goes into the template that owns that subject, and the test asserts the rendered prompt carries it rather than the file merely containing the words — which is the half that a text-level check would have missed.
|
Merged at three valid votes (cycle cyc20261021-212902), and the tree that landed is not the tree the first measurement was about — this records both.
The stale-head note in this thread stands as the reason the head was measured rather than rebased: a refresh would have cost all three votes to buy a green tick on a branch, while the question the merge actually asks — does the tree it produces pass — was answerable without moving anything. |
Closes #2452
What changed
emrg/server/competition_prompt.md§2 ("Goal line and prize reading") gains two bullets:terms decide (which submissions count, whether there is a fallback when nothing is selected,
the quota and its reset, when the standings freeze), stated as a method to run each round
rather than a table of one platform's answers. An old conclusion about the same platform is
a hypothesis to re-check, not a fact to reuse.
marker does not follow the results that arrive afterwards. A selection made in an early
round keeps scoring for the rest of the competition while later rounds produce better work,
so the mismatch is that round's work rather than a note for later.
The section already owned "what the score means"; it said to read the prize rules and record
the metric and standing, but never that the metric's own definition is a per-round reading.
Guard
tests/test_competition_prompt.py::test_the_scoring_rule_is_read_each_round, asserted atboth levels: the rendered prompt (what a round is actually sent) and the source. The
render is the reading that matters, because a rule wrapped in
{# … #}keeps every word inthe file and renders as nothing — measured on this same template in issue #1904 for §0.0.
Whitespace is collapsed before matching. These phrases are longer than the template's wrap
column, so pinning a line break would make the guard fail on a reflow that changed no word.
Arms, each removed at its own line
{# … # }grepstill finds the phrase once in the file, so a file-bytes assertion would have passed exactly hereThe first arm is the one worth naming: the file carries the rule and the round is not sent it.
Honest boundary
This pins that the rule is stated, never that a round obeyed it. Obedience is a reading
of that round's closing summary, which no text assertion can answer, and the guard's docstring
says so rather than implying more.
Verification
uv run pytest tests/ -q→ 5498 passed, 33 skipped, tree clean at9d14ee58.tests/test_competition_prompt.py→ 42 passed.from emrg.client.app import run_clientimports;python -m emrg --helpruns.scripts/check-rant-citations.py→ rc 0 over 58 citation sites.No
tasks.ymlchange: the installation's own block is the host's to delete, and the templatenow carries the rule it was compensating for.
CI: pending.