From 4c8e80c65a52b4beb34278c2b194dc50f8fbe410 Mon Sep 17 00:00:00 2001 From: luisleo526 <20812494+luisleo526@users.noreply.github.com> Date: Sun, 15 Nov 2026 22:25:28 +0800 Subject: [PATCH 1/7] Count unmatched trades on both sides in the corpus grader (#836) The corpus grader compared trade counts as a net difference, so an unmatched engine trade and an unmatched TradingView trade cancelled out. count_abs_delta now adds every unmatched gated TradingView and engine trade, and excellent requires it to be zero. The net difference stays available as net_count_abs_delta; count_delta keeps the legacy relative difference for the strong tier. The pinned parity headline moves to excellent=301, strong=10, the regrade measured in #836. All 312 pinned strategy trade outputs still match their baseline hashes. --- CHANGELOG.md | 2 + README.md | 15 +- scripts/check_corpus_parity.sh | 10 +- scripts/test_verify_corpus_count_rule.py | 177 ++++++++++++++++++++--- scripts/test_verify_corpus_metrics.py | 35 ++++- scripts/test_verify_corpus_open_mark.py | 4 + scripts/verify_corpus.py | 52 +++++-- 7 files changed, 259 insertions(+), 36 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index e0d992543..1aed83922 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -8,6 +8,8 @@ version number follows semantic versioning over the surfaces the ## Unreleased - Explicit `strategy.order()` quantities now floor executable units at declared decimal quantity steps while preserving the source quantity; the boundary control is not measured against TradingView (#716 item 2). +- Corrected the previously published parity grading: counting unmatched trades net instead of gross overstated the excellent tier by 10 of 312 reference probes (311 instead of 301) and by 24 of 7,989 graded probes on the recorded population. The trade data is unchanged. +- The corpus grader counts every unmatched gated trade on either side as a gross mismatch for the excellent gate. One engine-only trade and one TradingView-only trade count as two mismatches, so equal list sizes cannot satisfy that gate. The count percentage and strong-tier rules retain their legacy calculation from the difference in gated list sizes (#836). - The prebuilt Linux archives need glibc 2.35 or newer (for example Ubuntu 22.04 or Debian 12 and later), and the release check now refuses an archive that needs a newer glibc. - Multi-day charts restart their trading-date grid each year; measured on 24x7 daily input for chart bars, last_bar_index, barstate.islast and the open-trade report row. Streams keep the previous chart path; selected windows on generated N-day chart partitions are refused. Time offsets, exchange calendars and intraday input are not measured against TradingView. - `str.match` no longer reports "no match" after a memory failure; the failure now reaches `out_of_memory`. diff --git a/README.md b/README.md index 70f668ce7..490e0033d 100644 --- a/README.md +++ b/README.md @@ -304,6 +304,8 @@ the affected scripts and how to recognise it. **Measured 2026-11-12** on engine `6e27d0f2` with codegen-oss `126e055f` (baseline `pineforge-parity-baseline-20261112-release-tag-cd9c85a6`, snapshot `cd9c85a6`): **7,989 graded probes, 7,983 excellent + 6 strong**, 0 below strong and 0 engine errors across 18 market/timeframe lanes. The historical release 1.0.1 inventory holds 4,776,328 trades. +The historical gross-count regrade recorded in [#836](https://github.com/pineforge-4pass/pineforge-engine/issues/836) measured **7,959 excellent / 30 strong**: 24 probes moved from excellent to strong and none lower. That regrade and the dated scoreboard above describe their recorded pair and population; they are not a fresh measurement of this revision. The grader uses gross counting. Recorded release results remain historical measurements. + Release **1.4.0 (2026-10-08)** maps to **7,983 excellent / 6 strong** on 7,989 graded probes (baseline `pineforge-parity-baseline-20261007-engine-b3192bfc`, promoted 2026-10-07 UTC), with 0 below strong and 0 engine errors. These are recorded grading outcomes for that pair and population. Release **1.3.0 grades 7,982 excellent / 7 strong until the next release**, on 7,989 probes (baseline `pineforge-parity-baseline-20261006-engine-7a1f01c0`, 2026-10-06). A main scoreboard advance does not change release results. @@ -336,10 +338,19 @@ The hard lane combines 3092026-11-12** on engine `6e27d0f2` with codegen-oss `126e055f` (baseline `pineforge-parity-baseline-20261112-release-tag-cd9c85a6`, snapshot `cd9c85a6`): **7,989 graded probes, 7,983 excellent + 6 strong**, 0 below strong and 0 engine errors across 18 market/timeframe lanes. The historical release 1.0.1 inventory holds 4,776,328 trades. -The historical gross-count regrade recorded in [#836](https://github.com/pineforge-4pass/pineforge-engine/issues/836) measured **7,959 excellent / 30 strong**: 24 probes moved from excellent to strong and none lower. That regrade and the dated scoreboard above describe their recorded pair and population; they are not a fresh measurement of this revision. The grader uses gross counting. Recorded release results remain historical measurements. +For the population reported in [#836](https://github.com/pineforge-4pass/pineforge-engine/issues/836) on **2026-11-11**, the historical gross-count regrade measured **7,959 excellent / 30 strong**: 24 probes moved from excellent to strong and none lower. That regrade and the dated scoreboard above describe their recorded pair and population; they are not a fresh measurement of this revision. The grader uses gross counting. Recorded release results remain historical measurements. Release **1.4.0 (2026-10-08)** maps to **7,983 excellent / 6 strong** on 7,989 graded probes (baseline `pineforge-parity-baseline-20261007-engine-b3192bfc`, promoted 2026-10-07 UTC), with 0 below strong and 0 engine errors. These are recorded grading outcomes for that pair and population. From d25a4775a9a7f072899c21f5bc69fcc9430b6577 Mon Sep 17 00:00:00 2001 From: luisleo526 <20812494+luisleo526@users.noreply.github.com> Date: Mon, 16 Nov 2026 00:01:17 +0800 Subject: [PATCH 4/7] Use gross unmatched percentages for strong grading and FIFO MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Balanced unmatched trades can leave the two list sizes equal. Normalize the canonical gross unmatched count by the larger compared population in both aligned and zero-alignment paths. Strong grading and fragmented-FIFO metric eligibility share that percentage and retain their existing thresholds. The net count difference remains report-only. Add real-tape boundary regressions with below-limit controls, and correct the rescinded legacy-percentage expectations. Date the historical 24 movers and 7,959 excellent / 30 strong to the population reported on 2026-11-11. Keep the existing 312-strategy headline pin pending the actual regrade. Documentation provenance: intentionally replace or remove the following sentences. The earlier Excellent/Strong definitions are included for the net comparison with the preceding main revision. No other docs sentences are removed by this change. 1. README.md: "The historical gross-count regrade recorded in [#836](https://github.com/pineforge-4pass/pineforge-engine/issues/836) measured **7,959 excellent / 30 strong**: 24 probes moved from excellent to strong and none lower." 2. README.md: "- **strong** — ≥ 95% matched, difference in gated trade-list sizes below 6% of the larger count, entries within 0.1% and exits within 0.5% at p90;" 3. README.md: "The separate `net_count_abs_delta` (`net`) reports the difference in list sizes." 4. README.md: "The percentage metric, `count_delta`, retains the legacy relative difference in gated list sizes for the strong tier and other percentage comparisons." 5. README.md: "It can be zero when both sides have unmatched trades; that does not satisfy the excellent count gate." 6. README.md (earlier wording in the net comparison): "- **excellent** — the same number of trades, ≥ 99% of TradingView's trades matched, the distinct-entry identity check passed, entry and exit prices within 0.01% and PnL within 1% at the 90th percentile (trailing-stop scripts are graded on the *production* profile: exits within 0.02% and PnL within 1%, since a trail fill depends on TradingView's sub-bar path);" 7. README.md (earlier wording in the net comparison): "- **strong** — ≥ 95% matched, trade count within 6%, entries within 0.1% and exits within 0.5% at p90;" 8. CHANGELOG.md: "Corrected the previously published parity grading: counting unmatched trades net instead of gross overstated the excellent tier by 10 of 312 reference probes (311 instead of 301) and by 24 of 7,989 graded probes on the recorded population." 9. CHANGELOG.md: "The corpus grader counts every unmatched gated trade on either side as a gross mismatch for the excellent gate." 10. CHANGELOG.md: "One engine-only trade and one TradingView-only trade count as two mismatches, so equal list sizes cannot satisfy that gate." 11. CHANGELOG.md: "The count percentage and strong-tier rules retain their legacy calculation from the difference in gated list sizes (#836)." --- CHANGELOG.md | 2 +- README.md | 12 +- scripts/test_verify_corpus_count_rule.py | 71 +++++++++--- scripts/test_verify_corpus_metrics.py | 136 +++++++++++++++++++++-- scripts/verify_corpus.py | 15 +-- 5 files changed, 197 insertions(+), 39 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 63893726d..9db5a3de7 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -9,7 +9,7 @@ version number follows semantic versioning over the surfaces the - Explicit `strategy.order()` quantities now floor executable units at declared decimal quantity steps while preserving the source quantity; the boundary control is not measured against TradingView (#716 item 2). - Corrected the parity grading for the historical population reported in [#836](https://github.com/pineforge-4pass/pineforge-engine/issues/836) on 2026-11-11: counting unmatched trades net instead of gross overstated the excellent tier by 10 of 312 reference probes (311 instead of 301) and by 24 of 7,989 graded probes. The trade data is unchanged. -- The corpus grader counts every unmatched gated trade on either side as a gross mismatch for the excellent gate. One engine-only trade and one TradingView-only trade count as two mismatches, so equal list sizes cannot satisfy that gate. The count percentage and strong-tier rules retain their legacy calculation from the difference in gated list sizes (#836). +- The corpus grader counts every unmatched gated trade on either side as a gross mismatch. One engine-only trade and one TradingView-only trade count as two mismatches, so equal list sizes cannot satisfy the excellent count gate. The same gross count, divided by the larger gated trade count (zero for two empty pools), supplies the percentage for strong-tier grading and fragmented-FIFO eligibility; thresholds are unchanged (#836). - The prebuilt Linux archives need glibc 2.35 or newer (for example Ubuntu 22.04 or Debian 12 and later), and the release check now refuses an archive that needs a newer glibc. - Multi-day charts restart their trading-date grid each year; measured on 24x7 daily input for chart bars, last_bar_index, barstate.islast and the open-trade report row. Streams keep the previous chart path; selected windows on generated N-day chart partitions are refused. Time offsets, exchange calendars and intraday input are not measured against TradingView. - `str.match` no longer reports "no match" after a memory failure; the failure now reaches `out_of_memory`. diff --git a/README.md b/README.md index 162635fdb..9f3ef2465 100644 --- a/README.md +++ b/README.md @@ -339,17 +339,17 @@ The hard lane combines 309309 of them in the measured population: **309 excellent**" Documentation provenance for preceding changes in this PR (verbatim originals; these are earlier rewordings/deletions, not additional edits in this change): 1. README.md: "The historical gross-count regrade recorded in [#836](https://github.com/pineforge-4pass/pineforge-engine/issues/836) measured **7,959 excellent / 30 strong**: 24 probes moved from excellent to strong and none lower." 2. README.md: "- **strong** — ≥ 95% matched, difference in gated trade-list sizes below 6% of the larger count, entries within 0.1% and exits within 0.5% at p90;" 3. README.md: "The separate `net_count_abs_delta` (`net`) reports the difference in list sizes." 4. README.md: "The percentage metric, `count_delta`, retains the legacy relative difference in gated list sizes for the strong tier and other percentage comparisons." 5. README.md: "It can be zero when both sides have unmatched trades; that does not satisfy the excellent count gate." 6. README.md: "- **excellent** — the same number of trades, ≥ 99% of TradingView's trades matched, the distinct-entry identity check passed, entry and exit prices within 0.01% and PnL within 1% at the 90th percentile (trailing-stop scripts are graded on the *production* profile: exits within 0.02% and PnL within 1%, since a trail fill depends on TradingView's sub-bar path);" 7. README.md: "- **strong** — ≥ 95% matched, trade count within 6%, entries within 0.1% and exits within 0.5% at p90;" 8. CHANGELOG.md: "Corrected the previously published parity grading: counting unmatched trades net instead of gross overstated the excellent tier by 10 of 312 reference probes (311 instead of 301) and by 24 of 7,989 graded probes on the recorded population." 9. CHANGELOG.md: "The corpus grader counts every unmatched gated trade on either side as a gross mismatch for the excellent gate." 10. CHANGELOG.md: "One engine-only trade and one TradingView-only trade count as two mismatches, so equal list sizes cannot satisfy that gate." 11. CHANGELOG.md: "The count percentage and strong-tier rules retain their legacy calculation from the difference in gated list sizes (#836)." --- README.md | 4 +- scripts/check_corpus_parity.sh | 5 +- scripts/test_verify_corpus_count_rule.py | 140 +++++++++++++++++++++-- scripts/verify_corpus.py | 16 ++- 4 files changed, 152 insertions(+), 13 deletions(-) diff --git a/README.md b/README.md index 9f3ef2465..7c36fdf4f 100644 --- a/README.md +++ b/README.md @@ -314,9 +314,11 @@ The quantities above render from the public [facts tokens](https://github.com/pi | Board | Test set | Result | |---|---|---| -| **Public** — [open corpus](https://github.com/pineforge-4pass/pineforge-corpus) | 312 reference strategies on BINANCE:ETHUSDT.P 15m, Apache-2.0, reproducible by anyone | this repository's sweep of the v1.0.1 library: **311 excellent + 1 declared anomaly**; the 309 of them in the measured population: **309 excellent** | +| **Public** — [open corpus](https://github.com/pineforge-4pass/pineforge-corpus) | 312 reference strategies on BINANCE:ETHUSDT.P 15m, Apache-2.0, reproducible by anyone | reference sweep **2026-11-16** ([#836](https://github.com/pineforge-4pass/pineforge-engine/issues/836), [#883](https://github.com/pineforge-4pass/pineforge-engine/issues/883)): **301 excellent + 9 strong + 1 moderate + 1 declared anomaly**; the 309 of them in the dated scoreboard above: **309 excellent** | | **Closed test** | 7,680 probes of 741 TradingView scripts across the 18 lanes: 680 community-shared scripts (7,179 probes), private under TradingView's Terms of Service, and 61 probe scripts the maintainers wrote (501 probes) | **7,674 excellent + 6 strong** | +The gross count percentage in [#836](https://github.com/pineforge-4pass/pineforge-engine/issues/836) exposes an unchanged trade mismatch in `validation/mtf-triple-tf-macd-hist-confluence-01`, which remains **Moderate** in the 2026-11-16 reference sweep. Aggregate reference verification accepts only that full id at Moderate under a dated floor exception that ends when [#883](https://github.com/pineforge-4pass/pineforge-engine/issues/883) closes. Every other strategy keeps the Strong floor; missing inputs or lower tiers still fail. This reference measurement is separate from the dated scoreboard metadata above; baseline promotion requires its own verification. + ### Lane by lane diff --git a/scripts/check_corpus_parity.sh b/scripts/check_corpus_parity.sh index 14e22169d..cfc417752 100755 --- a/scripts/check_corpus_parity.sh +++ b/scripts/check_corpus_parity.sh @@ -91,8 +91,11 @@ PY="${PYTHON:-python3}" # expectation corrected: excellent=311, strong=0 -> excellent=301, strong=10, # because the excellent-only rule requires zero gross unmatched TV + engine # trades; all 312 pinned strategy trade outputs still match the baseline hashes. +# expectation corrected: excellent=301, strong=10, moderate=0 -> +# excellent=301, strong=9, moderate=1, because the gross count percentage exposes +# an unchanged trade mismatch (#836/#883; measured 2026-11-16). # scripts/verify_corpus.py prints this line verbatim. -EXPECTED_VERIFY="${EXPECTED_VERIFY:-Verified 312 strategies — excellent=301, strong=10, moderate=0, weak=0, minimal=0, anomaly=1, engine_only=0, missing=0}" +EXPECTED_VERIFY="${EXPECTED_VERIFY:-Verified 312 strategies — excellent=301, strong=9, moderate=1, weak=0, minimal=0, anomaly=1, engine_only=0, missing=0}" log() { printf '\033[1;34m[corpus-parity]\033[0m %s\n' "$*"; } warn() { printf '\033[1;33m[corpus-parity]\033[0m %s\n' "$*" >&2; } diff --git a/scripts/test_verify_corpus_count_rule.py b/scripts/test_verify_corpus_count_rule.py index 43d99c9cd..6a299fe3e 100644 --- a/scripts/test_verify_corpus_count_rule.py +++ b/scripts/test_verify_corpus_count_rule.py @@ -21,8 +21,11 @@ from pathlib import Path from unittest.mock import patch +import verify_corpus from verify_corpus import MATCH_WINDOW_SECONDS, analyze_strategy, main -from test_verify_corpus_trail_profile import TRADE_COLUMNS, TRAIL_STRATEGY +from test_verify_corpus_trail_profile import ( + STRICT_STRATEGY, TRADE_COLUMNS, TRAIL_STRATEGY, +) def _checked_path(path: Path) -> Path: @@ -47,6 +50,23 @@ def _checked_path(path: Path) -> Path: ClosedTrade = tuple[datetime, datetime, str] +REFERENCE_STRATEGY_ID = "validation/mtf-triple-tf-macd-hist-confluence-01" + + +def _reference_floor_tapes(replacements=(10,)): + start = datetime(2025, 1, 1, tzinfo=timezone.utc) + tv_trades = [ + (start + timedelta(days=index), + start + timedelta(days=index, hours=1), f"entry-{index + 1}") + for index in range(24) + ] + engine_trades = [ + (entry + timedelta(hours=12), exit_time + timedelta(hours=12), + identity + "-replacement") + if index in replacements else (entry, exit_time, identity) + for index, (entry, exit_time, identity) in enumerate(tv_trades) + ] + return tv_trades, engine_trades def _write_trades( @@ -84,6 +104,8 @@ def _analyze_tapes( run_root: Path, name: str, tv_trades: list, engine_trades: list, *, write_trades=_write_trades, + strategy_source=TRAIL_STRATEGY, aggregate=False, + omit_tv=False, omit_engine=False, ): strategy = _checked_path(run_root / name) files = { @@ -101,22 +123,25 @@ def _analyze_tapes( print(str(_checked_path(Path(__file__).with_name(filename))), flush=True) # The real analyzer probes these OHLCV candidates. They are deliberately # absent in the fresh fixture directory; no trim implementation is replaced. - strategy.mkdir() + strategy.mkdir(parents=True) with files["strategy.pine"].open("x", encoding="utf-8") as handle: - handle.write(TRAIL_STRATEGY) + handle.write(strategy_source) with files["inputs.json"].open("x", encoding="utf-8") as handle: json.dump({"tv_trades_csv_tz": "utc_plus_8"}, handle) - write_trades(files["tv_trades.csv"], tv_trades, tv_timezone=True) - write_trades(files["engine_trades.csv"], engine_trades, tv_timezone=False) + if not omit_tv: + write_trades(files["tv_trades.csv"], tv_trades, tv_timezone=True) + if not omit_engine: + write_trades(files["engine_trades.csv"], engine_trades, tv_timezone=False) result = analyze_strategy(strategy) row = result.report_row() cli_output = StringIO() + corpus_root = run_root if aggregate else strategy cli_argv = [ - "verify_corpus.py", str(strategy), - "--corpus-root", str(strategy), - "--derived-workspace", str(strategy), - "--harness-dir", str(strategy), + "verify_corpus.py", "--all" if aggregate else str(strategy), + "--corpus-root", str(corpus_root), + "--derived-workspace", str(corpus_root), + "--harness-dir", str(corpus_root), ] with patch.object(sys, "argv", cli_argv), redirect_stdout(cli_output): cli_exit = main() @@ -458,6 +483,103 @@ def test_strong_rejects_balanced_unmatched_at_gross_limit(self): self.assertEqual(result.count_delta, 2 * substitutions / 100) self.assertEqual(row["count_delta"], result.count_delta) + def _assert_reference_moderate(self, result, row): + self.assertEqual(result.label, "moderate") + self.assertEqual(row["tier"], "moderate") + self.assertEqual(result.profile, "strict") + self.assertEqual((result.tv_raw_count, result.eng_raw_count), (24, 24)) + self.assertEqual((result.tv_gate_count, result.eng_gate_count), (24, 24)) + self.assertEqual(result.gating_matched_count, 23) + self.assertEqual(result.count_abs_delta, 2) + self.assertEqual(result.net_count_abs_delta, 0) + self.assertEqual(result.count_delta, 2 / 24) + self.assertEqual(result.coverage, 23 / 24) + for metric in ("entry", "exit", "pnl"): + self.assertEqual(getattr(result, metric + "_p90"), 0.0) + + def test_dated_reference_floor_accepts_measured_moderate(self): + """#836/#883: aggregate acceptance preserves the measured Moderate tier.""" + result, row, cli = _analyze_tapes( + self.run_root, REFERENCE_STRATEGY_ID, *_reference_floor_tapes(), + strategy_source=STRICT_STRATEGY, aggregate=True, + ) + self._assert_reference_moderate(result, row) + self.assertIn("-> moderate", cli["stdout"]) + self.assertIn( + "Verified 1 strategies — excellent=0, strong=0, moderate=1, " + "weak=0, minimal=0, anomaly=0, engine_only=0, missing=0", + cli["stdout"], + ) + self.assertEqual( + cli["exit"], 0, + "dated reference floor must accept the measured Moderate state (#836/#883)", + ) + self.assertNotIn("Below strong tier:", cli["stdout"]) + + def test_reference_floor_rejects_other_moderate_ids(self): + """#836/#883: every other full id retains the Strong aggregate floor.""" + basename = REFERENCE_STRATEGY_ID.split("/", 1)[1] + for strategy_id in ( + "validation/other-moderate", f"validation/{basename}-variant", + f"validation/prefix-{basename}", f"basic/{basename}", + f"community/{basename}", + ): + with self.subTest(strategy_id=strategy_id): + result, row, cli = _analyze_tapes( + self.run_root / strategy_id.replace("/", "-"), + strategy_id, *_reference_floor_tapes(), + strategy_source=STRICT_STRATEGY, aggregate=True, + ) + self._assert_reference_moderate(result, row) + self.assertEqual(cli["exit"], 1) + self.assertIn("moderate=1", cli["stdout"]) + self.assertIn(f"Below strong tier:\n {strategy_id}", cli["stdout"]) + + def test_reference_floor_rejects_moderate_when_exception_removed(self): + """#836/#883: removing only the floor policy restores aggregate exit 1.""" + with patch.dict(verify_corpus.REFERENCE_FLOOR_EXCEPTIONS, clear=True): + result, row, cli = _analyze_tapes( + self.run_root, REFERENCE_STRATEGY_ID, *_reference_floor_tapes(), + strategy_source=STRICT_STRATEGY, aggregate=True, + ) + self._assert_reference_moderate(result, row) + self.assertEqual(cli["exit"], 1) + self.assertIn("moderate=1", cli["stdout"]) + self.assertIn(f"Below strong tier:\n {REFERENCE_STRATEGY_ID}", cli["stdout"]) + + def test_reference_floor_rejects_missing_or_lower_tiers(self): + """#836/#883: the exact id still fails with missing input or a lower tier.""" + for name, replacements, omit_tv, omit_engine, expected_tier in ( + ("weak", (3, 7, 12, 18), False, False, "weak"), + ("minimal", tuple(range(24)), False, False, "minimal"), + ("missing-tv", (10,), True, False, "missing"), + ("missing-engine", (10,), False, True, "missing"), + ): + with self.subTest(name=name): + result, row, cli = _analyze_tapes( + self.run_root / name, REFERENCE_STRATEGY_ID, + *_reference_floor_tapes(replacements), + strategy_source=STRICT_STRATEGY, aggregate=True, + omit_tv=omit_tv, omit_engine=omit_engine, + ) + self.assertEqual(result.label, expected_tier) + self.assertEqual(row["tier"], expected_tier) + self.assertEqual(cli["exit"], 1) + self.assertIn(f"{expected_tier}=1", cli["stdout"]) + self.assertIn( + f"Below strong tier:\n {REFERENCE_STRATEGY_ID}", cli["stdout"] + ) + + def test_reference_floor_does_not_change_single_strategy_exit(self): + """#836/#883: the dated allowance applies only to aggregate verification.""" + result, row, cli = _analyze_tapes( + self.run_root, REFERENCE_STRATEGY_ID, *_reference_floor_tapes(), + strategy_source=STRICT_STRATEGY, + ) + self._assert_reference_moderate(result, row) + self.assertEqual(cli["exit"], 1) + self.assertIn("-> moderate", cli["stdout"]) + if __name__ == "__main__": parser = argparse.ArgumentParser(add_help=False, allow_abbrev=False) diff --git a/scripts/verify_corpus.py b/scripts/verify_corpus.py index 9659a3677..8098ca7e5 100755 --- a/scripts/verify_corpus.py +++ b/scripts/verify_corpus.py @@ -1678,6 +1678,14 @@ def verify_one(strategy_dir: Path, *, verbose: bool = True, show_diffs: int = 0) return result.label +# 2026-11-16 (#836/#883): allow only this measured Moderate state through the +# aggregate reference floor; its grade and trades remain unchanged. The exception +# ends when #883 closes: https://github.com/pineforge-4pass/pineforge-engine/issues/883. +REFERENCE_FLOOR_EXCEPTIONS = { + "validation/mtf-triple-tf-macd-hist-confluence-01": "moderate", +} + + def main() -> int: ap = argparse.ArgumentParser(description=__doc__) ap.add_argument("strategy_dir", nargs="?", help="Single strategy folder, e.g. corpus/basic/greedy") @@ -1752,8 +1760,12 @@ def main() -> int: counts[label] = counts.get(label, 0) + 1 # `anomaly` and `engine_only` are documented success outcomes # (declared via inputs.json::expected_tier) — not failures. - if label not in {"excellent", "strong", "anomaly", "engine_only"}: - n_fail.append(f"{cat}/{strat.name}") + strategy_id = f"{cat}/{strat.name}" + if ( + label not in {"excellent", "strong", "anomaly", "engine_only"} + and label != REFERENCE_FLOOR_EXCEPTIONS.get(strategy_id) + ): + n_fail.append(strategy_id) if n_total == 0: print( f"error: refused corpus root {corpus_root}: " From ca7ffaaeb263685d117207ec3df62cfbe0653ee7 Mon Sep 17 00:00:00 2001 From: luisleo526 <20812494+luisleo526@users.noreply.github.com> Date: Mon, 16 Nov 2026 04:23:52 +0800 Subject: [PATCH 7/7] test(ci): keep corpus fixtures aligned with required build jobs Retain build-tests and build-tests-result in the independent corpus-toggle fixtures and their exact dependency diagnostics. Preserve the admission assertions, mutation cases, and single-match checks. Validation: complete corpus-admission module, 57 tests passed. Lane: ci-cr1152-corpus-fixture-1116-0401. --- scripts/test_ci_corpus_variant.py | 21 +++++++++++---------- 1 file changed, 11 insertions(+), 10 deletions(-) diff --git a/scripts/test_ci_corpus_variant.py b/scripts/test_ci_corpus_variant.py index 1fb18f713..1cefaadac 100644 --- a/scripts/test_ci_corpus_variant.py +++ b/scripts/test_ci_corpus_variant.py @@ -145,8 +145,8 @@ def without_corpus(self, workflow: str) -> str: workflow = workflow.replace(CORPUS_JOB, '', 1) workflow = self.replace_job( workflow, 'result', - ' needs: [admit, preflight, build-x86-gcc133, corpus-gitlink-result, legs]\n', - ' needs: [admit, preflight, build-x86-gcc133, legs]\n') + ' needs: [admit, preflight, build-x86-gcc133, build-tests, build-tests-result, corpus-gitlink-result, legs]\n', + ' needs: [admit, preflight, build-x86-gcc133, build-tests, build-tests-result, legs]\n') for addition in ( ' CORPUS_RESULT: ${{ needs.corpus-gitlink-result.result }}\n', ' test "$CORPUS_RESULT" = success || fail "corpus validation did not pass"\n'): @@ -161,8 +161,8 @@ def with_corpus(self, workflow: str) -> str: self.assertEqual(changed.replace(CORPUS_JOB, '', 1), workflow) changed = self.replace_job( changed, 'result', - ' needs: [admit, preflight, build-x86-gcc133, legs]\n', - ' needs: [admit, preflight, build-x86-gcc133, corpus-gitlink-result, legs]\n') + ' needs: [admit, preflight, build-x86-gcc133, build-tests, build-tests-result, legs]\n', + ' needs: [admit, preflight, build-x86-gcc133, build-tests, build-tests-result, corpus-gitlink-result, legs]\n') # Literal result hunk; these anchors are independent of checker templates. for anchor, addition in ( (' X86_RESULT: ${{ needs.build-x86-gcc133.result }}\n', @@ -240,11 +240,11 @@ def test_corpus_selector_only_selects_its_workflow_and_defined_job(self) -> None def test_job_present_with_old_result_needs_is_refused(self) -> None: changed = self.replace_job( self.workflow, 'result', - ' needs: [admit, preflight, build-x86-gcc133, corpus-gitlink-result, legs]\n', - ' needs: [admit, preflight, build-x86-gcc133, legs]\n') + ' needs: [admit, preflight, build-x86-gcc133, build-tests, build-tests-result, corpus-gitlink-result, legs]\n', + ' needs: [admit, preflight, build-x86-gcc133, build-tests, build-tests-result, legs]\n') self.assert_refused( changed, 'gke-pr.yml result must need admit, preflight, build-x86-gcc133, ' - 'corpus-gitlink-result and legs') + 'build-tests, build-tests-result, corpus-gitlink-result and legs') self.assertIn( 'gke-pr.yml result must require admission, preflight, native x86, ' 'corpus validation and consistent proof results', @@ -263,10 +263,11 @@ def test_job_present_with_old_result_template_is_refused(self) -> None: def test_job_absent_with_corpus_result_needs_is_refused(self) -> None: changed = self.replace_job( self.original, 'result', - ' needs: [admit, preflight, build-x86-gcc133, legs]\n', - ' needs: [admit, preflight, build-x86-gcc133, corpus-gitlink-result, legs]\n') + ' needs: [admit, preflight, build-x86-gcc133, build-tests, build-tests-result, legs]\n', + ' needs: [admit, preflight, build-x86-gcc133, build-tests, build-tests-result, corpus-gitlink-result, legs]\n') self.assert_refused( - changed, 'gke-pr.yml result must need admit, preflight, build-x86-gcc133 and legs', + changed, 'gke-pr.yml result must need admit, preflight, build-x86-gcc133, ' + 'build-tests, build-tests-result and legs', baseline=self.original) def test_job_absent_with_corpus_result_environment_is_refused(self) -> None: