From 02432c9c7bb9b415360ff8f1e987736667f09166 Mon Sep 17 00:00:00 2001 From: blacktoe Date: Wed, 16 Sep 2026 21:54:39 +1000 Subject: [PATCH 1/2] feat(eval): Fit the king attack zone weights on the archive MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The king attack zone landed at zero weight, counted but not scored. These are its eight weights, fitted on the whole archive with every other term held: knight 11 / 3 bishop 20 / -2 rook 27 / -3 queen 17 / 9 310 artifacts and 50,677 games, whose 6,623,970 post-book plies gave 6,227,777 positions and 2,890,773 quiet rows at the 820 slot layout. The corpus is sha256 a0798d51, built with the sealed pairs named, and those are sha256 a723887f. The rows are sha256 6001063b and the fitted vector sha256 f65aa57e. K was held at 1.0821. The games split by pair: 16,300 pairs and 32,516 games trained, 5,528 pairs and 11,030 games chose the ridge, and 3,498 pairs and 6,985 games were sealed. The seal file names 3,501 pairs over 6,995 games. Three of those pairs reach no row, which accounts for at most six of the ten missing games, and where the other four went is not established. The 768 table entries, the six material values and the mobility, shelter and pawn structure weights were all held, so these eight are the only thing that moved. The fit's figure is the sealed group's. It was opened once, at 2026-09-16T10:28:31Z, after the vector was frozen. Over 417,778 positions it reads -0.000143 against a standard error of 0.000059 at a design factor of 3.2. That is 2.42 standard errors from zero, outside its interval. The selection group reads -0.000395 against a standard error of 0.000052 at the same design factor: 0.085901 at zero, 0.085506 at the fit before rounding and 0.085496 after. By phase it is -0.000261 at six pieces or fewer, -0.000811 from seven to twelve and +0.000008 at thirteen or more. The two readings are about 3.2 standard errors apart. By the rule the fit was registered with, that a disagreement wider than the mobility refit's 1.85 standard errors means the sealed figure is the one quoted, the selection group's -0.000395 ±0.000052 overstated the fit. Two findings in the sealed reading were written down before any game was played. The first is its size: about a third of the selection group's, and about 3.2 standard errors from it, where the mobility refit's two readings were 1.85 standard errors apart and the king safety fit's 0.68. Most of the sealed games are the lazy mobility arms, whose candidates skipped the mobility term at most quiescence stand pats and so reached positions a different evaluation chose. The second is where it pays. By phase the sealed reading is -0.000088 at six pieces or fewer, -0.000414 from seven to twelve and +0.000366 at thirteen or more, so on those games the fullest boards read slightly worse at these weights. No standard error is printed per bucket. The ridge is zero, which the grid ranked first. It would have been overruled had the largest weight sat on the column with the least support. It did not: the largest is the rook's 27, and the rook column is the best supported of the four. A midgame coefficient is written in 16.2% of the training rows for the knight, 27.0% for the bishop, 30.5% for the rook and 17.4% for the queen, and little of that is on a full board. A sample of every twentieth of all 2,890,773 rows, across all three groups and reading positions only, finds a rook bearing on the enemy ring from 0.6% of the sides with thirteen or more pieces left, against 21.7% over the sampled rows. Because the sample spans every group, it is a different population from the training row shares. The learning curve is flat from an eighth of the training pairs. Five draws of 2,038 pairs read -0.000346 to -0.000397 against the whole set's -0.000395, so the corpus is not what limits eight weights. The match, played on this commit with the shared walk of the one after it, passed [0, 10] at 10+0.1 on 8moves_v3 over 1,000 games in two batches: +35 ±16 elo pooled against 4b1b80f, which plays the zero weight tree. The batches read +36 ±23 and +33 ±24, and the likelihood ratio reached 4.21 against a bound of 2.94. Under +40, so it is not re-measured. ## What moved The bench goes from 3,868,987 to 4,102,609, which is 6.04% more. Sixteen of the eighteen default pins and sixteen of the eighteen reference ones moved. `king and pawn` and `trebuchet` are unmoved in both sets. default before after start 68,341 73,075 italian 150,035 132,136 ruy lopez 161,892 127,220 kiwipete 243,106 275,925 perft 4 110,373 122,259 promotions 54,363 46,622 middlegame 100,507 167,457 sharp middlegame 180,608 184,701 bratko kopec 1 571,361 671,225 wac 4 1,944,432 1,946,406 rook and pawns 29,021 29,112 tarrasch 89,870 53,975 lucena 15,468 16,701 philidor 61,191 60,589 minor endgame 25,367 30,918 queen endgame 56,696 157,932 king and pawn 3,522 3,522 trebuchet 2,834 2,834 reference before after start 28,328 33,729 italian 227,641 120,260 ruy lopez 120,190 118,515 kiwipete 178,402 224,244 perft 4 232,961 255,435 promotions 95,254 95,210 middlegame 167,371 164,325 sharp middlegame 277,951 241,920 bratko kopec 1 44,544 52,545 wac 4 89,394 89,428 rook and pawns 24,020 22,615 tarrasch 33,182 48,576 lucena 21,127 24,290 philidor 51,857 29,763 minor endgame 34,141 30,627 queen endgame 96,523 175,117 king and pawn 1,601 1,601 trebuchet 968 968 The tactical suite goes from 224 to 237, which puts back more than the mobility refit took. The floor's comment now counts the history with 237 in it and the empty list of accepted losses counts sixty three misses. The strategic suite goes from 99,938 to 100,176. Eight themes rise and seven fall. AKPC rises most, by 432, then Undermine by 275 and Square Vacancy by 235. Pawn Play in the Center falls 322, Offer of Simplification 311 and Advancement of a/b/c pawns 222. The uci session's node budget moves with the tree, as its comment says it will. The swap is still at depth ten, which now ends at 2,114,707 and reports the new move from about 1,995,000, so the budget goes from 2,800,000 to 2,050,000. No other test moved: the pins in `engine.rs`, `reduction.rs`, `residual.rs` and the reductions session all pass as they stand. ## What it costs Callgrind over `arche bench` at depth 7, on a build with `king_attack::fold`, `king_attack::counts_of` and `mobility::fold` forced out of line: king_attack::fold, inclusive 558,628,090 11.51% of the run of which counts_of 512,071,008 10.55% mobility::fold, inclusive 553,880,961 11.41% the run 4,854,634,752 Those three attributes cost 33,558,174 instructions of call overhead the shipped build does not pay: the shipped build is 4,821,016,788. What the figure prices is this arrangement, a second walk over both sides' pieces at every leaf beside mobility's, and it costs about what mobility's walk costs, 136 instructions for each node the bench counts against mobility's 135. The run is 4,024,717,460 instructions over 3,868,987 nodes before this and 4,821,016,788 over 4,102,609 after, which is 1,040 a node against 1,175, or 13.0% more. That is over the 5% rule of thumb. The figure is recorded here and the counts are left as their own walk. Taking both readings in the mobility walk is a separate change and is not made here. On the clock, `scripts/speed.sh` against 4b1b80f over 21 interleaved rounds reads -7.6% on the median nodes per second and -8.8% on the fastest, against a spread of 8.9%. The tree is 6.0% larger, so the bench takes 14.8% longer to reach depth 7. The box was shared during the run, so the instruction count is the claim. ## What went The scaffolding the zero weights needed. The equality on the term's name in `every_term_writes_the_counts_its_width_claims` is back to asserting every term is priced. `the_table_names_the_terms_the_sum_adds` reads the shipped fold again rather than trial weights, on the same position, where all four folds are non-zero. `fold_with` is private again, and the fold test no longer asserts that the shipped fold is zero. The doc blocks and comments that said the term was unfitted, in `king_attack.rs`, `eval/mod.rs`, `tune.rs`, the architecture overview and the instruments guide, say what it is now. The readme counts four fitted leaf terms. The roadmap says what the term counts and what the fit read, and gives its cost. Bench: 4102609 Elo: +35 ±16 (sprt [0, 10] passed, 1000 games, 10+0.1, vs 4b1b80f) Co-Authored-By: Claude Opus 5 --- README.md | 6 +- arche-core/src/bench.rs | 64 +++++++------- arche-core/src/eval/king_attack.rs | 133 +++++++++++++++++++---------- arche-core/src/eval/mod.rs | 46 ++-------- arche-core/src/strategy.rs | 2 +- arche-core/src/tactics.rs | 23 ++--- arche-core/src/tune.rs | 20 ++--- docs/ARCHITECTURE.md | 12 ++- docs/INSTRUMENTS.md | 8 +- docs/ROADMAP.md | 26 ++++-- tests/uci_session.rs | 10 +-- 11 files changed, 187 insertions(+), 163 deletions(-) diff --git a/README.md b/README.md index 5c4e812b..33e5dcc2 100644 --- a/README.md +++ b/README.md @@ -15,9 +15,9 @@ search, principal variation search, reverse futility pruning, a null move pass a reduction. Captures are ordered by what a static exchange evaluation says the swap wins, with MVV-LVA breaking the ties between the ones it prices alike, and the quiet moves by the ones that have cut off before. Evaluation is material, piece square tables, piece mobility, the -pawns standing in front of each king, and pawn structure, tapered between a middlegame -and an endgame score. The tables and the three leaf terms were fitted to the engine's -own games; the material values are hand set. +pawns standing in front of each king, pawn structure, and the squares around each king the +enemy pieces attack, tapered between a middlegame and an endgame score. The tables and the +four leaf terms were fitted to the engine's own games; the material values are hand set. ### Background diff --git a/arche-core/src/bench.rs b/arche-core/src/bench.rs index f1c19111..bf8bfa5e 100644 --- a/arche-core/src/bench.rs +++ b/arche-core/src/bench.rs @@ -759,22 +759,22 @@ mod tests { assert_eq!( counted, vec![ - ("start", 68_341), - ("italian", 150_035), - ("ruy lopez", 161_892), - ("kiwipete", 243_106), - ("perft 4", 110_373), - ("promotions", 54_363), - ("middlegame", 100_507), - ("sharp middlegame", 180_608), - ("bratko kopec 1", 571_361), - ("wac 4", 1_944_432), - ("rook and pawns", 29_021), - ("tarrasch", 89_870), - ("lucena", 15_468), - ("philidor", 61_191), - ("minor endgame", 25_367), - ("queen endgame", 56_696), + ("start", 73_075), + ("italian", 132_136), + ("ruy lopez", 127_220), + ("kiwipete", 275_925), + ("perft 4", 122_259), + ("promotions", 46_622), + ("middlegame", 167_457), + ("sharp middlegame", 184_701), + ("bratko kopec 1", 671_225), + ("wac 4", 1_946_406), + ("rook and pawns", 29_112), + ("tarrasch", 53_975), + ("lucena", 16_701), + ("philidor", 60_589), + ("minor endgame", 30_918), + ("queen endgame", 157_932), ("king and pawn", 3_522), ("trebuchet", 2_834), ] @@ -807,22 +807,22 @@ mod tests { assert_eq!( counted, vec![ - ("start", 28_328), - ("italian", 227_641), - ("ruy lopez", 120_190), - ("kiwipete", 178_402), - ("perft 4", 232_961), - ("promotions", 95_254), - ("middlegame", 167_371), - ("sharp middlegame", 277_951), - ("bratko kopec 1", 44_544), - ("wac 4", 89_394), - ("rook and pawns", 24_020), - ("tarrasch", 33_182), - ("lucena", 21_127), - ("philidor", 51_857), - ("minor endgame", 34_141), - ("queen endgame", 96_523), + ("start", 33_729), + ("italian", 120_260), + ("ruy lopez", 118_515), + ("kiwipete", 224_244), + ("perft 4", 255_435), + ("promotions", 95_210), + ("middlegame", 164_325), + ("sharp middlegame", 241_920), + ("bratko kopec 1", 52_545), + ("wac 4", 89_428), + ("rook and pawns", 22_615), + ("tarrasch", 48_576), + ("lucena", 24_290), + ("philidor", 29_763), + ("minor endgame", 30_627), + ("queen endgame", 175_117), ("king and pawn", 1_601), ("trebuchet", 968), ] diff --git a/arche-core/src/eval/king_attack.rs b/arche-core/src/eval/king_attack.rs index c1dd61fb..0ba18148 100644 --- a/arche-core/src/eval/king_attack.rs +++ b/arche-core/src/eval/king_attack.rs @@ -25,18 +25,73 @@ pub(crate) const COUNTS: usize = mobility::PIECES.len(); /// What one attacked square of the enemy king's ring is worth to each of /// [`mobility::PIECES`], as the packed pairs the taper is read from. /// -/// Unfitted. All eight halves are zero, so [`SCORED`] is false, the leaf does -/// not take the counts at all, and the evaluation is the one the commit below -/// this counted: the same bench, the same two suite totals, and the claim of -/// the commit that added the counts is that they moved none of them. The -/// counts are here for the tuner's rows, which state a coefficient per column -/// whatever weight stands beside it, so a corpus extracted from this build is -/// what a fit prices them on. +/// Fitted 2026-09-16 by `scripts/tune.py` over the whole archived strength +/// run: 310 artifacts and 50,677 games, whose 6,623,970 post-book plies gave +/// 6,227,777 positions and 2,890,773 quiet rows, extracted by `arche terms` at +/// the 820 slot layout. The corpus is sha256 `a0798d51`, built with the sealed +/// pairs named, and those are sha256 `a723887f`. The rows are sha256 +/// `6001063b` and the fitted vector is sha256 `f65aa57e`. K was held at 1.0821 +/// and the games split by pair: 16,300 pairs and 32,516 games trained, 5,528 +/// pairs and 11,030 games chose the ridge, and 3,498 pairs and 6,985 games +/// were sealed. The sealed file names 3,501 pairs over 6,995 games. Three of +/// those pairs reach no row, which accounts for at most six of the ten missing +/// games, and where the other four went is not established. The 768 table +/// entries, the six material values, the eight mobility weights, the fourteen +/// shelter weights and the sixteen pawn structure ones were all held, so these +/// eight are the only thing that moved. /// -/// The fit replaces this block with the provenance one the other three terms -/// carry. `every_term_writes_the_counts_its_width_claims` in the module above -/// names this term as the unfitted one and fails until that happens. -static KING_ATTACK: [i32; COUNTS] = [pack(0, 0); COUNTS]; +/// The fit's figure is the sealed group's: a paired difference of -0.000143 +/// against a standard error of 0.000059 at a design factor of 3.2. The sealed +/// group was opened once, at 2026-09-16T10:28:31Z, after the vector was frozen +/// and over 417,778 positions that no fit and no ridge choice had read. It +/// scores 0.079520 at zero and 0.079377 at these rounded weights. That is 2.42 +/// standard errors from zero, outside its interval. +/// +/// The selection group read -0.000395 against a standard error of 0.000052 at +/// the same design factor, scoring 0.085901 at zero and 0.085506 at the fit +/// before it was rounded (0.085496 after). By the rule the fit was registered +/// with, that a disagreement wider than the mobility refit's 1.85 standard +/// errors means the sealed figure is the one quoted, the selection group's +/// -0.000395 overstated the fit. +/// +/// Two things in the sealed reading were recorded before any game was played. +/// The first is its size. It is about a third of the selection group's, and +/// about 3.2 standard errors from it, where the mobility refit's two readings +/// were 1.85 apart and the king safety fit's 0.68. The sealed games are the +/// strength runs played since the mobility refit's corpus was built, and most +/// of them are the lazy mobility arms, whose candidates skipped the mobility +/// term at most quiescence stand pats and so reached positions a different +/// evaluation chose. The second is where it pays. By phase the sealed reading +/// is -0.000088 at six pieces or fewer, -0.000414 from seven to twelve and +/// +0.000366 at thirteen or more, so on those games the fullest boards read +/// slightly worse at these weights. The selection group read -0.000261, +/// -0.000811 and +0.000008 over the same buckets, the fullest boards level. +/// Neither group prints a standard error per bucket. +/// +/// The ridge is zero, which the grid ranked first. The check that would have +/// overruled it is a largest weight on the column with the least support, and +/// that is not what happened: the largest weight is the rook's 27, and the +/// rook column is the best supported of the four. A midgame coefficient is +/// written in 16.2% of the training rows for the knight, 27.0% for the bishop, +/// 30.5% for the rook and 17.4% for the queen. Little of that is on a full +/// board. A sample of every twentieth of all 2,890,773 rows, across all three +/// groups and reading positions only, finds a rook bearing on the enemy ring +/// from 0.6% of the sides with thirteen or more pieces left, against 21.7% of +/// the sides over the sampled rows. The sample spans every group, so it is a +/// different population from the training row shares above. +/// +/// The learning curve is flat from an eighth of the training pairs. Five +/// draws of 2,038 pairs read -0.000346 to -0.000397 against the whole set's +/// -0.000395, so the corpus is not what limits eight weights. +/// +/// [`SCORED`] is true at these weights, so the leaf counts the ring at every +/// evaluation. What that costs is in the commit that landed this. +/// +/// `bounds_hold` charges one piece of each kind two squares of the ring for a +/// knight, three for a bishop, four for a rook and six for a queen, both sides +/// counted. That puts this term at 584 in the midgame and the whole vector's +/// boardful at 9,762, against the 32,767 a half has to stay inside. +static KING_ATTACK: [i32; COUNTS] = [pack(11, 3), pack(20, -2), pack(27, -3), pack(17, 9)]; /// The weight of one of the four pieces that carries one, as the packed pair. /// The tuner's seam asks through [`super::TERMS`], so that a slot names the @@ -47,13 +102,13 @@ pub(crate) const fn weight(index: usize) -> i32 { /// Whether [`super::sum`] takes this term at the leaf: true when one of the /// four [`KING_ATTACK`] weights is not zero at one end of the taper or the -/// other. All eight halves are zero today, so it is false and no leaf asks -/// [`counts_of`] anything. +/// other. The 2026-09-16 fit priced all eight halves, so it is true and every +/// leaf asks [`counts_of`] for both sides. /// /// Derived from the weights rather than written out, which is how -/// [`mobility::SCORED_KINDS`] is derived, and it means the fit that prices the -/// columns turns the term on by replacing the weights and editing nothing -/// else. +/// [`mobility::SCORED_KINDS`] is derived. The fit turned the term on by +/// replacing the weights and editing nothing else, and weights put back to +/// zero would turn it off the same way. /// /// A count at a zero weight is not folded away by the compiler. Multiplying /// the counts by nothing leaves the walk over the pieces standing, and llvm @@ -158,26 +213,17 @@ pub(crate) fn counts(board: &Board, color: Color, into: &mut [i32]) { /// What white's bearing on the black king stands ahead by, as a packed pair on /// the scale the piece square pair is on. /// -/// Nothing at the weights that ship, since all eight halves are zero, and the -/// sum does not call it at all while [`SCORED`] is false. The fold is written -/// now rather than with the fit so that the sum names this term from the -/// commit the counts arrive in, and so that what the fit moves is eight -/// numbers and not the shape of the evaluation. +/// The sum calls it only while [`SCORED`] is true. It was written in the +/// commit the counts arrived in, at zero weight, so that what the fit moved +/// was eight numbers and not the shape of the evaluation. #[inline] pub(crate) fn fold(board: &Board) -> i32 { fold_with(board, &KING_ATTACK) } /// The same fold against weights named by the caller. -/// -/// Visible to the module above rather than private, which is what the other -/// three leaf terms keep theirs. [`KING_ATTACK`] is zero, so the shipped fold -/// answers nothing on every position and can say nothing about how the four -/// pairs are read, and `the_table_names_the_terms_the_sum_adds` has to read it -/// against weights of its own to say anything at all. The fit puts that back -/// and this goes private again. #[inline] -pub(super) fn fold_with(board: &Board, weights: &[i32; COUNTS]) -> i32 { +fn fold_with(board: &Board, weights: &[i32; COUNTS]) -> i32 { let white = counts_of(board, Color::White); let black = counts_of(board, Color::Black); let mut packed = 0; @@ -196,10 +242,8 @@ mod tests { /// The counts by hand, square by square, because nothing else pins them. /// The tuner's identity folds a row against the live weights, and `eval` /// and the walk read this same function, so the two sides of the identity - /// move together whatever it answers. At the zero weights that ship the - /// identity says less than that: the coefficients are multiplied by - /// nothing, so a wrong sign or a wrong half of a pair is invisible to it. - /// These cases are the only check this term has. + /// move together whatever it answers. These cases are what pins the + /// counts themselves. /// /// The black king stands on e8 unless the case says otherwise, so its ring /// is d7, e7, f7, d8 and f8. The white king stands off every line the case @@ -383,7 +427,7 @@ mod tests { /// [`counts_of`]. /// /// Black's king on g8 has the ring f7, g7, h7, f8 and h8. White's knight - /// on e6 has f8 and g7. Its bishop on d3 has h7 up the long diagonal. Its + /// on e6 has f8 and g7. Its bishop on d3 has h7, through e4, f5 and g6. Its /// rook on a7 has d7 and e7 and f7 along the rank, of which f7 alone is in /// the ring, and then g7 and h7 as well, which is three. Its queen on h6 /// has h7 and h8 up the file and g7 and f8 up the diagonal, which is four. @@ -407,12 +451,12 @@ mod tests { /// What the fold does with weights that are not the shipped ones. /// - /// The shipped ones are zero, so they would say nothing here at all. What - /// the packing does with a pair, and which of the four pieces each pair - /// belongs to, stay invisible until a fit prices the columns. So this - /// hands the fold four pairs that differ from each other at both ends and - /// asserts the packed pair against the arithmetic: white's count less - /// black's, piece by piece, each half of the pair summed on its own. + /// Nothing holds the shipped weights apart from each other, so a pair + /// read into the wrong piece's slot or a half read off the wrong end could + /// land on the right number by accident. So this hands the fold four pairs + /// that differ at both ends and asserts the packed pair against the + /// arithmetic: white's count less black's, piece by piece, each half of the + /// pair summed on its own. #[test] fn the_king_attack_fold_reads_white_less_black_piece_by_piece() { let board = Board::from_fen(BEARING).unwrap(); @@ -431,11 +475,6 @@ mod tests { let packed = super::fold_with(&board, &TRIAL); assert_ne!(midgame, 0, "black less white would answer the same here"); assert_eq!((mg_value(packed), eg_value(packed)), (midgame, endgame)); - assert_eq!( - super::fold(&board), - 0, - "the shipped weights are not zero, so the bench pin and this term's own doc block are stale" - ); } /// What the leaf is allowed to leave out, which is the whole of the @@ -449,8 +488,8 @@ mod tests { /// else, read here off the array a second way, and then a weight put back /// into each half of each column in turn. /// - /// A fit that prices the term flips this to true and the leaf starts - /// counting, with no line here or in the sum to edit. + /// The fit priced the term, so the constant is true. Weights put back to + /// zero would turn it off with no line here or in the sum to edit. #[test] fn the_term_is_counted_exactly_when_a_weight_is_not_zero() { let priced = (0..COUNTS).any(|index| { diff --git a/arche-core/src/eval/mod.rs b/arche-core/src/eval/mod.rs index 139371cf..fbcb3b2d 100644 --- a/arche-core/src/eval/mod.rs +++ b/arche-core/src/eval/mod.rs @@ -196,11 +196,11 @@ impl Memo for Caches { /// on one scale, and it is what keeps one divide however many such terms /// there are. /// -/// The king attack zone is behind [`king_attack::SCORED`], which is false -/// while its eight weights are zero. A count multiplied by nothing scores -/// nothing, and llvm does not take the walk that produces it out on that -/// ground, so the summand is dropped here instead: at a constant false the -/// call is not compiled rather than computed and thrown away. +/// The king attack zone is behind [`king_attack::SCORED`], which is true at +/// the fitted weights and would be false if all eight were zero. A count +/// multiplied by nothing scores nothing, and llvm does not take the walk that +/// produces it out on that ground, so at a constant false the summand is not +/// compiled rather than computed and thrown away. /// /// Material that cannot mate reads zero before any of it. That is the one /// place in the evaluation that is not a dot product against the weights, @@ -838,28 +838,10 @@ mod evaluate { let priced = (0..term.width).any(|index| { mg_value((term.weight)(index)) != 0 || eg_value((term.weight)(index)) != 0 }); - // every term is priced but the king attack zone, which ships at - // zero weight until its fit lands. A nominal weight in the - // meantime would move the bench, and the commit that added the - // counts claims the bench is the one below it. Written as an - // equality rather than an exemption, so the fit that prices the - // term fails here until this line goes with it - assert_eq!( - priced, - term.name != "king_attack", - "{} is worth {} at either end", - term.name, - if priced { "something" } else { "nothing" } - ); + assert!(priced, "{} is worth nothing at either end", term.name); } } - /// Four weights that differ from each other at both ends of the taper, - /// for the king attack zone, whose shipped eight are zero and say nothing - /// about a fold. - const KING_ATTACK_TRIAL: [i32; king_attack::COUNTS] = - [pack(11, 2), pack(-7, 13), pack(3, -5), pack(29, 41)]; - /// The table names the four leaf terms the sum adds, which is what makes /// the tuner's row and the evaluation the same arithmetic. /// @@ -868,13 +850,6 @@ mod evaluate { /// added to the table and left out of the sum would print coefficients the /// evaluation never reads, and the tuner's identity would fail on the /// first position that touched it; this says which of the two is wrong. - /// - /// An unpriced term stands outside that guard until its fit. The king - /// attack zone is skipped at the leaf while its weights are zero, so a sum - /// that dropped it altogether would add up the same and this test would - /// pass. What it can say meanwhile is that the term is there and that its - /// counts are not level here, which is why it reads the fold against - /// weights of its own rather than the shipped ones. #[test] fn the_table_names_the_terms_the_sum_adds() { let names: Vec<&str> = TERMS.iter().map(|term| term.name).collect(); @@ -884,18 +859,15 @@ mod evaluate { ); // two queens and a rook against none, a king in each corner of the // board and pawns of both colours on six files, so that no one of the - // three folds to nothing and the test says something about each. The + // four folds to nothing and the test says something about each. The // queen on a4 bears on d7 and e8 of the black king's ring, which is - // what leaves the fourth term's counts unlevel too + // what leaves the king attack counts unlevel let board = Board::from_fen("3k4/P4p2/8/3P2p1/Q2P4/PP2p2p/1P6/1Q4KR w - - 0 1").unwrap(); for (name, term) in [ ("mobility", mobility::fold(&board)), ("shelter", shelter::fold(&board)), ("pawn structure", pawn_structure::fold(&board)), - ( - "king attack", - king_attack::fold_with(&board, &KING_ATTACK_TRIAL), - ), + ("king attack", king_attack::fold(&board)), ] { assert_ne!(term, 0, "{} is level here, so it says nothing", name); } diff --git a/arche-core/src/strategy.rs b/arche-core/src/strategy.rs index 0db9b9a5..7b51fe2c 100644 --- a/arche-core/src/strategy.rs +++ b/arche-core/src/strategy.rs @@ -51,7 +51,7 @@ pub const TABLE_BYTES: usize = 16 * 1024 * 1024; /// Exact, not a floor. A change that raises it has to update this number in /// the same commit, which is what puts the improvement in the diff rather /// than leaving it to be noticed later or not at all. -pub const EXPECTED_POINTS: u32 = 99938; +pub const EXPECTED_POINTS: u32 = 100176; const SUITE: &str = include_str!("../strategy.epd"); diff --git a/arche-core/src/tactics.rs b/arche-core/src/tactics.rs index 7918dbf1..f9b5bf9c 100644 --- a/arche-core/src/tactics.rs +++ b/arche-core/src/tactics.rs @@ -44,8 +44,9 @@ pub const TABLE_BYTES: usize = 16 * 1024 * 1024; /// the same duty and a heavier one: this is the tactical tripwire, and it had /// moved by one in each direction across every evaluation arm before the /// 2026-09-13 mobility refit took it down seven at once. What that bought is -/// in the refit's commit, and the suite is not a gate. -pub const EXPECTED_PASSES: usize = 224; +/// in the refit's commit, and the suite is not a gate. The 2026-09-16 king +/// attack fit took it up thirteen. +pub const EXPECTED_PASSES: usize = 237; /// The count the suite may not go under, whatever a commit says it meant to /// spend. @@ -57,14 +58,14 @@ pub const EXPECTED_PASSES: usize = 224; /// time until the suite says nothing at all, because each step is small and /// each is argued for on its own. /// -/// Two hundred and ten is fourteen under where the count stands and eleven -/// under the lowest it has been gated at (221, on the model gated two ply -/// reduction). Since the suite was first gated the count has been 243, 241, -/// 240, 237, 236, 229, 221, 226, 227, 228, 231 and 224, and the largest -/// single step in that list is eight. So one change spending fourteen is -/// spending most of two of the largest steps ever taken, and this fails -/// rather than being written down and rearmed. Lowering the floor is a -/// commit whose whole subject is lowering the floor. +/// Two hundred and ten was fourteen under where the count stood when the +/// floor was set, and is eleven under the lowest it has been gated at (221, on +/// the model gated two ply reduction). Since the suite was first gated the +/// count has been 243, 241, 240, 237, 236, 229, 221, 226, 227, 228, 231, 224 +/// and 237, and the largest single step down in that list is eight. So one +/// change spending fourteen is spending most of two of the largest steps ever +/// taken, and this fails rather than being written down and rearmed. Lowering +/// the floor is a commit whose whole subject is lowering the floor. pub const FLOOR: usize = 210; // The snapshot cannot be set under the floor without moving the floor, and @@ -98,7 +99,7 @@ pub struct AcceptedLoss { /// The losses accepted so far. /// -/// Empty. The suite misses seventy six positions at the pinned depth and +/// Empty. The suite misses sixty three positions at the pinned depth and /// none of them has been read one at a time, so there is nothing here that /// would be a record rather than a guess. The next change that lowers /// `EXPECTED_PASSES` is the first that writes here, naming what it spent. diff --git a/arche-core/src/tune.rs b/arche-core/src/tune.rs index 186fa298..59a8d033 100644 --- a/arche-core/src/tune.rs +++ b/arche-core/src/tune.rs @@ -57,11 +57,9 @@ //! is exactly the kinds that score nothing, and //! `eval_counts_a_kind_exactly_when_its_weight_is_not_zero` is what says the //! difference is that and not something else. None of the other three is -//! split that way. Both fits gave every one of their weights a value, the -//! shelter's fourteen and the pawn structure's sixteen, so there is nothing -//! in either to leave out, and the king attack zone's eight are all zero -//! until the fit that prices them, which leaves the walk and `eval` reading -//! the same four counts there too. +//! split that way. Their fits gave every one of their weights a value, the +//! shelter's fourteen, the pawn structure's sixteen and the king attack +//! zone's eight, so there is nothing in any of them to leave out. use crate::bench::Position; use crate::board::Board; @@ -741,12 +739,12 @@ mod tests { /// taper too, and its count is hand counted rather than read back off the /// board. /// - /// The identity says nothing at all about these eight slots. The eight - /// weights are zero until the fit, so a king attack coefficient written - /// to the wrong slot, doubled, or left out entirely reproduces every row - /// of the corpus and every reconstruction test above. What is asserted - /// here is the coefficient itself, against a count worked out by hand - /// from the position below. + /// The identity reads these eight slots against the fitted weights, so it + /// catches a coefficient written to the wrong slot only where the two + /// weights differ, and it cannot tell a count that is wrong in the walk + /// from the same count wrong in `eval`, since both read one function. What + /// is asserted here is the coefficient itself, against a count worked out + /// by hand from the position below. #[test] fn every_king_attack_count_writes_both_ends_of_the_taper() { // one white piece of each kind that carries a weight, and no black diff --git a/docs/ARCHITECTURE.md b/docs/ARCHITECTURE.md index 2cc49b57..5ed02384 100644 --- a/docs/ARCHITECTURE.md +++ b/docs/ARCHITECTURE.md @@ -27,10 +27,8 @@ the bishop, the rook and the queen; king safety, which counts the pawns in front of each king, the open files beside it and the enemy pawns coming for it; pawn structure, which counts each side's passed pawns by rank, its isolated pawns and its doubled ones; and the king attack zone, which counts -how many squares of each king's ring the other side's pieces bear on. Three -of the four are fitted to the engine's own archived games. The king attack -weights are all zero until their fit, and the leaf skips that term's counts -while they are. +how many squares of each king's ring the other side's pieces bear on. All +four are fitted to the engine's own archived games. ## Code map: arche-core @@ -133,9 +131,9 @@ while they are. king move as well. - **king_attack.rs**: How many squares of the enemy king's ring each side's knights, bishops, rooks and queens attack, off the same attack sets - mobility walks but with nothing taken out of them. Skipped at the leaf - while its weights are zero, and read at every leaf once a fit prices - them. Not remembered either way: what it depends on is the whole occupancy + mobility walks but with nothing taken out of them. Read at every leaf, + and skipped only if its weights are all zero. Not remembered: what it + depends on is the whole occupancy and a side's pieces, which is the position, and a table under that key was measured for mobility and turned down on the memory it cost. - **psqt.rs**: The piece square tables. Every piece has a second table diff --git a/docs/INSTRUMENTS.md b/docs/INSTRUMENTS.md index 0af00a0d..db1836dd 100644 --- a/docs/INSTRUMENTS.md +++ b/docs/INSTRUMENTS.md @@ -411,9 +411,9 @@ colour and counts that square if the ring holds it, and a square this side already stands on or an enemy pawn covers is still attacked. A ring square two pieces attack is counted twice, once per attacker, and two rooks on the seventh rank of a king on e8 read six between them rather than the three -squares they share. Pawns and kings carry no column. The eight weights are -zero until the fit that prices them, so the leaf does not take these counts at -all today and the rows carry their coefficients all the same. +squares they share. Pawns and kings carry no column. The rows carry the +coefficients whatever the weights hold, which is how the fit that priced the +eight weights read them before any of them was worth anything. The vector was 518 until a knight, a bishop, a rook and a queen were given an endgame table of their own, since each of the four had handed one array to @@ -816,7 +816,7 @@ reason, `--hold-shelter` the fourteen shelter ones, `--hold-pawn` the sixteen pawn structure ones and `--hold-king-attack` the eight king attack ones. Each term earns a hold of its own as it is fitted, and a fit of the newest term names every hold below it, so the shelter was fitted under two, the pawn -structure under three and the king attack zone is to be fitted under four. +structure under three and the king attack zone under four. A term already in the tree can be worth fitting again on a corpus grown since, and then the holds are not a ladder downward. Every other term is older than diff --git a/docs/ROADMAP.md b/docs/ROADMAP.md index b65ff780..f15ff57c 100644 --- a/docs/ROADMAP.md +++ b/docs/ROADMAP.md @@ -27,10 +27,20 @@ the engine plays. Roughly in the order they look worth doing. left no weight at zero, so nothing is skipped at the leaf now. King safety counts the pawns on the two ranks in front of the king, the open and half open files beside it, and the enemy pawns on the three ranks in front of it, and all fourteen of its weights - are fitted. What is not measured at all is the squares the enemy pieces attack around - the king, which wants the attack sets the mobility count already walks. The storm is followed - three ranks and no further, so a pawn four ranks out is not counted, and a storm pawn - blocked by one of ours counts the same as a free one. Pawn structure counts a side's + are fitted. The storm is followed three ranks and no further, so a pawn four ranks out + is not counted, and a storm pawn blocked by one of ours counts the same as a free one. + The squares the enemy pieces attack around the king are counted too: for each side, + how many squares of the other king's ring its knights, bishops, rooks and queens bear + on, off the attack sets the mobility count already walks, with a square two pieces + attack counted twice. All eight of its weights are fitted, trained on 32,516 of the + archive's 50,677 games with every other term held. The fit's figure is the sealed + group's -0.000143 against a standard error of 0.000059. The selection group read + -0.000395 against 0.000052, about 3.2 standard errors away, and by the rule the fit + was registered with that disagreement means the selection group overstated it. On the + sealed games the boards with thirteen or more pieces left read slightly worse. What it + leaves out is how the attack adds up: the count is priced per square, where an attack + by several pieces is usually taken as worth more than the sum of its parts, and neither + safe checks nor the ring's defenders are read. Pawn structure counts a side's passed pawns by the rank they have reached, its isolated pawns and its doubled ones, read off the two pawn boards alone and remembered under the pawn key, and all sixteen of its weights are fitted. The seventh rank is the one to distrust: a passed pawn there @@ -111,7 +121,13 @@ the engine plays. Roughly in the order they look worth doing. the refit that priced all four kinds; measured again at `378c148` the term is 13.5% of the bench. The 5% is a rule of thumb and nothing enforces it, which the games have now said outright: the term was made 6% cheaper across the whole search and 6,000 of them - could not see it. So the ceiling is the thing under question rather than the term. The + could not see it. So the ceiling is the thing under question rather than the term. + The king attack zone is over it as well. Measured at its fit with callgrind over + `arche bench` at depth 7, its fold and its counts forced out of line, the term is + 558,628,090 of 4,854,634,752 instructions, 11.5%, and mobility is 11.4% of the same run. + The run costs 1,175 instructions a node against 1,040 before the fit, 13.0% more. The + counts are a second walk over the pieces mobility already walks, and taking both in one + walk is the change that would bring the term down. The reasoning and the numbers are in the planning repository - a held-out loss on our own games cannot resolve a fit of the piece square tables one way or the other, so an sprt is what decides a re-tune. Measured 2026-09-10 over 1,812 diff --git a/tests/uci_session.rs b/tests/uci_session.rs index 703102a8..76713e4f 100644 --- a/tests/uci_session.rs +++ b/tests/uci_session.rs @@ -266,14 +266,14 @@ fn the_move_a_swap_answers_with_opens_the_last_line_said() { // own the last line an interface read would open with the move being // given up, and fastchess calls that out on every move it happens on. // The budget moves whenever the tree does, in the commit that moved it. - // The swap is at depth ten here, which finishes at 2,884,594 nodes, and - // the depth before it at 1,267,976, so the budget has to land between the + // The swap is at depth ten here, which finishes at 2,114,707 nodes, and + // the depth before it at 1,038,433, so the budget has to land between the // move being found and the iteration ending. Depth ten reports the new - // move from about 2,750,000 nodes on, so anything from there to the - // 2,884,594 does; this sits between the two with room either side + // move from about 1,995,000 nodes on, so anything from there to the + // 2,114,707 does; this sits between the two with room either side let mut s = Session::start(&[]); s.say(&format!("position fen {}", SHARP_MIDDLEGAME)); - s.say("go nodes 2800000"); + s.say("go nodes 2050000"); let answer = s.wait_for(|l| l.starts_with("bestmove")); let best = answer .strip_prefix("bestmove ") From 31394e327ae4691f7ff5222d4500c55673a11c25 Mon Sep 17 00:00:00 2001 From: blacktoe Date: Wed, 16 Sep 2026 22:17:06 +1000 Subject: [PATCH 2/2] perf(eval): Take the king attack counts in the mobility walk Mobility and the king attack zone each walked both sides' knights, bishops, rooks and queens at every leaf, and each probed the same attack sets: the knight table and the two magic lookups over the whole occupancy. Now one walk per side takes each piece's attack set once and reads it twice, against mobility's scope for the mobility counts and against the enemy king's ring for the king attack counts. The probes and the loops over the pieces are what is shared. The masks, the columns, the weights and the folds are not: each term folds its own counts through a new `fold_counts`. The skips still hold. The walk takes mobility's `SCORED_KINDS` and the king attack `SCORED` as compile time parameters, so a count that is not asked for is not compiled, down to the loop over a kind neither term wants. Read off the disassembly of the walk forced out of line: all four kinds with the ring is 227 instructions with 8 popcounts, without the ring 161 with 4, the knight alone without the ring 61 with 1 and no magic multiply, the knight with the ring 188 with 5, and nothing at all 9. The walk lives in `eval/mod.rs`, beside the sum, which is its one caller. It is the sum's arrangement of two terms rather than a part of either, and putting it in `mobility.rs` would have that file compute the other term's counts (and import from a file that already imports from it). A separate `eval/attacks.rs` was the other choice and was declined: it would be a file for one function with one caller and no counts, weights or fold of its own. `mobility::counts_of` and `king_attack::counts_of` are unchanged. The tuner still reads them through `TERMS`, the hand counts still pin them, and they are now what the walk is tested against. `fold` in each term file is kept for the tests, as the fold over the standalone counts the sum used to answer. ## What it costs Callgrind over `arche bench` at depth 7. The attributed builds force the measured functions out of line: at 02432c9 the two folds and `king_attack::counts_of`, here the walk and the two `fold_counts`. A third build is this commit with the walk's ring turned off and the king attack term read by its own walk again, which gives the walk without the ring over the same tree. attributed 02432c9 this commit the run 4,854,574,730 4,600,409,835 per node 1,183.3 1,121.3 king_attack::fold 558,628,090 11.51% of which counts_of 512,071,008 10.55% mobility::fold 553,880,961 11.41% the shared walk 777,417,417 16.90% the walk without the ring 576,199,267 the ring 201,218,150 4.37% king_attack::fold_counts 21,162,310 0.46% mobility::fold_counts 21,162,310 0.46% shipped 02432c9 this commit the run 4,821,016,788 4,452,853,888 per node 1,175.1 1,085.4 The node count is 4,102,609 in every build. The shipped run is 368,162,900 instructions shorter, 7.64%. What the ring adds to the walk is 49 instructions a node, and 54 with its fold, against the 125 the standalone counts cost. The walk without the ring costs more than 02432c9's `mobility::fold` because it is called once per side rather than once per leaf in that build and returns both arrays, so the attributed builds overstate the walk a little and the shipped figures are the ones to compare. Against 4b1b80f, before the king attack zone, measured again here on its shipped build at 4,024,751,355 instructions over 3,868,987 nodes, 1,040.3 a node, this commit is 4.34% more a node. The fit alone was 12.96% more. The ring's share of the attributed run is 4.37%, and 4.83% with its fold. Both are under the 5% rule of thumb. These price instructions over one tree, so they are what the arrangement saves and nothing about strength. The fit commit's attributed build reproduces its own figures to within 60,022 instructions of the run. It was measured as 4379b1b, whose code 02432c9 carries unchanged: the two differ in comments and docs alone. On the clock, `scripts/speed.sh` against 02432c9 over 21 interleaved rounds reads +5.7% on the median nodes per second and +5.3% on the fastest, against a spread of 12.2%, which is two slow candidate rounds (7 and 18). The box was shared, so the instruction count is the claim, and the clock agrees with it in sign. ## Tests `the_shared_walk_counts_what_each_term_counts_alone` holds the walk to the two standalone counts for both colours on every position of the bench, tactical and strategic suites and the shared fens. It asks five instantiations: all four kinds with the ring, the live constants, two that leave out different kinds with the ring and without it, and no kind without the ring. A skipped count has to answer zero. The hand count tests of both terms run the same check on their positions. Two mutations were run against it. Reading the ring through mobility's scope fails it, and so does skipping the queens. Each also fails both hand count tests and `the_table_names_the_terms_the_sum_adds`. The ARCHITECTURE entries say the two terms share one walk at the leaf and keep their own counts for the tuner. The doc comments that said `eval` and the tuner read one function for these counts now name the shared walk and the test that holds it to them. The roadmap's budget bullet carries the fused figures, with the standalone 11.5% kept as history. Bench: 4102609 Speed: +5.7% (bench nps, 21 interleaved rounds vs 02432c9, spread 12.2%) Co-Authored-By: Claude Opus 5 --- arche-core/src/eval/king_attack.rs | 62 +++++++---- arche-core/src/eval/mobility.rs | 100 +++++++++++------ arche-core/src/eval/mod.rs | 167 ++++++++++++++++++++++++++++- arche-core/src/tune.rs | 42 ++++---- docs/ARCHITECTURE.md | 23 ++-- docs/ROADMAP.md | 16 +-- 6 files changed, 318 insertions(+), 92 deletions(-) diff --git a/arche-core/src/eval/king_attack.rs b/arche-core/src/eval/king_attack.rs index 0ba18148..1cf4a60f 100644 --- a/arche-core/src/eval/king_attack.rs +++ b/arche-core/src/eval/king_attack.rs @@ -18,8 +18,8 @@ use crate::psqt::{eg_value, mg_value, pack}; /// How many counts the term is measured in, which is one per piece that /// carries a weight. The same four as mobility, in the order -/// [`mobility::PIECES`] names them, so a walk that one day takes both readings -/// at once writes each into the slot the other uses. +/// [`mobility::PIECES`] names them, so the walk in `eval/mod.rs` that takes +/// both readings at once writes each into the slot the other uses. pub(crate) const COUNTS: usize = mobility::PIECES.len(); /// What one attacked square of the enemy king's ring is worth to each of @@ -103,7 +103,7 @@ pub(crate) const fn weight(index: usize) -> i32 { /// Whether [`super::sum`] takes this term at the leaf: true when one of the /// four [`KING_ATTACK`] weights is not zero at one end of the taper or the /// other. The 2026-09-16 fit priced all eight halves, so it is true and every -/// leaf asks [`counts_of`] for both sides. +/// leaf counts the ring for both sides. /// /// Derived from the weights rather than written out, which is how /// [`mobility::SCORED_KINDS`] is derived. The fit turned the term on by @@ -165,13 +165,17 @@ const fn scored(weights: &[i32; COUNTS]) -> bool { /// mostly the same pawn one rank on; a king bearing on the other king's ring is /// the opposition, which the endgame table prices. /// -/// Both [`super::eval`] and the tuner's walk read this, so the identity -/// between them cannot see a wrong count here. What pins it is the hand counts -/// in the tests below, the way the mobility counts are pinned. +/// The tuner's walk reads this, and the evaluation reads the same counts off +/// the walk in `eval/mod.rs` it shares with mobility, which +/// `the_shared_walk_counts_what_each_term_counts_alone` holds to this +/// function. So neither the identity nor that test can see a wrong count +/// here. What pins it is the hand counts in the tests below, the way the +/// mobility counts are pinned. /// -/// Inlined by force, for the reason `mobility::counts_of` gives: the -/// evaluation asks for it twice at every leaf and every quiescence node, and -/// llvm leaves a walk this shape out of line when it is left to itself. +/// Inlined by force, for the reason `mobility::counts_of` gives: when the +/// evaluation read this it asked for it twice at every leaf and every +/// quiescence node, and llvm leaves a walk this shape out of line when it is +/// left to itself. #[inline(always)] pub(crate) fn counts_of(board: &Board, color: Color) -> [i32; COUNTS] { let occupied = board.occupied(); @@ -211,21 +215,38 @@ pub(crate) fn counts(board: &Board, color: Color, into: &mut [i32]) { } /// What white's bearing on the black king stands ahead by, as a packed pair on -/// the scale the piece square pair is on. +/// the scale the piece square pair is on, given each side's counts. /// -/// The sum calls it only while [`SCORED`] is true. It was written in the -/// commit the counts arrived in, at zero weight, so that what the fit moved -/// was eight numbers and not the shape of the evaluation. +/// The sum calls it only while [`SCORED`] is true, and does not call +/// [`counts_of`] for the counts. It takes them from the walk in `eval/mod.rs` +/// that reads mobility's counts off the same attack sets, so a piece's attack +/// set is probed once for the two terms rather than once for each. #[inline] +pub(crate) fn fold_counts(white: [i32; COUNTS], black: [i32; COUNTS]) -> i32 { + weigh(&KING_ATTACK, white, black) +} + +/// The fold with the counts taken by [`counts_of`], which is what the sum +/// answered before it shared a walk with mobility. The tests hold the sum to +/// it. +#[cfg(test)] pub(crate) fn fold(board: &Board) -> i32 { fold_with(board, &KING_ATTACK) } /// The same fold against weights named by the caller. -#[inline] +#[cfg(test)] fn fold_with(board: &Board, weights: &[i32; COUNTS]) -> i32 { - let white = counts_of(board, Color::White); - let black = counts_of(board, Color::Black); + weigh( + weights, + counts_of(board, Color::White), + counts_of(board, Color::Black), + ) +} + +/// White's counts less black's, piece by piece, against `weights`. +#[inline] +fn weigh(weights: &[i32; COUNTS], white: [i32; COUNTS], black: [i32; COUNTS]) -> i32 { let mut packed = 0; for ((weight, white), black) in weights.iter().zip(white).zip(black) { packed += weight * (white - black); @@ -240,10 +261,10 @@ mod tests { use pretty_assertions::assert_eq; /// The counts by hand, square by square, because nothing else pins them. - /// The tuner's identity folds a row against the live weights, and `eval` - /// and the walk read this same function, so the two sides of the identity - /// move together whatever it answers. These cases are what pins the - /// counts themselves. + /// The tuner's identity folds a row against the live weights, and the + /// tuner reads this function and `eval` reads a walk held to it, so the + /// two sides of the identity move together whatever it answers. These + /// cases are what pins the counts themselves. /// /// The black king stands on e8 unless the case says otherwise, so its ring /// is d7, e7, f7, d8 and f8. The white king stands off every line the case @@ -366,6 +387,7 @@ mod tests { ] { let board = Board::from_fen(fen).unwrap(); assert_eq!(counts_of(&board, color), counts, "{}", why); + super::super::evaluate::the_shared_walk_agrees(&board, why); } } diff --git a/arche-core/src/eval/mobility.rs b/arche-core/src/eval/mobility.rs index 3d79df2f..db47b48d 100644 --- a/arche-core/src/eval/mobility.rs +++ b/arche-core/src/eval/mobility.rs @@ -139,27 +139,36 @@ const fn scored_kinds() -> u8 { /// time. The king and the pawn have no count of their own: a king's is a /// danger signal rather than a scope one, and a pawn's is move generation. /// -/// Both [`super::eval`] and the tuner's walk read this, and they may ask for -/// different kinds. The walk always asks for all four, because it is offline -/// and its coefficients are what prices a kind; the evaluation asks for the -/// kinds whose weight is not zero, because a count multiplied by zero is not -/// worth the leaf it is taken at. Since the refit priced all four the two -/// sets are equal today. What keeps them honest whether or not they are is -/// that the difference is exactly the zero weights, which -/// `eval_counts_a_kind_exactly_when_its_weight_is_not_zero` pins. Within one -/// set of kinds the counts are still one answer rather than two, so a second -/// implementation of them would still be two chances to be wrong rather than a -/// check on one, and the hand counts below are what pins them. That is the -/// exception to the rule `tune.rs` states in its header, which names it. +/// The tuner's walk reads this. The evaluation does not: it reads the same +/// counts off the shared walk in `eval/mod.rs`, which takes the king attack +/// zone's counts off the same attack sets, and +/// `the_shared_walk_counts_what_each_term_counts_alone` holds that walk to +/// this function kind by kind. +/// +/// The two may ask for different kinds. The tuner's walk always asks for all +/// four, because it is offline and its coefficients are what prices a kind; +/// the evaluation asks for the kinds whose weight is not zero, because a count +/// multiplied by zero is not worth the leaf it is taken at. Since the refit +/// priced all four the two sets are equal today. What keeps them honest +/// whether or not they are is that the difference is exactly the zero +/// weights, which `eval_counts_a_kind_exactly_when_its_weight_is_not_zero` +/// pins. +/// +/// The shared walk is a second statement of these counts, and it is held to +/// this one position by position. A count wrong in the same way in both would +/// still pass that test and the tuner's identity, so the hand counts below are +/// what pins the counts themselves. That is the exception to the rule +/// `tune.rs` states in its header, which names it. /// /// `KINDS` is a compile time set, so a kind left out of it costs nothing: /// its loop is not compiled rather than skipped. /// /// Inlined by force. Left to itself llvm keeps this out of line even under -/// link time optimisation, and the evaluation asks for it twice at every leaf -/// and every quiescence node. That call was three fifths of what the term cost -/// over the bench: 4.30 billion instructions without the attribute against -/// 3.76 billion with it. +/// link time optimisation, and when the evaluation read this it asked for it +/// twice at every leaf and every quiescence node. That call was three fifths +/// of what the term cost over the bench: 4.30 billion instructions without the +/// attribute against 3.76 billion with it. The shared walk is inlined by force +/// for the same reason. #[inline(always)] pub(crate) fn counts_of(board: &Board, color: Color) -> [i32; COUNTS] { let occupied = board.occupied(); @@ -207,16 +216,26 @@ pub(crate) fn counts(board: &Board, color: Color, into: &mut [i32]) { } /// What white's mobility stands ahead by, as a packed pair on the scale the -/// piece square pair is on. +/// piece square pair is on, given each side's counts. /// -/// Only [`SCORED_KINDS`] are counted. A kind whose weight is zero contributes -/// nothing however many squares it covers, so counting it is work no score -/// can see. The rule is what is written down, not a list: when the first fit -/// left six of the eight weights at zero, leaving three of the four kinds out -/// took a bit over a third off what the term cost. The refit priced all four, -/// so today this counts every kind and the skip is waiting for a weight to -/// round to nothing again. +/// The sum does not call [`counts_of`] for these. It takes them from the walk +/// in `eval/mod.rs` that reads the king attack counts off the same attack +/// sets, and that walk counts only [`SCORED_KINDS`]. A kind whose weight is +/// zero contributes nothing however many squares it covers, so counting it is +/// work no score can see. The rule is what is written down, not a list: when +/// the first fit left six of the eight weights at zero, leaving three of the +/// four kinds out took a bit over a third off what the term cost. The refit +/// priced all four, so today every kind is counted and the skip is waiting for +/// a weight to round to nothing again. #[inline] +pub(crate) fn fold_counts(white: [i32; COUNTS], black: [i32; COUNTS]) -> i32 { + weigh(&MOBILITY, white, black) +} + +/// The fold with the counts taken by [`counts_of`] over [`SCORED_KINDS`], +/// which is what the sum answered before it shared a walk with the king +/// attack zone. The tests hold the sum to it. +#[cfg(test)] pub(crate) fn fold(board: &Board) -> i32 { fold_with::(board, &MOBILITY) } @@ -230,10 +249,18 @@ pub(crate) fn fold(board: &Board) -> i32 { /// own through here anyway, over all four kinds, because what they pin is the /// fold rather than the fit: a permuted [`MOBILITY`] would be a different /// evaluation and not a wrong one. -#[inline] +#[cfg(test)] fn fold_with(board: &Board, weights: &[i32; COUNTS]) -> i32 { - let white = counts_of::(board, Color::White); - let black = counts_of::(board, Color::Black); + weigh( + weights, + counts_of::(board, Color::White), + counts_of::(board, Color::Black), + ) +} + +/// White's counts less black's, piece by piece, against `weights`. +#[inline] +fn weigh(weights: &[i32; COUNTS], white: [i32; COUNTS], black: [i32; COUNTS]) -> i32 { let mut packed = 0; for ((weight, white), black) in weights.iter().zip(white).zip(black) { packed += weight * (white - black); @@ -249,10 +276,10 @@ mod tests { use pretty_assertions::assert_eq; /// The counts by hand, square by square, because nothing else pins them. - /// The tuner's identity folds a row against the live weights, and `eval` - /// and the walk read this same function, so the two sides of the identity - /// move together whatever it answers. These cases are the only check this - /// term has. + /// The tuner's identity folds a row against the live weights, and the + /// tuner reads this function and `eval` reads a walk held to it, so the + /// two sides of the identity move together whatever it answers. These + /// cases are the only check this term has. /// /// Each case names what the count is made of. The two kings stand in /// opposite corners and out of the way, so that nothing here is a count of @@ -326,6 +353,7 @@ mod tests { "{}", why ); + super::super::evaluate::the_shared_walk_agrees(&board, why); } } @@ -407,8 +435,12 @@ mod tests { /// What the evaluation is allowed to leave out, which is the whole of the /// contract between it and the tuner's walk. /// - /// The walk counts all four kinds and the evaluation counts - /// [`SCORED_KINDS`], so the two no longer read one answer. What makes that + /// The tuner's walk counts all four kinds and the evaluation counts + /// [`SCORED_KINDS`], so the two no longer read one answer. The evaluation + /// takes those counts off the shared walk in `eval/mod.rs`, and this reads + /// `counts_of::`, which + /// `the_shared_walk_counts_what_each_term_counts_alone` holds that walk + /// to. What makes that /// safe is the size of the difference and nothing else: a count multiplied /// by zero adds nothing, so a kind worth zero can go uncounted without /// moving a score, and any other kind cannot. So this asserts the @@ -440,7 +472,7 @@ mod tests { scored[index], if priced { all[index] } else { 0 }, "{:?} is worth {} in the midgame and {} in the ending, so it should be \ - {}, and eval counted {} of its {} squares", + {}, and counts_of:: counted {} of its {} squares", piece, mg_value(weight), eg_value(weight), diff --git a/arche-core/src/eval/mod.rs b/arche-core/src/eval/mod.rs index fbcb3b2d..934f2fe9 100644 --- a/arche-core/src/eval/mod.rs +++ b/arche-core/src/eval/mod.rs @@ -20,7 +20,8 @@ mod mobility; mod pawn_structure; mod shelter; -use crate::board::Board; +use crate::board::{Board, king_attacks, knight_attacks, pawn_attacks, pop_lsb}; +use crate::magic::MAGIC; use crate::misc::{Color, Piece, Score}; use crate::psqt::{PieceSquareTables, eg_value, mg_value}; @@ -196,11 +197,15 @@ impl Memo for Caches { /// on one scale, and it is what keeps one divide however many such terms /// there are. /// +/// Mobility and the king attack zone are read off one walk over each side's +/// pieces, [`attack_counts`], rather than a walk each. +/// /// The king attack zone is behind [`king_attack::SCORED`], which is true at /// the fitted weights and would be false if all eight were zero. A count /// multiplied by nothing scores nothing, and llvm does not take the walk that /// produces it out on that ground, so at a constant false the summand is not -/// compiled rather than computed and thrown away. +/// compiled rather than computed and thrown away, and the walk takes no ring +/// counts. /// /// Material that cannot mate reads zero before any of it. That is the one /// place in the evaluation that is not a dot product against the weights, @@ -214,17 +219,106 @@ fn sum(board: &Board, memo: &mut impl Memo) -> Score { if board.drawn_by_material() { return 0; } - let leaf = mobility::fold(board) + let (white_scope, white_ring) = + attack_counts::<{ mobility::SCORED_KINDS }, { king_attack::SCORED }>(board, Color::White); + let (black_scope, black_ring) = + attack_counts::<{ mobility::SCORED_KINDS }, { king_attack::SCORED }>(board, Color::Black); + let leaf = mobility::fold_counts(white_scope, black_scope) + memo.shelter(board) + memo.pawn_structure(board) + if king_attack::SCORED { - king_attack::fold(board) + king_attack::fold_counts(white_ring, black_ring) } else { 0 }; board.eval.score(board.active_color, leaf) } +/// One side's mobility counts and its king attack counts, from one walk over +/// its knights, bishops, rooks and queens. +/// +/// Each piece's attack set is taken once, the knight's mask or the magic +/// probes over the whole occupancy, and read twice: against mobility's scope +/// for the first array and against the enemy king's ring for the second. The +/// two terms ask the same question of every piece and differ only in the mask +/// the answer is read against, so the probes are what they share and each +/// keeps its own mask and its own column. +/// +/// `KINDS` says which of the four kinds mobility counts, as +/// [`mobility::counts_of`] reads it, and `RING` whether the ring is counted at +/// all. Both are compile time, so a count that is not asked for is not +/// compiled, down to the loop over a kind neither term wants. A count left out +/// answers zero. +/// +/// This is a second statement of the two terms' counts, and on purpose. Each +/// term keeps its own [`mobility::counts_of`] and [`king_attack::counts_of`], +/// which the tuner reads through [`TERMS`] and the hand counts pin, and +/// `the_shared_walk_counts_what_each_term_counts_alone` holds this walk to the +/// two of them over the bench, tactical and strategic suites. It lives here +/// rather than in either term's file because it is the sum's arrangement of +/// the two and has no other caller, and a term file holding it would compute +/// the other term's counts. +/// +/// Inlined by force, for the reason `mobility::counts_of` gives. +#[inline(always)] +fn attack_counts( + board: &Board, + color: Color, +) -> ([i32; mobility::COUNTS], [i32; king_attack::COUNTS]) { + let occupied = board.occupied(); + let (ours, theirs) = board.sides(color); + let scope = !(ours | pawn_attacks(board.pawns() & theirs, !color)); + let ring = if RING { + king_attacks(board.king_index(!color)) + } else { + 0 + }; + let magic = &MAGIC; + let mut scoped = [0; mobility::COUNTS]; + let mut bearing = [0; king_attack::COUNTS]; + let mut read = |index: usize, attacks: u64| { + if mobility::counted(KINDS, index) { + scoped[index] += (attacks & scope).count_ones() as i32; + } + if RING { + bearing[index] += (attacks & ring).count_ones() as i32; + } + }; + let walked = |index: usize| mobility::counted(KINDS, index) || RING; + if walked(0) { + let mut knights = board.knights() & ours; + while knights != 0 { + let from = pop_lsb(&mut knights); + read(0, knight_attacks(from)); + } + } + if walked(1) { + let mut bishops = board.bishops() & ours; + while bishops != 0 { + let from = pop_lsb(&mut bishops); + read(1, magic.get_diagonal_move(from, occupied)); + } + } + if walked(2) { + let mut rooks = board.rooks() & ours; + while rooks != 0 { + let from = pop_lsb(&mut rooks); + read(2, magic.get_straight_move(from, occupied)); + } + } + if walked(3) { + let mut queens = board.queens() & ours; + while queens != 0 { + let from = pop_lsb(&mut queens); + read( + 3, + magic.get_straight_move(from, occupied) | magic.get_diagonal_move(from, occupied), + ); + } + } + (scoped, bearing) +} + /// The score with the shelter and the pawn structure computed every time. /// /// The tuner's walk and the instruments ask this one, and so do the five @@ -877,4 +971,69 @@ mod evaluate { + king_attack::fold(&board); assert_eq!(eval(&board), board.eval.score(board.active_color, leaf)); } + + /// Holds [`super::attack_counts`] to [`mobility::counts_of`] and + /// [`king_attack::counts_of`] on one position, for both colours. + /// + /// Five instantiations are asked. All four kinds with the ring is the + /// walk with nothing skipped, and the live constants are the one the sum + /// compiles. Two more leave out a different half of the kinds each, one + /// with the ring and one without, so that the skips answer zero where they + /// skip and the standalone count everywhere else. The last asks for no + /// kind and no ring, where every count has to answer zero. The term tests + /// call this on their hand count positions, which is why it is not a test + /// of its own. + pub(super) fn the_shared_walk_agrees(board: &Board, why: &str) { + fn held(board: &Board, color: Color, why: &str) { + let scope = mobility::counts_of::<{ mobility::ALL_KINDS }>(board, color); + let ring = king_attack::counts_of(board, color); + let expected = ( + std::array::from_fn(|index| { + if mobility::counted(KINDS, index) { + scope[index] + } else { + 0 + } + }), + if RING { ring } else { [0; king_attack::COUNTS] }, + ); + assert_eq!( + super::attack_counts::(board, color), + expected, + "{} for {:?}, counting kinds {:04b} and the ring {}", + why, + color, + KINDS, + RING + ); + } + for color in [Color::White, Color::Black] { + held::<{ mobility::ALL_KINDS }, true>(board, color, why); + held::<{ mobility::SCORED_KINDS }, { king_attack::SCORED }>(board, color, why); + held::<0b0101, false>(board, color, why); + held::<0b1010, true>(board, color, why); + held::<0, false>(board, color, why); + } + } + + /// The walk the sum reads mobility and the king attack zone off answers + /// what the two terms' own counts answer, on every position of the bench, + /// tactical and strategic suites and the shared fens. + /// + /// The tuner reads the standalone counts and the evaluation reads the + /// walk, so the tuner's identity holds the two together only at the live + /// constants and only through the fold. This holds them count by count, + /// and under the skips as well. + #[test] + fn the_shared_walk_counts_what_each_term_counts_alone() { + let mut fens: Vec = fens::CORE.iter().map(|f| f.to_string()).collect(); + fens.extend(crate::bench::positions().into_iter().map(|p| p.fen)); + fens.extend(crate::tactics::positions().into_iter().map(|p| p.fen)); + fens.extend(crate::strategy::positions().into_iter().map(|p| p.fen)); + assert!(fens.len() > 1_500, "{} positions", fens.len()); + for fen in fens { + let board = Board::from_fen(&fen).unwrap_or_else(|e| panic!("{}: {}", fen, e)); + the_shared_walk_agrees(&board, &fen); + } + } } diff --git a/arche-core/src/tune.rs b/arche-core/src/tune.rs index 59a8d033..ca22d96d 100644 --- a/arche-core/src/tune.rs +++ b/arche-core/src/tune.rs @@ -35,26 +35,30 @@ //! is neither read nor duplicated. //! //! The four leaf terms are the exception, and it is deliberate. The walk -//! asks each term in `eval::TERMS` for its counts through the descriptor, and -//! what it reaches is the function `eval` reads, so the identity cannot see a -//! wrong count at any weights, fitted or zero. A second count here would be a -//! second chance to be wrong about a term that is read at every leaf rather -//! than a check on the first, so what pins them is the hand counts beside each -//! term in `eval/`. +//! asks each term in `eval::TERMS` for its counts through the descriptor. For +//! shelter and pawn structure what it reaches is the function `eval` reads, so +//! the identity cannot see a wrong count at any weights, fitted or zero. For +//! mobility and the king attack zone, `eval` reads a shared walk that +//! `the_shared_walk_counts_what_each_term_counts_alone` holds to these +//! functions. That test catches the two disagreeing, and neither it nor the +//! identity can see a count wrong the same way in both. A second count here +//! would be a second chance to be wrong about a term that is read at every +//! leaf rather than a check on the first, so what pins them is the hand counts +//! beside each term in `eval/`. //! //! The layout is that list too. Where a term stands in the vector, what its //! weights are and how many counts it carries all come off `eval::TERMS`, so a //! term added there is laid out, priced and walked here without an edit, and //! the run's layout line states the result for `scripts/tune.py` to read. //! -//! On mobility the two may ask for different kinds. The walk always asks for -//! all four, because it is offline and a coefficient for a kind worth nothing -//! today is what lets a later fit price it; `eval` asks only for the kinds -//! whose weight is not zero, because a count multiplied by zero is not worth -//! taking at every leaf. The refit priced all four, so the two sets are equal -//! today and the walk's row is the wider of the two only while some weight -//! rounds to nothing. The identity holds either way, because the difference -//! is exactly the kinds that score nothing, and +//! On mobility the two may ask for different kinds. The tuner's walk always +//! asks for all four, because it is offline and a coefficient for a kind worth +//! nothing today is what lets a later fit price it; `eval` asks only for the +//! kinds whose weight is not zero, because a count multiplied by zero is not +//! worth taking at every leaf. The refit priced all four, so the two sets are +//! equal today and the tuner's row is the wider of the two only while some +//! weight rounds to nothing. The identity holds either way, because the +//! difference is exactly the kinds that score nothing, and //! `eval_counts_a_kind_exactly_when_its_weight_is_not_zero` is what says the //! difference is that and not something else. None of the other three is //! split that way. Their fits gave every one of their weights a value, the @@ -741,10 +745,12 @@ mod tests { /// /// The identity reads these eight slots against the fitted weights, so it /// catches a coefficient written to the wrong slot only where the two - /// weights differ, and it cannot tell a count that is wrong in the walk - /// from the same count wrong in `eval`, since both read one function. What - /// is asserted here is the coefficient itself, against a count worked out - /// by hand from the position below. + /// weights differ. Nor can it see a count that is wrong in the tuner's walk + /// and wrong the same way in the shared walk `eval` reads, which + /// `the_shared_walk_counts_what_each_term_counts_alone` holds to the + /// function the tuner's walk reads. What is asserted here is the + /// coefficient itself, against a count worked out by hand from the + /// position below. #[test] fn every_king_attack_count_writes_both_ends_of_the_taper() { // one white piece of each kind that carries a weight, and no black diff --git a/docs/ARCHITECTURE.md b/docs/ARCHITECTURE.md index 5ed02384..d611de7e 100644 --- a/docs/ARCHITECTURE.md +++ b/docs/ARCHITECTURE.md @@ -115,10 +115,15 @@ four are fitted to the engine's own archived games. that, which is the one place the score is not a sum over the weights. Also here is `TERMS`, a descriptor per leaf term (its name, how many counts it is measured in, its weights and its counts), which is what the - tuner lays its slot vector out from. + tuner lays its slot vector out from. And the one walk over each side's + knights, bishops, rooks and queens that mobility and the king attack zone + both read at the leaf: each piece's attack set is probed once and read + against both terms' masks. - **mobility.rs**: How many squares each side's pieces cover. Read off the - board at every leaf and not remembered: a piece that moves changes what - every slider looking through its square sees. + board at every leaf, through the walk it shares with the king attack + zone, and not remembered: a piece that moves changes what every slider + looking through its square sees. The term keeps its own count of the same + squares, which the tuner reads and a test holds the shared walk to. - **shelter.rs**: What stands between each king and the board, its own pawns and the enemy pawns coming for it. Computed at the leaf and then remembered under the pawns and the two king squares it is a function of, @@ -131,11 +136,13 @@ four are fitted to the engine's own archived games. king move as well. - **king_attack.rs**: How many squares of the enemy king's ring each side's knights, bishops, rooks and queens attack, off the same attack sets - mobility walks but with nothing taken out of them. Read at every leaf, - and skipped only if its weights are all zero. Not remembered: what it - depends on is the whole occupancy - and a side's pieces, which is the position, and a table under that key was - measured for mobility and turned down on the memory it cost. + mobility walks but with nothing taken out of them. Read at every leaf off + the walk it shares with mobility, and skipped only if its weights are all + zero. The term keeps its own count as well, for the tuner's rows, and the + shared walk has to agree with it. Not remembered: what it depends on is + the whole occupancy and a side's pieces, which is the position, and a + table under that key was measured for mobility and turned down on the + memory it cost. - **psqt.rs**: The piece square tables. Every piece has a second table for the endgame; both phases are packed into one integer so the taper costs one multiply. diff --git a/docs/ROADMAP.md b/docs/ROADMAP.md index f15ff57c..9028e9f8 100644 --- a/docs/ROADMAP.md +++ b/docs/ROADMAP.md @@ -121,14 +121,14 @@ the engine plays. Roughly in the order they look worth doing. the refit that priced all four kinds; measured again at `378c148` the term is 13.5% of the bench. The 5% is a rule of thumb and nothing enforces it, which the games have now said outright: the term was made 6% cheaper across the whole search and 6,000 of them - could not see it. So the ceiling is the thing under question rather than the term. - The king attack zone is over it as well. Measured at its fit with callgrind over - `arche bench` at depth 7, its fold and its counts forced out of line, the term is - 558,628,090 of 4,854,634,752 instructions, 11.5%, and mobility is 11.4% of the same run. - The run costs 1,175 instructions a node against 1,040 before the fit, 13.0% more. The - counts are a second walk over the pieces mobility already walks, and taking both in one - walk is the change that would bring the term down. The - reasoning and the numbers are in the planning repository + could not see it. So the ceiling is the thing under question rather than the term. The + reasoning and the numbers are in the planning repository. The king attack zone costs + less than the 5%. It was 11.5% of the run as a walk of its own at its fit, and it now + takes its counts in mobility's walk. Measured with callgrind over `arche bench` at + depth 7, the shared walk forced out of line, the walk is 777,417,417 of 4,600,409,835 + instructions, 16.9%, and the same walk without the ring is 576,199,267, so the ring is + 201,218,150 of them, 4.4%, and 4.8% with its fold. The shipped build costs 1,085 + instructions a node against 1,040 before the term, 4.3% more - a held-out loss on our own games cannot resolve a fit of the piece square tables one way or the other, so an sprt is what decides a re-tune. Measured 2026-09-10 over 1,812 archived games, 100,726 quiet positions across 1,807 of them: the shipped weights score