perf: propagate top-k floor to conjunction children for block-max pruning

[LuQQiu]

Problem

A multi-term FTS AND (RequiredConjunctionScorer) never pushes the top-k competitive floor down to its MUST children. With ≥2 children, set_min_competitive_score returned without setting any child floor:

if self.children.len() == 1 {
    self.children[0].set_min_competitive_score(min_score)?;
}
Ok(())  // >=2 children: floor propagated to nothing

The guard is correct — propagating the full floor to one child is unsafe, since a MUST term can score below the floor while the terms sum to a competitive hit — but the effect was "propagate nothing", so each child's block-max skip machinery (skip_non_competitive_block) stayed inert for every multi-term AND. The scorer then intersected and fully scored every candidate.

Measured on a 2.86B-row corpus (analyze_plan index_comparisons, tight per-tier df control): single-term pruning was ~99.9%, but a 2nd term collapsed it to ~47-61% across all frequency tiers, and index_comparisons jumped 4x (rare terms) to ~1,100x (very common terms) — hundreds of millions of candidates scored to return 10 rows.

Fix

Give each child i a floor of min_score − Σ_{j≠i} upper_bound_j, rounded down (next_down) so float error can only under-prune. A child's block is skipped only when even the best possible contribution from every other MUST term cannot lift the sum to the top-k floor — so no competitive document is ever pruned.

Reuses existing infrastructure: sum_global_score_upper_bounds (already finiteness/non-negativity checked) and each child's global_score_upper_bound + skip_non_competitive_block. Falls back to the previous no-floor behavior when scores may be negative or a child lacks a finite upper bound. Single-term queries are unaffected (they use the block-max WAND path, not the conjunction path).

Tests

• required_conjunction_propagates_floor_to_children_for_block_skip: a hopeless first block (sum below floor) is skipped once the floor reaches the children — next() lands on the competitive doc instead of doc 0.
• required_conjunction_floor_never_prunes_a_competitive_sum: a child individually far below the floor (1.0 vs floor 10) is never pruned when the other term lifts every sum to 21 — all docs survive.
• Existing required_conjunction_uses_all_must_scores_for_competitive_bounds and the full inverted suite (518 tests) pass unchanged.

[LuQQiu]

Good catch — fixed in d7ce02f. Subtracting a child bound from the outward-rounded total wasn't a conservative sibling bound. I now derive each child's "others" upper directly as a provably conservative sum: accumulate the other children's f32 upper bounds in f64, widen by score_sum_upper_bound_factor(n-1) for the accumulation error, and round outward with outward_f32_upper_bound. This only over-estimates the rest of the sum, so the child floor only under-prunes and the equality boundary is preserved. Added your four-child float-tie reproducer as required_conjunction_keeps_four_child_floor_tie — it now passes, along with the full inverted suite (519 tests).

[lance-gatekeeper]

❌ Gate recommendation: request changes.

The reported four-child equality case is fixed, but the generalized floor derivation still over-prunes because the final f32 subtraction can round upward. The child floor must conservatively invert the complete f32 score accumulation, including that subtraction boundary.

A viable revision would derive and validate the floor against the actual accumulation semantics—for example, by bit-searching candidate f32 child scores against the existing outward sum bound—and cover cancellation-sensitive two-child equality cases.

[lance-gatekeeper]

The new others_upper is conservative, but min_score - others_upper is evaluated in f32 and can round upward. Here the inclusive floor is 0x472f12aa, while this line derives 0x3df00000 for a child scoring 0x3def75a1; that child is incorrectly skipped and the exactly competitive document disappears.

Please conservatively invert the full f32 accumulation instead of using raw subtraction. Bit-searching candidate f32 scores against the existing outward bound is one viable approach; decrementing by one ULP is not generally sufficient when subtraction loses multiple bits. This is the current projection of the predecessor finding.

Reproducer run on this head

#[test]
fn required_conjunction_keeps_two_child_floor_tie() {
    let child = f32::from_bits(0x3def_75a1);
    let other = f32::from_bits(0x472f_128c);
    let floor = child + other;
    assert_eq!(floor.to_bits(), 0x472f_12aa);

    let left = MaterializedScorer::try_new(rows(&[(0, child)]))
        .unwrap()
        .with_block_size(1);
    let right = MaterializedScorer::try_new(rows(&[(0, other)]))
        .unwrap()
        .with_block_size(1);
    let mut scorer = RequiredConjunctionScorer::try_new(vec![
        Box::new(left),
        Box::new(right),
    ])
    .unwrap();

    scorer.set_min_competitive_score(floor).unwrap();
    assert_eq!(scorer.next().unwrap(), Some(0));
}

cargo test -p lance-index required_conjunction_keeps_two_child_floor_tie -- --nocapture fails with left: None, right: Some(0).