Skip to content

fix(search): apply cosine radius across iterative recall paths - #451

Open
knqiufan wants to merge 1 commit into
EverMind-AI:mainfrom
knqiufan:codex/fix-search-radius
Open

knqiufan wants to merge 1 commit into
EverMind-AI:mainfrom
knqiufan:codex/fix-search-radius

Conversation

@knqiufan

Copy link
Copy Markdown

Summary

Fixes #394.

agentic and llm_multiround accepted radius but bypassed the manager's dense-recall filter. A request with dense hits at cosine 0.9 and 0.2 and radius=0.5 now removes the 0.2 hit before fusion on every iterative query. Existing vector and hybrid filtering remains in place.

  • Share request-default resolution and dense filtering across search orchestration. Resolve the default before capping top_k: explicit values win (including zero), unlimited searches default to 0.5, and positive top_k has no implicit floor.
  • Pass the resolved floor through user/agent AGENTIC and LLM_MULTIROUND. Filter both atomic-fact and episode-subject dense hits before user AGENTIC MaxSim pooling, and filter each multi-round dense pool before RRF.
  • Remove the three inert RankInput.radius arguments. Never map cosine thresholds to ahybrid_retrieve(min_score=...), which would compare them with a different score scale.
  • Document the dense-only contract and the episode HYBRID-only scope of min_score. Add 49 regression cases covering dispatch/defaults, boundaries, sparse preservation, empty dense pools, both agent kinds, and repeated queries with real RRF/MaxSim operators.

Area

  • Architecture method
  • Benchmark
  • Use case
  • Documentation
  • Developer experience
  • CI, build, or release

Verification

Pre-PR Linux validation uses the same source/test tree as this PR; validation branches change only workflow configuration:

  • Linux CI: make lint and make package passed; make test passed on Python 3.12 and 3.13 (2599 passed, 4 skipped each); make integration passed on both versions (183 passed, 5 skipped, 7 deselected each).
  • Local search suite: 310 passed. Windows required an import-only fcntl stand-in that raises on any attempted POSIX operation; the Linux runs above use the real platform without that stand-in.
  • Documentation links, Conventional Commit format, and git diff --check passed. OpenAPI export has no drift.
  • Milvus 2.6.22 validation: repository contract (5 passed), /get telemetry (1 passed), tiered API end-to-end tests (27 passed), and both-backend index contract (18 passed). The first run could not pull the upstream compose file's minio/minio image; the successful validation uses the same MinIO version tag from quay.io/minio/minio. This environment-only workaround is not part of the PR; the unchanged upstream workflow may encounter the same Docker Hub pull failure.

Checklist

  • I kept the change scoped to the relevant area.
  • I am opening this from a separate branch, not pushing directly to main.
  • I updated docs, examples, or setup notes when behavior changed.
  • I added or updated tests when the change affects behavior.
  • I did not commit secrets, .env files, dependency folders, or generated output.
  • Active relative links in Markdown files resolve.

Notes for Reviewers

This implements option (a) from the issue: a pre-fusion dense cosine floor. Keyword matches, cluster expansions, and case-to-skill linkage remain independent recall routes; radius is not a final-score floor or a promise of monotonically decreasing result counts. The existing min_score behavior is unchanged.

The intended behavior change includes default unlimited iterative searches: they now honor the documented 0.5 floor. Internal helper callers that omit the new optional argument retain unfiltered behavior. VECTOR and HYBRID already filtered dense hits before this change; the unused ranker argument was not evidence that those paths ignored radius.

By submitting this pull request, I agree that my contribution is licensed under the Apache License 2.0.

Share radius resolution and pre-fusion dense filtering across search lanes. Preserve sparse and linkage routes, remove inert RankInput arguments, and cover defaults, boundaries, iterative queries, and RRF score separation.

Fixes EverMind-AI#394
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

radius is documented and validated but has no effect on any search lane

1 participant