ctx_search: dismissed notes stay searchable and outrank memories (positional score 1.00); request search configurability (limit / score threshold / time decay / rerank)

[G0-0000]

ctx_search: dismissed notes stay searchable and outrank memories (positional score 1.00); request search configurability (limit / score threshold / time decay / rerank)

Bug: dismissed notes remain searchable and dominate results

In packages/plugin/src/features/magic-context/search.ts (shipped bundle dist/index-jc59f2q7.js, v0.41.4):

1. dismissed is in the searchable-status list.

// line 34487
var NOTE_SEARCHABLE_STATUSES = ["active", "pending", "ready", "dismissed"];

Dismissing a note only affects injection, not search. There is also no retention/TTL and no deletion path for notes at all — the only note write paths are INSERT (status transitions aside), so a dismissed note stays searchable forever.

2. Note scores are positional, not relevance-based.

After keyword matching in rankNotesForNeedle (line 34506), the score returned for each note is:

// line 34562
score: linearDecayScore(rank, ranked.length),   // i.e. 1 − rank/N

linearDecayScore (line 34282) is a function of rank position and result count only. Because the note pool is typically small, a single keyword hit is often the only hit — and the sole hit always scores 1.00, regardless of how weak the match is.

3. Note source boost is 1, so 1.00 is nearly unbeatable.

// lines 34702–34714 — getSourceBoost: note → 1, memory → 1.3

With the memory boost at 1.3 (line 33892), a memory needs a raw score ≥ ~0.77 (1.00 / 1.3) to outrank a note's 1.00. Typical semantic/FTS memory hits score well below that.

Reproduction (real project DB)

The same short dismissed note ranked #1 with score 1.00 in three separate ctx_search queries, while genuinely useful memory hits for those queries scored only 0.38–0.58. The note merely happened to contain one query token; its dismissed status and staleness had no effect.

Net effect: one dismissed note matching a single word permanently squats at the top of ctx_search for that term.

(Note: I have locally patched NOTE_SEARCHABLE_STATUSES to drop "dismissed" as a stopgap; the line numbers/behavior above are from the pristine published 0.41.4 tarball.)

Feature requests: search configurability

None of the following are currently configurable — the config schema is closed (additionalProperties: false) and exposes no search-tuning keys for explicit ctx_search:

a. Search knobs — configurable default limit (currently hardcoded DEFAULT_LIMIT = 10, line 36901) and a min_score threshold for explicit searches (the existing auto_search.score_threshold only gates the auto-search hint, not user-invoked ctx_search).

b. Note source control — options to disable the note source in ctx_search, exclude dismissed notes from search (or at minimum from top ranking), and choose the note scoring mode (positional vs. the raw keyword score already computed in rankNotesForNeedle).

c. Time dimension — filter by age (since_days / before) and/or configurable recency decay, so stale notes/memories don't outrank fresh, more relevant hits.

d. Second-stage rerank — optional rerank of the top-K unified candidates, e.g. an external reranker endpoint with OpenAI-compatible /rerank semantics (cross-encoder). This would fix the "positional 1.00 beats semantic 0.5" class of problems generically, across all sources.

Environment

• @cortexkit/pi-magic-context 0.41.4 (Pi extension)
• Config: ~/.config/cortexkit/magic-context.jsonc
• DB: ~/.local/share/cortexkit/magic-context/context.db

Happy to provide the reproducing queries/scores or test a patch — thanks for the quick turnaround on #421/#422.