docs(ctx_search): add hybrid-retrieval query-phrasing guidance to tool description - #430
docs(ctx_search): add hybrid-retrieval query-phrasing guidance to tool description#430G0-0000 wants to merge 3 commits into
Conversation
| export const CTX_SEARCH_TOOL_NAME = "ctx_search"; | ||
| export const CTX_SEARCH_DESCRIPTION = `Your long-term recall for this project — search everything that ever happened here, not just what's currently visible. | ||
|
|
||
| Retrieval is HYBRID: ~0.7 semantic (embedding cosine) + ~0.3 keyword (FTS5 BM25), fused with per-source boosts. Phrase \`query\` as a natural-language QUESTION that still embeds the key literal terms (paths, symbols, commands, config keys) — pure keyword stacks starve the semantic leg (0.7 weight), while pure prose without literal terms may miss exact FTS matches. Good: "Where is the dream log file and which error counters indicate failures?" Bad: "dream log error counter". |
There was a problem hiding this comment.
Changing the full ctx_search description without regenerating the A1 golden leaves the expected description stale. The full-preset registry test compares this value byte-for-byte with the golden, which still lacks this paragraph, so the test suite will fail until the golden is updated.
…c prompt-surface golden
|
Thanks Greptile — both review notes are addressed in 0edb532.
Thanks again for catching both. |
There was a problem hiding this comment.
2 issues found across 2 files (changes from recent commits).
Prompt for AI agents (unresolved issues)
Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.
<file name="packages/plugin/src/shared/prompt-surface-a1-golden.md">
<violation number="1" location="packages/plugin/src/shared/prompt-surface-a1-golden.md:403">
P3: The golden's section header still reads `### ctx_search — description ~309 tokens`, but this delta adds a substantial paragraph to the description without updating the token estimate. The current CTX_SEARCH_DESCRIPTION is ~296 words / ~1900 chars (roughly 380+ Claude BPE tokens), so ~309 understates it by a meaningful margin. Regenerate the golden with `export-agent-surface.ts` (or update the header) so the estimate matches the synced description.</violation>
</file>
<file name="packages/plugin/src/tools/ctx-search/constants.ts">
<violation number="1" location="packages/plugin/src/tools/ctx-search/constants.ts:4">
P2: Do not infer embedding availability from the absence of `match=semantic` labels. This can misclassify embedding-enabled searches when results are unlabeled, labeled `hybrid`, or omit semantic results; remove this fallback signal or expose an explicit embedding-status indicator.</violation>
</file>
Reply with feedback, questions, or to request a fix.
Re-trigger cubic
| ``` | ||
| Your long-term recall for this project — search everything that ever happened here, not just what's currently visible. | ||
|
|
||
| Retrieval is HYBRID when embeddings are enabled: ~0.7 semantic (embedding cosine) + ~0.3 keyword (FTS5 BM25), fused with per-source boosts; with embeddings disabled it silently falls back to keyword-only matching — tell by results carrying no `match=semantic` labels, and then prefer literal-heavy queries. Phrase `query` as a natural-language QUESTION that still embeds the key literal terms (paths, symbols, commands, config keys) — pure keyword stacks starve the semantic leg (0.7 weight), while pure prose without literal terms may miss exact FTS matches. Good: "Where is the dream log file and which error counters indicate failures?" Bad: "dream log error counter". |
There was a problem hiding this comment.
P3: The golden's section header still reads ### ctx_search — description ~309 tokens, but this delta adds a substantial paragraph to the description without updating the token estimate. The current CTX_SEARCH_DESCRIPTION is ~296 words / ~1900 chars (roughly 380+ Claude BPE tokens), so ~309 understates it by a meaningful margin. Regenerate the golden with export-agent-surface.ts (or update the header) so the estimate matches the synced description.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At packages/plugin/src/shared/prompt-surface-a1-golden.md, line 403:
<comment>The golden's section header still reads `### ctx_search — description ~309 tokens`, but this delta adds a substantial paragraph to the description without updating the token estimate. The current CTX_SEARCH_DESCRIPTION is ~296 words / ~1900 chars (roughly 380+ Claude BPE tokens), so ~309 understates it by a meaningful margin. Regenerate the golden with `export-agent-surface.ts` (or update the header) so the estimate matches the synced description.</comment>
<file context>
@@ -400,6 +400,8 @@ Example: ctx_memory(action="write", category="CONSTRAINTS", content="Pi stores s
Your long-term recall for this project — search everything that ever happened here, not just what's currently visible.
+Retrieval is HYBRID when embeddings are enabled: ~0.7 semantic (embedding cosine) + ~0.3 keyword (FTS5 BM25), fused with per-source boosts; with embeddings disabled it silently falls back to keyword-only matching — tell by results carrying no match=semantic labels, and then prefer literal-heavy queries. Phrase query as a natural-language QUESTION that still embeds the key literal terms (paths, symbols, commands, config keys) — pure keyword stacks starve the semantic leg (0.7 weight), while pure prose without literal terms may miss exact FTS matches. Good: "Where is the dream log file and which error counters indicate failures?" Bad: "dream log error counter".
+
Reach for it when something feels familiar but isn't in view: "did we solve this before?", "what did we decide about X?", "when did this break?", "where does Y live?". Results only contain things you CANNOT currently see — memories already shown in and the live conversation tail are filtered out. A query that is just one or more memory ids (e.g. #7234 or 12, 34) bypasses text search and resolves those ids directly.
</file context>
</details>
|
Confirmed: Dropped the self-check sentence that told models to infer embedding-off from missing A reliable embedding-status signal would be a code-level feature, outside this docs PR. Happy to discuss that separately if useful. |
Refs #429
Pure copy change: one insertion in
packages/plugin/src/tools/ctx-search/constants.ts(CTX_SEARCH_DESCRIPTION), between the opening paragraph and the Reach-for-it section. No schema, tests, or i18n updates (verified none needed for this description-only change).Documents that ctx_search is hybrid retrieval (~0.7 semantic / ~0.3 FTS) and that
queryshould be a natural-language question that still embeds literal terms. Bare keyword stacks starve the 0.7 semantic leg.Empirical (same intent, same target memory in a real project DB): question-form with embedded literals moved the true hit from score 0.55 (3rd) to 0.60 (2nd).
Whether the light-plugin description should be kept in sync is left to the maintainer.
Need help on this PR? Tag
@codesmith-botwith what you need. Autofix is disabled.Summary by cubic
Documents
ctx_search's hybrid retrieval and query-phrasing guidance, and syncs the prompt-surface golden so the exact-description test passes. Query phrasing now recommends natural-language questions that still embed literal terms.Written for commit 468a27d. Summary will update on new commits.
Greptile Summary
Updates the
ctx_searchdocumentation and its prompt-surface golden to explain hybrid retrieval and recommend natural-language queries containing relevant literal terms.Confidence Score: 5/5
The documentation-only change appears safe to merge, with the golden synchronized and no outstanding correctness issue.
The previously stale golden is now updated exactly, and the unreliable fallback-status inference has been removed. Two earlier non-blocking threads were manually resolved without explanatory replies; the current code nevertheless addresses their underlying concerns.
Important Files Changed
ctx_searchdescription.Reviews (3): Last reviewed commit: "docs(ctx_search): drop unreliable match-..." | Re-trigger Greptile