Skip to content

docs(ctx_search): add hybrid-retrieval query-phrasing guidance to tool description - #430

Open
G0-0000 wants to merge 3 commits into
cortexkit:masterfrom
G0-0000:docs/ctx-search-hybrid-guidance
Open

docs(ctx_search): add hybrid-retrieval query-phrasing guidance to tool description#430
G0-0000 wants to merge 3 commits into
cortexkit:masterfrom
G0-0000:docs/ctx-search-hybrid-guidance

Conversation

@G0-0000

@G0-0000 G0-0000 commented Sep 7, 2026

Copy link
Copy Markdown

Refs #429

Pure copy change: one insertion in packages/plugin/src/tools/ctx-search/constants.ts (CTX_SEARCH_DESCRIPTION), between the opening paragraph and the Reach-for-it section. No schema, tests, or i18n updates (verified none needed for this description-only change).

Documents that ctx_search is hybrid retrieval (~0.7 semantic / ~0.3 FTS) and that query should be a natural-language question that still embeds literal terms. Bare keyword stacks starve the 0.7 semantic leg.

Empirical (same intent, same target memory in a real project DB): question-form with embedded literals moved the true hit from score 0.55 (3rd) to 0.60 (2nd).

Whether the light-plugin description should be kept in sync is left to the maintainer.


View with [code]smith Autofix with [code]smith
Need help on this PR? Tag @codesmith-bot with what you need. Autofix is disabled.


Summary by cubic

Documents ctx_search's hybrid retrieval and query-phrasing guidance, and syncs the prompt-surface golden so the exact-description test passes. Query phrasing now recommends natural-language questions that still embed literal terms.

  • Covers the ~0.7 semantic / ~0.3 keyword mix and warns against pure keyword stacks or pure prose, and notes the keyword-only fallback when embeddings are disabled.
  • Leaves the light-plugin description untouched; syncing it is left to the maintainer.

Written for commit 468a27d. Summary will update on new commits.

Review in cubic

Greptile Summary

Updates the ctx_search documentation and its prompt-surface golden to explain hybrid retrieval and recommend natural-language queries containing relevant literal terms.

  • Keeps the runtime description and byte-for-byte golden fixture synchronized.
  • Qualifies hybrid retrieval based on embedding availability.
  • Removes the unreliable suggestion that missing semantic match labels reveal keyword-only fallback.

Confidence Score: 5/5

The documentation-only change appears safe to merge, with the golden synchronized and no outstanding correctness issue.

The previously stale golden is now updated exactly, and the unreliable fallback-status inference has been removed. Two earlier non-blocking threads were manually resolved without explanatory replies; the current code nevertheless addresses their underlying concerns.

Important Files Changed

Filename Overview
packages/plugin/src/tools/ctx-search/constants.ts Adds qualified hybrid-retrieval query guidance and removes the unreliable fallback-detection advice.
packages/plugin/src/shared/prompt-surface-a1-golden.md Synchronizes the golden prompt surface with the updated ctx_search description.

Reviews (3): Last reviewed commit: "docs(ctx_search): drop unreliable match-..." | Re-trigger Greptile

export const CTX_SEARCH_TOOL_NAME = "ctx_search";
export const CTX_SEARCH_DESCRIPTION = `Your long-term recall for this project — search everything that ever happened here, not just what's currently visible.

Retrieval is HYBRID: ~0.7 semantic (embedding cosine) + ~0.3 keyword (FTS5 BM25), fused with per-source boosts. Phrase \`query\` as a natural-language QUESTION that still embeds the key literal terms (paths, symbols, commands, config keys) — pure keyword stacks starve the semantic leg (0.7 weight), while pure prose without literal terms may miss exact FTS matches. Good: "Where is the dream log file and which error counters indicate failures?" Bad: "dream log error counter".

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Prompt golden is stale

Changing the full ctx_search description without regenerating the A1 golden leaves the expected description stale. The full-preset registry test compares this value byte-for-byte with the golden, which still lacks this paragraph, so the test suite will fail until the golden is updated.

Comment thread packages/plugin/src/tools/ctx-search/constants.ts Outdated

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No issues found across 1 file

Re-trigger cubic

@G0-0000

G0-0000 commented Sep 7, 2026

Copy link
Copy Markdown
Author

Thanks Greptile — both review notes are addressed in 0edb532.

  1. Golden sync: packages/plugin/src/shared/prompt-surface-a1-golden.md is now byte-for-byte aligned with the updated CTX_SEARCH_DESCRIPTION. This was done by hand (the generator packages/plugin/scripts/export-agent-surface.ts needs ai-tokenizer and other workspace deps that are not installed in this checkout; I did not run a large install). Please let CI confirm tool-registry.test.ts / the A1 golden match.

  2. Hybrid claim: the description now qualifies the mix as applying when embeddings are enabled, and documents the silent keyword-only fallback (no match=semantic labels → prefer literal-heavy queries).

Thanks again for catching both.

Comment thread packages/plugin/src/tools/ctx-search/constants.ts Outdated

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

2 issues found across 2 files (changes from recent commits).

Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="packages/plugin/src/shared/prompt-surface-a1-golden.md">

<violation number="1" location="packages/plugin/src/shared/prompt-surface-a1-golden.md:403">
P3: The golden's section header still reads `### ctx_search — description ~309 tokens`, but this delta adds a substantial paragraph to the description without updating the token estimate. The current CTX_SEARCH_DESCRIPTION is ~296 words / ~1900 chars (roughly 380+ Claude BPE tokens), so ~309 understates it by a meaningful margin. Regenerate the golden with `export-agent-surface.ts` (or update the header) so the estimate matches the synced description.</violation>
</file>

<file name="packages/plugin/src/tools/ctx-search/constants.ts">

<violation number="1" location="packages/plugin/src/tools/ctx-search/constants.ts:4">
P2: Do not infer embedding availability from the absence of `match=semantic` labels. This can misclassify embedding-enabled searches when results are unlabeled, labeled `hybrid`, or omit semantic results; remove this fallback signal or expose an explicit embedding-status indicator.</violation>
</file>

Reply with feedback, questions, or to request a fix.

Re-trigger cubic

Comment thread packages/plugin/src/tools/ctx-search/constants.ts Outdated
```
Your long-term recall for this project — search everything that ever happened here, not just what's currently visible.

Retrieval is HYBRID when embeddings are enabled: ~0.7 semantic (embedding cosine) + ~0.3 keyword (FTS5 BM25), fused with per-source boosts; with embeddings disabled it silently falls back to keyword-only matching — tell by results carrying no `match=semantic` labels, and then prefer literal-heavy queries. Phrase `query` as a natural-language QUESTION that still embeds the key literal terms (paths, symbols, commands, config keys) — pure keyword stacks starve the semantic leg (0.7 weight), while pure prose without literal terms may miss exact FTS matches. Good: "Where is the dream log file and which error counters indicate failures?" Bad: "dream log error counter".

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P3: The golden's section header still reads ### ctx_search — description ~309 tokens, but this delta adds a substantial paragraph to the description without updating the token estimate. The current CTX_SEARCH_DESCRIPTION is ~296 words / ~1900 chars (roughly 380+ Claude BPE tokens), so ~309 understates it by a meaningful margin. Regenerate the golden with export-agent-surface.ts (or update the header) so the estimate matches the synced description.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At packages/plugin/src/shared/prompt-surface-a1-golden.md, line 403:

<comment>The golden's section header still reads `### ctx_search — description ~309 tokens`, but this delta adds a substantial paragraph to the description without updating the token estimate. The current CTX_SEARCH_DESCRIPTION is ~296 words / ~1900 chars (roughly 380+ Claude BPE tokens), so ~309 understates it by a meaningful margin. Regenerate the golden with `export-agent-surface.ts` (or update the header) so the estimate matches the synced description.</comment>

<file context>
@@ -400,6 +400,8 @@ Example: ctx_memory(action="write", category="CONSTRAINTS", content="Pi stores s

Your long-term recall for this project — search everything that ever happened here, not just what's currently visible.

+Retrieval is HYBRID when embeddings are enabled: ~0.7 semantic (embedding cosine) + ~0.3 keyword (FTS5 BM25), fused with per-source boosts; with embeddings disabled it silently falls back to keyword-only matching — tell by results carrying no match=semantic labels, and then prefer literal-heavy queries. Phrase query as a natural-language QUESTION that still embeds the key literal terms (paths, symbols, commands, config keys) — pure keyword stacks starve the semantic leg (0.7 weight), while pure prose without literal terms may miss exact FTS matches. Good: "Where is the dream log file and which error counters indicate failures?" Bad: "dream log error counter".
+
Reach for it when something feels familiar but isn't in view: "did we solve this before?", "what did we decide about X?", "when did this break?", "where does Y live?". Results only contain things you CANNOT currently see — memories already shown in and the live conversation tail are filtered out. A query that is just one or more memory ids (e.g. #7234 or 12, 34) bypasses text search and resolves those ids directly.
</file context>


</details>

@G0-0000

G0-0000 commented Sep 7, 2026

Copy link
Copy Markdown
Author

Confirmed: match is a three-state label (hybrid / semantic / fts), notes omit it, and #id shortcuts always take the FTS path — checked against the bundled searcher.

Dropped the self-check sentence that told models to infer embedding-off from missing match=semantic labels (it was unreliable given those three states and the note/#id exceptions).

A reliable embedding-status signal would be a code-level feature, outside this docs PR. Happy to discuss that separately if useful.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant