From 202e8fc0bdebd00333bd06fed69645787959d533 Mon Sep 17 00:00:00 2001 From: oblomov-dev Date: Fri, 11 Sep 2026 12:11:28 +0000 Subject: [PATCH 1/2] Search: stop words, phrase bonus, group order, compounds, plural, relaxed queries Five ranking defects measured on the real index, each pinned in test/search.test.mjs: - "how do i install" answered with Client API > check_on_init: `a`, `i`, `to`, `do` are a prefix of something in every entry and each earned a title-hit's worth of points. Stop words are no longer search terms while a real word is left. - "abap cloud" preferred abap-cleaner over the heading "ABAP Cloud": the words next to each other now earn a bonus. - "dialog" opened on four pages that mention the word above 47 samples that are dialogs: groups come in the order of their best hit, the documentation leading only when it has a real answer. - "messagebox" missed the heading "Message Box", "tables" found 33 of 202: a field is also matched with its spaces taken out, and a plural `s` is taken off before the prefix match. - "tabel" and "readonly" found nothing: a query that finds nothing sets the rarest word aside, then tries one edit away against the titles, and the box says what it answered instead; synonyms are data in the index builder (readonly -> editable, f4 -> value help, ...). Co-Authored-By: Claude Fable 5.1 Claude-Session: https://claude.ai/code/session_017JMbwNQAZ65yn4GrgRdvsi --- AGENTS.md | 43 +++- docs/.vitepress/theme/SearchBox.vue | 9 +- docs/.vitepress/theme/search-engine.js | 317 +++++++++++++++++++++---- scripts/lib/search-index.mjs | 46 +++- test/search.test.mjs | 130 +++++++++- 5 files changed, 485 insertions(+), 60 deletions(-) diff --git a/AGENTS.md b/AGENTS.md index d9478c044..4bd4336c7 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -277,8 +277,47 @@ what a reader wants is a sample, in another repository, on another deployment. (`/docs/search-index.json`, built by `generate-search.mjs`): every page of the manual with its headings and its words, and all ~770 entries of the three sample catalogues with the titles, summaries and keywords those repositories -maintain. Results are grouped by area, documentation first, and a sample hit -opens that sample's own page in the catalogue. +maintain. Results are grouped by area, and a sample hit opens that sample's +own page in the catalogue. The groups come in the order of their best hit; +the documentation leads whenever it has a real answer (a hit within half of +the best hit anywhere), and a page that only MENTIONS the word comes after +the samples that are about it — "dialog" used to open on Value Help, Popup, +PDF and Lock above forty-seven samples that are dialogs. + +**What the matcher does with the words, and why.** Each rule is a query that +was measured wrong on the real index before it, and each is pinned in +`test/search.test.mjs`: + +- **Stop words are not search terms** while a real word is left. "how do i + install" was answered with Client API › check_on_init, and Installation + was not in the first four: `a` and `i` are a prefix of something in every + entry and each earned a title-hit's worth of points. A query that is + nothing but stop words still asks what it says. +- **The words next to each other earn a bonus.** "abap cloud" preferred + Toolchain › abap-cleaner (a title hit on `abap`, a mention of `cloud`) + over the heading that IS the two words typed. +- **A field with its spaces taken out is matched too**, at the weight of the + field, for terms of five letters and up: "messagebox" found the three + samples spelled that way and not the chapter whose heading is "Message + Box", nor "selectdialog" a single page. The index carries no compound + forms for this; the matcher looks (`joined( )`). +- **The plural is the singular with an `s` on it**, taken off before the + prefix match — "tables" found 33 entries and "table" 202, and all 169 it + missed were about tables. Letters only, never `ss`, never a class name. +- **Nothing found is not the end**: the word on the fewest entries is set + aside, then the next, until something answers; a single word that still + finds nothing is tried against every title word one edit away ("tabel" is + "table"; and the candidate that starts the way the reader started wins + over the one on more titles, or "tabel" would be "label"). Whatever + answered is on the result as `hits.relaxedTo` and both boxes say so above + the rows: a list that silently answers a different question than the one + typed is worse than an empty one. +- **Synonyms are data in the index, not code in the matcher**: `SYNONYMS` in + `scripts/lib/search-index.mjs` is `[what is typed, what the entry says]` + pairs ("readonly" for `editable`, "f4" for the value help), added to an + entry's `terms` — the long tail's weight, so the alias finds the entry and + the ranking stays with the words it really carries. One direction each, + so it is the reader's word that is mapped, not the project's. **The box states the type it would otherwise inherit.** Every rule inside the panel is identical to `.search-panel`'s in `catalogue.css` — and the two still diff --git a/docs/.vitepress/theme/SearchBox.vue b/docs/.vitepress/theme/SearchBox.vue index 30bfb1cc8..f936bc015 100644 --- a/docs/.vitepress/theme/SearchBox.vue +++ b/docs/.vitepress/theme/SearchBox.vue @@ -74,6 +74,10 @@ function hide() { * them is not measurable. */ const hits = computed(() => (entries.value.length ? search(entries.value, query.value, { limit: 500 }) : [])); const groups = computed(() => grouped(hits.value)); +/* What answered, when it was not what was typed: a typo corrected or a word + * set aside (search-engine.js). The list says so above the results, and the + * marks in the rows are of the query that answered, not the one that did not. */ +const relaxedTo = computed(() => hits.value.relaxedTo || ''); /* What is in the box, in the two numbers a reader recognises. Only once the * index has arrived - before that the box says what it is, not how much. */ @@ -185,7 +189,7 @@ onMounted(() => document.addEventListener('keydown', onKey)); onUnmounted(() => document.removeEventListener('keydown', onKey)); const indexOf = (hit) => rows.value.indexOf(hit); -const parts = (text) => highlight(text, query.value); +const parts = (text) => highlight(text, relaxedTo.value || query.value);