Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
39 changes: 39 additions & 0 deletions .github/pull_request_template.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
# Pull request

## What changed

<!-- One or two sentences. For a documentation change: state which code fact it reflects. -->

## Documentation = code (mandatory when any .md changes)

Every changed documentation line must point at the code that makes it true. A
change that only "fixes a number" without a source reference is rejected before
commit.

| Changed doc line (file) | Code fact it describes (file:symbol) | How it was verified |
|---|---|---|
| `docs/en/...` | `install.py::_print_config` | ran `python install.py --print-config --client generic` |

- [ ] No sentence in this PR describes behaviour the code does not have.
- [ ] `python scripts/check_docs_vs_installer.py` is green (flags and client
names in docs exist in `install.py` / `adapters/clients/snippets.py`).
- [ ] `install / uninstall / clean / doctor` are described and match the behaviour.
- [ ] The "What works today" table keeps its **Entry point** column.

## Gates

- [ ] `python -m pytest tests/ -q`
- [ ] `ruff check src/ tests/` (or `scripts/ruff_gate.py`)
- [ ] New behaviour has a test that fails when the behaviour is removed
(paste the red run below).
- [ ] Numbers in this PR are measured, with unit and denominator.

## Evidence

```text
<paste command output, not a summary>
```

## Risks / not verified

<!-- What is untested and why. "None" is a valid answer only when it is true. -->
12 changes: 11 additions & 1 deletion AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -339,7 +339,17 @@ impact-guard).

- **Канон:** `README.md` (EN) + `docs/ru/README.md` + `docs/zh/README.md`;
`docs/en/ARCHITECTURE.md`, `docs/en/SEARCH_PIPELINE.md`, `docs/en/INSTALL.md`,
`docs/en/CONFIGURATION.md`, `docs/FACTS.md`.
`docs/en/CLIENTS.md`, `docs/en/DATA_MAP.md`, `docs/en/CONFIGURATION.md`,
`docs/FACTS.md`.
- **Документация = код.** Каждое утверждение в `.md` выводимо из кода, и правка
документации в PR ссылается на факт кода (`файл:символ`), который она отражает.
Правка «только цифра» без ссылки отклоняется до коммита (чек-лист
`.github/pull_request_template.md`).
- **Guard дрейфа флагов/клиентов:** `scripts/check_docs_vs_installer.py` — каждый
`--flag` и `--client X` из `README.md`/`docs/*/{INSTALL,CLIENTS,DATA_MAP}.md`
обязан существовать в `install.py`/`adapters/clients/snippets.py` (`tests/
test_docs_vs_installer.py`). Колонка **Entry point** в «What works today»
обязательна.
- **Три языка обязательны** для канонических страниц: EN / RU / ZH. Перевод — полный
документ, не фрагмент; заголовки и числа совпадают с EN.
- **Числа** (инструменты, чанки, качество, тайминги) живут только в `docs/FACTS.md`
Expand Down
20 changes: 20 additions & 0 deletions AGENT_DIARY.md
Original file line number Diff line number Diff line change
Expand Up @@ -1765,3 +1765,23 @@ BREACH: MSC-P-19 | date: 2026-10-03 | rule: §12 | guard_at_time: no | detected_
**Автотест DoD:** `tests/test_install_cli.py::test_real_sync_install_then_doctor_then_clean_leaves_zero` — реальный `--sync-only` → `--doctor --json` (install_root.files > 100) → `--clean --yes` → `remaining_bytes == 0`, каталог удалён. Синтетический кейс отдельно проверяет dry-run (ничего не удалено), отказ без `--yes` (exit 2) и `doctor` после clean (`total_bytes == 0`).
**Найденный по ходу баг:** двойная обёртка stdout в UTF-8 (`_utf8_stdout()` в main + старый TUI-блок) закрывала буфер: `ValueError: I/O operation on closed file` на `--sync-only`. Убран дубль, оставлена одна обёртка на все режимы.
**Урок:** «очистка по слоям» без dry-run и guard'ов — это удаление чужого кэша по недопониманию; ноль остатка надо доказывать на файловой системе (`remaining_bytes == 0`), а не на числе вызванных `rmtree`.
## [2026-11-12] G7: документация = код — guard вместо обещаний
**Симптом:** `docs/{en,ru,zh}/INSTALL.md` семь месяцев документировали `python install.py --uninstall` — флага никогда не существовало; INSTALL_MODELS/SYSTEM_REQUIREMENTS/CONTRIBUTING описывали «спросит перед загрузкой моделей», хотя дефолт качал их без вопроса; README перечислял 3 слоя данных из 4.
**Fix:** правило в `AGENTS.md §8` (каждая изменённая строка docs → `файл:символ`) + чек-лист `.github/pull_request_template.md` + механический guard `scripts/check_docs_vs_installer.py`: каждый `install.py --flag` и `--client X` в `README.md`/`docs/*/{INSTALL,CLIENTS,DATA_MAP}.md`/`docs/en/FAQ.md` обязан существовать в `install.py`/`adapters/clients/snippets.py`; пустой корпус = exit 2 («не доказано»), а не «прошло».
**Проверка:** `158 flag reference(s) checked, 0 problem(s)`; тот же guard на дореформенном `docs/en/INSTALL.md` из `main` даёт `✗ unknown installer flag --uninstall` (воспроизведение исходного дефекта). Тесты: `tests/test_docs_vs_installer.py`, 5/5.
**Исправлено по коду:** INSTALL §7 (`--clean --yes`/uninstaller вместо `--uninstall`), INSTALL §2 и INSTALL_MODELS×3 (`--with-models` вместо prompt/`--skip-models`), SYSTEM_REQUIREMENTS×3, CONTRIBUTING×3, FAQ×3, README/README×2 (4 слоя + DATA_MAP), `adapters/clients/*.json` (`PROJECT_PATH`, `PYTHONUTF8`, `<install-root>`), `adapters/clients/README.md` (`cwd` → `PROJECT_PATH`), баннер в `AI_INSTALLATION_PROMPT.md`.
**Урок:** «документация соответствует коду» проверяется только исполнением: guard ловит исчезнувшие флаги, но не смысл. Смысловые утверждения (модели по умолчанию, Zed-обязательность) нашлись ручной сверкой с `step_models`/`step_zedcfg`.
## [2026-11-12] G5 addendum: полный suite поймал контракт адаптера
**Симптом:** полный прогон (`2488 passed, 1 failed`) упал на `tests/test_cli.py::test_stdio_ref_valid_entrypoint`: `['-u','-m','src.main'] != ['-m','src.main']`.
**Корень:** я привёл `adapters/clients/*.json` к форме сгенерированного сниппета и добавил `-u`. Но эти JSON — публичный контракт адаптеров, и тест фиксирует именно его; генератор сниппетов (`install.py --print-config`) — отдельная сущность.
**Fix:** в JSON возвращён `args: ["-m","src.main"]`, добавленные `PROJECT_PATH`/`PYTHONUTF8`/`<install-root>` оставлены. Тест не менялся (правило: не править проверку под код).
**Урок:** «shape» клиентского конфига — такая же публичная поверхность, как имя инструмента; `-u` в документации и в генераторе не даёт права менять статический адаптер.

## [2026-11-12] G5: холодный старт по README нашёл баг `--yes`
**Симптом:** `install.py --yes --no-zed` печатал итоговую рамку «Next steps», но на диске не появлялось ничего: `venv python exists: False`, install root пуст. Установка «успешно» завершалась за 0 секунд.
**Корень:** `for attempt in range(0 if auto_yes else max_retries + 1)` — при `--yes` `range(0)` не делает ни одной итерации, а ветка `for/else` помечает каждый шаг как failed, ни разу его не вызвав. Логика живёт с v3.0: все неинтерактивные установки (`--yes`) были пустыми.
**Fix:** `range(1 if auto_yes else max_retries + 1)` + регрессионный тест `test_auto_yes_runs_each_step_once` (до фикса красный: фейковый шаг не вызывался).
**Холодный старт (чистые временные `MSCODEBASE_INSTALL_ROOT`/`DATA_DIR`, `--no-zed`, без `--with-models`):** `install rc=0`, **12 ok / 0 skipped / 0 failed за 96 s**; venv Python 3.14.3; `src/main.py` и маркер на месте; **модельных каталогов нет** (`model dirs present: False`, install root 1.4 GB).
**Два клиента не-Zed/не-OpenCode, ровно по `docs/en/CLIENTS.md`:** `.mcp.json` (Claude Code §3.4) и `.cursor/mcp.json` (Cursor §3.5) записаны из `--print-config`, запущены как есть → `MSCodebase Intelligence Server`, `tools=17`, `investigate` в списке (4.8 s и 4.2 s); `--doctor` на этом же каталоге: `install_root 1.4 GB / 28755 файлов`, `index 0 chunk(s)`.
**Побочный вывод:** на ветке G5 (до G6) установщик ещё качает модели — README/CLIENTS G5-ветки обещали text-first и были приведены к фактическому поведению (`6010443`); text-first проверен на G6/G7.
**Урок:** `--yes` значит «не спрашивать», а не «не делать»; нулевой `range()` плюс `for/else` выглядит безобидно на ревью и полностью ломает установку. Ни один тест не запускал установщик целиком — поймал только прогон по README в пустом каталоге.
8 changes: 8 additions & 0 deletions AI_INSTALLATION_PROMPT.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,13 @@
# 🤖 One-Prompt Install Manifest for Zed Agent

> **Статус: Zed-специфичный сценарий (2026-08-12).** Актуальный, нейтральный к
> хосту порядок описан в `docs/en/INSTALL.md` (v3.2). Отличия, которые этот текст
> не отражает: установка больше не привязана к Zed (`MSCODEBASE_INSTALL_ROOT`,
> `--no-zed`, `install.py --print-config --client …`), модели стали opt-in
> (`--with-models`, `install.py::models_enabled`), а вместо ручного
> `printf 's\nn\n' | python install.py` используются `--yes` и `--clean/--doctor`.
> Проверить, что здесь устарело: `python scripts/check_docs_vs_installer.py`.

> Версия: 3.5.0 | Дата: 2026-08-12
>
> Скопируй этот текст в чат Агента Zed (`Ctrl+Shift+P` → `Agent Panel: Toggle`)
Expand Down
16 changes: 16 additions & 0 deletions docs/en/ARCHITECTURE.md
Original file line number Diff line number Diff line change
Expand Up @@ -151,3 +151,19 @@ replace/insert) with preview and impact guards.
- **Vector store:** LanceDB, only when dense retrieval is enabled.

None of them is required for the default text-first path.

## 9. Code references (audited 2026-11-12)

Every claim above resolves to a symbol, not to prose. Audit table:

| Claim in this page | Code |
|---|---|
| entry point builds the server, DI first | `src/main.py::main`, `src/mcp/server_factory.py::create_mcp_server` (line 156), `src/core/di_container.py` |
| artifact layout (`<data_root>/projects/<hash8>/…`) | `src/core/artifact_paths.py::get_data_root`, `::project_hash`, `::get_graph_db_path`, `::get_logs_dir` |
| text chunks in `<project>/.mscodebase/chunks.db` | `src/config/settings.py::SearchConfig.sqlite_chunks` (`MSCODEBASE_SQLITE_CHUNKS`) |
| `experiments/**` excluded from indexing | `src/core/indexing/file_guard.py` |
| retrieval owner | `src/core/search/engine.py::Searcher` |
| parsing / graph | `src/core/indexing/parser.py`, `src/core/graph.py` |
| profile `core` = 17 of 69 registered | `src/mcp/server_tools.py::CORE_PROFILE`; measured with `tools/list`: 17 (`core`) / 69 (`full`), 2026-11-12 |
| optional LSP layer | `src/mcp/tools/` `lsp_*` tools, failure cache 30 s (`src/config/settings.py::LSPConfig`) |

16 changes: 15 additions & 1 deletion docs/en/CONFIGURATION.md
Original file line number Diff line number Diff line change
Expand Up @@ -76,7 +76,7 @@ frozen panels must never appear as search hits.
| `MSCODEBASE_LINK_TIMEOUT_SEC` | — | Graph link-pass budget |
| `MSCODEBASE_REINDEX_STALL_SEC` | — | Stall watchdog for long reindexes |
| `MSCODEBASE_IVF_MAX_ABANDONS` | — | Auto-disable IVF after N failed optimize/create |
| `MSCODEBASE_MAX_FILE_SIZE_MB` (security) | `10` | Files above this are skipped |
| `MAX_FILE_SIZE_MB` (security, no prefix — as read by `SecurityConfig`, `src/config/settings.py:425`) | `10` | Files above this are skipped |

## 5. Providers (all optional)

Expand Down Expand Up @@ -128,3 +128,17 @@ python -c "from src.config.settings import get_config; c=get_config(); print(c.s
Config is built once per process: restart the MCP host after editing `.env`, then
`get_health_report()` should show the new values (or at least not the old provider
state).

## 9. Code references (audited 2026-11-12)

| Group | Where it is read |
|---|---|
| server/profile/storage (`MSCODEBASE_MCP_PROFILE`, `MSCODEBASE_DATA_DIR`, …) | `src/config/settings.py`, `src/mcp/server_tools.py::CORE_PROFILE`, `src/core/artifact_paths.py::get_data_root` |
| retrieval defaults (`MSCODEBASE_DOCS_MARGIN`, `_ONDEMAND_RERANK_*`, `_CODE_BM25_*`) | `src/config/settings.py::SearchConfig` (lines 189–268) |
| `MAX_FILE_SIZE_MB` (no prefix) | `src/config/settings.py::SecurityConfig` (line 425) |
| plugin gates (`MSCODEBASE_STALE_GATE`, `_ISOLATION_GATE`, `_GATE_LOG`, `MSCODEBASE_PY`) | `.opencode/plugin/mscodebase_gate.ts` (lines 25–27) |
| pre-commit budget (`MSCB_PRECOMMIT_FAST`, `MSCB_PRECOMMIT_GATE_TIMEOUT`) | `.githooks/pre-commit` (lines 33, 153) |

`python scripts/check_docs_vs_config.py` fails when a name in this table stops
existing in the code.

6 changes: 3 additions & 3 deletions docs/en/CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -35,9 +35,9 @@ python -m venv venv; venv\Scripts\activate; pip install -e ".[dev]"
```

`pip install -e .` is enough for the text-first runtime. Models are optional and
never download silently: `python install.py` asks before each download and
accepts `--skip-models`; `--sync-only` copies code into the extension directory
without touching pip or models.
never download silently: `install.py` is text-first by default and only downloads
with `--with-models` (`models_enabled`); `--sync-only` copies code into the install
root without touching pip or models.

## 3. Tests and lint

Expand Down
3 changes: 2 additions & 1 deletion docs/en/FAQ.md
Original file line number Diff line number Diff line change
Expand Up @@ -118,7 +118,8 @@ instead.

Yes. Nothing in the default path needs a GPU, a cloud API, or internet: FTS5,
the AST graph, and tree-sitter parsing are all local. The only network use is
downloading optional models (GGUF/ONNX via `install.py`) and the optional
downloading optional models (`install.py --with-models`, off by default:
`models_enabled()` in `install.py`) and the optional
`tree-sitter-language-pack`, whose parsers are fetched on first use
(`MSCODEBASE_LANGUAGE_PACK=true`). With that flag off, every grammar comes from
a pinned wheel and no download happens.
Expand Down
26 changes: 15 additions & 11 deletions docs/en/INSTALL_MODELS.md
Original file line number Diff line number Diff line change
Expand Up @@ -30,15 +30,17 @@ time; the download path does not compare them, so a hash here is a reference
value, not a verified-pin. (The llama.cpp **binary** archive *is* hash-checked
before extraction.)

## 2. Method 1 — `install.py` (interactive)
## 2. Method 1 — `install.py` (models are opt-in)

```bash
python install.py
python install.py --with-models
```

12 steps. Network steps ask before they act:
Without `--with-models` the installer is text-first: `step_models` and `step_gguf`
return immediately (`install.py::models_enabled`), so no model is downloaded, no
model directory is created and no prompt is shown. With `--with-models`:

- ONNX models: *"Download ONNX models (e5-small-int8 ~113 MB + reranker ~544 MB)? (Y/n)"*;
- ONNX: `multilingual-e5-small-int8` (~113 MB) + `bge-reranker-v2-m3` (~544 MB);
- GGUF + llama.cpp: the GGUF step downloads `multilingual-e5-small` and
`bge-reranker-v2-m3` and the llama.cpp binary for the selected backend;
- every step offers `[r]etry / [s]kip / [c]ancel` on failure.
Expand All @@ -47,13 +49,15 @@ Useful flags:

| Flag | Effect |
|---|---|
| `--sync-only` / `--sync` | copy the code into the extension directory, touch nothing else (use this after editing `src/`) |
| `--skip-models` / `--no-models` | skip the model steps entirely |
| `--yes` / `-y` | answer yes instead of prompting |
| `--sync-only` / `--sync` | copy the code into the install root, touch nothing else (use this after editing `src/`) |
| `--with-models` | run the model steps (ONNX + GGUF); this is the only way to get them |
| `--skip-models` / `--no-models` | accepted for old scripts; models are already skipped by default |
| `--no-zed` | do not touch Zed settings (host-neutral install) |
| `--yes` / `-y` | accept every step instead of prompting |
| `--verbose` / `-v` | full step output instead of the compact view |

Installation targets the extension directory
`%LOCALAPPDATA%\Zed\extensions\mscodebase-intelligence`:
Files land in the install root (`MSCODEBASE_INSTALL_ROOT`, default the historical
extension directory `%LOCALAPPDATA%\Zed\extensions\mscodebase-intelligence`):

- ONNX → `<ext>/.codebase_models/onnx/<slug>/model_quantized.onnx`
(its specialised reranker subfolder uses `model.onnx`);
Expand Down Expand Up @@ -142,8 +146,8 @@ git-ignored (`.codebase_models/`, `models/`), so nothing appears in `git status`

---

**Verified against:** `install.py` (`STEPS`, `step_models`, `step_gguf`,
`--sync-only`, `--skip-models`, `--yes`, prompt strings),
**Verified against:** `install.py` (`STEPS`, `models_enabled`, `step_models`,
`step_gguf`, `--sync-only`, `--with-models`, `--skip-models`, `--yes`, `--no-zed`),
`src/providers/reranker/llama_install.py` (`GGUF_MODELS` sizes/dims/SHA-256,
`LLAMA_VERSION`, `LLAMA_PORT`, `DEFAULT_EMBEDDING_MODEL`, `resolve_ubatch`,
`_patch_dll_imports`), `src/providers/reranker/llama_runner.py`
Expand Down
12 changes: 12 additions & 0 deletions docs/en/SEARCH_PIPELINE.md
Original file line number Diff line number Diff line change
Expand Up @@ -181,3 +181,15 @@ frozen before the run: `experiments/xerj_2026_11/evidence_gate/REPORT.md`.
`grep_code(..., scope="all")` → did retrieval miss, or did ranking reorder?
4. `investigate(...)` → which channels found the file (`[search]`, `[grep]`)?
5. Check the log tail: `%LOCALAPPDATA%/mscodebase/logs/mcp_global.log`.

## 9. Code references (audited 2026-11-12)

| Claim in this page | Code |
|---|---|
| long query refused above 512 chars | `src/mcp/tools/search_tools.py:288` |
| channel weights in `investigate` (search 0.6, grep 0.3, tests 0.1, memory 0.4) | `src/mcp/tools/investigate_tool.py:44`, `:1025`, `:1041` |
| output cap: 8 rows, 180 chars per row | `src/mcp/tools/investigate_tool.py:681` (`limit: int = 8`), `:1191` (`[:180]`) |
| docs routing margin, default 4.0 | `src/config/settings.py::SearchConfig.docs_margin` (line 268) |
| BM25 constants (`K1` 1.2, `B` 0.45) | `src/config/settings.py` lines 219, 222 |
| request guard, FTS5 candidate generation | `src/core/search/engine.py` |

6 changes: 3 additions & 3 deletions docs/en/SYSTEM_REQUIREMENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -56,9 +56,9 @@ After editing `src/`, run `python install.py --sync-only` and restart the host.
Rule of thumb: the text index is proportional to the indexed corpus and roughly
an order of magnitude smaller than a vector index over the same corpus. The
optional model directories dominate when they exist and are absent otherwise.
`install.py` downloads models only when asked (`--skip-models`, or answering "no"
to its prompt), so a text-only installation is the size of the code plus the
chunk store.
`install.py` downloads models only with `--with-models` (`models_enabled()` in
`install.py`); the default is text-first with no prompt and no download, so a
text-only installation is the size of the code plus the chunk store.

`artifact_paths.check_disk_space()` warns when the data root has less than
**500 MB** free (`MIN_FREE_DISK_MB`); the CI guard fails below 1 GB.
Expand Down
Loading
Loading