Skip to content
This repository was archived by the owner on Nov 10, 2026. It is now read-only.
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
28 changes: 13 additions & 15 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -104,9 +104,10 @@ counts those reports as unevidenced.

## Session-end distillation

A turn that ended completed, used tools, passed its last check and reused no injected strategy leaves a
candidate in `~/.evolver/state/distill-pending.json`; a later verified turn replaces it, and a next prompt
that reads as a correction drops it. When the session ends, one call to the session's own model writes a
A turn that ended completed, used tools and passed its last check leaves a candidate in
`~/.evolver/state/distill-pending.json`, a turn that reused an injected strategy too (the engine's intake
drops a Gene that repeats one it already has); a later verified turn replaces it, and a next prompt that
reads as a correction drops it. When the session ends, one call to the session's own model writes a
Gene from a digest of that turn (task, commands, touched paths, checks, final answer — never the whole
history) and sends it to the Proxy's `POST /conversation/distill` with `persist: true`, `publish: false`
and its own token cost as `distill_usage`. Keys and tokens are stripped from the candidate first, paths and
Expand All @@ -123,17 +124,14 @@ weighs each report by its evidence:

| Report | When | `task_id` | `evidence` |
|---|---|---|---|
| Turn end | the turn ends | `dsh:<session>:<turn>` | `tier` `L1` when the turn ran a test, build, lint or type check, else `none`; `applied`; `check` `{kind, passed, fail_then_pass}` of the last check; `aborted`; `stage: immediate` |
| Correction | the next prompt says the answer did not hold, or approves it | `…:correction` | `tier: L3`, `applied`, `stage: correction` |
| Retention | the next session in the same project finds the turn's edits kept or reverted | `…:retention` | `tier: L2`, `applied`, `stage: retention` |
| Aborted | the session goes away, or is killed, before the turn ends (a killed session's go out at the next session start, once its dsh process is gone) | `dsh:<session>:<turn>` | `tier: none`, `applied: partial`, `aborted: true`, `stage: immediate` |

- **`applied`** compares the strategy's numbered steps with the turn's shell commands,
touched paths, the code its edits wrote (200,000 characters a turn, kept in memory only)
and the final answer: `used` when half the steps show up, `partial` when fewer do,
`unused` when none do. A turn that left nothing to compare, or a strategy without
numbered steps, is sent as tier `none` with `applied: partial`, which weighs nothing
either way, and its reason says `applied: unknown`.
| Turn end | the turn ends | `dsh:<session>:<turn>` | `tier` `L1` when the turn ran a test, build, lint or type check, else `none`; `applied: used`; `check` `{kind, passed, fail_then_pass}` of the last check; `aborted`; `stage: immediate` |
| Correction | the next prompt says the answer did not hold, or approves it | `…:correction` | `tier: L3`, `applied: used`, `stage: correction` |
| Retention | the next session in the same project finds the turn's edits kept or reverted | `…:retention` | `tier: L2`, `applied: used`, `stage: retention` |
| Aborted | the session goes away, or is killed, before the turn ends (a killed session's go out at the next session start, once its dsh process is gone) | `dsh:<session>:<turn>` | `tier: none`, `applied: used`, `aborted: true`, `stage: immediate` |

- **`applied`** is always `used`: the model follows a strategy injected into its turn. Whether the
strategy helped is what the outcome says, judged by the turn's last check (or how the turn ended when it
ran none), the next prompt and whether its edits were kept.
- **Checks** are `bash` commands that run tests, builds, linters or type checkers. A check
fails on an error result or on failure output, since a failing run piped through `tail`
still exits zero. `fail_then_pass` compares commands with output pipes (`| tail -9`) and
Expand All @@ -148,7 +146,7 @@ weighs each report by its evidence:
- `/evolver-status` lists the Hub's verdict (`eligible`, `counted`, `weight`) on recent
reports and the reason for each rejected one. Reports with evidence are also written to
the local root event log once.
- Reasons name the turn, the applied verdict and the check kind, never the prompt, a
- Reasons name the turn and the check kind, never the prompt, a
command, code or a diff, and the prompt loses keys and tokens before recall sends it.

## Requirements
Expand Down
12 changes: 6 additions & 6 deletions README.zh.md
Original file line number Diff line number Diff line change
Expand Up @@ -94,17 +94,17 @@ dsh plugin --profile dsh-tui add -w @evomap/dsh-evolver@latest

| 上报 | 时机 | `task_id` | `evidence` |
|---|---|---|---|
| 回合结束 | 回合结束时 | `dsh:<会话>:<回合>` | 本回合跑过测试、构建、lint 或类型检查时 `tier` 为 `L1`,否则为 `none`;`applied`;最后一次检查的 `check` `{kind, passed, fail_then_pass}`;`aborted`;`stage: immediate` |
| 追问补报 | 下一条提示词表示结果不对,或表示认可 | `…:correction` | `tier: L3`、`applied`、`stage: correction` |
| 留存回访 | 同一项目的下一个会话发现本回合的改动被保留或被撤销 | `…:retention` | `tier: L2`、`applied`、`stage: retention` |
| 中止 | 回合结束前会话消失或被强杀(被强杀会话的上报在下次会话开始时发出) | `dsh:<session>:<turn>` | `tier: none`、`applied: partial`、`aborted: true`、`stage: immediate` |
| 回合结束 | 回合结束时 | `dsh:<会话>:<回合>` | 本回合跑过测试、构建、lint 或类型检查时 `tier` 为 `L1`,否则为 `none`;`applied: used`;最后一次检查的 `check` `{kind, passed, fail_then_pass}`;`aborted`;`stage: immediate` |
| 追问补报 | 下一条提示词表示结果不对,或表示认可 | `…:correction` | `tier: L3`、`applied: used`、`stage: correction` |
| 留存回访 | 同一项目的下一个会话发现本回合的改动被保留或被撤销 | `…:retention` | `tier: L2`、`applied: used`、`stage: retention` |
| 中止 | 回合结束前会话消失或被强杀(被强杀会话的上报在下次会话开始时发出) | `dsh:<session>:<turn>` | `tier: none`、`applied: used`、`aborted: true`、`stage: immediate` |

- **`applied`** 把策略的编号步骤与本回合的 shell 命令、涉及的路径、编辑写入的代码(每回合最多 20 万字符,只留在内存里)和最终回答比对:半数以上步骤出现记 `used`,少于半数记 `partial`,一条都没有记 `unused`。回合没留下可比对的内容,或策略没有编号步骤时,按 tier `none` 加 `applied: partial` 上报,两边都不计权重,reason 里写 `applied: unknown`。
- **`applied`** 固定为 `used`:注入到回合里的策略,模型就会照着用。策略有没有起作用看结果:本回合最后一次检查是否通过(没跑检查时看回合如何结束)、下一条提示词,以及改动是否被保留。
- **检查**指运行测试、构建、lint 或类型检查的 `bash` 命令。结果报错或输出里有失败信息都算失败,因为经 `tail` 管道的失败运行退出码仍是 0。判断 `fail_then_pass` 时会先去掉输出管道(`| tail -9`)和重定向(`2>&1`)再比较命令。
- **outcome** 跟随最后一次检查;没有检查时跟随回合的结束方式。
- **投递**:上报先存进 `~/.evolver/state/reports-outbox.json`,在每次回合结束、追问和会话开始时重试,保留 8 天。Hub 只要给出答复就算送达,包括 `duplicate_report` 和不计分的回执(`same_owner`、`daily_cap`)。Hub 判为无效的(`terminal: true`,或旧版 Proxy 返回的 `hub 400` / `hub 422`)记为被拒,不再重发。`hub 404`、`hub 5xx` 和网络失败会重试。
- `/evolver-status` 会列出最近上报的 Hub 判定(`eligible`、`counted`、`weight`)以及每条被拒上报的原因。带证据的上报还会在本地 root event 日志里记一次。
- reason 只写回合、采用判定和检查类型,不含提示词、命令、代码或 diff;召回前会先去掉提示词里的密钥和令牌。
- reason 只写回合和检查类型,不含提示词、命令、代码或 diff;召回前会先去掉提示词里的密钥和令牌。

## 环境要求

Expand Down
70 changes: 0 additions & 70 deletions src/applied.js

This file was deleted.

5 changes: 0 additions & 5 deletions src/edited-content.js
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,6 @@

export const EDIT_TOOL_NAMES = ['write', 'edit', 'str_replace_editor'];

const CONTENT_KEYS = ['content', 'new_string', 'new_str', 'file_text', 'text'];
const PATH_KEYS = ['path', 'file_path'];

function firstString(args, keys) {
Expand All @@ -13,10 +12,6 @@ function firstString(args, keys) {
return '';
}

export function editedContent(args) {
return firstString(args, CONTENT_KEYS);
}

export function editedPath(args) {
return firstString(args, PATH_KEYS);
}
9 changes: 3 additions & 6 deletions src/index.js
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,6 @@

import { boundContextSummary, createUserMessage } from '@deepseek-ai/dsh-llm';

import { appliedVerdictOf } from './applied.js';
import { evolverCommands } from './commands.js';
import { Config } from './config.js';
import { createRecallLog, createReportLog, pluginDiagnostics } from './diagnostics.js';
Expand Down Expand Up @@ -186,12 +185,11 @@ function reportsOf(assets, { sessionId, turn, reasonKind, record }) {
turn: turn ?? asset.turn,
reasonKind,
checks: record.checks,
applied: appliedVerdictOf({ strategyText: asset.strategyText, ...record }).verdict,
}));
}

// Every injected strategy is reported when its turn ends, with the turn's own evidence; the turn itself
// becomes this session's distillation candidate when it finished verified without reusing one.
// becomes this session's distillation candidate when it finished verified.
function reportTurns(ctx, records, deliver, labelable, projectDirOf, prompts) {
const closeTurn = (session, { turn, reason }) => {
const sessionId = sessionKeyOf(session);
Expand All @@ -203,8 +201,7 @@ function reportTurns(ctx, records, deliver, labelable, projectDirOf, prompts) {
labelable.set(sessionId, reuses);
deliver(reports.map((report) => ({ report, sessionId }))).catch(() => {});
rememberForRetention({ sessionId, projectDir: projectDirOf(session), reuses, editedBefore: finished.editedBefore });
const reusedAsset = reuses.some((reuse) => reuse.applied === 'used');
if (skipReason(reasonKind, finished, { reusedAsset })) return;
if (skipReason(reasonKind, finished)) return;
const model = modelOf(session?.deriveMessages?.() ?? []);
rememberCandidate(sessionId, candidateOf({ prompt: prompts.get(sessionId), record: finished, model, turn }));
};
Expand All @@ -218,7 +215,7 @@ function reportTurns(ctx, records, deliver, labelable, projectDirOf, prompts) {

function abortedReports(assets) {
return assets.map((asset) => ({
report: reportsOf([asset], { sessionId: asset.sessionId, reasonKind: 'aborted', record: { checks: [], commands: [], paths: [], edits: [], answer: '' } })[0],
report: reportsOf([asset], { sessionId: asset.sessionId, reasonKind: 'aborted', record: { checks: [] } })[0],
sessionId: asset.sessionId,
}));
}
Expand Down
20 changes: 9 additions & 11 deletions src/reuse.js
Original file line number Diff line number Diff line change
@@ -1,7 +1,6 @@
// SPDX-License-Identifier: MIT
// Copyright (c) 2026 EvoMap

import { UNKNOWN } from './applied.js';
import { checkEvidenceOf } from './checks.js';
import { REJECTED, RETRY, SENT, isSettled, withAttempt } from './report-outbox.js';
import { outcomeOfReason } from './turn-outcomes.js';
Expand All @@ -14,16 +13,15 @@ function turnTaskId(sessionId, turn) {
return `dsh:${sessionId || 'unknown'}:${turn ?? 'unknown'}`;
}

// The Hub has no `unknown`, and `unused` carries a soft negative there; a
// partial reuse at tier none weighs nothing either way.
const UNKNOWN_APPLIED_AS = 'partial';
// The model follows a strategy injected into its turn, so an injection is a use; whether it helped is
// what the outcome, the check, the next prompt and the edits' retention tell.
const APPLIED = 'used';

function immediateEvidenceOf({ checks, applied, reasonKind }) {
function immediateEvidenceOf({ checks, reasonKind }) {
const check = checkEvidenceOf(checks);
const isKnown = applied !== UNKNOWN;
return {
tier: check && isKnown ? 'L1' : 'none',
applied: isKnown ? applied : UNKNOWN_APPLIED_AS,
tier: check ? 'L1' : 'none',
applied: APPLIED,
...(check && { check }),
aborted: reasonKind === 'aborted',
stage: 'immediate',
Expand All @@ -45,13 +43,13 @@ function reportOf({ assetId, taskId, outcome, reason, evidence }) {
return { asset_id: assetId, outcome, task_id: taskId, reason, evidence };
}

export function immediateReportOf({ asset, sessionId, turn, reasonKind, checks, applied }) {
const evidence = immediateEvidenceOf({ checks, applied, reasonKind });
export function immediateReportOf({ asset, sessionId, turn, reasonKind, checks }) {
const evidence = immediateEvidenceOf({ checks, reasonKind });
return reportOf({
assetId: asset.assetId,
taskId: turnTaskId(sessionId, turn),
outcome: immediateOutcomeOf(evidence, reasonKind),
reason: `Injected by Evolver into dsh turn ${turn}; applied: ${applied}; ${checkPhrase(evidence.check)}.`,
reason: `Injected by Evolver into dsh turn ${turn}; ${checkPhrase(evidence.check)}.`,
evidence,
});
}
Expand Down
9 changes: 4 additions & 5 deletions src/session-distill.js
Original file line number Diff line number Diff line change
Expand Up @@ -26,16 +26,15 @@ const MAX_ATTEMPTS = 3;
const pendingPath = () => statePath('distill-pending.json');

/**
* Why a finished turn leaves no candidate, or null when it does. A turn that reused an injected strategy
* adds evidence to that asset through its reuse report rather than a near-copy of it; a chat reply without a
* single command has no method to keep.
* Why a finished turn leaves no candidate, or null when it does. A chat reply without a single command has no
* method to keep. A turn that reused an injected strategy is distilled too: the engine's intake drops a Gene
* that repeats one it already has.
*/
export function skipReason(reasonKind, record, { reusedAsset = false } = {}) {
export function skipReason(reasonKind, record) {
if (reasonKind !== 'completed') return 'not_completed';
if (!record.answer) return 'no_answer';
if (record.checks.at(-1)?.failed) return 'failed_check';
if (record.commands.length === 0 && record.edits.length === 0) return 'no_tool_use';
if (reusedAsset) return 'asset_reused';
return null;
}

Expand Down
15 changes: 6 additions & 9 deletions src/turn-record.js
Original file line number Diff line number Diff line change
Expand Up @@ -3,16 +3,15 @@

import path from 'node:path';

import { editedContent, editedPath } from './edited-content.js';
import { editedPath } from './edited-content.js';
import { contentHashOf } from './retention-pending.js';
import { checkCommandOf, verificationFailed } from './verification.js';

const TURN_EDITED_MAX_CHARS = 200_000;
const CHECK_COMMAND_MAX_CHARS = 200;
const PATH_KEYS = ['path', 'file_path'];

function freshTurnRecord() {
return { checks: [], commands: [], paths: [], edits: [], editedChars: 0, editedBefore: new Map(), answer: '' };
return { checks: [], commands: [], paths: [], edits: [], editedBefore: new Map(), answer: '' };
}

export function createTurnRecords() {
Expand Down Expand Up @@ -44,11 +43,9 @@ export function noteFileBeforeEdit(record, projectDir, args) {
if (!record.editedBefore.has(file)) record.editedBefore.set(file, contentHashOf(file));
}

function noteEditedContent(record, args) {
const kept = editedContent(args).slice(0, TURN_EDITED_MAX_CHARS - record.editedChars);
if (!kept) return;
record.edits.push(kept);
record.editedChars += kept.length;
function noteEditedFile(record, args) {
const edited = editedPath(args);
if (edited) record.edits.push(edited);
}

export function noteToolResult(record, exec, result, { isEdit }) {
Expand All @@ -57,7 +54,7 @@ export function noteToolResult(record, exec, result, { isEdit }) {
for (const key of PATH_KEYS) if (typeof args[key] === 'string') record.paths.push(args[key]);
const check = checkCommandOf(exec);
if (check) record.checks.push({ command: check.slice(0, CHECK_COMMAND_MAX_CHARS), failed: verificationFailed(result) });
if (isEdit && !result?.isError) noteEditedContent(record, args);
if (isEdit && !result?.isError) noteEditedFile(record, args);
}

export function noteAnswer(record, message) {
Expand Down
Loading
Loading