Skip to content
This repository was archived by the owner on Nov 10, 2026. It is now read-only.

Count an injected strategy as used, and distill turns that reused one - #55

Merged
forrestlinfeng merged 1 commit into
mainfrom
feat/injected-means-used
Oct 26, 2026
Merged

Count an injected strategy as used, and distill turns that reused one#55
forrestlinfeng merged 1 commit into
mainfrom
feat/injected-means-used

Conversation

@forrestlinfeng

@forrestlinfeng forrestlinfeng commented Oct 26, 2026

Copy link
Copy Markdown
Contributor

Why

The applied verdict guessed whether a turn followed an injected strategy by matching its steps against the turn's commands, paths, edited code and answer. On a real Codex run with a Chinese prompt (我想在小红书里面种草…) and a 4-step Chinese strategy, the answer followed every step but the report said applied: unknown; with any command in the turn it would have said unused. Fixing it per language (#54) adds tuning for every script, and the verdict feeds nothing locally: the engine reads only outcome.

Decision: the model follows what is injected into its turn, so an injection is a use.

Change

  • Every reuse report says applied: used (turn end, correction, retention, aborted). Whether the strategy helped stays with what already judges it: the outcome from the last check (or how the turn ended), the next prompt (L3) and whether the edits were kept (L2). tier is L1 whenever a check ran.
  • A turn that reused an injected strategy is a distillation candidate like any other verified turn. With every injection a use, the old asset_reused skip would have stopped nearly every turn from being distilled. The engine's Gene intake drops a Gene that repeats an existing one (from the intakeGene / ExistingGeneRef types in @evomap/evolver-core 2.0.46; the JS is obfuscated, so not checked line by line).
  • Deleted src/applied.js and its tests, and the per-turn edited-code buffer (up to 200,000 characters) that only fed it; a turn now records which files it edited.
  • README / README.zh: reports table and the applied note; distillation section.

Tests

npm test: 171 pass. New: a verified turn that reused an injected strategy is distilled (fails on main). Report tests now expect applied: used and reasons without the verdict.

Supersedes #54. The same change goes to the Codex plugin (EvoMap/evolver-codex-plugin#10), then the other ports.

🤖 Generated with Claude Code

forrestlinfeng @claude
Count an injected strategy as used, and distill turns that reused one
6432abc 
The applied verdict matched a strategy's steps against the turn's commands,
paths, edited code and answer. It failed on text without spaces (a turn
that followed every Chinese step read as `unknown`, or `unused` once it ran
a command) and would need tuning per language. The model follows what is
injected into its turn, so an injection is now a use: every report says
`applied: used`, and whether the strategy helped is left to what already
judges it — the outcome from the last check or how the turn ended, the
next prompt, and whether the edits were kept.

A turn that reused an injected strategy no longer skips distillation; with
every injection a use, that rule would have stopped nearly every turn from
being distilled. The engine's Gene intake drops one that repeats a Gene it
already has.

applied.js and the per-turn edited-code buffer (up to 200,000 characters)
existed only for the verdict and are removed; a turn records which files it
edited instead.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@forrestlinfeng forrestlinfeng mentioned this pull request Oct 26, 2026
Closed
@cursor

cursor Bot commented Oct 26, 2026

Copy link
Copy Markdown

Bugbot is paused — on-demand spend limit reached

Bugbot uses usage-based billing for this team and has hit its on-demand spend limit.

A team admin can raise the spend limit in the Cursor dashboard, or wait for the next billing cycle to continue.

@evomap-aiqa-bot

evomap-aiqa-bot Bot commented Oct 26, 2026 •
edited
Loading

Copy link
Copy Markdown

Sentinel 已审核此变更，未发现需要报告的问题。

开发 agent 查询详情： 调用 MCP 工具 sentinel_pr_review_context，传入 owner_repo="evomap/evolver-dsh-plugin"、pr_number=55、head_sha="6432abc96789bb7e74fdf51bdea1212714cc63e3"，并按条目追加 finding_id 或 finding_key。

由 Sentinel 审核 · 目标提交 6432abc96789

forrestlinfeng pushed a commit to EvoMap/evolver-codex-plugin that referenced this pull request Oct 26, 2026
forrestlinfeng @claude
feat: count an injected strategy as used, and distill turns that reus…
b70ad48 
…ed one

Port of EvoMap/evolver-dsh-plugin#55. The applied verdict matched a
strategy's steps against the turn's commands, paths, patched code and
answer; on a real Codex run with a Chinese strategy the answer followed
every step and the report still said `applied: unknown`. The model
follows what is injected into its turn, so every report now says
`applied: used`, and whether the strategy helped is left to the outcome
(last check, or how the turn ended), the next prompt and kept edits.

A turn that reused an injected strategy is a distillation candidate like
any verified turn; the engine's Gene intake drops a repeat of a Gene it
has. applied.mjs and the patched-code buffer (200,000 characters a turn)
existed only for the verdict and are removed; a turn records the files
its applied patches edited.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@forrestlinfeng
forrestlinfeng merged commit f7b4cde into main Oct 26, 2026
9 checks passed
forrestlinfeng pushed a commit to EvoMap/evolver-cursor-plugin that referenced this pull request Oct 26, 2026
forrestlinfeng @claude
feat: count an injected strategy as used, and distill turns that reus…
b783935 
…ed one

Port of EvoMap/evolver-dsh-plugin#55. Every reuse report (turn end,
correction, retention, aborted) now says `applied: used`: the model follows
what is injected into its turn, and whether the strategy helped is left to
the outcome (last check, or how the turn ended), the next prompt and kept
edits. `tier` is L1 whenever a check ran, and the reason no longer names a
verdict.

A turn that reused an injected strategy is a distillation candidate like any
verified turn; the engine's Gene intake drops a repeat of a Gene it has.
applied.mjs and the edited-text capture existed only for the verdict and are
removed; a turn records the files it edited (afterFileEdit writes, Delete).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
forrestlinfeng pushed a commit to EvoMap/evolver-workbuddy-plugin that referenced this pull request Oct 26, 2026
forrestlinfeng @claude
feat(reuse): count an injected strategy as used, and distill turns th…
1c9596d 
…at reused one (dsh #55)

Mirrors EvoMap/evolver-dsh-plugin#55:
- Every reuse report (turn end, correction L3, retention L2, aborted) sends evidence.applied "used".
  The applied verdict (lib/applied.mjs) is gone; the immediate report is tier L1 whenever a check ran,
  else none, and its reason is "Injected by Evolver into WorkBuddy turn N; <check phrase>." Whether the
  strategy helped is judged by the outcome, the next prompt and the kept edits.
- A verified turn that reused an injected strategy is a distillation candidate like any other: the
  reusedAsset option and the asset_reused skip reason are removed, since the engine's intake drops a
  Gene that repeats one it has.
- The per-turn buffer of written code (200,000 characters) existed only for the verdict; record.edits
  now holds the paths of the files the turn edited. Aborted reports only need checks and the answer.

WorkBuddy keeps its own stricter candidate gate (a passing check, plus edits or three commands) and
its "the turn left no answer" outcome.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
forrestlinfeng pushed a commit to EvoMap/evolver-antigravity-plugin that referenced this pull request Oct 26, 2026
forrestlinfeng @claude
feat: count an injected strategy as used, and distill turns that reus…
3f53237 
…ed one

Port of EvoMap/evolver-dsh-plugin#55. Every reuse report (turn end, correction,
retention, aborted) now says `applied: used`; whether the strategy helped is left to
the outcome (last check, or how the turn ended), the next prompt and kept edits. The
turn-end report is L1 whenever a check ran, and its reason drops the verdict.

A verified turn that reused an injected strategy is a distillation candidate like
any other; the engine's Gene intake drops a repeat of a Gene it has. applied.mjs and
the edited-code buffer existed only for the verdict and are removed; a turn records
the files its edits changed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
forrestlinfeng added a commit to EvoMap/evolver-workbuddy-plugin that referenced this pull request Oct 26, 2026
@forrestlinfeng forrestlinfeng @claude
feat(hooks): keep Evolver out of the model context — report at turn e…
e52b520 
…nd, distill at session end (0.3.0) (#10)

* feat(hooks): keep Evolver out of the model context — report at turn end, distill at session end (0.3.0)

No evolver_* MCP tools and no skill any more; the recalled strategy ends with "Apply them where they
fit." and session start adds only a one-off engine notice. Everything else runs in hooks:

- FinalStop closes the turn with its real end reason (completed / interrupted, cancelled → aborted /
  failed → error): reuse reports, and a distillation candidate when the turn completed, used tools,
  passed its last check and reused no injected strategy. A correction at the next prompt drops it.
- SessionEnd reports injections no turn end reported as aborted, closes the candidate and starts the
  background worker, which distills it once through the bundled CLI headless (`codebuddy -p`, no
  tools) and POSTs /conversation/distill (persist, no publish). Three failed tries give it up; the
  next session start claims leftovers and reports a killed session's open injections.
- State goes through the exclusive state lock; candidates live in workbuddy-distill-pending.json.
- Commands run plugin scripts inline (status, search, distill, sync, review, run, solidify) with
  arguments on stdin; /evolver-evolve is the one prompt for the agent. Hooks no-op inside the
  headless distillation run.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(commands): exec the inline script and hand it the plugin data folder

The host fills $ARGUMENTS in raw before it runs an inline command, and a quoted argument keeps its
newlines, so an argument line reading EVOLVER_ARGUMENTS ended the heredoc and the shell ran what
followed (Sentinel finding_e0e1ec1748486ed6ea47c874). `exec node …` replaces the shell, so nothing
after the heredoc ever runs.

The host's inline-command shell has no per-plugin environment, so the scripts fell back to
~/.evolver/workbuddy-hooks while the hooks keep their state in CODEBUDDY_PLUGIN_DATA:
/evolver-status, sync, run, review and solidify could not see the engine or the reports. Each
command now sets CODEBUDDY_PLUGIN_DATA from the ${CODEBUDDY_PLUGIN_DATA} the host substitutes
when it loads command content.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* feat(distill): /evolver-distill falls back to the session's own model

Inside the WorkBuddy app the headless CLI cannot sign in: the app keeps its sign-in field-encrypted and its
CLI removes the bootstrap socket and policy variables from the environment hooks and commands inherit. So
when /evolver-distill gets no Gene from the headless model (no_model, the new model_failed, or an unparsed
answer), the script prints the distillation request under "EVOLVER: WRITE THE GENE YOURSELF" with an
`evolver-distill.mjs --post` command. The agent writes the JSON with its own model and runs that command,
which posts the Gene for the same candidate. Such a headless failure costs the candidate no try during the
command; the session-end worker still counts it. The headless path stays first.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(distill): keep WorkBuddy's candidates in the plugin data folder

WorkBuddy's sandbox lets /evolver-* commands write ~/.workbuddy/plugins/ but not ~/.evolver, so a command
that claims or settles a candidate in ~/.evolver/state/workbuddy-distill-pending.json fails or prompts. The
candidate file now lives in CODEBUDDY_PLUGIN_DATA with the hooks' other state, under the same lock. A
session start merges a file an earlier build left in ~/.evolver/state into it (a candidate already there
wins) and removes the old one. The --post command printed for the agent names the plugin data folder, since
the agent's shell does not carry it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* feat(distill): distill only knowledge-dense turns, and only count what the Proxy stored

Three gates against low-value Genes:
- Candidate: a completed turn with an answer must have run a check (test, lint, typecheck or build) and
  passed the last one, and must have changed files or run at least three commands. Small talk, lookups
  and one-off commands no longer qualify (new skip reasons unverified and too_little_work).
- Model: the distillation prompt asks to keep only reusable knowledge and to answer {"skip": "<why>"}
  for routine work; the candidate is then dropped without a Proxy call, for the headless model and for
  the agent's own model through /evolver-distill --post alike.
- Proxy: an answer of status "draft" means the core stored nothing (quality gate, or a validation trace
  no host verifier confirmed). It now counts as not stored, is reported with its score, and is not
  retried, instead of being taken for a distilled Gene.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* feat(reuse): count an injected strategy as used, and distill turns that reused one (dsh #55)

Mirrors EvoMap/evolver-dsh-plugin#55:
- Every reuse report (turn end, correction L3, retention L2, aborted) sends evidence.applied "used".
  The applied verdict (lib/applied.mjs) is gone; the immediate report is tier L1 whenever a check ran,
  else none, and its reason is "Injected by Evolver into WorkBuddy turn N; <check phrase>." Whether the
  strategy helped is judged by the outcome, the next prompt and the kept edits.
- A verified turn that reused an injected strategy is a distillation candidate like any other: the
  reusedAsset option and the asset_reused skip reason are removed, since the engine's intake drops a
  Gene that repeats one it has.
- The per-turn buffer of written code (200,000 characters) existed only for the verdict; record.edits
  now holds the paths of the files the turn edited. Aborted reports only need checks and the answer.

WorkBuddy keeps its own stricter candidate gate (a passing check, plus edits or three commands) and
its "the turn left no answer" outcome.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* test: drop the stale 'reused no strategy' from a candidate test title

Since 1c9596d a reused strategy no longer matters for candidacy.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix: address Sentinel findings on PR #10

- /evolver-distill --post: the heredoc the agent runs ends on a random line per hand-over, so a Gene
  line steered by the turn's record cannot close it early and run as shell; the command names the
  candidate it was written for, and a Gene for a replaced candidate is not sent.
- A timed-out evolver run on Windows ends its process tree with taskkill /T /F, so the Proxy and
  workers it started stop too.
- takeUnreported fills in the session for ledger entries an earlier build wrote without one, so their
  aborted reports are not filed under workbuddy:unknown.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: forrestlinfeng <linfengchen@tencent.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
forrestlinfeng added a commit to EvoMap/evolver-antigravity-plugin that referenced this pull request Oct 26, 2026
@forrestlinfeng forrestlinfeng @claude
Run Evolver from Antigravity hooks, with nothing in the model context…
ee3b677 
… but the recalled steps (0.2.0) (#6)

* feat(hooks): run Evolver from Antigravity hooks, with nothing in the model context but the recalled steps (0.2.0)

Align with dsh-evolver 0.3.0 (#51/#52): no MCP server, tools or skills; recall injected
in PreInvocation ending "Apply them where they fit."; turn-end and aborted reuse reports;
the session's latest verified turn distilled by agy run headless once its Antigravity
process exits. Turn evidence comes from the transcript, since hooks get no prompt,
output or answer.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* feat: count an injected strategy as used, and distill turns that reused one

Port of EvoMap/evolver-dsh-plugin#55. Every reuse report (turn end, correction,
retention, aborted) now says `applied: used`; whether the strategy helped is left to
the outcome (last check, or how the turn ended), the next prompt and kept edits. The
turn-end report is L1 whenever a check ran, and its reason drops the verdict.

A verified turn that reused an injected strategy is a distillation candidate like
any other; the engine's Gene intake drops a repeat of a Gene it has. applied.mjs and
the edited-code buffer existed only for the verdict and are removed; a turn records
the files its edits changed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix: queue reports before marking them, wait out unknown end reasons, run hooks only from install folders

Sentinel review of #6:
- Injections were marked reported before their reports were queued, so a crash in
  between lost them. reportUnreported/reportOrphaned queue first, then mark; a crash
  now sends a report twice, which the Hub takes as a duplicate.
- An unknown terminationReason closed the turn as an error and sent failed reports.
  Such a Stop now settles nothing; a later Stop, or the next prompt, closes the turn.
- Hook commands probed $PWD first, so a project holding these files could have its
  own scripts run automatically, and a partial copy blocked the fallback. They now
  run only from the two install folders and test for the hook file itself.
- Only SessionStart and PreInvocation get injectSteps; PreToolUse and Stop always
  keep their required decision.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: forrestlinfeng <linfengchen@tencent.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
forrestlinfeng added a commit to EvoMap/evolver-cursor-plugin that referenced this pull request Oct 26, 2026
@forrestlinfeng forrestlinfeng @claude
feat(hooks): run Evolver from Cursor hooks, with nothing in the model…
0a93aaa 
… context but the recalled steps (2.2.0) (#10)

* feat(hooks): run Evolver from Cursor hooks, with nothing in the model context but the recalled steps (2.2.0)

Aligns the Cursor plugin with dsh-evolver 0.3.0 (#51/#52, check readers from #53):

- beforeSubmitPrompt recalls one strategy by the prompt text and adds it as
  additional_context, ending with "Apply them where they fit."; inline edits
  (no composer_mode) are left alone.
- Tool, edit and response hooks record the turn per generation; stop reports
  every injected strategy with the turn's evidence and its own end status;
  aborted, correction and retention follow-ups go through the Proxy outbox.
- A verified turn becomes the chat's candidate, dropped by a correction, and
  is distilled at session end in a detached worker through cursor-agent,
  headless (-p --output-format json --mode ask, empty workspace), with an env
  marker that turns this plugin's own hooks off inside that run.
- /evolver-* commands are answered by the beforeSubmitPrompt hook itself
  (continue:false), before any model sees them; /evolver-evolve stays the dsh
  prompt.
- Removes the MCP server, skill, rule, Configure variables, the session-start
  memory summary, the [Evolution Signal] notice and the git-diff outcome
  posted to /a2a/evolution/record.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* feat: count an injected strategy as used, and distill turns that reused one

Port of EvoMap/evolver-dsh-plugin#55. Every reuse report (turn end,
correction, retention, aborted) now says `applied: used`: the model follows
what is injected into its turn, and whether the strategy helped is left to
the outcome (last check, or how the turn ended), the next prompt and kept
edits. `tier` is L1 whenever a check ran, and the reason no longer names a
verdict.

A turn that reused an injected strategy is a distillation candidate like any
verified turn; the engine's Gene intake drops a repeat of a Gene it has.
applied.mjs and the edited-text capture existed only for the verdict and are
removed; a turn records the files it edited (afterFileEdit writes, Delete).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* test: retitle the candidate test now that reuse no longer skips a turn

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* feat(status): lead /evolver-status with the tokens reused strategies saved

A turn that reused an injected strategy and whose last check passed adds
its saving to the chat's and to the running total in
~/.evolver/state/cursor-savings.json (chats unseen for 8 days drop out of
the per-chat map, not out of the total). The saving is savingsCore's
reference-reuse estimate, as the Claude Code and Codex plugins show it,
from the lines the turn's edits changed, capped at the turn's own tokens
from stop (input + output + cache writes). Cursor hooks have no message
for the person only, so the figures are shown by /evolver-status, which
the plugin answers without the model.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* feat(state): keep plugin state for three days, distillation candidates included

Undelivered reports, unreported injections, retention checks, chat files and
per-chat savings now live 3 days instead of 7-8 (one STATE_TTL_MS), and the
background worker drops distillation candidates older than that whoever owns
them, so a chat that never closes on a host without a process table no longer
keeps its candidate for good.

README: Cursor 3.27 loads plugin hooks only while "Include Third-Party Plugins,
Skills, and Other Configs" is on; with it off none of this plugin's hooks run.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* feat(status): list this chat's reused assets under the SAVED line

Each turn that reused an injected strategy is recorded with its turn
number, the asset's name and id, the turn's last check and its saving
(or none counted), and /evolver-status lists a chat's latest 20 below
the totals. The injection ledger keeps the recalled asset's name so the
turn end can show it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix: address Sentinel's advisories on the Cursor hook-only plugin

- sessionEnd ends only the turn it saw settle: a prompt that opened a new
  turn in the same chat meanwhile keeps its session, injections and
  candidate (endSession returns false and nothing is cleaned up).
- Tool, edit and response hooks record only under their own generation_id;
  one without it no longer lands in the current turn.
- A distillation claim counts only once it is on disk, and candidates are
  told apart by turn as well as timestamp.
- The background worker runs engine setup beside report delivery and
  distillation, so a slow install no longer holds them back.
- Stripe-style keys (sk_live_…, rk_test_…) are redacted.
- The global Evolver core is also looked for under %APPDATA%\npm on Windows.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: forrestlinfeng <linfengchen@tencent.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@forrestlinfeng