Skip to content

stabilize LLM prefix caching context #1

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
sunheyi6 wants to merge 1 commit into
base: main
Choose a base branch
from
Draft

stabilize LLM prefix caching context #1

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions docs/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@
| 文档 | 说明 |
|------|------|
| [项目信息](project-info.md) | 运行时路径、数据目录、日志路径、构建信息、架构约定（Agent 回答"日志在哪""路径是什么"时优先读这个） |
| [系统提示词设计](system-prompt-design.md) | System Prompt 构建：角色定义、Tool Discipline 反循环规则、一行式工具摘要、XML 结构化上下文、Token 优化 |
| [系统提示词设计](system-prompt-design.md) | 稳定 System Prompt + 可追加 Runtime Context：KV 前缀缓存、真实用户识别、主 Agent/Subagent 继承与缓存 epoch |
| [工具调用设计](tool-calling-design.md) | 工具架构：回合事件 (turn_start/end)、参数验证、前端工具卡片 (Command/File/Inspect)、数据流 |
| [图片理解模型路由](vision-model-routing-design.md) | 视觉旁路：上下文隔离、自动能力识别、手动覆盖、同厂商模型选择与 `inspect_image` 调用链 |
| [上下文压缩设计](context-compaction-design.md) | 容量保护 pipeline + 缓存友好的主动语义 projection：A/B/C 请求结构、失败保护、配置与 Terminal-Bench 验证 |
Expand Down Expand Up @@ -46,7 +46,7 @@
| Agent turn 计数 | Agent 全局 turnCount 跨 run 累计,StatusBar 显示当前轮次 |
| Tool Usage Discipline | 新增反循环规则章节,防止模型无限调用工具 |
| 工具一行式摘要 | 替代完整 JSON Schema,节省 ~60% 工具 token |
| `<project_context>` XML | 结构化注入 .agents.md / Skills / Memory / Lessons,遵循 pi/Codex 约定 |
| 结构化模型上下文 | `.agents.md` / Skills 位于稳定 system envelope；Memory / Lessons / Language 位于可追加 runtime context |
| Skills 技能系统 | `~/.suncode/skills/` + 项目 `.suncode/skills/` 双层加载,Markdown 格式,注入 system prompt |
| Memory 记忆系统 | 每次 session 结束自动生成记忆摘要存入 `.suncode/memories/`,下次对话根据语义检索注入上下文 |
| Lessons 教训系统 | 失败时自动提取教训存入 `.suncode/lessons/`,后续相关任务检索注入,避免重复踩坑 |
Expand Down
14 changes: 14 additions & 0 deletions docs/failure-lessons-design.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -124,6 +124,20 @@ export interface LessonExtractionContext {

---

## 检索结果注入

`Agent.runLoop()` 和 `runGoalLoop()` 根据当前真实用户目标调用 `loadRelevantLessons()`。返回文本不再写入 system prompt，而是进入：

```text
suncode.runtime_context.snapshot.relevantLessons
```

这样 lesson 检索结果变化时只追加新的运行时快照，不会破坏稳定的 system 前缀。主 Agent 会在每次运行开始前通过 `SubagentDispatcher.updateOptions()` 同步同一份 `relevantLessonsContent`，Subagent 再交由公共 `runAgentLoop()` 注入。

运行时上下文使用 user provider role，但带有 `contextKind='runtime_context'`；教训提取、用户纠错识别和记忆摘要必须通过 `isUserAuthoredMessage()` 排除该内部消息。

---

## Settings 扩展

```typescript
Expand Down
11 changes: 8 additions & 3 deletions docs/memory-system-design.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -61,6 +61,8 @@ SunCode 的记忆系统旨在帮助 AI agent 持久化和复用重要信息。
│ │ Agent 集成 │ │
│ │ agent.ts → loadMemoriesWithEntries() + relevanceJudge │ │
│ │ agent.ts → saveSessionMemory() / promoteExplicitDurableFacts() │ │
│ │ agent-loop.ts → 检索内容进入 runtime_context │ │
│ │ subagent.ts → 主 Agent 将同一份检索结果传给 Subagent │ │
│ │ agent-loop.ts → memoryReferences 随 finalMessage 持久化 │ │
│ └─────────────────────────────────────────────────────────────────────┘ │
└─────────────────────────────────────────────────────────────────────────┘
Expand Down Expand Up @@ -233,7 +235,7 @@ LLM 语义精排（可选）
记录访问（仅内存计数，写时落盘）
│
▼
返回 { content: 注入 system prompt, entries: 引用展示 }
返回 { content: 注入 runtime_context, entries: 引用展示 }
```

### 混合评分算法
Expand Down Expand Up @@ -460,7 +462,7 @@ Agent.prompt()
├── loadMemoriesWithEntries(query, { relevanceJudge })
│ ├── 常驻通道（主题门控）→ 普通检索 → LLM 精排 → 同源合并
│ └── 返回 { content, entries }
│ ├── content → 注入 system prompt
│ ├── content → 注入 runtime_context.snapshot.memory
│ └── entries → recordMemoryAccess()（内存计数）+ 传递 UI
│
▼
Expand Down Expand Up @@ -488,6 +490,7 @@ Agent.saveSessionMemory()
│
▼
message_end 事件
├── runtime_context: memoryContent（主 Agent 与 Subagent 共用）
└── memoryReferences: input.memoryEntries
│
▼
Expand Down Expand Up @@ -563,7 +566,9 @@ src/
│ │ └── getAllMemories() / getMemScenes()
│ ├── agent-data-dir.ts ← SUNCODE_APP_DATA 路径统一解析
│ ├── agent.ts ← 记忆加载、saveSessionMemory、promoteExplicitDurableFacts
│ └── agent-loop.ts ← finalMessage 携带 memoryReferences 持久化
│ ├── runtime-context.ts ← memory / lessons / language 运行时快照
│ ├── subagent.ts ← 继承主 Agent 检索结果
│ └── agent-loop.ts ← 注入 runtime_context；finalMessage 携带引用
├── main/
│ ├── index.ts ← 设置 SUNCODE_APP_DATA、flat 记忆迁移
│ ├── preload.ts ← IPC API 定义
Expand Down
Loading
Loading