Skip to content

perf(authority): reuse File proofs and bound quota observation output - #5222

Merged
huangruiteng merged 4 commits into
mainfrom
codex/goal-read-cost-0928
Sep 28, 2026
Merged

huangruiteng merged 4 commits into
mainfrom
codex/goal-read-cost-0928

Conversation

@huangruiteng

@huangruiteng huangruiteng commented Sep 27, 2026 •

Copy link
Copy Markdown
Collaborator

Repeated observations of two File-backed Goals were evicting each other's single cached history proof. Quota status/plan also emitted complete Todo lists even though should-run already had bounded display helpers. This change addresses those two measured costs within the local-authority qualification work in the retirement checkpoint.

  • File now retains up to four verified stores in LRU order under one 128 MiB encoded-data budget, retaining only a bounded head/receipt view for large journals. Every hit still reads/hashes the bytes and checks store identity. Failed publication invalidates only that store. The TS authority contract, on-disk format and original receipts are unchanged.
  • Intentional CLI default change: quota status/plan JSON uses the existing Todo summary projections after complete planning. Counts, quota ordering/decisions, health and omission markers remain available. Consumers needing complete item metadata use --include-detail agent-todos, user-todos, or all. Markdown is unchanged. A shared command/section map owns validation and all expansion; combining all with an unsupported section now fails before collection.
  • The CLI presentation adapter remains Python; File proof ownership remains TypeScript. The display hook is CLI-only, so App/Lark and shared semantic API payloads need no companion change. No new decision owner, provider default, migration or legacy-writer removal is introduced. This is independent of fix(quota): scope read-only plans to the selected Goal #5220's Goal-selector fix.

On detached copies retaining 379 original commits, alternating two File stores took 6.30–6.49 seconds per read before; after each store's first proof, the candidate took 9.8–11.2 milliseconds with matching hashes/cursors. First proof remains about 6 seconds. Three fresh-process samples measured File head reads at 5.98–6.32 seconds and SQLite at 34.5–36.0 milliseconds; these are different integrity workloads, not full-history throughput parity. A captured single Goal JSON row shrank from 1,252,747 to 78,688 UTF-8 bytes, and explicit detail restores it. Display size does not measure collection latency.

Validation on c56c0dc8b1d93a74166abfa57bde3a895936a569, integrated with main 00be5772f:

  • 323 Python tests pass: real isolated File/SQLite CLI, exact Unicode Todo metadata, selective/full detail, negative selectors, scheduler compatibility and the registry I/O census. Markdown parity uses one observation basis so freshness timestamps from separate calls do not create a false failure.
  • 304 File authority tests pass, including corruption, identity, bounded multi-store eviction and original-receipt recovery. TS typecheck, configured mypy (19 files), changed Python Ruff, whitespace and public-boundary checks pass.
  • The release-baseline publication smoke passes after integrating current main. Registry census changes only refresh source line locations; no exemptions were added.
  • Exact-scope quality receipt cqr_84eb7acb844bd7c53fda verifies valid for fingerprint 84eb7acb844bd7c53fdab60f7c481d968a9355da3a8c5747f48ca5783e754b88 (15 files). Standard premerge passed: 19 selected checks, zero failures. The configured managed review does not wait for remote CI; exact-head local validation and published review were required before the owner-authorized admin merge.
  • Prior unchanged-storage qualification retained 379 original commits and ran 12 fresh-process writes per provider across a checkpoint, with replay/conflict/hash parity (148 seconds). This historical bounded result is not D2 elapsed soak or a fresh Host acceptance claim.

The RFC now names the next shared cold-read investigation: history artifact lookup, active-contract validation and public-boundary scanning, with same-input base/head comparison. Consumer closure and supported onboarding/upgrade remain independent acceptance exits; remaining work is not estimated by a fixed PR count.

Remaining qualification: cold File verification, aggregate RSS, sustained workload/capacity/platform coverage, and consumer/onboarding upgrade acceptance. SQLite is a stronger long-history candidate, but these observations do not change the release default or authorize legacy-writer deletion. Raw snapshots and one-off diagnostic scripts are excluded.

Signed-off-by: huangruiteng <14976749+huangruiteng@users.noreply.github.com>
Signed-off-by: huangruiteng <14976749+huangruiteng@users.noreply.github.com>
Signed-off-by: huangruiteng <14976749+huangruiteng@users.noreply.github.com>

@huangruiteng huangruiteng left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approval conclusion (author-owned PR; GitHub blocks formal self-approval)
Reviewed exact head: 8b1e25849bdadccd366cd67732378e7f1489f56a
Comparison merge-base: 82dfe2e2c8c85d3fd63490313597886eba709aa0

动机

这份 PR 对应 shared-goal authority B 本地档的两个可量化观察成本:多 Goal 交替使用 File store 时,单项缓存不断失效;quota status/plan 默认 JSON 带回完整 Todo 数组,输出随项目规模膨胀。它是有界性能修复,不把 File 改成默认提供者,也不宣称完成 D2 十天稳定性验收。既有 checkpoint 的后续多日比较仍待完成。

改动思路

沿用两个既有 owner:quota planner 先对完整权威状态作决策,最后才生成有界 JSON 摘要;需要旧完整数组的 caller 可用 --include-detail agent-todos|user-todos|all 取回。FileAuthorityStore 把单项已验证读视图改成至多四库、128 MiB 编码数据预算的 LRU;命中仍重读文件、计算 digest 并核对 store identity,绝不让缓存决定写入或替代持久 journal。没有新增第二个资格/权限 owner。

具体改动

完整 PR 是 13 文件、+346/-37:现有 CLI/投影与 File store 小幅修改;新增 7 项 quota 观察测试、两项 File 交替/LRU 测试;文档同时说明旧/新默认、full-detail 冷路径、缓存界限和 D2 未闭合。CLI 参数按命令白名单校验,all 不能夹带不属于该命令的 selector;Markdown 路径保持原样。File 缓存超过四库或预算即逐出最久未用项,篡改或身份不符会重新验证并拒绝。

新增测试未改动公共 journal 格式或 quota 决策逻辑。

对主干的风险

最重要的兼容风险是默认 JSON 少了逐条 Todo,不能把它称为完全无变化。真实 File/SQLite CLI 测试证实完整 detail 保留原 note/status,摘要保留 count/plan,Markdown 输出相同;公开 quota 文档和 help 明示这项默认变化。独立复核又用同一个 12-Todo synthetic File Goal 跑旧/新 CLI:旧版 status/plan 完整输出分别 115,667/115,665 字节,新版 --include-detail all 与旧版逐条 Todo、quota 和 summary 一致;新版默认分别 16,298/16,292 字节。未发现决策或持久状态变化。

缓存方面,我把 head 新增的两项测试原样放到一次性的不可变基线测试位置运行:旧单项缓存分别出现验证计数 6(期望 2)与 8(期望 5),精确 head 两项通过;篡改一个 File store 不影响另一个,五库场景证实逐出。基线/head 原有 quota suite 各 26 passed,File store suite 302/304 passed;head 新 quota 测试 7 passed、完整 TS 控制面 3359 passed/30 skipped/0 failed、Ruff 与 TS typecheck 通过。最初隔离 worktree 缺 pg 时的八个导入失败在基线/head 同形,npm ci 后消失,不计为 PR 缺陷。未查询或等待远端 CI;本机无 PostgreSQL 测试 URL,30 项跳过及多日 D2 均不外推成通过。

我的整体评价

APPROVE(author-owned PR,故以 COMMENTED review 记录正式结论,不伪造 GitHub self-approval)。此变更改善本地档反复观察成本,同时保留权威读验证与完整详情回退;用基线反例和真实 CLI 双后端验证了核心承诺。Future-facing pass 看过相邻 owner:命令/section 映射与既有摘要 helper 已复用,继续提取抽象没有当前价值;D2 sustained File/SQLite 对照仍由既有 checkpoint 负责。它更动 runtime/control-plane 行为,按仓库规则应留给维护者合并,我不自合并。

English verdict: APPROVE - 8b1e258. The bounded display and four-store verified-view cache retain decision/authority semantics. Same-fixture CLI and base-vs-head counterfactuals passed; prior missing-pg worktree failures were setup-only. D2 soak and unconfigured PostgreSQL integration are not certified; no remote CI or merge action was used.

Signed-off-by: huangruiteng <14976749+huangruiteng@users.noreply.github.com>

@huangruiteng huangruiteng left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approval conclusion (author-owned PR; GitHub blocks formal self-approval)

Reviewed exact head: c56c0dc8b1d93a74166abfa57bde3a895936a569; comparison base: 00be5772fd63677983ec0de566adb597463eac1e.

动机

未发现阻塞项。本 PR 处理两个长期运行中会累积的观察成本:多 Goal 轮流读 File 时,单项证明缓存反复淘汰;quota status/plan 输出完整 Todo 数组,使普通观察包不断增大。它交付的是有用且可回退的读取优化,不把缓存命中或输出变小等同于默认 provider 验收。对照是既有 shared-authority B 本地 profile 与 quota 展示合同,后续消费者、升级和持续运行出口仍保留。

改动思路

复用现有 owner。完整规划先读取权威输入,最后由 CLI 展示适配器调用原有 Todo compactors;显式 detail 返回完整原记录,不让展示限额进入资格判断。File 仍由 TS 验证实际字节和 store identity,再复用至多四库的证明视图。缓存只影响重复工作,不生成权限、事务或回执。换成 SQLite 本身无法消除公共展示成本;为此另建缓存框架或第二套 Python 决策源均无必要。App/Lark 调用的语义载荷未经过这个 CLI-only hook,无需配套修改。

具体改动

完整差异为 15 文件、+367/-43,覆盖 CLI 分派、参数合同、展示、File 存储、三处测试和五份文档;source census 仅校正源码行号。与上次 head 相比,本次集成主线并修复旧诊断断言、源码清单漂移以及两次 CLI 的 freshness 时间戳造成的错误 Markdown 对比。没有更新数据格式、隐藏迁移或引入额外版本分支。

关键代码讲解

  • loopx/control_plane/coordination/file_authority_store.ts:63 的 rememberVerifiedDocument 用同一个 Map 保存最多四项,并对序列化历史及读视图共同计费;大文件只保留有界 head/receipt view。超出容量会淘汰最近最少使用项,未保留的历史仍重新验证。
  • 同文件 readVerified(278 行)每次都读取并计算文件摘要,校验 identity 和 digest 后才更新 LRU 顺序;调用方取得的是既有复制结果,不能通过改返回对象污染证明。发生不确定提交结果时只失效本库缓存,其他库仍可正常观察。
  • loopx/control_plane/quota/cli_projection.py:799 的 compact_quota_plan_cli_payload 在完整 payload 上浅复制行、复用原有角色摘要,保留 quota、计数和遗漏声明;角色 detail 则保留完整数组。quota_context 与 quota_request 共用命令到 section 的映射,all 无法夹带别的命令选项。

对主干的风险

核心风险是默认 JSON 形状有意变化,旧脚本不能继续假定每次都有 items。帮助、quota 文档和接口合同已明确要求这类调用保留原 registry/Goal 选择并加 --include-detail all。当前 base/head 同一 60-Todo fixture 的 File/SQLite 实际 CLI 对照覆盖 status/plan:完整角色摘要、metadata、quota 和 summary 相等,默认数组收敛;旧默认仍含 items,因此明确触发独立的压缩预期反例。未把两个不同时刻的 freshness 当作数据丢失。

缓存的最强反例是命中掩盖损坏、返回对象污染、或增加 Goal 后无限增长。304 项真实 File 测试覆盖字节与 identity、篡改、LRU 逐出、重放和竞争;323 项 Python 检查覆盖真实双后端 CLI、完整 Unicode metadata、错误参数、旧命令及 source census。标准 premerge 19 项零失败,TS 类型检查、mypy 19 文件、Ruff、publication smoke 和公开边界通过。过去的跨 checkpoint 隔离演练只作为既有证据,未冒充本轮 Host 或十天 D2 验收。128 MiB 是编码数据预算,不是进程 RSS 限额;首次 File 全历史验证仍存在。

语义与 CI 对齐

本 PR 复用已有 selector、payload_compaction 和 File journal 词汇,不新增行为权限。主线整合后,参数诊断测试改为断言真正不受支持的 section;Markdown 测试在同一个观测基准上比较,未归一化掉 Todo 或 quota 语义。没有放宽预算或绕过损坏验证。当前 Goal 的 review 配置为 wait_for_ci=false,本次审查和合并判断使用精确源码本地验证,不将远端排队状态当成已通过;质量回执 cqr_84eb7acb844bd7c53fda 对当前 15 文件范围有效。

我的整体评价

APPROVE。这项增量改善长程工作反复观察的成本;用户体验接受了一个已明确授权并披露的默认输出变化,同时保留可操作的完整详情恢复。既有 TS authority 和 Python 展示职责清晰,相关重构已经收拢 selector 规则和 compactor 复用,不需要再引入通用层。剩余冷历史读取、RSS、跨平台与自然时间 soak 继续由原 RFC 出口验收,不能据此删除 legacy writer。本次用户明确授权当前 head 自审后合并及本机升级;合并前仍要求精确 head 的 published review 与 merge-readiness 检查。

English verdict: APPROVE - c56c0dc. Same-fixture real File/SQLite CLI comparisons preserve full metadata and quota; bounded display and four-store proof reuse retain authority checks. 323 Python tests, 304 File tests, types, publication smoke, exact-scope quality and 19 premerge checks passed. Default JSON change is disclosed; cold verification, RSS and D2 qualification remain separate. Owner explicitly authorized self-merge and local adoption.

@huangruiteng
huangruiteng merged commit f679911 into main Sep 28, 2026
26 of 27 checks passed
@huangruiteng
huangruiteng deleted the codex/goal-read-cost-0928 branch September 28, 2026 03:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant