Compact File authority history with recognized, backed-up format upgrades

[huangruiteng]

Goal And Delivered Outcome

File authority repeatedly retained a complete projection for every transaction. This made a long-lived store expensive to rewrite. Reuse the existing TS checkpoint/delta codec while preserving original revisions, events, receipts and complete historical readbacks. Normal File readers/writers now accept only the current format; old decoding belongs to the explicit upgrade tool.

Related to #4574 (R5/G2), the shared-authority RFC and merged #5063. Initially stacked on #5063; now based on main eaa0c0fd0.

Scope And Continuation

• Add content-based authority-archive inspect --source PATH: distinguish File/SQLite stores, logical archives, backup packages and provider selectors. Metadata identification is explicitly weaker than complete history verification. Reject unknown formats, provider-directory mismatches and conflicting SQLite version markers.
• Add authority-archive upgrade [--all-known] [--execute | --require-current]. File backups retain exact source bytes and identity under the writer lock. SQLite uses a consistent online backup, validates a disposable copy through the existing converter, and fences source adoption against that backup's logical history digest.
• Wire upgrade into default installation and package updates. Interrupted publication is retriable; partial multi-store progress is reported. Gate binary rollback on target format compatibility. Keep candidate recovery material when post-upgrade launcher work fails.
• Reuse the portable logical archive for File/SQLite interchange. Do not change provider selection, registry/fence/lease authority, or migrate legacy Markdown sources through a physical-file converter.

This completes the local format-upgrade/storage slice. The RFC retains three named planned boundaries: external-effect interval fencing; whole-Goal supported-source/cutover/rollback closure (reconcile #5054); default entrypoints and bounded Python business-owner retirement. Existing #4931 and D1–D3 evidence remain separate. These are work packages, not a guaranteed remaining PR count. PostgreSQL service activation is not claimed.

Validation

• Tested revision: b4c4d3f971868498ba3dd79c7095be415f3b9633 for final typecheck/format-inspection tests. Broader File and installer runs precede the last recognition/error-reporting changes; affected recognition/CLI checks were rerun.
• Run state: finished
• Input classes: synthetic, public_fixture, authorized_private_read_only

Check kind	Result	Evidence / limitation
static	passed	npm run typecheck:control-plane, focused Python Ruff and git diff --check
real_backend	passed	Real filesystem File conformance; 291-test run including then-current upgrade cases. Final migration/recognition/journal/SQLite set: 20 passed, plus final discovery set: 6 passed. Node 24.21.0 / SQLite 3.53.4; isolated temporary stores.
real_entrypoint	passed	tests/control_plane/test_authority_archive.py: 3 CLI journeys through managed TS, including inspect, preview, migration, backup and readback. Native Todo update/journal readback: 25 passed.
integration	passed	Update/installer Python set: 29 passed, 5 platform skips. POSIX examples/release/local-install-promotion-boundary-smoke.py and examples/loopx-update-smoke.py passed.
regression_parity	passed	Detached authorized snapshot versus immutable #5063 baseline: exact backup verified, every retained transaction/receipt compared after further isolated commits, original source unchanged. Synthetic production-shaped history and deliberate history/version/backup corruption cover public regression sensitivity. No private artifacts included.
manual	not_run	Native Windows execution: require platform CI before Windows activation. PostgreSQL backend unchanged and not requalified by this PR.

Measured on the same detached workload: about 110.8 MiB becomes 5.9 MiB; steady writes fall from roughly 1.7–1.8 s to 0.25–0.27 s. Full cold verification increases from roughly 2.9 s to 3.9 s. Backup/upgrade/verification takes about 21 s. These are local comparative observations, not a D2 capacity qualification or changed budget. File still rewrites a single retained document.

Frontend / Visual Evidence

UI impact: none. CLI and installer entrypoints change; frontend/Lark business callers retain the same AuthorityStore contract and provider selection. No settings, page, or first-screen presentation change.

Type Of Change / Area

Breaking physical-format upgrade; control-plane storage refactor, installer/CLI behavior, documentation and tests. Old binaries cannot read new File data; migration, backup and downgrade boundaries are documented in docs/reference/file-authority-state-log.md.

Shared-authority RFC Fixture Impact

Reuse productionScaleHistoryProjection for retained mixed Todo/lease history; no new business transition rules. File and real SQLite arms pass, including both directions of archive interchange. No promotion/runtime-routing/compatibility-projection change; no three-arm promotion qualification claimed.

Boundary Checklist

• No private state, credentials, raw traces, local paths or internal links in the diff or PR.
• Existing codec, lock, SQLite converter and archive owners reused; no speculative migration framework or new provider.
• DCO sign-off on every commit.
• Exact-scope change-quality receipt recorded and verified; cold-read cost and native Windows evidence gap disclosed.
• No self-merge. Maintainer review and applicable CI remain required.

[huangruiteng]

Approval conclusion (author-owned PR; GitHub blocks formal self-approval)

Exact head: 14b90ca898eafe019e02d45b7a7b3a8d83760720. No blocking finding remains after self-review/refinement. 本次按维护者明确授权执行自审及 admin-bypass 合并；不以该授权替代以下验证。

动机

旧 File 把每一轮完整 projection 都写入历史，Goal 越久运行，重复序列化和写盘成本越高。#5063 已改善读取与等待边界，但没有消除物理重复。这次在保留 append-only 逻辑历史、原始回执和版本身份的前提下压缩存储，并补齐安装升级的备份与迁移入口。

改动思路

复用现有 TS authority_state_log 检查点/差量规则，正常 File 读写只维护一种当前格式。旧格式解析保留在显式迁移工具内，既能升级旧数据，也能恢复旧备份。格式识别依据内容而非文件后缀，识别结果不冒充全历史验证。Python 仅负责 CLI 和安装环境；迁移决定、历史校验、锁和持久化仍归 TS 存储边界。跨 provider 继续复用逻辑归档，避免维护成对转换器。

具体改动

File 每 64 条记录保留检查点，其余保留精确差量；事件、operation ID、receipt、cursor 与 provider revision 不改写。冷读仍校验完整历史，不能用正确 head 掩盖旧记录损坏。File 转换持有正常业务写锁，先保存并读回原始字节和 identity、同步目录，再原子发布；SQLite 用真实在线备份，并在采用新表的事务内核对备份对应的逻辑历史，拒绝期间发生的推进。升级覆盖注册表关联的多个 runtime、未选中的本地 store 和 rollback 文档，失败保留已完成结果；不修改 Goal 选择、lease 或 writer fence。

本轮 refine 增加了跨项目根目录去重、断开项目处理和不创建 Markdown Goal 存储的回归测试。POSIX、Windows、pip/pipx 安装更新都接入统一入口；失败不伪称整体回滚，也不删除仍可用于恢复的候选版本。二进制回退必须先证明格式兼容。参考文档与双语 RFC 同步说明了旧格式退出及 D1–D3 尚未完成的边界。

关键代码讲解

• FileAuthorityJournal.decode：重建每条逻辑事务，重新核对版本链和最终 head，得到经过验证的当前格式视图；scan 从最近检查点还原原始历史页面。
• migrateFileAuthorityStore：同一写锁覆盖原始读取、备份校验和发布；发布前失败可重试，发布后重试返回 already_current，不制造新业务提交。
• upgradeAuthorityFormats：以已知 runtime 为范围，识别 provider 与 Goal 身份后选择转换器；发现或迁移失败时如实返回此前成功项，不覆盖后来写入。

对主干的风险

主要风险是物理格式不可被旧二进制读取，以及冷校验需要重建历史，可能比原格式更慢。本 PR 没有提高缓存预算、跳过校验或宣称 File 的整文件读写变成常数成本。升级后须重启仍驻留旧版本的进程；恢复旧备份应在隔离目录先验证，不能把旧文件覆盖到已有后续提交的线上目录。真实副本演练额外检查了旧写入器拒绝新格式、备份能被旧读取器恢复、全部历史/回执相等，以及新版本可继续提交和读回。

本地证据包括完整 File store 合同测试、检查点跨页/历史损坏/竞争迁移/发布前后中断测试、真实 SQLite 备份和 File⇄SQLite 归档恢复、实际 CLI 迁移、安装与 update smoke、TS typecheck、相关 Python 测试、diff 和公开内容扫描。已重放当前两份实际存储的隔离副本。原生 Windows 未在本机执行，不能把跨平台 Python 测试视为其 OS 资格证明；PostgreSQL 路径未修改，本次也不宣称完成 PostgreSQL 或长期 soak 验收。

语义与 CI 对齐

file_v0 是 provider 路由身份，loopx_file_authority_store_v1 是物理 schema，二者不矛盾。升级不意味着 Goal 晋升；没有本地 authority store 的 Goal 不会因此创建 File/SQLite。当前 Goal 配置 wait_for_ci=false，本次不查询或等待远端 CI，依据本地验证和精确 head 评审作判断；合并前仍执行能力的 merge-readiness 检查。

我的整体评价

这是一项完整的存储成本修复和可运维升级切片，复用了已有 TS 编码与归档规则，没有新建业务权威。比提高预算或永久保留两套正常读写规则更容易维护。当前没有阻断项；剩余风险是已披露的冷读成本、旧进程重启和未执行的原生 Windows 资格验证。支持在维护者授权下合并，并按已演练的备份、升级、逐 Goal 读回流程落地。

English verdict: APPROVE - 14b90ca. Compact File history preserves logical transactions and receipts; explicit verified migration and installer gates pass local File/SQLite/CLI validation. Native Windows and long-soak qualification are not claimed.

[huangruiteng]

Approval conclusion (author-owned PR; GitHub blocks formal self-approval)

Exact head: 116b58ecfc8777d8be3f9478d7627b1c72be84bd. No blocking finding remains after self-review/refinement. 本次按维护者明确授权执行自审及 admin-bypass 合并；不以该授权替代以下验证。

动机

旧 File 把每一轮完整 projection 都写入历史，Goal 越久运行，重复序列化和写盘成本越高。#5063 已改善读取与等待边界，但没有消除物理重复。这次在保留 append-only 逻辑历史、原始回执和版本身份的前提下压缩存储，并补齐安装升级的备份与迁移入口。

改动思路

复用现有 TS authority_state_log 检查点/差量规则，正常 File 读写只维护一种当前格式。旧格式解析保留在显式迁移工具内，既能升级旧数据，也能恢复旧备份。格式识别依据内容而非文件后缀，识别结果不冒充全历史验证。Python 仅负责 CLI 和安装环境；迁移决定、历史校验、锁和持久化仍归 TS 存储边界。跨 provider 继续复用逻辑归档，避免维护成对转换器。

具体改动

File 每 64 条记录保留检查点，其余保留精确差量；事件、operation ID、receipt、cursor 与 provider revision 不改写。冷读仍校验完整历史，不能用正确 head 掩盖旧记录损坏。File 转换持有正常业务写锁，先保存并读回原始字节和 identity、同步目录，再原子发布；SQLite 用真实在线备份，并在采用新表的事务内核对备份对应的逻辑历史，拒绝期间发生的推进。升级覆盖注册表关联的多个 runtime、未选中的本地 store 和 rollback 文档，失败保留已完成结果；不修改 Goal 选择、lease 或 writer fence。

本轮 refine 增加了跨项目根目录去重、断开项目处理和不创建 Markdown Goal 存储的回归测试。POSIX、Windows、pip/pipx 安装更新都接入统一入口；失败不伪称整体回滚，也不删除仍可用于恢复的候选版本。二进制回退必须先证明格式兼容。参考文档与双语 RFC 同步说明了旧格式退出及 D1–D3 尚未完成的边界。

关键代码讲解

• FileAuthorityJournal.decode：重建每条逻辑事务，重新核对版本链和最终 head，得到经过验证的当前格式视图；scan 从最近检查点还原原始历史页面。
• migrateFileAuthorityStore：同一写锁覆盖原始读取、备份校验和发布；发布前失败可重试，发布后重试返回 already_current，不制造新业务提交。
• upgradeAuthorityFormats：以已知 runtime 为范围，识别 provider 与 Goal 身份后选择转换器；发现或迁移失败时如实返回此前成功项，不覆盖后来写入。

对主干的风险

主要风险是物理格式不可被旧二进制读取，以及冷校验需要重建历史，可能比原格式更慢。本 PR 没有提高缓存预算、跳过校验或宣称 File 的整文件读写变成常数成本。升级后须重启仍驻留旧版本的进程；恢复旧备份应在隔离目录先验证，不能把旧文件覆盖到已有后续提交的线上目录。真实副本演练额外检查了旧写入器拒绝新格式、备份能被旧读取器恢复、全部历史/回执相等，以及新版本可继续提交和读回。

本地证据包括完整 File store 合同测试、检查点跨页/历史损坏/竞争迁移/发布前后中断测试、真实 SQLite 备份和 File⇄SQLite 归档恢复、实际 CLI 迁移、安装与 update smoke、TS typecheck、相关 Python 测试、diff 和公开内容扫描。已重放当前两份实际存储的隔离副本。原生 Windows 未在本机执行，不能把跨平台 Python 测试视为其 OS 资格证明；PostgreSQL 路径未修改，本次也不宣称完成 PostgreSQL 或长期 soak 验收。

语义与 CI 对齐

file_v0 是 provider 路由身份，loopx_file_authority_store_v1 是物理 schema，二者不矛盾。升级不意味着 Goal 晋升；没有本地 authority store 的 Goal 不会因此创建 File/SQLite。当前 Goal 配置 wait_for_ci=false，本次不查询或等待远端 CI，依据本地验证和精确 head 评审作判断；合并前仍执行能力的 merge-readiness 检查。

我的整体评价

这是一项完整的存储成本修复和可运维升级切片，复用了已有 TS 编码与归档规则，没有新建业务权威。比提高预算或永久保留两套正常读写规则更容易维护。当前没有阻断项；剩余风险是已披露的冷读成本、旧进程重启和未执行的原生 Windows 资格验证。支持在维护者授权下合并，并按已演练的备份、升级、逐 Goal 读回流程落地。

English verdict: APPROVE - 116b58e. Compact File history preserves logical transactions and receipts; explicit verified migration and installer gates pass local File/SQLite/CLI validation. Native Windows and long-soak qualification are not claimed.