[Jackmeson1]

Summary

At caebd268d, Save can fail when its selected delta base alone exceeds a dependency-work budget: 512 KiB canonical or 256 KiB encoded. Acquisition returns Integrity("dependency canonical work") or Integrity("dependency encoded work") before selection reaches the existing independent-storage decision.

Stores created with supported cutoffs above the default can contain whole files large enough to trigger this. I observed failures through fs.apply, fs.write, a private Workspace Commit fixture, and Project Init of a directory containing two similar files. The failure requires selection of such a base.

Selection now checks the base chain's own stored costs before dependency reconstruction/authentication. An over-budget base produces independent storage with work_exceeded. A base whose own chain fits is acquired and authenticated first, followed by the unchanged prospective-budget check.

Reproduction and cause

The issue contains a runnable storage reproduction and separate SDK observations.

With StoragePolicy::new(1, 1_048_576, 16, 8), save a 300,000-byte incompressible whole-file object, then a near-identical version declaring the first as its base. On caebd268d, the second Save returns Integrity("dependency encoded work"). A 600,000-byte compressible pair returns Integrity("dependency canonical work").

On that parent, encoding/delta/select.rs:433 acquires the base before the prospective test at line 446. Resolver::resolve_dependency charges the base itself and every dependency against the same budgets. A valid base that cannot fit alone therefore aborts Save before its work_exceeded branch.

With parent production code and this PR's tests, five of the twelve storage cases and four of the five SDK cases fail with these budget errors. The remaining cases are controls or guards.

Implementation

• ChainBases::edge returns the stored base identity and the encoded width charged by Resolver::decode_at, using the shared selected_record_width calculation.
• ChainCost gains encoded. The bounded DepthCache accumulates it alongside canonical cost. Eligibility already walks this chain, so the selected base's subsequent cost lookup normally hits the cache.
• select refuses a base whose own chain exceeds either budget before acquisition. Otherwise it acquires and authenticates the base and applies the parent's prospective test unchanged: acquisition counters plus the dependent's canonical length against both budgets.
• Both budget refusals use refused_by_budget, preserving the counter, independent-record result, candidate insertion and depth-zero accounting.
• EncodedRecord::charged_width records the payload width after assembly. Compact whole-file assembly drops eight bytes of length fields; chunk and singleton records keep their framed widths. This helper is not an ordinary-group accounting function.
• Pooled metadata retains its separate reader accounting. Its cost-walk edge supplies zero encoded width, while selection continues using the pool reader's encoded total.
• Storage architecture section 6.8 and the delta handbook's selection diagram and chain-budget explanation describe the two checks.

The budgets, candidate ordering, resolver checks and strict > boundary remain unchanged. An exact fill is admitted. The prospective check still uses authenticated acquisition counters; the preliminary cost walk does not add a second charge to them.

Canonical identities, stored formats and the JavaScript SDK method surface are unchanged. The Rust ChainCost and DepthCache source surface changes, and the in-tree callers are updated.

Authentication boundary

A selected base whose measured own-chain costs fit both budgets is acquired and authenticated before prospective-budget refusal. Payload corruption that changes the reconstructed identity therefore still fails with Integrity("dependency identity"), including when the dependent would push the chain over a budget. Other errors encountered during acquisition also propagate.

An over-budget base is neither reconstructed nor canonically authenticated. Establishing its cost still reads and parses stored records, and failures encountered during that walk propagate.

This is not a guarantee of identical error precedence for every malformed over-budget chain. In a positive-depth chain, the parent could encounter identity or decoding corruption before reaching the cumulative budget crossing. The early refusal can skip that error, as well as resolver-only structural checks. The producer does not emit over-budget PREFIX chains under the same limits, but corruption can create malformed stored chains. The new object is stored independently and does not depend on the refused base.

Missing advisory candidates retain the existing eligibility behavior; this change does not make every missing candidate or descendant a Save error.

Tests

core/crates/layerfs-storage/tests/delta_base_budget.rs adds twelve cases.

Five regressions fail on parent production code:

• A 300,000-byte incompressible base exceeds the encoded budget.
• A 600,000-byte compressible base exceeds the canonical budget.
• A 10,000-byte dependent still refuses an oversized base.
• Canonical base-boundary pair: 524,265 payload bytes fit alone; 524,266 do not.
• Encoded base-boundary pair: 262,135 payload bytes fit alone in the singleton representation; 262,136 do not.

The oversized-base cases assert independent storage, work_exceeded = 1, no delta trial and complete canonical readback.

Controls and guards cover exact prospective fills and one-byte excesses, equivalent refusal boundaries within one Save and across separate Saves, ordinary chaining, disabled whole-file deltas, and two corruption cases.

The corruption tests flip a stored payload byte and recompute the pack digest. One pair fits both prospective budgets. In the other, the base fits alone but the dependent exceeds the prospective encoded limit:

base payload:                  131,056 bytes
base encoded record:           131,057 bytes
base canonical object:         131,079 bytes
dependent canonical object:    131,088 bytes
prospective encoded charge:    262,145 bytes
encoded limit:                 262,144 bytes

Both corruption guards pass on parent production code and this change. With 1165ad746 production code, the second guard fails because Save silently returns an independent FULL result.

engine/crates/layerfs-api/tests/sdk_large_file_budget.rs adds five cases:

Case	Route and assertions
fs.apply, second 600,000-byte text version	Rust method-table call; size and changed line
fs.write, second 300,000-byte incompressible version	Rust method-table call; Committed result and size; first version installed through fs.apply
Workspace Commit, ten-byte edit	Private fixture Workspace and captured producer; size and changed line; no FUSE mount
Init, two similar 600,000-byte files	Direct Project Init; both sizes
Default-cutoff control	Same text fs.apply pair; size

The fixture gains built_with to select the Store policy and retains explicit Disposable persistence. These tests do not provide complete byte-for-byte SDK readback or JavaScript-package integration coverage.

Validation

I ran these checks outside the sandbox on Google Colab Linux, x86_64 with 8 vCPUs, using Rust 1.85.1 and LAYERFS_CONSTRUCTION_WORKERS=1. Persistent Stores used Disposable.

Rust test targets were built with --no-run before execution. Focused tests ran under 100-second timeouts, the storage reproduction under 30 seconds, and nextest selections under 120 seconds.

Check	Parent production, caebd268d	This change
layerfs-storage --test delta_base_budget	7 passed, 5 failed	12 passed
layerfs-api --test sdk_large_file_budget	1 passed, 4 failed	5 passed
Issue's a_delta_budget_repro	2 passed, 3 failed	5 passed
Local SDK reproduction target	4 passed, 3 failed; 11/24 grid writes refused	7 passed; 0/24 refused
Storage nextest, 28 binaries	85 passed, 5 failed	90 passed
Persistence nextest, 35 binaries	99 passed, 1 skipped	99 passed, 1 skipped
Fast lane, 239 binaries	1,081 passed, 10 failed, 11 skipped; one binary skipped	1,090 passed, 1 failed, 11 skipped; one binary skipped

The crate runs used:

timeout 120 cargo +1.85.1 nextest run --locked -p layerfs-storage --no-fail-fast
timeout 120 cargo +1.85.1 nextest run --locked -p layerfs-persistence --no-fail-fast

The fast lane used:

timeout 120 cargo +1.85.1 nextest run --locked --workspace --exclude layerfs-node \
  $(python3 -B tools/ci_tests.py targets fast) --profile ci \
  -E "$(python3 -B tools/ci_tests.py filter fast)" --no-fail-fast

The remaining failure on both arms is layerfs-api::local_install import_installs_the_store_in_process. It reports that filesystem 0x794c7630 does not carry SQLite locks between kernels and that direct Store access requires ext4, xfs or btrfs. The fast lane is not green. The parent's other nine failures are the five storage and four SDK regressions.

Additional checks:

• cargo +1.85.1 clippy --locked -p layerfs-storage --all-targets -- -D warnings: clean.
• The same scoped Clippy command for layerfs-api: clean.
• cargo +1.85.1 fmt --manifest-path Cargo.toml --all -- --check: clean.
• timeout 110s python3 -B tools/check_product_boundary.py --skip-sdk: PASS on archived parent and final snapshots.
• timeout 110s python3 -B -m unittest discover -s tools -p 'test_*.py': 155 passed.
• Fuser integrity check: authorized patch verified; registry-package verification was not performed.

The earlier boundary-guard run in /root/lfs/src reports violations in unchanged codec.rs and file_control.rs. That failure is retained separately; clean archived snapshots pass.

Two throwaway boundary targets, not included in this PR, compare parent production code and this change. After two 100,000-byte versions, the third version is refused from 162,059 bytes on both, within one Save or across separate Saves. Across 26 size pairs, outcomes agree except for five parent budget errors corrected here.

A separate throwaway mutation replaced charged_width() with width() for newly selected PREFIX cache entries. All twelve storage tests still passed. I retain charged_width to keep cached payload costs equal to stored-record costs; the current tests do not distinguish that adjustment.

I did not measure Save time. Mounted FUSE Commit, macOS and Windows were not run. Durable: NOT_RUN — disabled by owner until explicit reauthorization.

Source size and related work

Production LOC: 102,200 -> 102,253 (delta +53)

All growth is in layerfs-storage: 8,993 -> 9,046. Core totals are 38,241 -> 38,294; engine remains 61,313 and API remains 2,646.

Reference absent (retired R9c; last counted 65,417 at tree 498dd1917812).

Method: tools/production_loc.py, blob 3341e4d0362c822b6d9a08419c432942b3a13800, with --root <snapshot> --detail, on archived caebd268d and that parent with the final patch applied. The retained final patch matches the committed tree. Tests, documentation and tooling are excluded.

The retained search found no matching open duplicate; it included open roadmap #446. Closed #228 fixed charging unrelated grouped bytes and retained these budgets. #415 also edits 05-storage.md, which is a possible textual conflict.

Fixes #472

AI disclosure: this was prepared with AI assistance (Claude Opus 5.5).

🤖 Generated with Claude Code

https://claude.ai/code/session_019TTGAv6fe859ebbCLLfLHP