Skip to content

fix(content): keep text a merge writes off a new file's other links - #471

Merged
yifanxuaaa merged 2 commits into
Ephemeral-AI-Lab:mainfrom
Jackmeson1:fix/merge-hardlink-alias
Oct 27, 2026
Merged

yifanxuaaa merged 2 commits into
Ephemeral-AI-Lab:mainfrom
Jackmeson1:fix/merge-hardlink-alias

Conversation

@Jackmeson1

@Jackmeson1 Jackmeson1 commented Oct 27, 2026 •

Copy link
Copy Markdown
Contributor

Summary

A merge that synthesized text at one name of a new hard-linked file also assigned those bytes to its untouched aliases. Preserve the original file at surviving bindings and give the synthesized value a separate carrier.

This fixes successful Branch merges that changed an untouched alias, and conflicted Workspace Commits whose locally installed root put conflict markers at a clean alias. A conflicted Commit still publishes no Commit.

Fixes #470

Reproduction and cause

Issue #470 contains the standalone repro_hardlinkmerge.rs.

Outside-sandbox Linux validation on caebd268d reproduced both failures:

  • branch.merge(branch=ours, with=theirs) put OUR_LINE1\nline2\nTHEIR_LINE3\n at /uncontested.txt, although its expected bytes were OUR_LINE1\nline2\nline3\n.
  • A conflicted Workspace Commit installed markers at /clean_alias.txt, whose expected bytes were our new file\n. It reported one conflict at /conflict.txt, history=None, and written=["/clean_alias.txt"].

The reverse Branch merge preserves the alias before and after this change. This example does not imply general merge symmetry.

At caebd268d, core/crates/layerfs-content/src/merge/build.rs:197 reused a source serial absent from into for synthesized content. merge/serial.rs:400 then suppressed the original new-file value because that source had an assignment. Every surviving alias still bound the same serial and therefore read the synthesized bytes.

Change

  • Pass EntryFacts to Plan::assign, so it can inspect the source value's link count.
  • When the source serial is absent from into and its source link count exceeds 1, allocate a fresh serial for a new synthesized result. An existing equal assignment is still reused.
  • In emit_change, check whether the original serial itself has a planned value (valued), rather than whether any carrier was assigned for that source. Emit the original value when surviving names still require it.
  • Remove the now-unused Plan::assigned method.
  • Update 81-compare-and-merge.md.

The link-count predicate is conservative: the synthesized result receives a fresh serial even if other decisions eventually remove every original binding. Existing collide/drop_new/unlinked accounting drops the original source once all its links are lost, before its value is emitted. Final stored link counts are derived from the selected bindings for both the original and synthesized carriers.

Single-link sources absent from into retain the existing reuse behavior. The existing rules for a serial already present in into, and decisions that select an existing entry instead of synthesizing bytes, are unchanged.

Construction continues through the existing merge records and backed filesystem update. This change adds no whole-tree traversal or resident collection.

Tests

Four external tests are added to layerfs-api::merge_by_serial using the existing Workspace fixture:

  • text_merged_at_one_link_of_a_new_file_leaves_its_other_link_alone: fails on base because /uncontested.txt contains the other side's line-3 edit. Checks both contents, separate link groups and stored counts of 1.
  • text_merged_at_one_link_of_a_new_file_leaves_its_other_link_alone_in_the_other_order: reverse-order control; passes before and after.
  • a_new_file_with_every_name_merged_keeps_none_of_its_own_bytes: all-original-bindings-lost control; passes before and after. Its two variants check different synthesized values as separate files with counts of 1, and equal assigned values as one file with count 2.
  • a_conflict_at_one_link_of_a_new_file_leaves_its_other_link_alone: fails on base because /clean_alias.txt contains markers. Checks the locally installed root, stored counts, unchanged Branch contents, history=None, the single conflict, and written=["/conflict.txt"].

support/view.rs gains stored-count and text readers plus a public hex helper. No test is deleted.

Validation

Outside-sandbox validation was run on Colab CPU, Ubuntu 24.04.5 LTS, Linux 6.6.159+, x86_64, with Rust/Cargo 1.85.1 and cargo-nextest 0.9.150. The root filesystem was overlay; the mount selections ran as root with /dev/fuse present. Stores used Disposable and LAYERFS_CONSTRUCTION_WORKERS=1.

Base is caebd268dc376865e964f33f78047f7918763190. The final targeted run records patch SHA-256 1dff2c642df1f670a8fa2715397d7497971067c3ddefd48804aec434061d5e9a, matching this change's retained patch.

Rust targets were built before execution. Recorded test invocations had explicit wall timeouts of at most 120 seconds.

Selection Base Fix
merge_by_serial, with the added test files 17 passed, 2 failed: the clean-alias and conflict-alias regressions above 19 passed
Issue's standalone repro_hardlinkmerge 2 failed 2 passed
Content/API fast lane 584 passed, 1 failed, 8 skipped 588 passed, 1 failed, 8 skipped
Selected slow/heavy Content/API targets Selected tests passed Selected tests passed
Four selected API mount-lane targets 13 passed 13 passed
Ten selected daemon targets 40 passed, 1 failed 40 passed, 1 failed
Formatting and scoped warning-denying Clippy Clean Clean
check_product_boundary.py --skip-sdk 2 violations Same 2 violations
Fuser integrity check and tool unit tests Check passed; 155 tests passed Check passed; 155 tests passed

The fast-lane failure on both sides is layerfs-api::local_install::import_installs_the_store_in_process: direct Store installation refuses this environment's overlay filesystem and requires ext4, xfs or btrfs.

The daemon failure on both sides is mounted_commit::r5_9_stores_through_shared_mappings_are_committed_once_written_back, with PermissionDenied at engine/crates/layerfs-daemon/tests/mounted_commit.rs:758.

The boundary violations on both sides report unsafe outside the audited module at:

  • core/crates/layerfs-storage/src/encoding/codec.rs:1
  • engine/crates/layerfs-persistence/src/backend/sqlite/file_control.rs:1

The fuser check verified the authorized patch; it did not verify an installed registry package.

The slow/heavy selections were:

  • API: scale_lineage_cap (3), sdk_merge_scale (2 passed, 9 skipped), history_ancestry_oracle (1), history_diff_oracle (1), sdk_diff_entries (2), sdk_list_entries (1), sdk_text_scan (1), mount_control (9).
  • Content: filesystem_sorted_localized (5).

The API mount-lane selections were kernel_mount_routes (2), kernel_application (1), observed_application (9), and r5_1_actual_daemon_commit (1).

The daemon selections were mounted_links (1), mounted_commit (9), mounted_install (4), r5_5_install_survival (4), mounted_parallel_commits (1), mounted_failure_scope (2), mounted_low_water (3), negative_entries (2), store_commit (10), and store_reader_lifecycle (5).

Core commands, from the repository root:

export LAYERFS_CONSTRUCTION_WORKERS=1

cargo +1.85.1 test --locked -p layerfs-api --test merge_by_serial --no-run
timeout 120 cargo +1.85.1 test --locked -p layerfs-api --test merge_by_serial -- --test-threads=1

# After placing the issue's reproducer in the API tests directory:
cargo +1.85.1 test --locked -p layerfs-api --test repro_hardlinkmerge --no-run
timeout 60 cargo +1.85.1 test --locked -p layerfs-api --test repro_hardlinkmerge -- --nocapture --test-threads=1

cargo +1.85.1 test --locked -p layerfs-content -p layerfs-api --tests --no-run
fast="$(python3 -B tools/ci_tests.py filter fast)"
timeout 120 cargo +1.85.1 nextest run --locked -p layerfs-content -p layerfs-api \
  --tests --profile ci --no-fail-fast \
  -E "($fast) & (package(layerfs-content) | package(layerfs-api))"

cargo +1.85.1 fmt --manifest-path Cargo.toml --all -- --check
cargo +1.85.1 clippy --locked -p layerfs-content -p layerfs-api --all-targets -- -D warnings
python3 -B tools/check_product_boundary.py --skip-sdk
python3 -B tools/check_fuser_integrity.py
timeout 120 python3 -B -m unittest discover -s tools -p 'test_*.py'

These are scoped results, not an all-green workspace claim. Durable execution was NOT_RUN — disabled by owner until explicit reauthorization.

Coverage limits

The ordinary Workspace merge and the in-lane step share merge_roots and Plan::assign, but the added hard-link tests do not force the in-lane step. They also do not check a mounted alias's cached st_nlink after this conflicted install; they verify the installed root's stored counts.

The standalone issue reproducer and the scratch randomized differential target are not included in this PR. No new performance or unrestricted resident-memory qualification is claimed.

After the merge of main (303246deb)

Main at 303246deb (after #445) was merged into this branch in 7f1c824a7; the only conflict was in merge_by_serial.rs, where #445 and this change both appended tests. I checked an equivalent tree, this change rebased onto 303246deb, which differs from 7f1c824a7 only in where #445's a_one_sided_mode_change_is_kept sits in that file. On it, on the same kind of Colab CPU host: cargo fmt --all -- --check passes, merge_by_serial runs 27 tests and all pass, the selected slow/heavy Content/API targets pass, and the Content/API fast lane gives 612 passed, 1 failed, 8 skipped. The failure is local_install::import_installs_the_store_in_process, which reports that the host's overlay root filesystem does not carry SQLite locks; the same test failed the same way on the base in the runs above.

Related work

Duplicate-search snapshot: saved searches at 2026-10-27 04:43:26 UTC and 05:20:12 UTC, refreshed at 06:21:25 UTC, found no matching item for these synthesized-content alias failures. At that snapshot, #409 was merged, #418 contained lost-link tests, and #419 addressed replaced directories.

The saved #418/#419 diffs overlap the test insertion area and use the API test path from before the layer relocation. #419 also touches nearby merge code and document 81. Neither changes Plan::assign or this emission guard; integration may still need conflict resolution.

Production LOC: 102,200 -> 102,199 (delta -1)

Counted with tools/production_loc.py, blob 3341e4d0362c822b6d9a08419c432942b3a13800, using --root <snapshot> --json on the parent archive and the parent with the final patch applied. Core changes from 38,241 to 38,240; layerfs-content from 25,978 to 25,977. Engine 61,313 and API 2,646 are unchanged; the dsh scope remains absent. Against 303246deb, the same count gives 102,528 -> 102,527 (delta -1): core 38,335 -> 38,334, layerfs-content 26,058 -> 26,057.

Reference absent (retired R9c; last counted 65,417 at tree 498dd1917812).

AI disclosure: this was prepared with AI assistance (Claude Opus 5.5).

🤖 Generated with Claude Code

https://claude.ai/code/session_019TTGAv6fe859ebbCLLfLHP

Jackmeson1 and others added 2 commits October 26, 2026 23:49
A clean Branch merge could replace the bytes of an untouched alias with
text synthesized for another name. A conflicted Workspace Commit could
likewise install conflict markers at a clean alias, while publishing no
Commit.

At caebd26, Plan::assign reused a source serial absent from into for the
written value. emit_change then suppressed the original new-file value
because the source had an assignment. Surviving aliases still bound that
serial and therefore read the synthesized bytes.

Pass EntryFacts to assign. For a source absent from into with more than
one source binding, give a new synthesized result a fresh serial.
Existing equal assignments are still reused. Check valued in emit_change
so an assignment carried by another serial does not suppress the
original value. Remove the unused Plan::assigned method.

The source link-count check is conservative: a fresh carrier is also used
when later decisions lose every original binding. Existing drop_new and
unlinked accounting suppresses the original source once all its links
are lost. The filesystem update derives both carriers' stored link counts
from their bindings.

Update 81-compare-and-merge.md and add four external API tests. The clean
merge and conflicted-install alias tests fail before the fix. The reverse
merge and all-original-bindings-merged cases are controls that pass
before and after. Add stored-count and text readers to support/view.rs.
No tests are deleted.

Outside-sandbox validation on Colab CPU, Ubuntu 24.04.5 LTS, Linux
6.6.159+, x86_64, Rust/Cargo 1.85.1, Disposable Stores:
- merge_by_serial: base plus tests 17 passed / 2 failed; fix 19 passed.
- Standalone issue reproducer: base 2 failed; fix 2 passed.
- Content/API fast lane: base 584 passed / 1 failed / 8 skipped;
  fix 588 passed / 1 failed / 8 skipped. Both fail the same local_install
  case because the environment's overlay filesystem is refused.
- Formatting and scoped warning-denying Clippy pass. The boundary guard
  reports the same two violations on both sides; tool tests pass.

The added cases do not force an in-lane hard-link merge or inspect a
mounted alias's cached st_nlink after conflicted installation. The
standalone reproducer and scratch differential target are not committed.
Durable execution: NOT_RUN — disabled by owner until explicit
reauthorization.

Production LOC: 102,200 -> 102,199 (delta -1)
  core 38,241 -> 38,240 (-1): layerfs-content 25,978 -> 25,977 (-1).
  engine 61,313 and api 2,646 unchanged; dsh scope absent.
  Reference absent (retired R9c; last counted 65,417 at tree 498dd19).
  Method: tools/production_loc.py
  (blob 3341e4d)
  --root <snapshot> --json on an archive of parent caebd26 and that
  archive with the final patch applied. The retained patch matches the
  committed parent-to-HEAD diff. Tests, docs and tools are excluded.

Fixes #ISSUE

AI disclosure: this was prepared with AI assistance (Claude Opus 5.5).

Signed-off-by: Jackmeson1 <130693642+Jackmeson1@users.noreply.github.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019TTGAv6fe859ebbCLLfLHP
)

Main gained Ephemeral-AI-Lab#418/Ephemeral-AI-Lab#419 merge tests and Ephemeral-AI-Lab#445 since this branch's base
caebd26. The only conflict was engine/crates/layerfs-api/tests/
merge_by_serial.rs, where both sides appended tests after the same test
(a_new_file_is_dropped_once_every_link_is_lost). Resolved by keeping both:
this branch's helper changes, then main's added tests, then this branch's
added tests. No line of either side was changed. Production source merged
without conflict.

Production LOC: 102199 -> 102527 (delta +328)
  First-parent comparison (bdef701 -> this merge); the +328 is main's.
  Against main 303246d (102528) this branch is delta -1: core
  layerfs-content 26058 -> 26057. Merged scopes: api 2659, core 38334,
  engine 61534; reference absent (retired R9c; last counted 65,417 at tree
  498dd19). Counted with tools/production_loc.py (blob 3341e4d)
  --json --root on git archive extracts of the parent, the merged tree and
  main.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@yifanxuaaa
yifanxuaaa merged commit 924f782 into Ephemeral-AI-Lab:main Oct 27, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

A merge that writes text for one name of a new hard-linked file also overwrites the file's other, untouched name

2 participants