Skip to content

fix(apply): close output fd synchronously to avoid ETXTBSY on subsequent spawn - #37

Merged
BYK merged 2 commits into
mainfrom
fix/applyReaderToFile-fd-leak
Jul 31, 2026
Merged

fix(apply): close output fd synchronously to avoid ETXTBSY on subsequent spawn#37
BYK merged 2 commits into
mainfrom
fix/applyReaderToFile-fd-leak

Conversation

@BYK

@BYK BYK commented Jul 31, 2026

Copy link
Copy Markdown
Owner

Problem

applyPatchChainInMemory (and downstream consumers like
getsentry/cli) chain the apply
output immediately into a spawn to run the patched binary. On
Linux this intermittently fails with ETXTBSY ("text file busy")
even after writer.end() resolves — Node's writable stream fires
its end callback on the 'finish' event (data flushed), but
the underlying fd can still be held at the kernel level when the
caller reaches the spawn line.

fs.writeFile does not exhibit the bug because it closes the fd
synchronously before returning, which is why the issue only surfaced
in the field once a fallback path used a different API.

Fix

applyReaderToFile now writes via fs.openSync + fs.writeSync

  • fs.closeSync instead of fs.createWriteStream / writer.end.
    The fd is guaranteed to be released before the function returns, so
    the caller can spawn the output file without any race.

Trade-off: we lose WriteStream's 1 MiB highWaterMark buffering, so
write syscall count scales with the bspatch chunk size (typically
16–64 KiB). Linux's write path coalesces adjacent small writes
effectively for sequential output, so the cost is bounded;
correctness is the property that matters here.

Regression test

test/bspatch.test.ts adds a case that runs applyPatchChainInMemory
then verifies no fd in /proc/self/fd points at the output file
(Linux-only check; skipped elsewhere). The bug is intermittent on
the spawn path itself, but the fd-not-leaked invariant is the
property the fix guarantees and is the directly testable one.

Documentation

While investigating I confirmed binpatch itself is correct — it
returns the SHA, throws on mismatch, falls back. Two related findings
are now documented because they're easy to miss:

  1. from-version annotation is a chain pointer, not a content
    hash.
    binpatch only verifies the final output SHA; it cannot
    detect a publish pipeline that records from-version: A but
    generated the patch bytes from B. The CI workflow that
    publishes patches MUST pass the version from the generate step,
    not recompute it independently (this is exactly what bit
    getsentry/cli — see
    getsentry/cli#1327).
    New Security → 'from-version' annotation trust section +
    callout in Wire Contract.
  2. Synchronous output-fd release is a documented guarantee. New
    Security → 'Output fd release' section explaining why the
    stream API was wrong and what the new contract is.

Files

  • src/bspatch.ts — replace createWriteStream with
    openSync/writeSync/closeSync in applyReaderToFile.
  • test/bspatch.test.ts — regression test verifying no fd points
    at the output after apply returns.
  • website/src/content/docs/security.md — new sections:
    from-version annotation trust, Output fd release; threat
    model row for compromised publish pipeline.
  • website/src/content/docs/wire-contract.md — annotation callout
    in the OCI patches table.
  • CHANGELOG.md[Unreleased] > Fixed and Documentation
    entries.

…ent spawn

Node's `fs.createWriteStream(path).end()` callback fires on the
`'finish'` event — which signals data has been flushed, NOT that the
underlying fd has been released at the kernel level. On Linux, a
subsequent `spawn` of the output file (which calls `execve`) then
fails with `ETXTBSY` ("text file busy") in a small but reproducible
race window.

Switch `applyReaderToFile` from the stream API to
`fs.openSync` + `fs.writeSync` + `fs.closeSync`, which closes the
fd synchronously before the function returns. The caller can now
`spawn` (or otherwise `execve`) the output file immediately without
racing the kernel's fd-release path.

Also documents two related findings from this investigation:
- The OCI patch manifest's `from-version` annotation is a chain
  pointer, not a content hash — `binpatch` only verifies the final
  output SHA. A publish pipeline that records a `from-version` that
  doesn't match the actual patch bytes will silently bypass the user's
  expected upgrade path. The CI workflow that publishes patches MUST
  pass the version from the generate step, not recompute it.
- The synchronous output-fd release is now a documented guarantee.
@github-actions

github-actions Bot commented Jul 31, 2026

Copy link
Copy Markdown
Contributor
PR Preview Action v1.8.1
Preview removed because the pull request was closed.
2026-07-31 17:44 UTC

Adversarial review findings:

- S1 (error precedence): the original stream-based code surfaced the
  'finish'-callback error over any prior writeError (`err ?? writeError`).
  The initial switch to openSync+closeSync silently inverted this:
  writeError won because the close path's catch was guarded by
  `if (!writeError)`. Restored the old precedence — closeError now
  wins (it's typically the more diagnostic EIO at flush time).
  Throw closeError immediately instead of stashing into writeError.

- S2 (doc determinism framing): the original commit message and
  security.md phrased the ETXTBSY race as 'enough to' / 'the bug only
  surfaced'. Softened to reflect that the race is intermittent
  (~5-50% in standalone Node repros) and depends on write pressure.

- S3 (stale highWaterMark docstring): the top-of-file block in
  src/bspatch.ts claimed 'written incrementally to disk via
  openSync+writeSync with a large highWaterMark to collapse
  thousands of small write syscalls' — but the new code does NOT use
  highWaterMark. Reworded to describe the actual contract
  (synchronous closeSync) instead.

- N5 (unreachable hash): added a comment near the `return
  hasher.digest('hex')` line noting it's unreachable on writeError
  (writeError implies we threw out of the loop early, never reaching
  the digest).

- B1 (regression test in vitest): the test now spawns the output 20×
  times in a loop, which is a stronger contract assertion (applies +
  spawns without ever hitting ETXTBSY). Honest comment added that the
  bug does NOT reliably reproduce inside vitest's microtask scheduler
  (~0% in vitest, ~5-50% in standalone Node repros), so the test is
  a contract assertion that passes with the fix; the user's actual
  upgrade runs in a Node SEA binary where the bug does reproduce. The
  fix itself (synchronous closeSync) is unambiguously correct
  regardless.

Tests: 58 passed across 3 files. Build, typecheck, lint (no lint
script) all green.
@BYK
BYK merged commit 534cd3c into main Jul 31, 2026
3 checks passed
@BYK
BYK deleted the fix/applyReaderToFile-fd-leak branch July 31, 2026 17:44
@craft-deployer craft-deployer Bot mentioned this pull request Jul 31, 2026
2 tasks
BYK added a commit to getsentry/cli that referenced this pull request Jul 31, 2026
Pulls in the synchronous output fd release from
[BYK/binpatch#37](BYK/binpatch#37) —
`applyReaderToFile` now closes the output fd via `fs.closeSync`
before returning, eliminating the ETXTBSY race that broke the
delta-upgrade → full-download fallback on Linux.

Verified locally:
- `pnpm --filter sentry run typecheck` — clean
- `pnpm --filter sentry run test:unit` — 8754 passed | 15 skipped

Also depends on
[BYK/binpatch#38](BYK/binpatch#38) (adds the
github target to .craft.yml — internal change, no runtime impact).
BYK added a commit to getsentry/cli that referenced this pull request Jul 31, 2026
…hronously (#1327)

Two related fixes for the self-upgrade flow on nightly and stable
channels.

## 1. CI workflow annotation mismatch (SHA mismatch on delta upgrade)

`generate-patches` and `publish-nightly` independently recomputed
`PREV_TAG` from `oras repo tags`. They could — and did — disagree: for
the published `patch-0.40.0-dev.1785511994` image, `generate-patches`
generated bytes against `0.41.0-dev.1785504113` (the semver-previous in
`sort -V` order), but `publish-nightly` stamped
`from-version: 0.40.0-dev.1785506255` in the annotation. `binpatch`
verifies only the final output SHA, so this silently bypassed the user's
expected upgrade path and produced a SHA mismatch on apply — the
patched binary's SHA didn't match the annotation, so the delta failed
and the upgrade fell back to full-download.

Fix: thread `PREV_TAG` through as job outputs. `generate-patches`
declares `outputs.prev-tag` (nightly main) and
`outputs.prev-release-tag`
(stable release branches) and writes them via the step IDs that already
compute the tag. `publish-nightly` reads the value via
`${{ needs.generate-patches.outputs.prev-tag }}` and strips the
appropriate prefix (`nightly-` for main, `v` for release tags — bash
parameter expansion is a no-op when the prefix doesn't match).

Verified by inspecting the published `:patch-0.40.0-dev.1785511994`
manifest on GHCR: `from-version` annotation was
`0.40.0-dev.1785506255` but the patch bytes apply correctly only to
the `0.41.0-dev.1785504113` binary. After the fix, the annotation
will match the actual source for every future build.

## 2. ETXTBSY on full-download fallback

`streamDecompressToFile` (the .gz streaming path used for nightly
manifest blobs and GitHub Release .gz assets) used
`fs.createWriteStream` + `await writer.end()`. Node resolves the
`end` callback on the `'finish'` event (data flushed) but the
underlying fd release is async — a subsequent `spawn(tempPath)` then
intermittently failed with `ETXTBSY` ("text file busy"). The
`fs.writeFile` raw fallback path didn't exhibit the bug because it
closes the fd synchronously before returning, which is why direct
full-version downloads worked but the delta fallback didn't.

Fix: replace `createWriteStream` with `fs.openSync` + manual
`writeSync` + `fs.closeSync` so the fd is released before the
function returns. Same pattern as the binpatch fix in
[BYK/binpatch#37](BYK/binpatch#37).

Also drops the stale 'Bun.file().writer() workaround for Bun
event-loop GC bug' docstring — the project migrated to Node, the
referenced Bun issue no longer applies.

## Verification

- `pnpm --filter sentry run typecheck` — clean
- `pnpm --filter sentry run test:unit` — 8754 passed | 15 skipped
- Manual reproduction: `sentry cli upgrade nightly` on
  `/home/byk/.local/bin/sentry` (currently at
  `0.40.0-dev.1785506255$) reproduced both errors before the fix;
  will re-verify against the merged workflow once a new nightly with
  the corrected `from-version` lands.

## Files

- `.github/workflows/ci.yml` — `generate-patches` declares
  `outputs.prev-tag`/`prev-release-tag`, writes them via
  `set-prev-tag`/`set-prev-release-tag` step IDs;
  `publish-nightly` reads from outputs instead of recomputing.
- `packages/cli/src/lib/upgrade.ts` — `streamDecompressToFile`
  rewritten with synchronous fd close.
BYK added a commit that referenced this pull request Aug 1, 2026
Closes #40. The `publish-ghcr` mode re-derived the previous version from
the registry independently of the `generate-ghcr` step that actually
downloaded the source binary and ran bsdiff. If the registry state
changed between the two steps, the two values could disagree and
`binpatch` consumers would silently hit SHA mismatch (the library only
verifies the final output SHA, not the source binary identity — the
annotation is a chain pointer, not a content hash).

## Fix

Add a new `from-version` input on `publish-ghcr`. When set, the
annotation on the pushed patch manifest uses THIS value verbatim. When
empty (back-compat), the old registry re-derive runs and prints a hint
suggesting the caller thread the value from `generate-ghcr`.

## Consumer wiring

```yaml
- uses: BYK/binpatch/action@vX.Y.Z
  id: gen
  with: { mode: generate-ghcr, ... }

- uses: BYK/binpatch/action@vX.Y.Z
  with:
    mode: publish-ghcr
    from-version: ${{ steps.gen.outputs.from-version }}
```

## Files

- action/action.yml: add `from-version` input, thread to push_patches
via env, use input when set else re-derive
- action/README.md: document the new input and the gen→publish wiring

## Relationship to other fixes

- #37 — fd-close fix in applyReaderToFile (released as
0.4.0)
- getsentry/cli#1329 — same-series filter in custom workflow (now
merged)
- getsentry/cli#1327 — CI workflow annotation threading (now merged)

Combined, the chain-side fixes (binpatch fd-close + getsentry/cli CI
workflow) and this action-side fix mean the consumer can stop running
its own hand-rolled patch generation/publish and consume the binpatch
action directly.

## Verification

- pnpm test: 58 passed
- pnpm build: clean
- pnpm typecheck: clean
- pnpm lint: (no script)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant