diff --git a/crates/socket-patch-cli/CLI_CONTRACT.md b/crates/socket-patch-cli/CLI_CONTRACT.md index 3756e7abf..5fcddd7ee 100644 --- a/crates/socket-patch-cli/CLI_CONTRACT.md +++ b/crates/socket-patch-cli/CLI_CONTRACT.md @@ -173,7 +173,7 @@ For a **9.0 root lock**, the CLI ensures `pnpm-workspace.yaml` carries `trustLoc **vlt hosted-mode contract**: `scan` / `get --mode hosted` rewrite, in `vlt-lock.json`, every default-registry node of a granted `name@version` (the `''` / `npm` segment or a URL segment equal to the lock's scalar `registry`, both DepID grammars, every peer and modifier variant): slot [2] becomes the granted sha512 and slot [3] the hosted URL (appended to a 3-tuple); the DepID, flags and trailing slots, the line ending and every other byte stay. `options` is never edited and `vlt.json` is only read. A lock with another `lockfileVersion` (decided on the raw JSON token), a BOM, a non-object body or a `nodes` section outside vlt's one-node-per-line layout refuses the whole lock (`redirect_vlt_lock_unsupported`). **Confirmation**: vlt drives when its install state (`node_modules/.vlt-lock.json` or `node_modules/.vlt/`) is present or no other npm-family lock is; then only `vlt-lock.json` confirms a uuid. Otherwise every lock is rewritten, `redirect_vlt_sibling_lockfiles` warns, and the other locks' rules confirm, including a dep `vlt-lock.json` merely does not wire (`redirect_vlt_entry_not_found`, `redirect_vlt_entry_vendored`). Whichever lock drives, a dep the vlt rewriter refuses (`redirect_vlt_missing_sha512`, `redirect_vlt_unsupported_lock_key`) is never confirmed by any lock, although a sibling lock may already carry its rewritten URL. **Artifact preflight**: before any takeover or write (dry runs included), each granted artifact with a default-registry instance is fetched once as vlt fetches it and must verify, else the dep is withheld (`redirect_vlt_artifact_unverifiable`, see the tag table). **Heal**: stale installed copies of Socket-owned nodes are removed so the next `vlt install` extracts the patched bytes, and `rollback` / `remove` do the same for the registry bytes (`--no-vlt-install-cleanup` keeps them; optional dependencies' copies are always kept); `redirect_vlt_reinstall_required` says what happened and what to run. The same-run `--vex` never attests a vlt package whose installed copy is stale or unchecked, whose lock a vlt release may ignore (`redirect_vlt_lockfile_version_missing`, `redirect_vlt_old_lockfile_ignored`, `redirect_vlt_scalar_registry_ignored`), or which also resolves from a non-default registry (`redirect_vlt_custom_registry_skipped`). `vlt.json` or vlt install state without `vlt-lock.json` warns `redirect_vlt_no_lockfile` instead of `redirect_npm_no_lockfile`. `rollback` / `remove` restore each hosted node's slots [2] and [3] from the version document of the registry the node resolves against (slot [3] is its `dist.tarball`, as vlt writes it; `upstream_registry_fallback` when that registry can't be read), following the lock's own slot-[3] convention (see "Hosted unwind coverage"). Tested releases: `docs/testing/vlt-compatibility.md`. -**Takeover reconciliation (every hosted ecosystem, v5.0)**: vendoring over a hosted pin (`vendor`, `scan --mode vendored`, `get --mode vendored`) first RESTORES that purl's lock entries to their default upstream registry entry — the same restore `rollback` runs (core `patch::redirect::upstream::restore_upstream`; see "Hosted unwind coverage"), over the hosted pins lockfile discovery finds (v5 keeps no hosted ledger) — and then vendors, so the vendor ledger records the PRISTINE registry entry as its wiring `original` and `vendor --revert` lands back on upstream registry state, never on hosted. The run that takes over records a `vendor_takeover_reverted_redirect` advisory event (`skipped` action beside the purl's genuine outcome; detail ` was hosted; restored its upstream registry entry () before vendoring (mode takeover)`; the human path prints `Warning: …`), plus any advisory the restore raised (`npm_allow_remote_left`, …). **A takeover the vendored backend does not carry through keeps the hosted pin (#853, #944)**: the wet run holds the restore in its group commit, and when the backend then refuses the purl — whatever the code: a pnpm `catalog:` dependency (`vendor_lock_entry_unsupported`), a CRLF `pnpm-lock.yaml` (`vendor_lockfile_crlf_unsupported`), a conflicting `pnpm-workspace.yaml` override — a range, another version or a `>` selector (`vendor_override_conflict`), a uv inline `[tool.uv] sources` table, a prebuilt download that fails, … — or its apply fails with nothing recorded, the restore is rolled back before anything reaches disk: the purl is reported `failed ` with the backend's own code and detail, neither `vendor_takeover_reverted_redirect` nor the restore's advisories are recorded for it, and the hosted wiring stays byte-for-byte (exit 1 / `partialFailure`), so the package stays hosted-patched instead of being un-hosted and then refused. `vendor --dry-run` previews that same `failed ` by running the backend's dry run over the restored project staged in memory (nothing written). A restore that writes a file outside the group commit's captured set (`.socket/gradle/hosted-index.tsv`) is not rolled back. The `scan` / `get --mode vendored --dry-run` preview, which stages no restore (it never touches the registry), predicts the pnpm (lockfileVersion 9) part of this (#853): a hosted pin the pnpm vendored backend refuses on lock or manifest text (a `catalog:` dependency, a CRLF `pnpm-lock.yaml` / `pnpm-workspace.yaml`, a conflicting override, …) is listed `would_refuse` with the wet run's `errorCode` and the backend's own `error` detail, evaluated on the still-hosted project (the pnpm restore only rewrites the entry's `resolution:`, which none of those gates reads); a hosted pin of any other lock flavor or ecosystem — a legacy pnpm 7/8 lock (`lockfileVersion` 5.4 / 6.0) included, whose CRLF, override-conflict and entry refusals are not lock-text gated — still previews `would_vendor` even when the wet takeover refuses it (the wet refusal keeps the hosted pin), except for the gem preflight below. **Gem preflight before the takeover**: `scan` / `get --mode vendored` ask the gem vendored backend's own refusals BEFORE the upstream restore — the manifest gate (`gemfile_not_loaded`: a `gems.rb` twin or a `BUNDLE_GEMFILE`-configured manifest) and the Gemfile declaration gate evaluated on the Gemfile and Gemfile.lock text the restore would leave (`gemfile_declaration_not_editable`: a declaration inside a `group` / `platforms` / conditional block, a parenthesized or duplicate declaration, …) — so a hosted gem vendored mode cannot wire is reported `failed ` with the hosted `Gemfile` / `Gemfile.lock` byte-untouched (exit 1 / `partialFailure`), and their `--dry-run` preview reports that gem as `would_refuse` with the same `errorCode` (exit 0, like the Bun / vlt `would_refuse` rows), never `would_vendor`. Pinned against real Bundler by `tests/e2e_redirect_gem_build.rs`. `--dry-run` resolves the same restore without writing (registry lookups included): a pin that would restore reports `vendor_would_revert_redirect`, and one that would be refused surfaces in the preview with the wet run's `redirect_revert_failed` code and detail (for bun, whose hosted rewrite replaces the entry's `name@version` spec, the preview first runs the Bun vendored preflight described below and then stops at the advisory instead of reading the still-hosted lock — a lock the vendored backend would refuse is previewed as the wet run's `failed `, never as `vendor_would_revert_redirect`). A purl whose upstream entry cannot be restored — `--offline`, a registry that does not answer, a lock the restore refuses (see "Hosted unwind coverage"; a hosted binary `bun.lockb` pin IS restored for the takeover — its npm registry record is rebuilt natively — while `rollback` / `remove` refuse it) — fails `redirect_revert_failed` with the detail `cannot vendor over the live hosted pin: cannot restore to its upstream registry entry: ; restore it from version control instead (`git checkout -- `)` (for `bun.lock` / `bun.lockb` the detail also adds `, then run \`bun install --force\` (a plain \`bun install\` keeps the patched copy)`) (exit 1 / `partialFailure`, nothing vendored for it, the hosted wiring left in place). The cargo backend's `hosted_redirect_live` refusal backstops a crate whose hosted residue is still in place when it is reached; its detail names `socket-patch rollback` and `git checkout -- Cargo.toml Cargo.lock`. **Bun vendored preflight before the takeover**: `vendor` — like `scan` / `get --mode vendored`, whose pre-download preflight runs earlier — checks `bun.lock` / `bun.lockb` with the shared Bun vendored preflight BEFORE the upstream restore, so a hosted purl on a lock the vendored backend refuses (a pre-version-2 `workspace:` lock → `vendor_bun_workspace_unsupported`; a malformed or unsupported binary lock → `vendor_bun_lockb_invalid`; an unsupported text-lock version → its code) is reported `failed ` with the hosted wiring and active Bun lock byte-untouched (exit 1 / `partialFailure`): the package stays hosted-patched instead of being un-hosted and then refused. `vendor --dry-run` previews that same `failed` code (exit-code parity with the wet run, nothing written) instead of promising `vendor_would_revert_redirect`. Pinned by `tests/in_process_vendor_bun_takeover.rs` and, against real Bun, `tests/mode_migration_bun.rs`. The npm package-lock backend's lock gate gets the same placement: a hosted pin in a project whose `npm-shrinkwrap.json` / `package-lock.json` is not a v2/v3 lock (npm 6's lockfileVersion 1) is refused `failed vendor_lockfile_version_unsupported` BEFORE the restore, in `vendor`, `scan --mode vendored` and `get --mode vendored` alike, so the package stays hosted-patched; the vendored dry-run preview lists every npm purl of such a project as `would_refuse` with that code. Pinned by `tests/in_process_vendor_npm_v1_takeover.rs`. Hosted → vendored and vendored → hosted (`redirect_takeover_reverted_vendored` in `warnings[]`) both work in place on the locks the target mode accepts. **Removed in v5.0**: the run-level `vendor_supersedes_redirect` warning and its reconcile of the redirect ledger (a live lock that already proved vendored won over a stale hosted ledger record) — once the lock routes a package to `.socket/vendor/`, no hosted state is left to go stale. Which way the live lock points is decided by the same lockfile discovery rules `vex` gates attestations on (see "Manifest-less VEX (lockfile discovery)"), for `redirect_supersedes_vendored` and `hosted_wiring_retained` alike. +**Takeover reconciliation (every hosted ecosystem, v5.0)**: plain `vendor` with at least one hosted pin in scope is not a per-purl takeover: it EJECTS exactly the lockfile's hosted pins as one planned transition (see "Vendor command contract") and ignores `.socket/manifest.json` even when it exists — a manifest-only patch is not vendored, and a manifest naming a different patch for a hosted purl does not displace the hosted one (vendoring the manifest's patches takes a full `scan --mode vendored`, whose edits then land in the lockfiles). The eject runs the same upstream restore and the same backend previews and refusals described below, but reports them as an eject (`eject_planned` / `eject_refused` / `eject_rolled_back`) rather than through the per-purl takeover advisories. Vendoring over a hosted pin with `scan --mode vendored` / `get --mode vendored` (and the legacy manifest-driven `vendor` path, reached only when no hosted pin is in scope) first RESTORES that purl's lock entries to their default upstream registry entry — the same restore `rollback` runs (core `patch::redirect::upstream::restore_upstream`; see "Hosted unwind coverage"), over the hosted pins lockfile discovery finds (v5 keeps no hosted ledger) — and then vendors, so the vendor ledger records the PRISTINE registry entry as its wiring `original` and `vendor --revert` lands back on upstream registry state, never on hosted. The run that takes over records a `vendor_takeover_reverted_redirect` advisory event (`skipped` action beside the purl's genuine outcome; detail ` was hosted; restored its upstream registry entry () before vendoring (mode takeover)`; the human path prints `Warning: …`), plus any advisory the restore raised (`npm_allow_remote_left`, …). **A takeover the vendored backend does not carry through keeps the hosted pin (#853, #944)**: the wet run holds the restore in its group commit, and when the backend then refuses the purl — whatever the code: a pnpm `catalog:` dependency (`vendor_lock_entry_unsupported`), a CRLF `pnpm-lock.yaml` (`vendor_lockfile_crlf_unsupported`), a conflicting `pnpm-workspace.yaml` override — a range, another version or a `>` selector (`vendor_override_conflict`), a uv inline `[tool.uv] sources` table, a prebuilt download that fails, … — or its apply fails with nothing recorded, the restore is rolled back before anything reaches disk: the purl is reported `failed ` with the backend's own code and detail, neither `vendor_takeover_reverted_redirect` nor the restore's advisories are recorded for it, and the hosted wiring stays byte-for-byte (exit 1 / `partialFailure`), so the package stays hosted-patched instead of being un-hosted and then refused. `vendor --dry-run` previews that same `failed ` by running the backend's dry run over the restored project staged in memory (nothing written). A restore that writes a file outside the group commit's captured set (`.socket/gradle/hosted-index.tsv`) is not rolled back. The `scan` / `get --mode vendored --dry-run` preview, which stages no restore (it never touches the registry), predicts the pnpm (lockfileVersion 9) part of this (#853): a hosted pin the pnpm vendored backend refuses on lock or manifest text (a `catalog:` dependency, a CRLF `pnpm-lock.yaml` / `pnpm-workspace.yaml`, a conflicting override, …) is listed `would_refuse` with the wet run's `errorCode` and the backend's own `error` detail, evaluated on the still-hosted project (the pnpm restore only rewrites the entry's `resolution:`, which none of those gates reads); a hosted pin of any other lock flavor or ecosystem — a legacy pnpm 7/8 lock (`lockfileVersion` 5.4 / 6.0) included, whose CRLF, override-conflict and entry refusals are not lock-text gated — still previews `would_vendor` even when the wet takeover refuses it (the wet refusal keeps the hosted pin), except for the gem preflight below. **Gem preflight before the takeover**: `scan` / `get --mode vendored` ask the gem vendored backend's own refusals BEFORE the upstream restore — the manifest gate (`gemfile_not_loaded`: a `gems.rb` twin or a `BUNDLE_GEMFILE`-configured manifest) and the Gemfile declaration gate evaluated on the Gemfile and Gemfile.lock text the restore would leave (`gemfile_declaration_not_editable`: a declaration inside a `group` / `platforms` / conditional block, a parenthesized or duplicate declaration, …) — so a hosted gem vendored mode cannot wire is reported `failed ` with the hosted `Gemfile` / `Gemfile.lock` byte-untouched (exit 1 / `partialFailure`), and their `--dry-run` preview reports that gem as `would_refuse` with the same `errorCode` (exit 0, like the Bun / vlt `would_refuse` rows), never `would_vendor`. Pinned against real Bundler by `tests/e2e_redirect_gem_build.rs`. `--dry-run` resolves the same restore without writing (registry lookups included): a pin that would restore reports `vendor_would_revert_redirect`, and one that would be refused surfaces in the preview with the wet run's `redirect_revert_failed` code and detail (for bun, whose hosted rewrite replaces the entry's `name@version` spec, the preview first runs the Bun vendored preflight described below and then stops at the advisory instead of reading the still-hosted lock — a lock the vendored backend would refuse is previewed as the wet run's `failed `, never as `vendor_would_revert_redirect`). A purl whose upstream entry cannot be restored — `--offline`, a registry that does not answer, a lock the restore refuses (see "Hosted unwind coverage"; a hosted binary `bun.lockb` pin IS restored for the takeover — its npm registry record is rebuilt natively — while `rollback` / `remove` refuse it) — fails `redirect_revert_failed` with the detail `cannot vendor over the live hosted pin: cannot restore to its upstream registry entry: ; restore it from version control instead (`git checkout -- `)` (for `bun.lock` / `bun.lockb` the detail also adds `, then run \`bun install --force\` (a plain \`bun install\` keeps the patched copy)`) (exit 1 / `partialFailure`, nothing vendored for it, the hosted wiring left in place). The cargo backend's `hosted_redirect_live` refusal backstops a crate whose hosted residue is still in place when it is reached; its detail names `socket-patch rollback` and `git checkout -- Cargo.toml Cargo.lock`. **Bun vendored preflight before the takeover**: `vendor` — like `scan` / `get --mode vendored`, whose pre-download preflight runs earlier — checks `bun.lock` / `bun.lockb` with the shared Bun vendored preflight BEFORE the upstream restore, so a hosted purl on a lock the vendored backend refuses (a pre-version-2 `workspace:` lock → `vendor_bun_workspace_unsupported`; a malformed or unsupported binary lock → `vendor_bun_lockb_invalid`; an unsupported text-lock version → its code) is reported `failed ` with the hosted wiring and active Bun lock byte-untouched (exit 1 / `partialFailure`): the package stays hosted-patched instead of being un-hosted and then refused. `vendor --dry-run` previews that same `failed` code (exit-code parity with the wet run, nothing written) instead of promising `vendor_would_revert_redirect`. Pinned by `tests/in_process_vendor_bun_takeover.rs` and, against real Bun, `tests/mode_migration_bun.rs`. The npm package-lock backend's lock gate gets the same placement: a hosted pin in a project whose `npm-shrinkwrap.json` / `package-lock.json` is not a v2/v3 lock (npm 6's lockfileVersion 1) is refused `failed vendor_lockfile_version_unsupported` BEFORE the restore, in `vendor`, `scan --mode vendored` and `get --mode vendored` alike, so the package stays hosted-patched; the vendored dry-run preview lists every npm purl of such a project as `would_refuse` with that code. Pinned by `tests/in_process_vendor_npm_v1_takeover.rs`. Hosted → vendored and vendored → hosted (`redirect_takeover_reverted_vendored` in `warnings[]`) both work in place on the locks the target mode accepts. **Removed in v5.0**: the run-level `vendor_supersedes_redirect` warning and its reconcile of the redirect ledger (a live lock that already proved vendored won over a stale hosted ledger record) — once the lock routes a package to `.socket/vendor/`, no hosted state is left to go stale. Which way the live lock points is decided by the same lockfile discovery rules `vex` gates attestations on (see "Manifest-less VEX (lockfile discovery)"), for `redirect_supersedes_vendored` and `hosted_wiring_retained` alike. **Withdrawn pins (`scan --mode hosted`, additive/MINOR)**: before the rewrite, a hosted scan checks every patch uuid the lockfiles pin, for a package this run covered, against the reference endpoint. A pin the server reports `withdrawn` (unpublished) or `not_found` (no such patch) no longer installs (the patch server answers 410), so it is restored to its default upstream registry entry: the same restore `rollback` runs (see "Hosted unwind coverage"), under the apply lock, `--dry-run` previewing it without writing. If the package has a replacement patch on offer, the rewrite that follows pins the replacement. Each restored pin is one `rolledBack` event (`verified` on a dry run) with `uuid`, `errorCode: "patch_withdrawn"` and `details.mode: "hosted"`; the restore's advisories (`redirect_bun_reinstall_required`, `redirect_pypi_reinstall_required`, `redirect_vlt_reinstall_required`) join `warnings[]`. Only those two statuses unpin: a pin merely absent from the search results (a failed batch, the free-only proxy fallback, a lapsed paid entitlement), or one reported `forbidden`, `pending_build` or `build_failed`, is kept. If the reference lookup fails, every pin is kept with a `redirect_withdrawn_check_failed` warning; a pin the restore refuses (or the apply lock being held) is kept with a `redirect_withdrawn_restore_failed` warning naming the remedy. Neither changes the status or the exit code. A run whose every search batch failed exits before the check. @@ -720,15 +720,15 @@ Not covered: the agent-mode blob cache and manifest are checked from `.socket` d ## Vendor command contract -`vendor` is `apply`'s committable sibling: instead of patching installed packages in place -(machine-local state), it ejects each patched package into `.socket/vendor/` and rewires the +`vendor` converts the hosted patches pinned in the project's lockfiles into committable +`.socket/vendor/` artifacts, without installing dependencies first. It rewires the ecosystem's lockfile/config so the project consumes the vendored copy. After committing `.socket/vendor/` + the lockfile edits, a fresh checkout builds with the patched dependency on machines with **no socket-patch installed and no Socket API access** (registry access for other, unvendored dependencies may still be needed). Every mechanism below was validated against the real package managers (`spikes/PHASE0-FINDINGS.txt`). -**Eject a hosted project (v5.0)**: standalone `vendor` with NO manifest in a project whose lockfiles +**Eject a hosted project (v5.0)**: standalone `vendor` in a project whose lockfiles pin hosted patches (not under `--global`; `--ecosystems` narrows the pins) takes its patch set from those pins — each pin's purl plus the patch uuid in its hosted URL — fetches each record from the patch API (`GET …/patches/view/`, the same client and public-proxy fallback as `get`), vendors @@ -736,15 +736,23 @@ into `.socket/vendor/` exactly like `scan --mode vendored`, and rewires each pac vendored (the upstream registry entry is restored first, so a later `vendor --revert` returns the project to upstream, never to hosted). The human output opens with `Ejecting N hosted package(s) into .socket/vendor/...` (`Would eject …` under `--dry-run`); the -JSON is the vendor envelope. It needs no installed `node_modules` / site-packages: the sources are -fetched from the upstream registry, so a fresh hosted checkout ejects. +JSON is the vendor envelope. Hosted pins take precedence even when `.socket/manifest.json` +exists: the manifest is neither used to select patches nor rewritten. The conversion skips +installed-package discovery and variant probes, so an unrelated virtualenv or global cache +cannot reject a lockfile-selected patch. It uses the shared lockfile readers, without VEX +attestation or installed-byte verification; downloaded artifacts still pass integrity and +patched-member verification. Upstream registry metadata is resolved for reversal, while the +patched archives come from the artifact service. No installed `node_modules`, site-packages, +or upstream source download is needed. When no hosted pins are in scope, legacy manifest-driven +vendoring remains available. The eject is ONE planned transition, all-or-nothing: (1) every record is fetched first — a failed (or 404) view fetch is a `failed` event with `errorCode: "patch_fetch_failed"` and the run stops with `status: "error"`, `error.code: "eject_refused"`, exit 1, nothing touched; (2) the upstream restore of every pin is resolved against staged copies — a refused pin (offline registry, a non-derivable field; a binary `bun.lockb` record is rebuilt like the takeover's) is `eject_refused` with the `git checkout -- ` -remedy, nothing touched; (3) `--dry-run` stops here and reports each pin as an `applied` event with +remedy, nothing touched. Backend refusals are also previewed against the restored text in memory +(for example, unsupported pnpm catalog entries); (3) `--dry-run` stops here and reports each pin as an `applied` event with reason `eject_planned` — no file is written and no `.socket/` is created; (4) the wet run snapshots every file the eject may touch under one `apply.lock`, restores upstream, then vendors. If any package then fails, exactly the files the eject wrote are put back (the pins' files, the upstream @@ -1475,7 +1483,7 @@ command's integration tests check its output. | `cargo_copy_untaggable` | `failed` (error prefix) | vendor / scan / get `--mode vendored` (cargo, v5.0): the copy's `Cargo.toml` has no literal `[package] version` string that can be rewritten byte-exactly (or it names another version); nothing is swapped in. A dry run over an already-vendored copy reports the same failure; a patch-service crate that cannot be tagged fails with `vendor_prebuilt_required`. | | `cargo_wiring_restored` | `skipped` (advisory note) | repair (v5.0): a vendored crate's Cargo.lock entry was detached with no Socket-owned `[patch]` pointing at its committed copy (a pre-v5 release overwrote its crate-named config key when a second version was vendored); the manifest entry is written back and the ledger updated (dry run: "would restore"). A `vendor` re-run heals the same state as a plain re-vendor. | | `cargo_manifest_unreadable` / `cargo_manifest_unparseable` / `cargo_manifest_not_workspace_root` / `cargo_manifest_patch_source_alias` | `failed` | vendor / scan / get `--mode vendored` (cargo, v5.0): the workspace-root `Cargo.toml` cannot carry the vendored `[patch.crates-io]` entry (or cargo would ignore it there) — see the cargo caveat under "Vendored mode". Refused before any write. | -| `vendor_would_revert_redirect` / `vendor_takeover_reverted_redirect` | `skipped` (advisory event) | vendor / scan / get `--mode vendored` over a hosted pin (every ecosystem, v5.0): dry run — the upstream restore was resolved (registry lookups included) and would succeed (for bun, only after the Bun vendored preflight accepted the lock; a refused lock is previewed as the wet run's `failed ` instead) / wet run — the pin's lock entries were restored to their upstream registry entry before vendoring (mode takeover; detail ` was hosted; restored its upstream registry entry () before vendoring (mode takeover)`), so `vendor --revert` later returns to upstream. Fires on the run that takes over, not on re-runs, and not for a purl whose takeover was rolled back because the backend refused it (see "Takeover reconciliation"). | +| `vendor_would_revert_redirect` / `vendor_takeover_reverted_redirect` | `skipped` (advisory event) | scan / get `--mode vendored` over a hosted pin (every ecosystem, v5.0; plain `vendor` ejects hosted pins instead and reports `eject_planned` / `eject_refused`): dry run — the upstream restore was resolved (registry lookups included) and would succeed (for bun, only after the Bun vendored preflight accepted the lock; a refused lock is previewed as the wet run's `failed ` instead) / wet run — the pin's lock entries were restored to their upstream registry entry before vendoring (mode takeover; detail ` was hosted; restored its upstream registry entry () before vendoring (mode takeover)`), so `vendor --revert` later returns to upstream. Fires on the run that takes over, not on re-runs, and not for a purl whose takeover was rolled back because the backend refused it (see "Takeover reconciliation"). | | `redirect_revert_failed` | `failed` | vendor / scan / get `--mode vendored` (dry and wet): the upstream restore of a hosted pin was refused (`--offline`, a registry that does not answer, a lock shape the restore refuses — for `bun.lockb`, a record the codec cannot rebuild) — detail `cannot vendor over the live hosted pin: cannot restore to its upstream registry entry: ; restore it from version control instead (`git checkout -- `)` (for `bun.lock` / `bun.lockb` the detail also adds `, then run \`bun install --force\` (a plain \`bun install\` keeps the patched copy)`); nothing vendored for the purl, hosted wiring left in place, exit 1 `partialFailure`. | | `patch_fetch_failed` (eject) | `failed` | vendor eject (v5.0): a hosted pin's patch record could not be fetched from `…/patches/view/`; the whole eject is refused (`eject_refused`), nothing touched, exit 1. | | `redirect_pnpm_lockfile_elsewhere` / `redirect_workspace_lockfile_elsewhere` / `cargo_manifest_not_workspace_root` (hosted) | top-level `error.code` (`status: "error"`) | scan / get `--mode hosted` (v5.0): the project directory is a workspace member whose lock lives in another directory, so the rewriters, which read only the project directory, would pin nothing (pnpm: no npm-family lock here, and the nearest ancestor `pnpm-workspace.yaml` or the project's `lockfile-dir` (`.npmrc`) / `lockfileDir` (`pnpm-workspace.yaml`) puts `pnpm-lock.yaml` elsewhere; npm / yarn / Bun, `redirect_workspace_lockfile_elsewhere`: no npm-family lock here, and the nearest ancestor `package.json` whose `workspaces` (array, or the object form's `packages`) matches the directory holds `package-lock.json`, `npm-shrinkwrap.json`, `yarn.lock`, `bun.lock` or `bun.lockb`; a matching root with none of them that is itself listed by an outer root's `workspaces` hands the check to that root; vlt, same code: the nearest ancestor `vlt.json` whose `workspaces` (a string, an array, or an object of groups) matches the directory holds `vlt-lock.json`, or, as vlt falls back to it when `vlt.json` has no `workspaces` field, the `package.json` `workspaces` root above holds `vlt-lock.json`, and the nearer of a `vlt.json` and a `package.json` root is named; `workspaces` patterns use the glob grammar the package managers share: `*`, `?`, `**`, brace sets and sequences (`{a,b}`, `{1..3}`) and character classes (`[a-c]`, `[!a]`), read the way the root's lock's package manager reads them (#1097): extglobs (`@(a|b)`, `?(…)`, `*(…)`, `+(…)`, `!(…)`) match for npm and yarn but not Bun or vlt; yarn berry (picomatch) also reads regex groups (`packages/(a|b)`; a group without `|` only in a pattern fast-glob deems dynamic, or in a `!` pattern), the regex quantifiers `?` / `+` after a group, `+` after a class and `+` inside a group, POSIX classes (`[[:alpha:]]`) and `..` segments in a static pattern or after a static prefix (`packages/a/../b` lists `packages/b`; literal in a `!` pattern); a group alternative holding `/` (`(packages/ui|apps/web)`) is matched across components, and a `?` or `+` quantifying such a group fails closed (the pattern lists every directory, and excludes none as a `!` pattern) (#1490); a `!` pattern excludes what earlier patterns matched for npm, Bun 1.3+ (a `bun.lock` with a top-level `configVersion`) and vlt, excludes whatever the order for yarn berry (a `yarn.lock` with `__metadata`, where a leading `!(` is an extglob), and is ignored by yarn classic and older Bun (a `bun.lock` without `configVersion`, or a `bun.lockb` alone without the trailing `cNfGvRsN` config-version extension, its binary twin, #1492); only the root's locks whose package manager lists the directory govern it (and are named), and a lockless root lists it when any of them would; when a pnpm workspace also governs the directory, the nearer root is named and a tie goes to `redirect_pnpm_lockfile_elsewhere`; uv (pypi, #1138), `redirect_workspace_lockfile_elsewhere`: the directory holds a `pyproject.toml`, and the nearest ancestor `pyproject.toml` declaring `[tool.uv.workspace]` lists it in `members` (and not in `exclude`), with no standalone project (`[project]`, no workspace) in between, so uv installs it from that root's `uv.lock` (a `uv.lock`, `poetry.lock`, `pdm.lock`, `Pipfile.lock` or pylock left in the member is never read and does not exempt it); the message says uv workspaces are not patched from their root yet either, and vendored refuses the same layout with `pypi_uv_workspace_unsupported` instead of rewriting a member with Hatch configuration as a lockless Hatch project; a directory whose own locks are all ones its manager never reads inside a workspace member is refused the same way, naming the ignored locks: `package-lock.json` / `npm-shrinkwrap.json` when its `package.json` `workspaces` root holds `package-lock.json` or `npm-shrinkwrap.json` (npm, #1094), `bun.lock` / `bun.lockb` when that root holds `bun.lock` or `bun.lockb` (Bun, #1101), and `vlt-lock.json` in a directory with no `vlt.json` of its own when its vlt workspace root (as above) holds `vlt-lock.json` (vlt, #1134); vendored refuses it with `vendor_lockfile_missing`, and `vex` reads the ignored lock as absent, with one `patched_ref_unattributable` warning naming it when it holds Socket references) or rewrite the member as a lockless project (cargo: the vendored workspace-root check). Refused before any takeover or write, `--dry-run` included; the message names the directory to run from; exit 1. Disk runs only (an in-memory project has no ancestors). | diff --git a/crates/socket-patch-cli/src/commands/vendor.rs b/crates/socket-patch-cli/src/commands/vendor.rs index 3516e7c49..eb1b8e6e4 100644 --- a/crates/socket-patch-cli/src/commands/vendor.rs +++ b/crates/socket-patch-cli/src/commands/vendor.rs @@ -1,7 +1,7 @@ //! `socket-patch vendor` — committable vendoring of patched dependencies. //! -//! Works like `apply`, but instead of patching installed packages in place it -//! ejects each patched package into `.socket/vendor///…` and +//! Converts the hosted patches pinned in the project's lockfiles into +//! `.socket/vendor///…` without an installed package tree, and //! rewires the ecosystem's lockfile/config so the project consumes the //! vendored copy. After committing `.socket/vendor/` + the lockfile edits, a //! fresh checkout builds with the patched dependency on machines with no @@ -15,7 +15,8 @@ //! vendored` drive this module's [`vendor_records`] engine directly in //! DETACHED mode: every entry they write carries its patch record embedded //! in the ledger and no `.socket/manifest.json` is ever written — this -//! command is the one manifest-driven writer (`detached: false`). See +//! command also supports legacy manifest-driven vendoring when no hosted +//! pins are in scope (`detached: false`). See //! CLI_CONTRACT.md "Ownership, state, and reversal". use clap::Args; @@ -987,15 +988,10 @@ pub async fn run(args: VendorArgs) -> i32 { let manifest_path = args.common.resolved_manifest_path(); let socket_dir = crate::args::socket_dir_of(&manifest_path, &args.common.cwd); - // `--revert` derives everything from state.json + the vendor tree; it - // must work after the manifest was deleted. Plain vendor needs the - // manifest and exits clean without one (same contract as apply). This - // is a MANIFEST check, not a `.socket/` check: `scan`/`get --mode - // vendored` projects have `.socket/` but never a manifest. Nothing is - // locked or written on this path. - if !args.revert && tokio::fs::metadata(&manifest_path).await.is_err() { - // A hosted project (no manifest, hosted pins in its lockfiles) - // ejects: its patch set is the lockfiles' hosted pins. + // Hosted pins are the input, even when an agent manifest also exists. + // The raw lockfile inventory does not require VEX eligibility or an + // installed copy of a patch. Revert needs only the vendor ledger. + if !args.revert { let inventory = crate::commands::hosted_inventory(&args.common, &args.common.cwd).await; // Contested hosted wiring: the patch set cannot be read off the // lockfiles, and a "nothing to vendor" answer would hide it. @@ -1020,6 +1016,12 @@ pub async fn run(args: VendorArgs) -> i32 { } return run_eject(&args, pins).await; } + } + + // Preserve legacy manifest-only vendoring when there are no hosted + // pins. A missing manifest is a no-op; detached vendored projects keep + // their records in the ledger instead. + if !args.revert && tokio::fs::metadata(&manifest_path).await.is_err() { // A requested `--vex` still attests what the `.socket/vendor` // ledgers and lockfiles already wire. Same contract as `apply --vex` // with no manifest: nothing referenced anywhere keeps exit 0; any @@ -1719,18 +1721,22 @@ impl EjectSnapshot { } } -/// Standalone `vendor` in a hosted project — no manifest, hosted pins in the -/// lockfiles: EJECT. The patch set is the pins themselves (purl + the uuid -/// in each hosted URL); each record is fetched from the API, vendored into -/// `.socket/vendor/` exactly like `scan --mode vendored`, and the lock is -/// rewired from hosted to vendored (the engine's takeover restores each -/// pin's upstream registry entry first, so `vendor --revert` later returns -/// the project to upstream, not to hosted). +/// Standalone `vendor` with hosted pins in scope: EJECT. The patch set comes +/// only from the lockfile pins (purl + the uuid in each hosted URL), even +/// when `.socket/manifest.json` exists: manifest-only patches are not +/// vendored and the manifest is not rewritten (vendoring those takes a full +/// `scan --mode vendored`). Each record is fetched from the API, vendored +/// into `.socket/vendor/` exactly like `scan --mode vendored`, and the lock +/// is rewired from hosted to vendored in one planned transition: every +/// pin's upstream registry entry is restored first (not per purl by the +/// engine's takeover), so `vendor --revert` later returns the project to +/// upstream, not to hosted. async fn run_eject(args: &VendorArgs, pins: Vec) -> i32 { let common = &args.common; let (client, use_public_proxy) = get_api_client_with_overrides(common.api_client_overrides()).await; let telemetry = TelemetryAuth::for_client(&client); + let service = common.vendor_service_config(Some(client.clone()), use_public_proxy); if !common.json && !common.silent { println!( "{} {} into .socket/vendor/...", @@ -1827,11 +1833,45 @@ async fn run_eject(args: &VendorArgs, pins: Vec) -> i32 { .refused() .map(|(pin, why)| (pin.purl.clone(), "redirect_revert_failed", why.to_string())) .collect(); - // An sbt / scala-cli pin is vendored only past the build-evidence gate - // (no `target/` evidence in a fresh clone, stale evidence, …). Run it - // now, while the hosted pin still serves the patch: a pin the gate - // would stop after the restore would end neither hosted nor vendored. + // Reuse the takeover's read-only backend preview over the restored + // lockfile text. A manifest's presence must not change whether a dry + // run catches an unsupported lock entry (for example a pnpm catalog). + let uninstalled = common.cwd.join(".socket/vendor/.uninstalled"); + let blobs = common.socket_dir().join("blobs"); + let sources = PatchSources { + blobs_path: &blobs, + mem_blobs: None, + }; + let pipenv_version = tokio::sync::OnceCell::new(); + let installed_sites = vendor::pypi::InstalledSiteListings::default(); + let vendored_at = now_rfc3339(); + let restore_refused: HashSet = refused.iter().map(|(p, _, _)| p.clone()).collect(); for pin in &pins { + // A pin the restore already refused keeps its first, accurate + // reason; previewing it again would add a second failure event. + if restore_refused.contains(&pin.purl) { + continue; + } + if let Some((code, detail)) = takeover_dry_refusal( + &plan, + &pin.purl, + PackageSource::Installed(&uninstalled), + &common.cwd, + &records[&pin.purl], + &sources, + &vendored_at, + args.force, + Some(&service), + &pipenv_version, + &installed_sites, + ) + .await + { + refused.push((pin.purl.clone(), code, detail)); + } + // sbt / scala-cli also require build evidence. The backend treats + // that gate as a benign skip, but an eject must retain the hosted + // pin if it cannot vendor it. if let Err((code, detail)) = socket_patch_core::vendor::maven_repo::jvm_gate_preflight(&common.cwd, &pin.purl).await { @@ -1912,11 +1952,10 @@ async fn run_eject(args: &VendorArgs, pins: Vec) -> i32 { } // One transaction under one apply lock: snapshot what the eject can - // touch, restore every pin upstream (so the vendor engine resolves the - // pristine registry package even in a fresh checkout with nothing - // installed), vendor, and on ANY failure put back what the eject wrote - // — a failed eject leaves the project hosted, exactly as it was, and - // every file it never wrote untouched. + // touch, restore upstream metadata for the vendor ledger's reversal, + // download and wire the prebuilt patches, and on ANY failure put back + // what the eject wrote. A failed eject leaves the project hosted, + // exactly as it was, and every file it never wrote untouched. let socket_dir = common.socket_dir(); let timeout = Duration::from_secs(common.lock_timeout.unwrap_or(0)); let guard = match crate::commands::lock_cli::acquire_with_status(&socket_dir, timeout) { @@ -1949,6 +1988,20 @@ async fn run_eject(args: &VendorArgs, pins: Vec) -> i32 { &client, )); } + // The vlt store entries the hosted pins installed, read before the + // restore unpins them: once the eject stands they hold hosted bytes the + // vendored lock no longer names, and are healed like a takeover's. + let pin_purls: Vec = pins.iter().map(|p| p.purl.clone()).collect(); + let vlt_targets = socket_patch_core::utils::fs::read_regular_to_string( + &common + .cwd + .join(socket_patch_core::constants::npm_family::VLT_LOCK), + ) + .await + .map(|lock| { + socket_patch_core::patch::redirect::vlt_heal::lock_targets(&lock, &origins, &pin_purls) + }) + .unwrap_or_default(); let restore = socket_patch_core::patch::redirect::upstream::restore_upstream( &common.cwd, &pins, @@ -1987,9 +2040,8 @@ async fn run_eject(args: &VendorArgs, pins: Vec) -> i32 { } let manifest = crate::commands::vendored_backend::records_manifest(records); // The same vendored apply as `scan --mode vendored`: detached (the - // eject is manifest-free) over this run's client and the shared - // service-config assembler. - let service = common.vendor_service_config(Some(client.clone()), use_public_proxy); + // eject does not use the agent manifest) over this run's client + // and the shared service-config assembler. let applied = VendoredBackend::new(common, Some(&service)) .apply( ApplyRequest { @@ -1997,7 +2049,7 @@ async fn run_eject(args: &VendorArgs, pins: Vec) -> i32 { socket_dir: &socket_dir, ledger: load_state(&common.cwd).await, detached: true, - force: false, + force: args.force, prior: None, eject: Some(&mut capture), }, @@ -2072,6 +2124,21 @@ async fn run_eject(args: &VendorArgs, pins: Vec) -> i32 { env.mark_partial_failure(); } } else { + // The eject stands: invalidate the vlt store copies still holding + // the hosted bytes (as the per-purl takeover does), so a later + // `vendor --revert` cannot leave patched bytes under a registry pin. + if let Some(detail) = crate::commands::vlt_heal::takeover_heal(common, &vlt_targets).await { + let code = crate::commands::vlt_heal::REINSTALL_REQUIRED; + if !common.json && !common.silent { + if let Some(line) = format_advisory(code, &detail, common.verbose) { + eprintln!("{line}"); + } + } + env.warnings.push(RunWarning { + code: code.to_string(), + detail, + }); + } // The vendored summary and its "Next steps:", deferred until the // eject is known to stand: a failed one is rolled back (or needs // the manual restore its error names), so neither applies. @@ -2082,7 +2149,7 @@ async fn run_eject(args: &VendorArgs, pins: Vec) -> i32 { // Embedded VEX: same contract as the manifest-driven arm — only on // success, never on a dry run, and a requested-but-failed VEX flips the - // exit code. The ejected project has no manifest. + // exit code. The ejected patches are recorded in the vendor ledger. if exit == 0 { if let Some(vex_path) = args.vex.vex.as_ref() { if common.dry_run { @@ -2769,41 +2836,48 @@ pub(crate) async fn vendor_records_reusing( vendor::prestage::sweep_stale(&common.cwd).await; } - let crawler_options = common.crawler_options(); - // Resolve installed packages with the qualified-purl-aware resolver, never - // a base-keyed one: the manifest keys release-variant ecosystems (gem - // `?platform=`, pypi `?artifact_id=`, maven `?classifier=&ext=`) by - // *qualified* purls, but the crawler only knows the *base* purl, so a - // base-keyed map would misclassify every installed qualified purl as - // "not installed". The rollback variant fans each base path back out to - // every qualified manifest purl. - let mut all_packages: HashMap = find_packages_for_rollback_reusing( - &vendorable_partition, - &crawler_options, - common.silent || common.json, - prior, - ) - .await - .into_iter() - .map(|(purl, dir)| (purl, StagedSource::Installed(dir))) - .collect(); - - // An npm alias is installed under its dependency key, not its actual - // package name. The targeted resolver probes canonical paths; before - // fetching a supposedly missing source, resolve aliases by the installed - // package.json identity. This also permits offline binary Bun vendoring. - let missing_npm: Vec<&String> = vendorable_partition - .get(&Ecosystem::Npm) + // An eject already selected exact patch UUIDs from the lockfiles. Its + // artifacts come from the service: installed copies (including an + // unrelated global Python/Ruby environment) cannot select or reject + // them. Leave their sources missing and let the backends read the locks. + let mut all_packages: HashMap = HashMap::new(); + if eject.is_none() { + let crawler_options = common.crawler_options(); + // Resolve installed packages with the qualified-purl-aware resolver, never + // a base-keyed one: the manifest keys release-variant ecosystems (gem + // `?platform=`, pypi `?artifact_id=`, maven `?classifier=&ext=`) by + // *qualified* purls, but the crawler only knows the *base* purl, so a + // base-keyed map would misclassify every installed qualified purl as + // "not installed". The rollback variant fans each base path back out to + // every qualified manifest purl. + all_packages = find_packages_for_rollback_reusing( + &vendorable_partition, + &crawler_options, + common.silent || common.json, + prior, + ) + .await .into_iter() - .flatten() - .filter(|p| !all_packages.contains_key(*p)) + .map(|(purl, dir)| (purl, StagedSource::Installed(dir))) .collect(); - let by_identity = match prior.and_then(|p| p.packages_for(&crawler_options)) { - Some(installed) => npm_paths_by_identity_in(installed, &missing_npm), - None => npm_paths_by_identity(&crawler_options, &missing_npm).await, - }; - for (purl, paths) in by_identity { - all_packages.insert(purl, StagedSource::Installed(paths[0].clone())); + + // An npm alias is installed under its dependency key, not its actual + // package name. The targeted resolver probes canonical paths; before + // fetching a supposedly missing source, resolve aliases by the installed + // package.json identity. This also permits offline binary Bun vendoring. + let missing_npm: Vec<&String> = vendorable_partition + .get(&Ecosystem::Npm) + .into_iter() + .flatten() + .filter(|p| !all_packages.contains_key(*p)) + .collect(); + let by_identity = match prior.and_then(|p| p.packages_for(&crawler_options)) { + Some(installed) => npm_paths_by_identity_in(installed, &missing_npm), + None => npm_paths_by_identity(&crawler_options, &missing_npm).await, + }; + for (purl, paths) in by_identity { + all_packages.insert(purl, StagedSource::Installed(paths[0].clone())); + } } let vendored_installs = drop_vendored_installs_by(&common.cwd, &mut all_packages, |source| match source { @@ -2884,10 +2958,13 @@ pub(crate) async fn vendor_records_reusing( // the lockfiles still pin hosted must restore its upstream registry // entry FIRST (see the dispatch loop below). Discovered once, before any // write of this run. - let hosted_pins: Vec = - socket_patch_core::patch::redirect::upstream::HostedPin::all( - &crate::commands::discover_wiring(common, &common.cwd).await, - ); + // Eject restored all its pins in one transaction before calling us; + // there is no per-package takeover or second discovery to perform. + let hosted_pins = if eject.is_some() { + Vec::new() + } else { + HostedPin::all(&crate::commands::discover_wiring(common, &common.cwd).await) + }; let hosted_pin_of = |purl: &str| { hosted_pins .iter() diff --git a/crates/socket-patch-cli/src/commands/vendored_backend/mod.rs b/crates/socket-patch-cli/src/commands/vendored_backend/mod.rs index 0830ec6b1..edf085bdb 100644 --- a/crates/socket-patch-cli/src/commands/vendored_backend/mod.rs +++ b/crates/socket-patch-cli/src/commands/vendored_backend/mod.rs @@ -72,6 +72,9 @@ pub(crate) struct ApplyRequest<'a> { /// rollback undoes exactly these), and the wired flavors. With it, the /// run's human summary and "Next steps:" are left to the eject, which /// prints them only once it knows the eject stands. + /// The records are exact lockfile-selected patches with their hosted + /// wiring already restored upstream; skip installed-package discovery + /// and per-package hosted takeover. pub(crate) eject: Option<&'a mut EjectCapture>, } diff --git a/crates/socket-patch-cli/src/lib.rs b/crates/socket-patch-cli/src/lib.rs index 9b04330af..66d68faad 100644 --- a/crates/socket-patch-cli/src/lib.rs +++ b/crates/socket-patch-cli/src/lib.rs @@ -87,11 +87,12 @@ pub enum Commands { /// Generate OpenVEX for vulnerabilities addressed by verified patches Vex(commands::vex::VexArgs), - /// Store patched dependencies in .socket/vendor/ for offline installs + /// Download hosted patches from lockfiles into .socket/vendor/ /// - /// Rewire dependency files to use the committed artifacts. Unpatched - /// dependencies still need their normal registry or cache. Use --revert - /// to undo vendoring. + /// Works before installing dependencies and rewires dependency files to + /// use the committed artifacts. Falls back to the agent manifest when no + /// hosted patches are in scope. Unpatched dependencies still need their + /// normal registry or cache. Use --revert to undo vendoring. Vendor(commands::vendor::VendorArgs), /// Reapply agent-mode patches after installing dependencies diff --git a/crates/socket-patch-cli/tests/e2e_golang_hosted_state.rs b/crates/socket-patch-cli/tests/e2e_golang_hosted_state.rs index d41f404e0..bbcda4245 100644 --- a/crates/socket-patch-cli/tests/e2e_golang_hosted_state.rs +++ b/crates/socket-patch-cli/tests/e2e_golang_hosted_state.rs @@ -18,6 +18,7 @@ use std::path::Path; #[path = "common/mod.rs"] mod common; +use base64::Engine as _; use socket_patch_core::hash::git_sha256::compute_git_sha256_from_bytes; use wiremock::matchers::{method, path}; use wiremock::{Mock, MockServer, ResponseTemplate}; @@ -468,7 +469,8 @@ async fn hosted_rollback_restores_go_sum_byte_for_byte() { /// to its upstream entry (drop the hosted replace and the socket go.sum /// lines, re-derive the pruned upstream pair from the checksum database), /// so the project is fully vendored — never a vendored go.mod beside a -/// hosted pin. No ledger is involved at any point. +/// hosted pin. No ledger is involved at any point. The vendored patch is +/// the hosted pin's own: a manifest naming another patch is ignored. #[tokio::test(flavor = "multi_thread")] async fn vendored_takeover_of_hosted_module_unwinds_the_redirect() { let tmp = tempfile::tempdir().unwrap(); @@ -516,11 +518,50 @@ async fn vendored_takeover_of_hosted_module_unwinds_the_redirect() { .unwrap(); std::fs::write(socket.join("blobs").join(&after), PATCHED_LIB).unwrap(); - // Online: the takeover's upstream restore consults the (mocked) - // checksum database; the patch itself comes from the local manifest. - let (code, stdout, stderr) = run_with_prebuilt( + let manifest = std::fs::read(socket.join("manifest.json")).unwrap(); + + // The manifest above names a DIFFERENT patch (UUID_V), but a hosted pin + // is in scope: plain `vendor` ejects exactly the go.mod's hosted patch + // (UUID_H) and ignores the manifest (vendoring manifest patches takes a + // full `scan --mode vendored`). The eject fetches the record from the + // API and downloads the prebuilt module from the artifact service; the + // upstream restore consults the (mocked) checksum database. + let view = serde_json::json!({ + "uuid": UUID_H, + "purl": UPURL, + "publishedAt": "2026-01-01T00:00:00Z", + "files": { "lib.go": { + "beforeHash": compute_git_sha256_from_bytes(PRISTINE_LIB.as_bytes()), + "afterHash": &after, + "blobContent": base64::engine::general_purpose::STANDARD.encode(PATCHED_LIB), + }}, + "vulnerabilities": {}, + "description": "golang hosted state fixture", + "license": "MIT", + "tier": "free", + }); + prebuilt_common::mount_view_from_source( + &server, + &view, + None, + Some(&modcache.join(format!("{UMOD}@{UVER}"))), + ) + .await; + let uri = server.uri(); + let (code, stdout, stderr) = common::run_with_env( &consumer, - &["vendor", "--json", "--cwd", consumer.to_str().unwrap()], + &[ + "vendor", + "--json", + "--cwd", + consumer.to_str().unwrap(), + "--api-url", + &uri, + "--org", + ORG, + "--api-token", + "fake", + ], &[ ("GOMODCACHE", modcache.to_str().unwrap()), ("SOCKET_GOSUMDB_URL", sumdb.as_str()), @@ -531,13 +572,24 @@ async fn vendored_takeover_of_hosted_module_unwinds_the_redirect() { "vendor failed\nstdout:\n{stdout}\nstderr:\n{stderr}" ); assert!( - stdout.contains("vendor_takeover_reverted_redirect"), - "the takeover is reported: {stdout}" + !stdout.contains(UUID_V), + "the manifest-only patch is not part of the eject: {stdout}" + ); + assert!( + !consumer + .join(format!(".socket/vendor/golang/{UUID_V}")) + .exists(), + "the manifest-only patch must not be vendored" + ); + assert_eq!( + std::fs::read(socket.join("manifest.json")).unwrap(), + manifest, + "the eject never rewrites the manifest" ); let go_mod = std::fs::read_to_string(consumer.join("go.mod")).unwrap(); assert!( go_mod.contains(&format!( - "replace {UMOD} {UVER} => ./.socket/vendor/golang/{UUID_V}/{UMOD}@{UVER}" + "replace {UMOD} {UVER} => ./.socket/vendor/golang/{UUID_H}/{UMOD}@{UVER}" )), "{go_mod}" ); diff --git a/crates/socket-patch-cli/tests/e2e_sbt_vendor.rs b/crates/socket-patch-cli/tests/e2e_sbt_vendor.rs index 404ac329e..0fc1cd0c9 100644 --- a/crates/socket-patch-cli/tests/e2e_sbt_vendor.rs +++ b/crates/socket-patch-cli/tests/e2e_sbt_vendor.rs @@ -786,17 +786,29 @@ fn drop_evidence(proj: &Path) { } } -/// A takeover (vendoring over a hosted pin) the vendored gate would skip -/// (no evidence: a fresh clone) is refused BEFORE the hosted pin is -/// restored: the project stays hosted, never neither hosted nor vendored. +/// The `vendor` flags naming the mock patch API of [`mock_gson_api`]. +fn api_flags(uri: &str) -> [&str; 6] { + ["--api-url", uri, "--org", "test-org", "--api-token", "fake"] +} + +/// Plain `vendor` over a hosted pin ejects it even beside an agent +/// manifest. An eject the vendored gate would skip (no evidence: a fresh +/// clone) is refused BEFORE the hosted pin is restored: the project stays +/// hosted, never neither hosted nor vendored, and the manifest is untouched. #[test] fn vendor_takeover_the_gate_would_skip_keeps_the_hosted_pin() { let tmp = tempfile::tempdir().unwrap(); let fx = Fixture::new(tmp.path(), &[(GSON, GSON_UUID)]); let hosted = write_hosted_pin(&fx.proj(), GSON_UUID); + let manifest = std::fs::read(fx.proj().join(".socket/manifest.json")).unwrap(); drop_evidence(&fx.proj()); - let (exit, env) = fx.run(&["vendor"]); + let (_rt, server) = mock_gson_api(); + let uri = server.uri(); + let mut args = vec!["vendor"]; + args.extend(api_flags(&uri)); + let (exit, env) = fx.run(&args); assert_eq!(exit, Some(1), "{env}"); + assert_eq!(env["error"]["code"], "eject_refused", "{env}"); assert!( event_triples(&env).contains(&( GSON.purl(), @@ -807,47 +819,47 @@ fn vendor_takeover_the_gate_would_skip_keeps_the_hosted_pin() { ); assert_eq!(std::fs::read(fx.proj().join(HOSTED_FILE)).unwrap(), hosted); assert!(!fx.proj().join(VENDORED_FILE).exists()); + assert_eq!( + std::fs::read(fx.proj().join(".socket/manifest.json")).unwrap(), + manifest + ); } -/// A dry-run takeover previews (the restore is dry, so the hosted file -/// still pins the GA on disk; the planner's hosted-conflict refusal the wet -/// run never sees is not raised), and the wet run takes over. +/// A dry-run eject of a hosted sbt pin (beside an agent manifest) previews +/// the plan and writes nothing; the wet run vendors the pinned patch and +/// drops the hosted file, leaving the manifest as it was. #[test] fn vendor_takeover_of_a_hosted_sbt_pin_dry_and_wet() { let tmp = tempfile::tempdir().unwrap(); let fx = Fixture::new(tmp.path(), &[(GSON, GSON_UUID)]); let hosted = write_hosted_pin(&fx.proj(), GSON_UUID); - let env = fx.ok(&["vendor", "--dry-run"]); - assert!( - env.to_string().contains("vendor_would_revert_redirect"), - "{env}" - ); - assert!( - !env.to_string().contains("vendor_sbt_hosted_conflict"), - "{env}" - ); + let manifest = std::fs::read(fx.proj().join(".socket/manifest.json")).unwrap(); + let (_rt, server) = mock_gson_api(); + let uri = server.uri(); + let mut args = vec!["vendor"]; + args.extend(api_flags(&uri)); + let mut dry = args.clone(); + dry.push("--dry-run"); + let env = fx.ok(&dry); + assert!(env.to_string().contains("eject_planned"), "{env}"); assert_eq!(std::fs::read(fx.proj().join(HOSTED_FILE)).unwrap(), hosted); - let env = fx.ok(&["vendor"]); - assert!( - env.to_string() - .contains("vendor_takeover_reverted_redirect"), - "{env}" - ); + assert!(!fx.proj().join(VENDORED_FILE).exists()); + let env = fx.ok(&args); assert!(!fx.proj().join(HOSTED_FILE).exists(), "{env}"); assert!(fx.generated().contains(GSON_UUID)); + assert_eq!( + std::fs::read(fx.proj().join(".socket/manifest.json")).unwrap(), + manifest + ); } -/// An eject (plain `vendor` in a hosted project) the vendored gate would -/// skip is refused whole before anything is restored. -#[test] -fn vendor_eject_the_gate_would_skip_is_refused() { +/// A mock patch API serving gson's patch record by `GSON_UUID`: plain +/// `vendor` over a hosted pin ejects it, fetching the pinned record from +/// the API (the agent manifest is not its input). Returns the runtime that +/// keeps the server alive and the server. +fn mock_gson_api() -> (tokio::runtime::Runtime, wiremock::MockServer) { use wiremock::matchers::{method, path}; use wiremock::{Mock, MockServer, ResponseTemplate}; - let tmp = tempfile::tempdir().unwrap(); - let fx = Fixture::new(tmp.path(), &[(GSON, GSON_UUID)]); - std::fs::remove_file(fx.proj().join(".socket/manifest.json")).unwrap(); - let hosted = write_hosted_pin(&fx.proj(), GSON_UUID); - drop_evidence(&fx.proj()); let rt = tokio::runtime::Builder::new_multi_thread() .worker_threads(1) .enable_all() @@ -868,6 +880,19 @@ fn vendor_eject_the_gate_would_skip_is_refused() { .respond_with(ResponseTemplate::new(200).set_body_json(view)) .mount(&server), ); + (rt, server) +} + +/// An eject (plain `vendor` in a hosted project) the vendored gate would +/// skip is refused whole before anything is restored. +#[test] +fn vendor_eject_the_gate_would_skip_is_refused() { + let tmp = tempfile::tempdir().unwrap(); + let fx = Fixture::new(tmp.path(), &[(GSON, GSON_UUID)]); + std::fs::remove_file(fx.proj().join(".socket/manifest.json")).unwrap(); + let hosted = write_hosted_pin(&fx.proj(), GSON_UUID); + drop_evidence(&fx.proj()); + let (_rt, server) = mock_gson_api(); let uri = server.uri(); let (exit, env) = fx.run(&[ "vendor", diff --git a/crates/socket-patch-cli/tests/help_text_hygiene.rs b/crates/socket-patch-cli/tests/help_text_hygiene.rs index 80b995709..e0bb485c8 100644 --- a/crates/socket-patch-cli/tests/help_text_hygiene.rs +++ b/crates/socket-patch-cli/tests/help_text_hygiene.rs @@ -227,7 +227,7 @@ fn vendor_and_repair_summaries_read_as_one_line() { let text = long_help(&[]); assert!( text.lines().any(|l| l - == " vendor Store patched dependencies in .socket/vendor/ for offline installs"), + == " vendor Download hosted patches from lockfiles into .socket/vendor/"), "{text}" ); assert!( diff --git a/crates/socket-patch-cli/tests/in_process_vendor.rs b/crates/socket-patch-cli/tests/in_process_vendor.rs index bd3377f85..c97f3e301 100644 --- a/crates/socket-patch-cli/tests/in_process_vendor.rs +++ b/crates/socket-patch-cli/tests/in_process_vendor.rs @@ -1656,7 +1656,7 @@ fn split_berry_checksum(lock: &str) -> (String, String) { } /// Mode takeovers on the Windows shapes, both directions, hermetic. Hosted → -/// vendored: `vendor` restores the CRLF hosted pin's upstream registry +/// vendored: `vendor` ejects the CRLF hosted pin, restoring its upstream registry /// entry (v5: re-resolved from the registry, no ledger) before wiring, and /// `vendor --revert` then lands on the upstream CRLF + BOM pair — not the /// hosted URL. Vendored → hosted: `scan --mode hosted` reverts the vendored @@ -1706,14 +1706,23 @@ async fn berry_crlf_takeovers_round_trip_both_directions() { ); prebuilt_common::mount_project(&server, root).await; - let (code, env) = vendor_online_cli(root, &server.uri(), &server.uri(), &[]); + // An eject: the hosted pin's record comes from the API. + let uri = server.uri(); + let (code, env) = vendor_online_cli( + root, + &uri, + &uri, + &[ + "--api-url", + &uri, + "--org", + "test-org", + "--api-token", + "fake", + ], + ); assert_eq!(code, 0, "vendor over the hosted pin: {env:#}"); assert_eq!(env["summary"]["applied"], 1, "{env:#}"); - assert!( - env.to_string() - .contains("vendor_takeover_reverted_redirect"), - "the takeover is surfaced: {env:#}" - ); let vendored_lock = std::fs::read_to_string(root.join("yarn.lock")).unwrap(); assert!( !vendored_lock.contains("__archiveUrl") && vendored_lock.contains(".socket/vendor/npm/"), @@ -2218,6 +2227,7 @@ async fn berry_hosted_to_vendored_takeover_runs_package_gates_first() { // The hosted pin's origin must count as the patch server, or // the vendor run never sees it as a takeover. let uri = server.uri(); + // An eject: the hosted pin's record comes from the API. let mut args = vec![ "vendor", "--json", @@ -2225,6 +2235,12 @@ async fn berry_hosted_to_vendored_takeover_runs_package_gates_first() { root.to_str().unwrap(), "--patch-server-url", &uri, + "--api-url", + &uri, + "--org", + "test-org", + "--api-token", + "fake", ]; if dry { args.push("--dry-run"); @@ -4202,6 +4218,33 @@ snapshots: /// count as hosted, so every post-scan run passes it. const PATCH_ORIGIN: &str = "https://patch.socket.dev"; + /// `vendor` over the hosted pin: an eject. Its patch set is the + /// lockfile's hosted pins (the staged manifest is not its input), so it + /// fetches each pinned record from `server`'s API (`view/{uuid}`, see + /// [`mock_hosted_api`]) and downloads the prebuilt artifact from a + /// fixture service; online against `registry` for the upstream restore. + fn vendor_eject_cli( + root: &Path, + server: &MockServer, + registry: &str, + extra: &[&str], + ) -> (i32, Value) { + let fixture = prebuilt_common::Server::project(root); + let uri = server.uri(); + let mut args = vec![ + "--api-url", + uri.as_str(), + "--org", + ORG, + "--api-token", + "fake", + "--vendor-url", + fixture.uri.as_str(), + ]; + args.extend_from_slice(extra); + vendor_online_cli(root, registry, PATCH_ORIGIN, &args) + } + /// The npm registry's version document for the fixture package, as the /// v5 upstream restore reads it (`SOCKET_NPM_REGISTRY`): it hands back /// the pristine integrity the fixture lock started with. @@ -4225,9 +4268,8 @@ snapshots: } /// Hosted → vendored on a pnpm project, v5: the hosted run writes ONLY - /// the lock (no ledger); `vendor` over the hosted pin restores the - /// upstream registry entry first (`vendor_takeover_reverted_redirect`), - /// so the vendor ledger records the PRISTINE registry fragment as the + /// the lock (no ledger); `vendor` over the hosted pin ejects it, + /// restoring the upstream registry entry first, so the vendor ledger records the PRISTINE registry fragment as the /// original; a re-vendor is a quiet no-op; `vendor --revert` /// byte-restores the registry lock — never the hosted splice. #[tokio::test] @@ -4257,13 +4299,12 @@ snapshots: // 2. Vendor over the hosted lock (staged blob, online only for the // registry lookup of the restore). seed_manifest_and_blob(root); - let (code, env1) = vendor_online_cli(root, ®istry, PATCH_ORIGIN, &[]); + let (code, env1) = vendor_eject_cli(root, &server, ®istry, &[]); assert_eq!( code, 0, "vendor over the hosted lock must succeed: {env1:#}" ); find_event(&env1, "applied", None); - find_event(&env1, "skipped", Some("vendor_takeover_reverted_redirect")); let vendored_lock_text = std::fs::read_to_string(root.join("pnpm-lock.yaml")).unwrap(); assert!( @@ -4295,7 +4336,7 @@ snapshots: // 3. Re-vendor: an `already_vendored` no-op with NO takeover event — // the lock no longer pins anything hosted. - let (code, env2) = vendor_online_cli(root, ®istry, PATCH_ORIGIN, &[]); + let (code, env2) = vendor_eject_cli(root, &server, ®istry, &[]); assert_eq!(code, 0, "re-vendor must succeed: {env2:#}"); find_event(&env2, "skipped", Some("already_vendored")); assert!( @@ -4315,9 +4356,10 @@ snapshots: ); } - /// Offline, the hosted pin's upstream entry cannot be re-resolved: the - /// takeover REFUSES the purl (`redirect_revert_failed`, checkout remedy) - /// and leaves the hosted lock exactly as found. + /// Offline, neither the hosted pin's record nor its upstream entry can + /// be fetched: the eject is REFUSED before anything is read or written + /// (`offline_eject_unavailable`) and the hosted lock stays exactly as + /// found. #[tokio::test] #[serial] async fn offline_vendor_over_hosted_pin_is_refused() { @@ -4332,12 +4374,9 @@ snapshots: seed_manifest_and_blob(root); let (code, env) = vendor_cli(root, &["--offline", "--patch-server-url", PATCH_ORIGIN]); assert_eq!(code, 1, "{env:#}"); - let failed = find_event(&env, "failed", Some("redirect_revert_failed")); - assert!( - failed - .to_string() - .contains("git checkout -- pnpm-lock.yaml"), - "the refusal names the checkout remedy: {failed:#}" + assert_eq!( + env["error"]["code"], "offline_eject_unavailable", + "the eject needs the pinned record from the API: {env:#}" ); assert_eq!( std::fs::read(root.join("pnpm-lock.yaml")).unwrap(), @@ -4409,10 +4448,9 @@ snapshots: ); seed_manifest_and_blob(root); - let (code, env) = vendor_online_cli(root, ®istry, PATCH_ORIGIN, &[]); + let (code, env) = vendor_eject_cli(root, &server, ®istry, &[]); assert_eq!(code, 0, "vendor over the hosted lock must succeed: {env:#}"); find_event(&env, "applied", None); - find_event(&env, "skipped", Some("vendor_takeover_reverted_redirect")); assert!( !root.join(".npmrc").exists(), "the takeover must remove the .npmrc the hosted run created: {env:#}" @@ -4557,8 +4595,7 @@ snapshots: /// REGRESSION (#828): the hosted → vendored takeover over a hosted pin /// beside a bundled copy restores the upstream entry first, exactly as - /// without the bundled copy: `vendor_takeover_reverted_redirect`, the - /// `.npmrc` deleted, the vendor ledger's original the REGISTRY entry + /// without the bundled copy: the pin is ejected, the `.npmrc` deleted, the vendor ledger's original the REGISTRY entry /// (it recorded the grant-tokenized hosted URL), and `vendor --revert` /// lands on upstream, never back on hosted. #[tokio::test] @@ -4595,9 +4632,9 @@ snapshots: .contains(HOSTED_URL)); seed_manifest_and_blob(root); - let (code, env) = vendor_online_cli(root, ®istry, PATCH_ORIGIN, &[]); + let (code, env) = vendor_eject_cli(root, &server, ®istry, &[]); assert_eq!(code, 0, "vendor over the hosted lock must succeed: {env:#}"); - find_event(&env, "skipped", Some("vendor_takeover_reverted_redirect")); + find_event(&env, "applied", None); assert!( !root.join(".npmrc").exists(), "the takeover must remove the .npmrc the hosted run created: {env:#}" @@ -4649,7 +4686,7 @@ snapshots: assert!(String::from_utf8_lossy(&hosted_lock).contains(HOSTED_URL)); seed_manifest_and_blob(root); - let (code, env) = vendor_online_cli(root, ®istry, PATCH_ORIGIN, &[]); + let (code, env) = vendor_eject_cli(root, &server, ®istry, &[]); assert_eq!(code, 1, "{env:#}"); find_event(&env, "failed", Some("vendor_artifact_gitignored")); assert!( @@ -4700,7 +4737,7 @@ snapshots: std::fs::create_dir_all(root.join(".socket/vendor")).unwrap(); std::os::unix::fs::symlink(&shared, root.join(".socket/vendor/npm")).unwrap(); - let (code, env) = vendor_online_cli(root, ®istry, PATCH_ORIGIN, &[]); + let (code, env) = vendor_eject_cli(root, &server, ®istry, &[]); assert_eq!(code, 1, "{env:#}"); find_event(&env, "failed", Some("vendor_dir_symlink_unsupported")); assert!( diff --git a/crates/socket-patch-cli/tests/in_process_vendor_bun_takeover.rs b/crates/socket-patch-cli/tests/in_process_vendor_bun_takeover.rs index 51d2017a8..0c8865d74 100644 --- a/crates/socket-patch-cli/tests/in_process_vendor_bun_takeover.rs +++ b/crates/socket-patch-cli/tests/in_process_vendor_bun_takeover.rs @@ -353,6 +353,64 @@ fn registry_uri() -> &'static str { }) } +/// The patch API's view of `uuid` over `purl`: [`patch_record`] plus the +/// patched file's `blobContent`. +fn patch_view(uuid: &str, purl: &str) -> Value { + let mut view = patch_record(uuid); + view["purl"] = json!(purl); + view["publishedAt"] = json!("2024-01-01T00:00:00Z"); + view["files"]["package/index.js"]["blobContent"] = + json!(base64::engine::general_purpose::STANDARD.encode(PATCHED_INDEX)); + view +} + +/// One patch API shared by every `vendor` run in this binary, serving the +/// view of each hosted record the fixtures pin. Plain `vendor` over a +/// hosted pin ejects it: the patch set is the lockfile's hosted pins (the +/// agent manifest is not read), and each pinned record is fetched from the +/// API. Runs on its own thread + runtime for the life of the process, like +/// [`registry_uri`]. +fn patch_api_uri() -> &'static str { + static URI: std::sync::OnceLock = std::sync::OnceLock::new(); + URI.get_or_init(|| { + let (tx, rx) = std::sync::mpsc::channel(); + std::thread::spawn(move || { + let rt = tokio::runtime::Builder::new_current_thread() + .enable_all() + .build() + .expect("patch API runtime"); + rt.block_on(async move { + let server = MockServer::start().await; + for (uuid, purl) in [(UUID, PURL), (registry::SCOPED_UUID, registry::SCOPED_PURL)] { + Mock::given(method("GET")) + .and(path(format!("/v0/orgs/{ORG}/patches/view/{uuid}"))) + .respond_with( + ResponseTemplate::new(200).set_body_json(patch_view(uuid, purl)), + ) + .mount(&server) + .await; + } + tx.send(server.uri()).expect("hand back the patch API uri"); + std::future::pending::<()>().await; + }); + }); + rx.recv().expect("patch API started") + }) +} + +/// The flags pointing a `vendor` run's patch-record fetches at +/// [`patch_api_uri`]. +fn patch_api_args() -> [&'static str; 6] { + [ + "--api-url", + patch_api_uri(), + "--org", + ORG, + "--api-token", + "fake", + ] +} + /// Run the built `socket-patch` binary in the hermetic environment /// (`tests/common/hermetic.rs`); `https://patch.socket.dev` counts as the patch server /// (`SOCKET_PATCH_SERVER_URL`) and the registry is the shared mirror @@ -441,10 +499,11 @@ fn scan_mode(cwd: &Path, api_url: &str, mode: &str, extra: &[&str]) -> (i32, Val run_json(cwd, &args) } -/// `vendor --json` — online: a takeover's upstream restore reads the -/// registry mirror. +/// `vendor --json` — online: an eject's upstream restore reads the +/// registry mirror and its record fetch the shared [`patch_api_uri`]. fn vendor_cli(cwd: &Path, extra: &[&str]) -> (i32, Value) { let mut args = vec!["vendor", "--json", "--cwd", cwd.to_str().unwrap()]; + args.extend(patch_api_args()); args.extend_from_slice(extra); run_json(cwd, &args) } @@ -930,14 +989,15 @@ async fn bun_vendor_dry_run_previews_the_takeover_then_wet_vendor_completes_it() // The manifest record `vendor` acts on (the staged blob). seed_manifest_and_blob(root); - // Dry run: the takeover's upstream restore is resolved write-free and - // previewed; the backend preview does not run against the still-hosted - // lock, so no false `vendor_lock_entry_not_found` and no refusal. - // Nothing written. + // Dry run: a hosted pin is in scope, so `vendor` previews the eject of + // the lockfile's pin (its upstream restore resolved write-free); the + // backend preview runs over the restored text, so no false + // `vendor_lock_entry_not_found` and no refusal. Nothing written. + let manifest = std::fs::read(root.join(".socket/manifest.json")).unwrap(); let (code, env) = vendor_cli(root, &["--dry-run"]); assert_eq!(code, 0, "vendor --dry-run must succeed: {env:#}"); - let advisory = find_event(&env, "skipped", Some("vendor_would_revert_redirect")); - assert_eq!(advisory["purl"], PURL, "{env:#}"); + let planned = find_event(&env, "applied", Some("eject_planned")); + assert_eq!(planned["purl"], PURL, "{env:#}"); assert_no_event_code(&env, "vendor_lock_entry_not_found"); assert_no_event_code(&env, "redirect_revert_failed"); assert_eq!( @@ -955,13 +1015,18 @@ async fn bun_vendor_dry_run_previews_the_takeover_then_wet_vendor_completes_it() "a dry run must not create the vendor ledger" ); - // Wet `vendor`: the takeover the preview promised. + // Wet `vendor`: the eject the preview promised. The manifest is + // neither the eject's input nor rewritten by it. let (code, env) = vendor_cli(root, &[]); assert_eq!(code, 0, "vendor must succeed: {env:#}"); assert_eq!(env["summary"]["applied"], 1, "{env:#}"); - find_event(&env, "skipped", Some("vendor_takeover_reverted_redirect")); find_event(&env, "applied", None); assert_pure_vendored(root); + assert_eq!( + std::fs::read(root.join(".socket/manifest.json")).unwrap(), + manifest, + "the eject must not rewrite the manifest" + ); } // ───────────────────────────────────────────────────────────────────── @@ -1178,6 +1243,7 @@ fn bun_vendor_silent_refusal_keeps_error_diagnosis() { "--cwd", root.to_str().unwrap(), ]; + args.extend(patch_api_args()); if dry_run { args.push("--dry-run"); } @@ -1266,7 +1332,8 @@ fn bun_vendor_over_hosted_v1_workspace_lock_refuses_before_unhosting() { code, 1, "the preview must exit like the wet run it predicts: {env:#}" ); - assert_eq!(env["status"], "partialFailure", "{env:#}"); + assert_eq!(env["status"], "error", "{env:#}"); + assert_eq!(env["error"]["code"], "eject_refused", "{env:#}"); assert_eq!(env["dryRun"], true, "{env:#}"); let failed = find_event(&env, "failed", Some(WS_CODE)); assert_eq!(failed["purl"], PURL, "{env:#}"); @@ -1282,7 +1349,8 @@ fn bun_vendor_over_hosted_v1_workspace_lock_refuses_before_unhosting() { // `.socket/vendor/` empty. let (code, env) = vendor_cli(root, &[]); assert_eq!(code, 1, "the wet run refuses: {env:#}"); - assert_eq!(env["status"], "partialFailure", "{env:#}"); + assert_eq!(env["status"], "error", "{env:#}"); + assert_eq!(env["error"]["code"], "eject_refused", "{env:#}"); let failed = find_event(&env, "failed", Some(WS_CODE)); assert_eq!(failed["purl"], PURL, "{env:#}"); assert!( @@ -1315,13 +1383,12 @@ fn bun_vendor_over_hosted_v2_workspace_lock_still_takes_over() { let (code, env) = vendor_cli(root, &["--dry-run"]); assert_eq!(code, 0, "{env:#}"); - find_event(&env, "skipped", Some("vendor_would_revert_redirect")); + find_event(&env, "applied", Some("eject_planned")); assert_no_event_code(&env, WS_CODE); let (code, env) = vendor_cli(root, &[]); - assert_eq!(code, 0, "the v2 takeover must succeed: {env:#}"); + assert_eq!(code, 0, "the v2 eject must succeed: {env:#}"); assert_eq!(env["summary"]["applied"], 1, "{env:#}"); - find_event(&env, "skipped", Some("vendor_takeover_reverted_redirect")); find_event(&env, "applied", None); assert_no_event_code(&env, WS_CODE); assert_pure_vendored(root); @@ -1362,16 +1429,26 @@ fn assert_bun_reinstall_event(env: &Value) { } /// A hoisted bun project with left-pad vendored over its (v5) hosted pin -/// by a plain `vendor` run; returns the pristine registry lock. +/// by a plain `vendor` run (an eject: a detached ledger entry beside the +/// untouched manifest); returns the pristine registry lock. fn write_vendored_project(root: &Path) -> String { + write_vendored_project_from(root, true) +} + +/// [`write_vendored_project`]; with `hosted: false` the lock stays on the +/// registry line, so the plain `vendor` run is the manifest-driven one and +/// its ledger entry is tracked by the manifest. +fn write_vendored_project_from(root: &Path, hosted: bool) -> String { let pristine = pristine_lock(); write_bun_project(root, &pristine, &[(NAME, VERSION)]); - let hosted = pristine.replace( - LEFT_PAD_REGISTRY_LINE, - &hosted_line(NAME, NAME, HOSTED_URL, PATCHED_SHA512), - ); - assert_ne!(hosted, pristine, "the hosted splice must hit"); - std::fs::write(root.join("bun.lock"), &hosted).unwrap(); + if hosted { + let hosted = pristine.replace( + LEFT_PAD_REGISTRY_LINE, + &hosted_line(NAME, NAME, HOSTED_URL, PATCHED_SHA512), + ); + assert_ne!(hosted, pristine, "the hosted splice must hit"); + std::fs::write(root.join("bun.lock"), &hosted).unwrap(); + } seed_manifest_and_blob(root); let (code, env) = vendor_cli(root, &[]); assert_eq!(code, 0, "vendor must succeed: {env:#}"); @@ -1473,7 +1550,8 @@ async fn bun_scan_prune_revert_advises_a_forced_reinstall() { .await; let tmp = tempfile::tempdir().unwrap(); let root = tmp.path(); - let pristine = write_vendored_project(root); + // A manifest-tracked entry: plain `vendor` with no hosted pin in scope. + let pristine = write_vendored_project_from(root, false); std::fs::write(root.join(".socket/manifest.json"), "{\"patches\": {}}\n").unwrap(); let (code, env) = scan_mode(root, &server.uri(), "vendored", &["--prune"]); diff --git a/crates/socket-patch-cli/tests/in_process_vendor_bun_takeover/registry.rs b/crates/socket-patch-cli/tests/in_process_vendor_bun_takeover/registry.rs index f2281a6f0..05b3c2565 100644 --- a/crates/socket-patch-cli/tests/in_process_vendor_bun_takeover/registry.rs +++ b/crates/socket-patch-cli/tests/in_process_vendor_bun_takeover/registry.rs @@ -23,15 +23,15 @@ use wiremock::matchers::{header, method, path, path_regex}; use wiremock::{Mock, MockServer, ResponseTemplate}; use super::{ - assert_no_event_code, find_event, hosted_line, line_integrity, lock_line, patch_record, read, - run_json, run_json_env, vendor_cli, write_bun_project, HOSTED_URL, LEFT_PAD_REGISTRY_LINE, - NAME, PATCHED_INDEX, PATCHED_SHA512, PURL, UUID, VERSION, + assert_no_event_code, events, find_event, hosted_line, line_integrity, lock_line, patch_record, + read, run_json, run_json_env, vendor_cli, write_bun_project, HOSTED_URL, + LEFT_PAD_REGISTRY_LINE, NAME, PATCHED_INDEX, PATCHED_SHA512, PURL, UUID, VERSION, }; const SCOPED_NAME: &str = "@corp/widget"; const SCOPED_VERSION: &str = "2.0.0"; -const SCOPED_PURL: &str = "pkg:npm/@corp/widget@2.0.0"; -const SCOPED_UUID: &str = "3c4d5e6f-7a8b-4c9d-8e0f-1a2b3c4d5e6f"; +pub(super) const SCOPED_PURL: &str = "pkg:npm/@corp/widget@2.0.0"; +pub(super) const SCOPED_UUID: &str = "3c4d5e6f-7a8b-4c9d-8e0f-1a2b3c4d5e6f"; const SCOPED_HOSTED_URL: &str = "https://patch.socket.dev/patch/npm/@corp/widget/2.0.0/55555555-5555-4555-8555-555555555555/3c4d5e6f-7a8b-4c9d-8e0f-1a2b3c4d5e6f/widget-2.0.0.tgz"; const SCOPED_INTEGRITY: &str = "sha512-corpWIDGETcorp0123456789=="; const SCOPED_PATCHED_SHA512: &str = "sha512-corpPATCHEDcorp0123456789=="; @@ -267,7 +267,15 @@ async fn bun_takeover_then_vendor_revert_keeps_the_bunfig_and_scope_registry_url let (code, env) = vendor_cli(root, &[]); assert_eq!(code, 0, "vendor over the hosted pins: {env:#}"); assert_eq!(env["summary"]["applied"], 2, "{env:#}"); - find_event(&env, "skipped", Some("vendor_takeover_reverted_redirect")); + // An eject of both hosted pins: each is restored upstream, then vendored. + assert_eq!( + events(&env) + .iter() + .filter(|e| e["action"] == "applied") + .count(), + 2, + "{env:#}" + ); assert_no_event_code(&env, "redirect_revert_failed"); assert_no_registry_fallback(&env); let lock = read(root, "bun.lock"); diff --git a/crates/socket-patch-cli/tests/in_process_vendor_bun_takeover/vlt.rs b/crates/socket-patch-cli/tests/in_process_vendor_bun_takeover/vlt.rs index 89c2faf20..63b7dedb0 100644 --- a/crates/socket-patch-cli/tests/in_process_vendor_bun_takeover/vlt.rs +++ b/crates/socket-patch-cli/tests/in_process_vendor_bun_takeover/vlt.rs @@ -72,6 +72,16 @@ fn seed_manifest(root: &Path) { /// Discovery, grant, the artifact, and a view carrying the blob (the /// vendored download phase stages it from `blobContent`). async fn mock_api(server: &MockServer) { + mock_view(server).await; + hosted::mock_discovery(server).await; + hosted::mock_reference(server).await; + hosted::mock_artifact(server).await; + mock_registry(server).await; +} + +/// The patch record's view, carrying the blob: what an eject fetches for +/// the hosted pin's UUID. +async fn mock_view(server: &MockServer) { let mut view = hosted::view_body(); view["files"]["package/index.js"]["blobContent"] = json!(base64::engine::general_purpose::STANDARD.encode(PATCHED)); @@ -80,10 +90,6 @@ async fn mock_api(server: &MockServer) { .respond_with(ResponseTemplate::new(200).set_body_json(view)) .mount(server) .await; - hosted::mock_discovery(server).await; - hosted::mock_reference(server).await; - hosted::mock_artifact(server).await; - mock_registry(server).await; } /// The npm registry's version document the v5 upstream restore reads for @@ -314,13 +320,13 @@ async fn vlt_vendor_dry_run_previews_the_takeover_then_wet_vendor_completes_it() seed_manifest(root); assert_no_redirect_ledger(root); + // A hosted pin is in scope: `vendor` previews the eject of the + // lockfile's pin; the manifest is neither its input nor rewritten. + let manifest = hosted::read(root, ".socket/manifest.json"); let (code, env, stderr) = vendor_online(root, &server, &["--dry-run"]); assert_eq!(code, 0, "{env:#}\n{stderr}"); let codes = all_codes(&env); - assert!( - codes.contains(&"vendor_would_revert_redirect".to_string()), - "{env:#}" - ); + assert!(codes.contains(&"eject_planned".to_string()), "{env:#}"); assert!( !codes.contains(&"vendor_lock_entry_not_found".to_string()), "the hosted node is registry-shaped: {env:#}" @@ -334,11 +340,13 @@ async fn vlt_vendor_dry_run_previews_the_takeover_then_wet_vendor_completes_it() let (code, env, stderr) = vendor_online(root, &server, &[]); assert_eq!(code, 0, "{env:#}\n{stderr}"); + assert_eq!(env["summary"]["applied"], 1, "{env:#}"); assert!( - all_codes(&env).contains(&"vendor_takeover_reverted_redirect".to_string()), + all_codes(&env).contains(&"redirect_vlt_reinstall_required".to_string()), "{env:#}" ); assert_vendored_from_registry(root); + assert_eq!(hosted::read(root, ".socket/manifest.json"), manifest); revert_lands_on_registry(root); } @@ -524,6 +532,8 @@ async fn vlt_failed_vendor_after_the_takeover_revert_keeps_the_hosted_pin() { ) .await; hosted::mock_artifact(&service).await; + // The eject fetches the hosted pin's record from this API. + mock_view(&service).await; let cwd = root.to_str().unwrap().to_string(); let uri = service.uri(); let argv = [ diff --git a/crates/socket-patch-cli/tests/in_process_vendor_pnpm_takeover.rs b/crates/socket-patch-cli/tests/in_process_vendor_pnpm_takeover.rs index d8b9c172a..7d4bcc49d 100644 --- a/crates/socket-patch-cli/tests/in_process_vendor_pnpm_takeover.rs +++ b/crates/socket-patch-cli/tests/in_process_vendor_pnpm_takeover.rs @@ -637,7 +637,21 @@ async fn vendor_dry_run_over_hosted_pnpm_catalog_dep_previews_the_refusal() { .unwrap(); let hosted = snapshot(root); let vendor = |extra: &[&str]| { - let mut args = vec!["vendor", "--json", "--cwd", root.to_str().unwrap()]; + let uri = server.uri(); + let mut args = vec![ + "vendor", + "--json", + "--cwd", + root.to_str().unwrap(), + "--api-url", + &uri, + "--api-token", + "fake", + "--org", + ORG, + "--patch-server-url", + &uri, + ]; args.extend_from_slice(extra); run_json(root, &server.uri(), &args) }; diff --git a/crates/socket-patch-cli/tests/mode_migration_bun.rs b/crates/socket-patch-cli/tests/mode_migration_bun.rs index d4d2ec20b..c25b4b68d 100644 --- a/crates/socket-patch-cli/tests/mode_migration_bun.rs +++ b/crates/socket-patch-cli/tests/mode_migration_bun.rs @@ -432,9 +432,20 @@ fn vendored_scan(proj: &Path, api: &str, extra: &[&str]) -> (i32, String, String /// `vendor --json` (+ extra) over the staged manifest. Online: vendoring /// over a hosted pin restores its upstream registry entry first, which reads /// the registry (the [`ONLINE_ENV`] mirror); the staged manifest + blob keep -/// everything else local. +/// everything else local. With a hosted pin in scope `vendor` instead ejects +/// that pin (the manifest is ignored) and fetches its record from the API, +/// so an online run also points `--api-url` at the mock server. fn vendor_cmd(proj: &Path, extra: &[&str]) -> (i32, String, String) { + let server = ONLINE_ENV + .lock() + .unwrap_or_else(|e| e.into_inner()) + .iter() + .find(|(k, _)| k == "SOCKET_PATCH_SERVER_URL") + .map(|(_, v)| v.clone()); let mut args = vec!["vendor", "--json", "--cwd", proj.to_str().unwrap()]; + if let Some(server) = server.as_deref() { + args.extend_from_slice(&["--api-url", server, "--org", ORG, "--api-token", "fake"]); + } args.extend_from_slice(extra); run_socket(proj, &args) } @@ -1359,15 +1370,17 @@ fn take_over_to_hosted(fx: &Fixture, proj: &Path, api: &str, hp: &HostedPatch, t } /// The hosted → vendored takeover on `proj` (already hosted for DEP_A), -/// driven by `driver`: the takeover must be announced, the project left -/// purely vendored at the uuid the driver vendors under, the vendor ledger -/// must hold that record (manifest-free for `scan --mode vendored`; the -/// manifest-fed `vendor` driver keeps its manifest record too), and a fresh -/// frozen install from an empty cache must land the MARKER bytes from the -/// committed artifact. Returns that uuid. +/// driven by `driver`: the project must be left purely vendored at the +/// HOSTED patch's uuid (both drivers vendor uuid_h), the vendor ledger must +/// hold that record as a detached entry, and a fresh frozen install from an +/// empty cache must land the MARKER bytes from the committed artifact. +/// Returns that uuid. #[derive(Clone, Copy, PartialEq, Debug)] enum VendoredDriver { - /// `vendor --json` over a hand-staged manifest (uuid_v). + /// `vendor --json` with a hand-staged manifest naming uuid_v. A hosted + /// pin is in scope, so `vendor` ejects the lockfile's hosted patch + /// (uuid_h) and ignores the manifest: vendoring the manifest's patches + /// takes a full `scan --mode vendored`. VendorManifest, /// `scan --mode vendored --json --yes` — discovery + download from the /// mock API (uuid_h), then the same vendor engine. @@ -1386,13 +1399,12 @@ fn take_over_to_vendored( // Kept apart from the envelope on purpose (no tuple): CodeQL's // cleartext-logging heuristic would otherwise taint every `{vendor_env}` // assertion message with the `uuid`-named half. - let uuid = match driver { - VendoredDriver::VendorManifest => dep.uuid_v, - VendoredDriver::ScanVendored => dep.uuid_h, - }; + let uuid = dep.uuid_h; + let mut staged_manifest = None; let vendor_env = match driver { VendoredDriver::VendorManifest => { stage_manifest(fx, proj, dep); + staged_manifest = Some(read(proj, ".socket/manifest.json")); let (code, stdout, stderr) = vendor_cmd(proj, &[]); assert_eq!(code, 0, "vendor failed ({tag}): {stdout}\n{stderr}"); envelope(&stdout, &stderr) @@ -1414,13 +1426,15 @@ fn take_over_to_vendored( assert_eq!(vendor_env["summary"]["applied"], 1, "{vendor_env:#}"); assert_eq!(vendor_env["summary"]["failed"], 0, "{vendor_env:#}"); let codes = event_codes(&vendor_env); - assert!( - codes - .iter() - .any(|c| c == "vendor_takeover_reverted_redirect"), - "the takeover must be announced ({tag}, {driver:?}): event codes = {codes:?}\n\ - {vendor_env:#}" - ); + if driver == VendoredDriver::ScanVendored { + assert!( + codes + .iter() + .any(|c| c == "vendor_takeover_reverted_redirect"), + "the takeover must be announced ({tag}, {driver:?}): event codes = {codes:?}\n\ + {vendor_env:#}" + ); + } assert!( !codes.iter().any(|c| c == "redirect_revert_failed"), "the hosted revert must not be refused ({tag}, {driver:?}): {vendor_env:#}" @@ -1431,33 +1445,40 @@ fn take_over_to_vendored( state["entries"][dep.purl]["uuid"], uuid, "the vendor ledger must record the vendored patch ({tag}, {driver:?}): {state:#}" ); - match driver { - // Standalone `vendor` is fed by the staged manifest and leaves its - // record in place — the legacy manifest-tracked shape. - VendoredDriver::VendorManifest => { - let manifest = read_json(proj, ".socket/manifest.json"); - assert_eq!( - manifest["patches"][dep.purl]["uuid"], uuid, - "the manifest must record the vendored patch ({tag}, {driver:?}): {manifest:#}" - ); - } - // Vendored mode is manifest-free: the ledger entry embeds the record - // and nothing else is written under `.socket/`. - VendoredDriver::ScanVendored => { + // Both drivers write a detached entry embedding the hosted record. + assert_eq!( + state["entries"][dep.purl]["detached"], + json!(true), + "the vendored entry is detached ({tag}, {driver:?}): {state:#}" + ); + assert_eq!( + state["entries"][dep.purl]["record"]["uuid"], uuid, + "the embedded record is the vendored patch ({tag}, {driver:?}): {state:#}" + ); + match staged_manifest { + // The eject neither used nor rewrote the manifest, and its + // manifest-only patch was not vendored or reported. + Some(staged) => { assert_eq!( - state["entries"][dep.purl]["detached"], - json!(true), - "scan --mode vendored writes a detached entry ({tag}): {state:#}" + read(proj, ".socket/manifest.json"), + staged, + "the eject must not rewrite the manifest ({tag})" ); - assert_eq!( - state["entries"][dep.purl]["record"]["uuid"], uuid, - "the embedded record is the vendored patch ({tag}): {state:#}" + assert!( + !vendor_env.to_string().contains(dep.uuid_v), + "the manifest-only patch is not part of the eject ({tag}): {vendor_env:#}" ); assert!( - !proj.join(".socket/manifest.json").exists(), - "scan --mode vendored must not write a manifest ({tag})" + !proj.join(dep.vendored_rel(dep.uuid_v)).exists(), + "the manifest-only patch must not be vendored ({tag})" ); } + // Vendored mode is manifest-free: nothing else is written under + // `.socket/`. + None => assert!( + !proj.join(".socket/manifest.json").exists(), + "scan --mode vendored must not write a manifest ({tag})" + ), } let fresh = fresh_frozen_install(fx, proj, &format!("fresh-vendored-{tag}")); assert_installed(&fresh, &DEP_A, &fx.a.patched, "vendored fresh install"); @@ -1468,16 +1489,12 @@ fn take_over_to_vendored( "vendored fresh install (bystander)", ); eprintln!("HOSTED→VENDORED OK ({tag}, {driver:?}, bun {})", fx.bun_raw); - let cve = match driver { - VendoredDriver::VendorManifest => CVE_MANIFEST, - VendoredDriver::ScanVendored => CVE_VIEW, - }; manifestless_vex( fx, proj, bun_vex::BunMode::Vendored, uuid, - cve, + CVE_VIEW, api, &format!("vendored-{tag}"), ); @@ -1642,24 +1659,31 @@ async fn bun_dry_run_previews_match_wet_outcomes() { stage_manifest(&fx, &hosted, &DEP_A); let before = snapshot(&hosted); - // `vendor --dry-run`: the takeover's upstream restore is resolved - // write-free (registry lookup included) and previewed; the backend - // preview does not run against the still-hosted lock, so no false - // `vendor_lock_entry_not_found` and no refusal. + // `vendor --dry-run`: a hosted pin is in scope, so `vendor` plans an + // EJECT of the lockfile's hosted patch (the staged manifest is ignored). + // The upstream restore is resolved write-free (registry lookup + // included) and the backend preview runs against the restored text, so + // no false `vendor_lock_entry_not_found` and no refusal. let (code, stdout, stderr) = vendor_cmd(&hosted, &["--dry-run"]); assert_eq!(code, 0, "vendor --dry-run must succeed: {stdout}\n{stderr}"); let env = envelope(&stdout, &stderr); assert_eq!(env["dryRun"], true, "{env:#}"); assert_eq!(env["summary"]["failed"], 0, "{env:#}"); - let advisory = env["events"] + let planned = env["events"] .as_array() .and_then(|evs| { evs.iter() - .find(|e| e["errorCode"] == "vendor_would_revert_redirect") + .find(|e| e["action"] == "applied" && e["purl"] == DEP_A.purl) }) - .unwrap_or_else(|| panic!("expected a `vendor_would_revert_redirect` preview: {env:#}")); - assert_eq!(advisory["action"], "skipped", "{advisory:#}"); - assert_eq!(advisory["purl"], DEP_A.purl, "{advisory:#}"); + .unwrap_or_else(|| panic!("expected an eject preview for {}: {env:#}", DEP_A.purl)); + assert!( + planned.to_string().contains("eject_planned"), + "the preview is the planned eject: {planned:#}" + ); + assert!( + !env.to_string().contains(DEP_A.uuid_v), + "the manifest-only patch is not previewed: {env:#}" + ); let codes = event_codes(&env); for forbidden in ["vendor_lock_entry_not_found", "redirect_revert_failed"] { assert!( @@ -1706,18 +1730,17 @@ async fn bun_dry_run_previews_match_wet_outcomes() { ); assert_unchanged(&before, &hosted, "scan --mode vendored --dry-run"); - // The WET vendor lands exactly the takeover previewed. + // The WET vendor lands exactly the eject previewed: the hosted patch, + // not the manifest's. let (code, stdout, stderr) = vendor_cmd(&hosted, &[]); assert_eq!(code, 0, "wet vendor failed: {stdout}\n{stderr}"); let env = envelope(&stdout, &stderr); assert_eq!(env["summary"]["applied"], 1, "{env:#}"); assert!( - event_codes(&env) - .iter() - .any(|c| c == "vendor_takeover_reverted_redirect"), - "the wet vendor must perform the takeover the preview promised: {env:#}" + !env.to_string().contains(DEP_A.uuid_v), + "the wet vendor ejects only the hosted pin: {env:#}" ); - assert_pure_vendored(&fx, &hosted, &DEP_A, DEP_A.uuid_v, &hp.url); + assert_pure_vendored(&fx, &hosted, &DEP_A, DEP_A.uuid_h, &hp.url); // ── over a LIVE VENDORED state ─────────────────────────────────────── let vendored = fx.dir("live-vendored"); diff --git a/crates/socket-patch-cli/tests/mode_migration_cargo.rs b/crates/socket-patch-cli/tests/mode_migration_cargo.rs index 2f38abde6..1b6688281 100644 --- a/crates/socket-patch-cli/tests/mode_migration_cargo.rs +++ b/crates/socket-patch-cli/tests/mode_migration_cargo.rs @@ -48,6 +48,7 @@ mod prebuilt_common; use std::path::{Path, PathBuf}; use std::process::Output; +use base64::Engine as _; use sha2::{Digest, Sha256}; use socket_patch_core::hash::git_sha256::compute_git_sha256_from_bytes; use wiremock::matchers::{method, path, path_regex}; @@ -296,6 +297,10 @@ async fn mount_hosted_mocks( ) .await; let cksum = sha256_hex(crate_bytes); + let sri512 = format!( + "sha512-{}", + base64::engine::general_purpose::STANDARD.encode(sha2::Sha512::digest(crate_bytes)) + ); // Production-shaped index path: manifest-less VEX reads the hosted // uuid out of the lock's `source` (with `--patch-server-url`). let index_rel = format!("/patch-registry/cargo/{TOKEN}/{UUID_H}/index"); @@ -342,10 +347,12 @@ async fn mount_hosted_mocks( "status": "granted", "url": hosted_url, "purl": purl, + // sha512 too: a plain `vendor` eject downloads this + // same prebuilt artifact into `.socket/vendor/`. "artifacts": [{ "kind": "tarball", "url": hosted_url, - "integrity": { "sha256": cksum } + "integrity": { "sha256": cksum, "sha512": sri512 } }], "registryOverride": { "kind": "cargo-sparse", @@ -401,6 +408,17 @@ async fn mount_hosted_mocks( .respond_with(ResponseTemplate::new(200).set_body_raw(index_line, "text/plain")) .mount(server) .await; + // The granted artifact itself (what a `vendor` eject downloads). + Mock::given(method("GET")) + .and(path(format!( + "/patch/cargo/{DEP}/{version}/{TOKEN}/{UUID_H}/{DEP}-{version}.crate" + ))) + .respond_with( + ResponseTemplate::new(200) + .set_body_raw(crate_bytes.to_vec(), "application/octet-stream"), + ) + .mount(server) + .await; Mock::given(method("GET")) .and(path(format!("/dl/{DEP}/{version}/download"))) .respond_with( @@ -437,7 +455,10 @@ async fn mount_crates_index(server: &MockServer, version: &str, pristine_lock: & } /// Plain `vendor --json` ONLINE over a hosted pin on `server`'s origin -/// (`--patch-server-url`), the crates index mirrored at `index`. +/// (`--patch-server-url`, which is also the artifact service), the crates +/// index mirrored at `index`. With a hosted pin in scope, `vendor` EJECTS +/// that pin and ignores the manifest, so the patch record comes from +/// `server`'s API (`patches/view/UUID_H`). fn vendor_over_hosted( proj: &Path, cargo_home: &Path, @@ -451,6 +472,12 @@ fn vendor_over_hosted( "--json", "--patch-server-url", server.uri().as_str(), + "--api-url", + server.uri().as_str(), + "--org", + ORG, + "--api-token", + "fake", "--cwd", proj.to_str().unwrap(), ], @@ -459,6 +486,33 @@ fn vendor_over_hosted( ) } +/// The decided contract of a plain `vendor` over a hosted pin while +/// `.socket/manifest.json` also names a patch (UUID_V) for the same purl: +/// only the hosted lockfile pin (UUID_H) is vendored; the manifest is +/// neither used to select patches nor rewritten. Vendoring the manifest's +/// patches instead takes a full `scan --mode vendored`. +fn assert_ejected_hosted_pin_only(proj: &Path, envelope: &serde_json::Value, manifest: &str) { + assert_eq!(envelope["status"], "success", "{envelope:#}"); + assert_eq!(envelope["summary"]["applied"], 1, "{envelope:#}"); + assert!( + proj.join(format!(".socket/vendor/cargo/{UUID_H}")).is_dir(), + "the hosted pin's patch is vendored: {envelope:#}" + ); + assert!( + !proj.join(format!(".socket/vendor/cargo/{UUID_V}")).exists(), + "the manifest-only patch must not be vendored while a hosted pin is in scope" + ); + assert!( + !envelope.to_string().contains(UUID_V), + "the manifest-only patch is not part of the eject: {envelope:#}" + ); + assert_eq!( + read(proj, ".socket/manifest.json"), + manifest, + "the eject never rewrites the manifest" + ); +} + /// Manifest-less VEX over a post-takeover fresh checkout: `wired` (uuid + /// marker) must attest, `displaced` never. Runs on a blocking thread (the /// shared `PatchApi` brings its own runtime). @@ -957,19 +1011,19 @@ async fn hosted_then_vendored_takeover_leaves_pure_vendored() { "hosted mode writes no ledger" ); - // B: plain `vendor` over the hosted state — the takeover. + // B: plain `vendor` over the hosted state. A manifest naming a + // DIFFERENT patch (UUID_V) for the same purl is staged too: with a + // hosted pin in scope, `vendor` ejects exactly the lockfile's hosted + // patch (UUID_H) and ignores the manifest (vendoring the manifest's + // patches takes a full `scan --mode vendored`). let index = mount_crates_index(&server, &version, &String::from_utf8_lossy(&lock_pristine)).await; stage_patch(&proj, &purl, &orig, &patched); + let manifest = read(&proj, ".socket/manifest.json"); let (code, stdout, stderr) = vendor_over_hosted(&proj, &cargo_home, &server, &index); assert_eq!(code, 0, "vendor failed: {stdout}\n{stderr}"); let envelope: serde_json::Value = serde_json::from_str(&stdout).expect("json envelope"); - assert_eq!(envelope["summary"]["applied"], 1, "{stdout}"); - // C7: the PLAIN vendor command surfaces the takeover (pre-fix: silent). - assert!( - stdout.contains("vendor_takeover_reverted_redirect"), - "takeover advisory missing from the plain vendor envelope: {stdout}" - ); + assert_ejected_hosted_pin_only(&proj, &envelope, &manifest); // The project is FULLY vendored: the hosted Cargo.toml pin and the // registries block are gone, [patch.crates-io] + detached lock are in, @@ -982,7 +1036,7 @@ async fn hosted_then_vendored_takeover_leaves_pure_vendored() { ); assert!( toml.contains("[patch.crates-io]") - && toml.contains(&format!(".socket/vendor/cargo/{UUID_V}/")), + && toml.contains(&format!(".socket/vendor/cargo/{UUID_H}/")), "the vendored wiring lives in Cargo.toml: {toml}" ); let config = read(&proj, ".cargo/config.toml"); @@ -1001,10 +1055,11 @@ async fn hosted_then_vendored_takeover_leaves_pure_vendored() { !lock_block.contains("source ="), "vendored lock entry is detached: {lock_block}" ); - assert_lock_version(&proj, &tagged(&version, UUID_V), "hosted -> vendored"); + assert_lock_version(&proj, &tagged(&version, UUID_H), "hosted -> vendored"); // C: the vendored contract — fresh checkout, EMPTY home, offline locked - // build (pre-fix: "no matching package named cfg-if found"). + // build (pre-fix: "no matching package named cfg-if found"). The + // ejected hosted patch attests; the manifest-only one never does. let (fresh, home) = fresh_checkout(&proj, tmp.path(), "c"); assert_build_ok( "cargo build --locked --offline (vendored fresh checkout)", @@ -1015,8 +1070,8 @@ async fn hosted_then_vendored_takeover_leaves_pure_vendored() { home: home.clone(), purl: purl.clone(), patched: patched.clone(), - wired: (UUID_V, Marker::Vendored), - displaced: UUID_H, + wired: (UUID_H, Marker::Vendored), + displaced: UUID_V, server_uri: server.uri(), } .run() @@ -1115,11 +1170,15 @@ async fn double_takeover_a_b_a_preserves_lock_originals() { assert_eq!(code, 0, "hosted scan failed: {stdout}\n{stderr}"); // A again: vendor back (restoring the hosted pin's crates.io entry - // first). + // first). The hosted pin is in scope, so plain `vendor` ejects the + // lockfile's hosted patch (UUID_H); the manifest's UUID_V is ignored. let index = mount_crates_index(&server, &version, &String::from_utf8_lossy(&lock_pristine)).await; + let manifest = read(&proj, ".socket/manifest.json"); let (code, stdout, stderr) = vendor_over_hosted(&proj, &cargo_home, &server, &index); assert_eq!(code, 0, "re-vendor failed: {stdout}\n{stderr}"); + let envelope: serde_json::Value = serde_json::from_str(&stdout).expect("json envelope"); + assert_ejected_hosted_pin_only(&proj, &envelope, &manifest); // The vendored ledger's lock originals are the PRISTINE crates.io values // — the only offline-recoverable home of the registry checksum. @@ -1138,8 +1197,9 @@ async fn double_takeover_a_b_a_preserves_lock_originals() { ); // The vendored contract still holds after the round trip, and the lock - // names the vendored patch again (tagged), not the hosted one. - assert_lock_version(&proj, &tagged(&version, UUID_V), "A -> B -> A"); + // names the ejected hosted patch, now vendored (tagged), not the + // manifest's original vendored one. + assert_lock_version(&proj, &tagged(&version, UUID_H), "A -> B -> A"); let (fresh, home) = fresh_checkout(&proj, tmp.path(), "aba"); assert_build_ok( "cargo build --locked --offline (A->B->A fresh checkout)", @@ -1150,8 +1210,8 @@ async fn double_takeover_a_b_a_preserves_lock_originals() { home: home.clone(), purl: purl.clone(), patched: patched.clone(), - wired: (UUID_V, Marker::Vendored), - displaced: UUID_H, + wired: (UUID_H, Marker::Vendored), + displaced: UUID_V, server_uri: server.uri(), } .run() @@ -1185,8 +1245,9 @@ async fn double_takeover_a_b_a_preserves_lock_originals() { } // ── FAIL CLOSED: vendoring over an unrestorable hosted pin refuses ───────── -// When the hosted pin's crates.io entry cannot be re-resolved (here: -// `--offline`), the vendor run must refuse the purl with an actionable error +// When the hosted pin cannot be ejected (`--offline`: no patch record; online +// with an index that cannot re-resolve the crates.io entry), the vendor run +// must refuse the purl with an actionable error // instead of creating the mixed unbuildable state and reporting success. The // cargo backend's `hosted_redirect_live` guard backstops a half-reverted // project whose lock is back on crates.io but whose manifest still pins. @@ -1246,11 +1307,30 @@ async fn vendor_over_unrestorable_hosted_pin_is_refused() { ], &cargo_home, ); + // With a hosted pin in scope, plain `vendor` ejects that pin (the staged + // manifest is ignored), and an offline eject cannot fetch its record: + // refused up front, before any restore. + assert_eq!(code, 1, "must fail closed: {stdout}\n{stderr}"); + assert!( + stdout.contains("offline_eject_unavailable") && stdout.contains("without --offline"), + "actionable offline refusal missing: {stdout}" + ); + assert_eq!(read(&proj, "Cargo.toml"), toml_before); + assert_eq!(read(&proj, "Cargo.lock"), lock_before); + + // Online, but the crates.io entry cannot be re-resolved (the index has + // no row for the crate): the eject's upstream restore refuses the pin + // with the checkout remedy instead of creating the mixed unbuildable + // state. + let (code, stdout, stderr) = vendor_over_hosted( + &proj, + &cargo_home, + &server, + &format!("{}/no-such-index", server.uri()), + ); assert_eq!(code, 1, "must fail closed: {stdout}\n{stderr}"); assert!( - stdout.contains("redirect_revert_failed") - && stdout.contains("cannot vendor over the live hosted pin") - && stdout.contains("git checkout --"), + stdout.contains("redirect_revert_failed") && stdout.contains("git checkout --"), "actionable refusal (with the checkout remedy) missing: {stdout}" ); // Nothing was half-applied: the hosted wiring is untouched and no diff --git a/crates/socket-patch-cli/tests/mode_migration_npm.rs b/crates/socket-patch-cli/tests/mode_migration_npm.rs index c8091f68e..1af488acc 100644 --- a/crates/socket-patch-cli/tests/mode_migration_npm.rs +++ b/crates/socket-patch-cli/tests/mode_migration_npm.rs @@ -616,10 +616,23 @@ fn assert_pure_vendored_and_round_trip( ) { let proj = &fx.proj; - // The takeover is surfaced on the vendor envelope (C7 twin). + // A hosted pin is in scope, so plain `vendor` ejected the lockfile's + // hosted patch (UUID_H) and ignored the staged manifest's UUID_V: + // vendoring manifest patches takes a full `scan --mode vendored`. The + // manifest-only patch is neither reported nor vendored, and the + // manifest is not rewritten. assert!( - vendor_stdout.contains("vendor_takeover_reverted_redirect"), - "takeover advisory missing from the vendor envelope ({tag}): {vendor_stdout}" + !vendor_stdout.contains(UUID_V), + "the manifest-only patch is not part of the eject ({tag}): {vendor_stdout}" + ); + assert!( + proj.join(format!(".socket/vendor/npm/{UUID_H}")).is_dir() + && !proj.join(format!(".socket/vendor/npm/{UUID_V}")).exists(), + "only the hosted pin's patch is vendored ({tag})" + ); + assert!( + read(proj, ".socket/manifest.json").contains(UUID_V), + "the eject must not rewrite the manifest ({tag})" ); // v5 hosted mode keeps no ledger: the lock is the only hosted state, @@ -865,6 +878,9 @@ async fn classic_hosted_then_vendored_takeover_round_trips_to_registry() { // --patch-server-url. let (registry, lock_upstream) = mount_registry_from_classic_lock(&server, &String::from_utf8_lossy(&lock_pristine)).await; + // The staged manifest names UUID_V, but the hosted pin is in scope: + // `vendor` ejects the hosted patch (UUID_H), fetching its record from + // the mock API. stage_patch(&proj, &fx.orig, &fx.patched); let (code, stdout, stderr) = run_socket_env( &proj, @@ -873,6 +889,12 @@ async fn classic_hosted_then_vendored_takeover_round_trips_to_registry() { "--json", "--patch-server-url", server.uri().as_str(), + "--api-url", + server.uri().as_str(), + "--org", + ORG, + "--api-token", + "fake", "--cwd", proj.to_str().unwrap(), ], @@ -882,15 +904,16 @@ async fn classic_hosted_then_vendored_takeover_round_trips_to_registry() { let envelope: serde_json::Value = serde_json::from_str(&stdout).expect("json envelope"); assert_eq!(envelope["summary"]["applied"], 1, "{stdout}"); - // Manifest-less VEX of the takeover's terminal (vendored) state. + // Manifest-less VEX of the eject's terminal (vendored) state: the + // hosted patch attests, the manifest-only one never does. tokio::task::block_in_place(|| { classic_manifestless_vex( &fx, "classic-h2v-vex", yarn_classic_vex::Wiring::Vendored, - UUID_V, - "CVE-2026-99999", UUID_H, + "CVE-2026-3333", + UUID_V, None, &lock_pristine, ) @@ -939,7 +962,9 @@ async fn berry_hosted_then_vendored_takeover_round_trips_to_registry() { // B: vendor over the live hosted pin — the takeover. Online against the // REAL registry: berry's restore re-derives the 10c0 checksum from the // registry tarball. The mock origin is named hosted via - // --patch-server-url. + // --patch-server-url. The staged manifest names UUID_V, but the hosted + // pin is in scope: `vendor` ejects the hosted patch (UUID_H), fetching + // its record from the mock API. stage_patch(&proj, &fx.orig, &fx.patched); let (code, stdout, stderr) = run_socket( &proj, @@ -948,6 +973,12 @@ async fn berry_hosted_then_vendored_takeover_round_trips_to_registry() { "--json", "--patch-server-url", server.uri().as_str(), + "--api-url", + server.uri().as_str(), + "--org", + ORG, + "--api-token", + "fake", "--cwd", proj.to_str().unwrap(), ], @@ -957,10 +988,11 @@ async fn berry_hosted_then_vendored_takeover_round_trips_to_registry() { assert_eq!(envelope["summary"]["applied"], 1, "{stdout}"); // MANIFEST-LESS VEX over the post-takeover (pure vendored) state, before - // the round trip below reverts it: the superseded HOSTED patch (UUID_H, - // same advisory) must never be attested alongside the vendored one — the - // matrix's exact-statement-set assertions would see it. - let tgz_rel = format!(".socket/vendor/npm/{UUID_V}/{DEP}-{DEP_VERSION}.tgz"); + // the round trip below reverts it: the ejected hosted patch (UUID_H) is + // now the vendored one, and the manifest-only UUID_V (same advisory) + // must never be attested alongside it — the matrix's + // exact-statement-set assertions would see it. + let tgz_rel = format!(".socket/vendor/npm/{UUID_H}/{DEP}-{DEP_VERSION}.tgz"); let registry_state = [ ("yarn.lock", lock_pristine.clone()), ("package.json", pkg_json_pristine.clone().into_bytes()), @@ -980,8 +1012,8 @@ async fn berry_hosted_then_vendored_takeover_round_trips_to_registry() { yarnrc: &yarnrc, registry_state: ®istry_state, purl: PURL, - uuid: UUID_V, - vulns: &[(GHSA, &["CVE-2026-99999"])], + uuid: UUID_H, + vulns: &[(GHSA, &["CVE-2026-3333"])], patched: &fx.patched, pristine: &fx.orig, installed: "node_modules/left-pad/index.js", diff --git a/crates/socket-patch-cli/tests/mode_migration_pypi.rs b/crates/socket-patch-cli/tests/mode_migration_pypi.rs index 7618c5653..6cd3dcf16 100644 --- a/crates/socket-patch-cli/tests/mode_migration_pypi.rs +++ b/crates/socket-patch-cli/tests/mode_migration_pypi.rs @@ -233,6 +233,34 @@ async fn mount_hosted_api_serving( hosted_url } +/// The patch API's record of `UUID` (the one [`stage_manifest`] stages): +/// plain `vendor` over a hosted pin ejects it, fetching the pinned UUID's +/// record from the API rather than reading the agent manifest. +async fn mount_patch_view(server: &MockServer) { + Mock::given(method("GET")) + .and(path(format!("/v0/orgs/{ORG}/patches/view/{UUID}"))) + .respond_with(ResponseTemplate::new(200).set_body_json(json!({ + "uuid": UUID, + "purl": PURL, + "publishedAt": "2026-01-01T00:00:00Z", + "files": { "six.py": { + "beforeHash": compute_git_sha256_from_bytes(ORIG), + "afterHash": compute_git_sha256_from_bytes(PATCHED), + }}, + "vulnerabilities": {}, + "description": "pypi mode takeover fixture", + "license": "MIT", + "tier": "free" + }))) + .mount(server) + .await; +} + +/// The flags pointing a `vendor` eject's record fetch at `uri`. +fn api_args(uri: &str) -> [&str; 6] { + ["--api-url", uri, "--org", ORG, "--api-token", "fake-token"] +} + /// Stage the manifest and vendor the project; `files` carry the wiring. fn vendor_project(root: &Path, files: &[&str]) { stage_manifest(root); @@ -458,12 +486,17 @@ async fn assert_all_hosted_requirements_unwind(pristine: &str) { if unwind[0] == "vendor" { stage_manifest(&root); - // `--patch-server-url` (which names the hosted origin to take - // over) also moves the vendored download onto this server. + // `--patch-server-url` (which names the hosted origin) also + // moves the vendored download onto this server; the eject + // fetches the pinned record from its API. prebuilt_common::mount_project(&server, &root).await; + mount_patch_view(&server).await; } let mut args = unwind.clone(); args.extend(["--patch-server-url", uri.as_str()]); + if unwind[0] == "vendor" { + args.extend(api_args(&uri)); + } let (code, env) = run_cli(&root, &args, &[]); assert_eq!(code, 0, "{unwind:?} over {pristine:?}: {env:#}"); let after = std::fs::read_to_string(root.join("requirements.txt")).unwrap(); @@ -471,7 +504,7 @@ async fn assert_all_hosted_requirements_unwind(pristine: &str) { assert!( after.contains(&format!(".socket/vendor/pypi/{UUID}/")) && !after.contains(&hosted_url), - "the takeover leaves the project vendored:\n{after}" + "the eject leaves the project vendored:\n{after}" ); } else { assert_eq!(after, pristine, "{unwind:?} restores the pristine file"); @@ -2597,12 +2630,11 @@ async fn pipenv_hosted_to_vendored_keeps_the_requirements_patched() { // which also rehosts the prebuilt download: serve the vendored wheel // from the same mock. prebuilt_common::mount_project(&server, &root).await; + mount_patch_view(&server).await; let uri = server.uri(); - let (code, env) = run_cli( - &root, - &["vendor", "--patch-server-url", &uri, "--vendor-url", &uri], - &[], - ); + let mut args = vec!["vendor", "--patch-server-url", &uri, "--vendor-url", &uri]; + args.extend(api_args(&uri)); + let (code, env) = run_cli(&root, &args, &[]); assert_eq!(code, 0, "vendor takeover: {env:#}"); let lock = std::fs::read_to_string(root.join("Pipfile.lock")).unwrap(); assert!( @@ -2727,7 +2759,10 @@ async fn pipenv_vendor_wires_the_exported_requirements_too() { } stage_manifest(&root); prebuilt_common::mount_project(&server, &root).await; - let (code, env) = run_cli(&root, &["vendor", "--patch-server-url", uri.as_str()], &[]); + mount_patch_view(&server).await; + let mut args = vec!["vendor", "--patch-server-url", uri.as_str()]; + args.extend(api_args(&uri)); + let (code, env) = run_cli(&root, &args, &[]); assert_eq!(code, 0, "takeover: {env:#}"); assert_both_vendored(&root, "takeover"); } diff --git a/crates/socket-patch-cli/tests/vendor/npm_fixture.rs b/crates/socket-patch-cli/tests/vendor/npm_fixture.rs index c0c6ba9c1..bb9b7c6f1 100644 --- a/crates/socket-patch-cli/tests/vendor/npm_fixture.rs +++ b/crates/socket-patch-cli/tests/vendor/npm_fixture.rs @@ -289,6 +289,22 @@ pub(crate) fn pin_hosted(fx: &NpmFixture) -> Vec { /// A wiremock npm registry answering left-pad@1.3.0's version document. pub(crate) async fn mock_npm_registry() -> MockServer { let server = MockServer::start().await; + let mut view = patch_record( + &compute_git_sha256_from_bytes(ORIG_INDEX), + &compute_git_sha256_from_bytes(PATCHED_INDEX), + ); + view["purl"] = json!(PURL); + view["publishedAt"] = json!("2026-01-01T00:00:00Z"); + view["files"]["package/index.js"]["blobContent"] = json!(base64::Engine::encode( + &base64::engine::general_purpose::STANDARD, + PATCHED_INDEX + )); + prebuilt_common::mount_view(&server, &view, None).await; + Mock::given(method("GET")) + .and(path(format!("/v0/orgs/test-org/patches/view/{UUID}"))) + .respond_with(ResponseTemplate::new(200).set_body_json(view)) + .mount(&server) + .await; Mock::given(method("GET")) .and(path("/left-pad/1.3.0")) .respond_with(ResponseTemplate::new(200).set_body_json(json!({ @@ -306,15 +322,14 @@ pub(crate) async fn mock_npm_registry() -> MockServer { } /// `vendor --json --patch-server-url ` through the binary, -/// ONLINE against the mock registry (the upstream restore needs it) but -/// anonymous, so no other network path opens. +/// ONLINE against the mock registry and patch service. pub(crate) fn vendor_online(fx: &NpmFixture, registry: &str, extra: &[&str]) -> (i32, Value) { let root = fx.root().to_str().unwrap(); let mut args = vec![ "vendor", "--json", "--patch-server-url", - HOSTED_ORIGIN, + registry, "--cwd", root, ]; @@ -323,7 +338,10 @@ pub(crate) fn vendor_online(fx: &NpmFixture, registry: &str, extra: &[&str]) -> fx.root(), &args, &[ - ("SOCKET_NO_API_TOKEN", "1"), + ("SOCKET_API_URL", registry), + ("SOCKET_VENDOR_URL", registry), + ("SOCKET_API_TOKEN", "fake-token"), + ("SOCKET_ORG_SLUG", "test-org"), ("SOCKET_NPM_REGISTRY", registry), ], ); diff --git a/crates/socket-patch-cli/tests/vendor/vendor_failure_paths.rs b/crates/socket-patch-cli/tests/vendor/vendor_failure_paths.rs index 2530d937e..8934b7608 100644 --- a/crates/socket-patch-cli/tests/vendor/vendor_failure_paths.rs +++ b/crates/socket-patch-cli/tests/vendor/vendor_failure_paths.rs @@ -207,23 +207,21 @@ fn corrupt_pre_v5_redirect_ledger_is_ignored_by_vendor() { ); } -/// Dry run over a purl the lockfile pins hosted: the run warns -/// `vendor_would_revert_redirect` (an UNCOUNTED advisory — dry/wet takeover -/// parity) after resolving the upstream entry exactly like a wet run -/// would, and writes nothing — no lock edit, no artifact, no ledger. +/// Dry run over a hosted pin with an agent manifest uses the same eject +/// preview as a manifest-free checkout, and writes nothing. #[tokio::test] -async fn dry_run_over_hosted_pin_warns_and_writes_nothing() { +async fn dry_run_over_hosted_pin_previews_eject_and_writes_nothing() { let fx = npm_fixture(); let hosted_lock = pin_hosted(&fx); let registry = mock_npm_registry().await; let (code, env) = vendor_online(&fx, ®istry.uri(), &["--dry-run"]); assert_eq!(code, 0, "the dry run itself succeeds: {env:#}"); - let warned = find_event(&env, "skipped", Some("vendor_would_revert_redirect")); + let warned = find_event(&env, "applied", Some("eject_planned")); assert_eq!(warned["purl"], PURL); assert_eq!( env["summary"]["skipped"], 0, - "the takeover advisory is uncounted: {env:#}" + "no package is skipped: {env:#}" ); assert_eq!( fx.lock_bytes(), @@ -242,7 +240,7 @@ async fn dry_run_over_hosted_pin_warns_and_writes_nothing() { } /// The wet twin: vendoring over the hosted pin first restores the upstream -/// registry entry (`vendor_takeover_reverted_redirect`), then vendors; the +/// registry entry, then vendors; the /// vendor ledger records the UPSTREAM entry as the original, so `vendor /// --revert` lands on the registry entry — never back on the hosted URL. #[tokio::test] @@ -254,12 +252,11 @@ async fn vendor_over_hosted_pin_restores_upstream_then_revert_returns_to_registr let (code, env) = vendor_online(&fx, ®istry.uri(), &[]); assert_eq!(code, 0, "{env:#}"); find_event(&env, "applied", None); - let warned = find_event(&env, "skipped", Some("vendor_takeover_reverted_redirect")); assert!( - warned["reason"] - .as_str() - .is_some_and(|m| m.contains("package-lock.json")), - "the advisory names the restored lockfile: {warned:#}" + !events(&env) + .iter() + .any(|e| e["errorCode"] == "vendor_takeover_reverted_redirect"), + "eject restores upstream as its own step: {env:#}" ); let lock: Value = serde_json::from_slice(&fx.lock_bytes()).unwrap(); let entry = &lock["packages"]["node_modules/left-pad"]; @@ -288,10 +285,8 @@ async fn vendor_over_hosted_pin_restores_upstream_then_revert_returns_to_registr assert_eq!(entry["integrity"], UPSTREAM_INTEGRITY, "{lock:#}"); } -/// A hosted pin whose upstream entry cannot be restored (here: `--offline`, -/// so the registry lookup is impossible) fails the purl CLOSED with -/// `redirect_revert_failed` naming the checkout remedy — vendoring over a -/// live hosted pin would record the hosted fragment as the "original". +/// An offline hosted conversion refuses before fetching records or +/// restoring upstream, even when an agent manifest is present. #[test] fn unrestorable_hosted_pin_fails_closed() { let fx = npm_fixture(); @@ -302,12 +297,11 @@ fn unrestorable_hosted_pin_fails_closed() { &["--offline", "--patch-server-url", HOSTED_ORIGIN], ); assert_eq!(code, 1, "{env:#}"); - let failed = find_event(&env, "failed", Some("redirect_revert_failed")); - assert_eq!(failed["purl"], PURL); - let detail = failed["error"].as_str().unwrap_or_default(); + assert_eq!(env["error"]["code"], "offline_eject_unavailable"); + let detail = env["error"]["message"].as_str().unwrap_or_default(); assert!( - detail.contains("git checkout") && detail.contains("package-lock.json"), - "the refusal names the checkout remedy for the lockfile: {env:#}" + detail.contains("offline") && detail.contains("re-run without --offline"), + "{env:#}" ); assert_eq!( fx.lock_bytes(), diff --git a/crates/socket-patch-cli/tests/vendor/vendor_render.rs b/crates/socket-patch-cli/tests/vendor/vendor_render.rs index 471f9871a..bfba61ca1 100644 --- a/crates/socket-patch-cli/tests/vendor/vendor_render.rs +++ b/crates/socket-patch-cli/tests/vendor/vendor_render.rs @@ -464,11 +464,9 @@ fn human_corrupt_pre_v5_redirect_ledger_vendors_normally() { ); } -/// Human unrestorable-hosted-pin surface: an offline run over a hosted pin -/// prints the `Cannot vendor …: cannot restore the upstream entry` line and -/// counts the refusal. +/// Human offline-hosted surface: an early refusal names the offline gate. #[test] -fn human_unrestorable_hosted_pin_prints_cannot_restore() { +fn human_offline_hosted_pin_reports_eject_refusal() { let fx = npm_fixture(); let hosted_lock = pin_hosted(&fx); @@ -476,13 +474,12 @@ fn human_unrestorable_hosted_pin_prints_cannot_restore() { human_vendor(&fx, &["--offline", "--patch-server-url", HOSTED_ORIGIN]); assert_eq!(code, 1, "stdout:\n{stdout}\nstderr:\n{stderr}"); assert!( - stderr.contains("Cannot vendor pkg:npm/left-pad@1.3.0:") - && stderr.contains("cannot restore the upstream entry"), - "the human takeover-refusal line: {stderr}" + stderr.contains("offline_eject_unavailable") && stderr.contains("re-run without --offline"), + "the human eject-refusal line: {stderr}" ); assert!( - stdout.contains("Vendored 0 packages; 1 failed."), - "the fail-closed refusal is counted: {stdout}" + !stdout.contains("Vendored"), + "an early refusal never starts vendoring: {stdout}" ); assert_eq!(fx.lock_bytes(), hosted_lock, "lock untouched"); } diff --git a/crates/socket-patch-cli/tests/vendor_eject.rs b/crates/socket-patch-cli/tests/vendor_eject.rs index 8218d4ef4..3c72cf75b 100644 --- a/crates/socket-patch-cli/tests/vendor_eject.rs +++ b/crates/socket-patch-cli/tests/vendor_eject.rs @@ -10,8 +10,7 @@ //! Every run goes through the built binary with a scrubbed environment: the //! API, the npm registry (`SOCKET_NPM_REGISTRY`) and the patch-server origin //! (`SOCKET_PATCH_SERVER_URL`, what makes the mock hosted URL count as -//! hosted) all point at a wiremock; `SOCKET_VENDOR_SOURCE=build` keeps the -//! vendoring service out of it. +//! hosted) all point at a wiremock, including the prebuilt artifact service. #[path = "common/envelope.rs"] mod envelope; @@ -271,6 +270,45 @@ async fn eject_vendors_hosted_pins_and_revert_returns_to_upstream() { assert!(!p.redirect_state().exists()); } +/// Hosted pins remain the input when an agent manifest is present. The +/// manifest does not list this patch and nothing is installed locally. +#[tokio::test] +async fn eject_reads_hosted_lockfiles_even_with_a_manifest() { + let p = hosted_project(true).await; + std::fs::remove_dir_all(p.root().join("node_modules")).unwrap(); + std::fs::create_dir(p.root().join(".socket")).unwrap(); + let manifest_path = p.root().join(".socket/manifest.json"); + let manifest = b"{\"patches\": {}}\n"; + std::fs::write(&manifest_path, manifest).unwrap(); + mock_registry(&p).await; + mock_view(&p).await; + + let original_lock = p.lock_bytes(); + let (code, preview) = p.run_json(&["vendor", "--dry-run"]); + assert_eq!(code, 0, "{preview:#}"); + assert_eq!( + find_event(&preview, "applied", Some("eject_planned"))["purl"], + PURL + ); + assert_eq!(p.lock_bytes(), original_lock); + assert!(!p.artifact().exists()); + + let (code, env) = p.run_json(&["vendor"]); + assert_eq!(code, 0, "{env:#}"); + assert_eq!(find_event(&env, "applied", None)["purl"], PURL); + assert!(p.artifact().is_file()); + assert!(p.lock()["packages"]["node_modules/left-pad"]["resolved"] + .as_str() + .unwrap() + .contains(".socket/vendor/")); + assert_eq!(std::fs::read(&manifest_path).unwrap(), manifest); + assert!(!p.root().join("node_modules").exists()); + let state: Value = + serde_json::from_slice(&std::fs::read(p.root().join(".socket/vendor/state.json")).unwrap()) + .unwrap(); + assert_eq!(state["entries"][PURL]["detached"], true); +} + /// Human eject: the first line announces the eject. #[tokio::test] async fn eject_human_output_announces_the_eject() { diff --git a/crates/socket-patch-cli/tests/vendor_eject_bun_lockb.rs b/crates/socket-patch-cli/tests/vendor_eject_bun_lockb.rs index 6214e7314..5b9c6f763 100644 --- a/crates/socket-patch-cli/tests/vendor_eject_bun_lockb.rs +++ b/crates/socket-patch-cli/tests/vendor_eject_bun_lockb.rs @@ -199,8 +199,8 @@ async fn hosted_project(writer: &str) -> Project { project } -/// `.socket/manifest.json` + the after-hash blob: the per-purl takeover -/// (instead of the manifest-less eject). +/// Legacy agent state alongside the hosted pins. Vendor still selects +/// the lockfile patches and runs the same eject as a manifest-free repo. fn stage_record(root: &Path) { let socket = root.join(".socket"); std::fs::create_dir_all(socket.join("blobs")).unwrap(); @@ -279,9 +279,7 @@ async fn takeover_vendors_over_a_hosted_bun_lockb_and_reverts_exactly() { let (code, env) = p.run_json(&["vendor", "--dry-run"]); assert_eq!(code, 0, "{writer}: dry run: {env:#}"); assert!( - codes(&env) - .iter() - .any(|c| c == "vendor_would_revert_redirect"), + codes(&env).iter().any(|c| c == "eject_planned"), "{writer}: {env:#}" ); assert_eq!(p.lock(), hosted, "{writer}: a dry run writes nothing"); @@ -292,7 +290,7 @@ async fn takeover_vendors_over_a_hosted_bun_lockb_and_reverts_exactly() { assert_eq!(env["summary"]["applied"], 1, "{writer}: {env:#}"); let codes = codes(&env); assert!( - codes + !codes .iter() .any(|c| c == "vendor_takeover_reverted_redirect"), "{writer}: {env:#}" @@ -395,18 +393,11 @@ async fn rollback_and_offline_vendor_refuse_with_the_checkout_remedy() { let (code, env) = p.run_json(&["vendor", "--offline"]); assert_eq!(code, 1, "{env:#}"); - let refused = env["events"] - .as_array() - .and_then(|events| { - events - .iter() - .find(|e| e["errorCode"] == "redirect_revert_failed") - }) - .unwrap_or_else(|| panic!("expected redirect_revert_failed: {env:#}")); + assert_eq!(env["error"]["code"], "offline_eject_unavailable", "{env:#}"); assert!( - refused["error"] + env["error"]["message"] .as_str() - .is_some_and(|e| e.contains("offline") && e.contains("git checkout -- bun.lockb")), + .is_some_and(|e| e.contains("re-run without --offline")), "{env:#}" ); assert_eq!(p.lock(), hosted, "a refused vendor writes nothing"); @@ -487,7 +478,7 @@ async fn takeover_and_revert_keep_the_bunfig_registry_tarball_url() { assert_eq!(env["summary"]["applied"], 1, "round {round}: {env:#}"); let codes = codes(&env); assert!( - codes + !codes .iter() .any(|c| c == "vendor_takeover_reverted_redirect"), "round {round}: {env:#}" diff --git a/crates/socket-patch-cli/tests/vendor_eject_fresh_checkout.rs b/crates/socket-patch-cli/tests/vendor_eject_fresh_checkout.rs index dce1abede..28e0d3244 100644 --- a/crates/socket-patch-cli/tests/vendor_eject_fresh_checkout.rs +++ b/crates/socket-patch-cli/tests/vendor_eject_fresh_checkout.rs @@ -2,10 +2,8 @@ //! and lockfiles exist — no `node_modules`, no Cargo home, no `.socket/`. //! //! The eject is lockfile-only by contract: it restores each hosted pin's -//! upstream registry entry first, so the vendor engine sees an ordinary -//! registry pin with a registry checksum and fetches the pristine source -//! from the registry (verified against that checksum) instead of requiring -//! an installed tree. Every registry and API endpoint is a wiremock. +//! upstream registry metadata for reversal, then downloads the prebuilt +//! patched artifact. Every registry and API endpoint is a wiremock. #[path = "common/hermetic.rs"] mod hermetic; @@ -135,8 +133,7 @@ fn applied(env: &Value, purl: &str) -> bool { }) } -/// npm: a hosted package-lock.json with NO `node_modules`. The pristine -/// tarball comes from the registry the restored entry names. +/// npm: a hosted package-lock.json with NO `node_modules`. #[tokio::test] async fn npm_eject_needs_no_installed_tree() { const UUID: &str = "9f6b2c4e-1d3a-4f6b-8c2d-7e5a9b1c3d5f"; @@ -215,9 +212,114 @@ async fn npm_eject_needs_no_installed_tree() { assert!(!lock.contains(&hosted), "no hosted residue: {lock}"); } +#[tokio::test] +async fn npm_eject_downloads_every_hosted_patch_from_the_lockfile() { + let server = MockServer::start().await; + let tmp = tempfile::tempdir().unwrap(); + let root = tmp.path().join("proj"); + std::fs::create_dir(&root).unwrap(); + let patches = [ + ("left-pad", "1.3.0", "9f6b2c4e-1d3a-4f6b-8c2d-7e5a9b1c3d5f"), + ("ms", "2.1.3", "2b1f6c1e-8d3a-4f6b-9c2d-7e5a9b1c3d11"), + ]; + let mut lock = json!({ + "lockfileVersion": 3, + "packages": { + "": { "dependencies": { "left-pad": "1.3.0", "ms": "2.1.3", "unpatched": "1.0.0" } }, + "node_modules/unpatched": { "version": "1.0.0", "resolved": "https://registry.npmjs.org/unpatched/-/unpatched-1.0.0.tgz" } + } + }); + let untouched = lock["packages"]["node_modules/unpatched"].clone(); + for (name, version, uuid) in patches { + let purl = format!("pkg:npm/{name}@{version}"); + mock_view( + &server, + uuid, + &purl, + "package/index.js", + b"original\n", + b"patched\n", + ) + .await; + Mock::given(method("GET")) + .and(path(format!("/{name}/{version}"))) + .respond_with(ResponseTemplate::new(200).set_body_json(json!({ + "name": name, "version": version, + "dist": { + "tarball": format!("{}/{name}/-/{name}-{version}.tgz", server.uri()), + "integrity": "sha512-UPSTREAMupstreamUPSTREAM==" + } + }))) + .mount(&server) + .await; + // No upstream tarball endpoint: vendor needs only its metadata + // for --revert, and downloads the patched service artifact. + lock["packages"][format!("node_modules/{name}")] = json!({ + "version": version, + "resolved": format!("{}/patch/npm/{name}/{version}/55555555-5555-4555-8555-555555555555/{uuid}/{name}-{version}.tgz", server.uri()), + "integrity": "sha512-HOSTEDpatchedHOSTEDpatched==" + }); + } + std::fs::write(root.join("package.json"), serde_json::to_vec(&json!({ + "name": "fixture", "version": "1.0.0", "dependencies": lock["packages"][""]["dependencies"] + })).unwrap()).unwrap(); + std::fs::write( + root.join("package-lock.json"), + serde_json::to_vec(&lock).unwrap(), + ) + .unwrap(); + + let (code, env) = run_json(&root, &server, &["vendor"]); + assert_eq!(code, 0, "{env:#}"); + assert_eq!(env["summary"]["applied"], 2, "{env:#}"); + let lock: Value = + serde_json::from_slice(&std::fs::read(root.join("package-lock.json")).unwrap()).unwrap(); + let requests = server.received_requests().await.unwrap(); + for (name, version, uuid) in patches { + assert!( + applied(&env, &format!("pkg:npm/{name}@{version}")), + "{env:#}" + ); + let relative = format!(".socket/vendor/npm/{uuid}/{name}-{version}.tgz"); + let bytes = std::fs::read(root.join(&relative)).unwrap(); + let entry = &lock["packages"][format!("node_modules/{name}")]; + assert_eq!(entry["resolved"], format!("file:{relative}")); + assert_eq!( + entry["integrity"], + format!( + "sha512-{}", + base64::engine::general_purpose::STANDARD.encode(Sha512::digest(&bytes)) + ) + ); + let mut archive = tar::Archive::new(flate2::read::GzDecoder::new(bytes.as_slice())); + let mut patched = archive + .entries() + .unwrap() + .map(Result::unwrap) + .find(|file| file.path().unwrap() == Path::new("package/index.js")) + .unwrap(); + let mut content = Vec::new(); + std::io::Read::read_to_end(&mut patched, &mut content).unwrap(); + assert_eq!(content, b"patched\n"); + assert_eq!( + requests + .iter() + .filter(|r| r.url.path() == format!("/v0/orgs/{ORG}/patches/view/{uuid}")) + .count(), + 1 + ); + assert!(!requests + .iter() + .any(|r| r.url.path() == format!("/{name}/-/{name}-{version}.tgz"))); + } + assert_eq!(lock["packages"]["node_modules/unpatched"], untouched); + assert!(!root.join("node_modules").exists()); + assert!(!root.join(".socket/manifest.json").exists()); +} + /// cargo: the hosted `Cargo.lock` / `Cargo.toml` pin with an EMPTY Cargo /// home. The restore re-resolves the crates.io checksum from the sparse -/// index; the pristine `.crate` is downloaded and verified against it. +/// index for reversal; the service supplies the patched crate. #[tokio::test] async fn cargo_eject_needs_no_cargo_home() { const UUID: &str = "55555555-5555-5555-5555-555555555555"; @@ -308,10 +410,21 @@ fn wheel(files: &[(&str, &[u8])]) -> Vec { } /// pypi: a hash-pinned hosted `requirements.txt` with NO virtualenv. The -/// restore re-resolves the release file hash from the PyPI JSON API; the -/// pristine wheel is downloaded and verified against it. +/// restore re-resolves the release file hash from the PyPI JSON API for +/// reversal; the service supplies the patched wheel. #[tokio::test] async fn pypi_eject_needs_no_virtualenv() { + pypi_eject_with_virtualenv(false).await; +} + +/// An unrelated environment must not select or reject a patch already +/// identified by the project's lockfile, even if it has the same release. +#[tokio::test] +async fn pypi_eject_ignores_unrelated_virtualenv() { + pypi_eject_with_virtualenv(true).await; +} + +async fn pypi_eject_with_virtualenv(unrelated_install: bool) { const UUID: &str = "2b1f6c1e-8d3a-4f6b-9c2d-7e5a9b1c3d11"; const PURL: &str = "pkg:pypi/six@1.16.0"; const WHEEL: &str = "six-1.16.0-py2.py3-none-any.whl"; @@ -374,28 +487,38 @@ async fn pypi_eject_needs_no_virtualenv() { ) .unwrap(); - // Nothing installed for the project: VIRTUAL_ENV names an EMPTY - // virtualenv. With no venv at all, a Python project falls back to the - // global interpreters, and on Ubuntu those carry apt's python3-six - // 1.16.0 (`six-1.16.0.egg-info`), whose bytes are not this fixture's. - let empty_venv = tmp.path().join("empty-venv"); - std::fs::create_dir_all(empty_venv.join(if cfg!(windows) { + // Nothing is installed in the project. An ambient VIRTUAL_ENV may + // belong to another project and contain unrelated bytes for six. + let venv = tmp.path().join("ambient-venv"); + let site = venv.join(if cfg!(windows) { "Lib/site-packages" } else { "lib/python3.11/site-packages" - })) - .unwrap(); + }); + std::fs::create_dir_all(&site).unwrap(); + if unrelated_install { + let metadata = site.join("six-1.16.0.dist-info"); + std::fs::create_dir(&metadata).unwrap(); + std::fs::write(metadata.join("METADATA"), "Name: six\nVersion: 1.16.0\n").unwrap(); + std::fs::write(site.join("six.py"), b"# unrelated installation\n").unwrap(); + } let (code, env) = run_json_with( &root, &server, &["vendor"], &[ ("SOCKET_PYPI_JSON_API", format!("{}/pypi", server.uri())), - ("VIRTUAL_ENV", empty_venv.display().to_string()), + ("VIRTUAL_ENV", venv.display().to_string()), ], ); assert_eq!(code, 0, "a fresh hosted pypi checkout ejects: {env:#}"); assert!(applied(&env, PURL), "{env:#}"); + if unrelated_install { + assert_eq!( + std::fs::read(site.join("six.py")).unwrap(), + b"# unrelated installation\n" + ); + } let reqs = std::fs::read_to_string(root.join("requirements.txt")).unwrap(); assert!( !reqs.contains("patch.socket.dev"), diff --git a/docs/usage.md b/docs/usage.md index 5f9841df8..bd316b637 100644 --- a/docs/usage.md +++ b/docs/usage.md @@ -78,16 +78,18 @@ existing patches. See [gradual rollout](configuration.md#gradual-rollout). ## Vendoring and offline installs -For an existing hosted project without an agent manifest: +For an existing hosted project, including a clean checkout with no dependencies installed: ```sh socket-patch vendor --dry-run socket-patch vendor ``` -`vendor` ejects the patches referenced by the project's hosted dependency files. -When an agent manifest exists, it uses that manifest's patch set. To discover and -vendor available patches in one run, use: +`vendor` reads the hosted patch pins from the project's lockfiles, downloads their +prebuilt patched artifacts, and rewrites the dependency files to use `.socket/vendor/`. +An agent manifest or an installed package tree does not change that selection. +When no hosted patches are in scope, it retains the legacy fallback to the agent +manifest. To discover and vendor available patches in one run, use: ```sh socket-patch scan --mode vendored