Conversation
✅ Snyk checks have passed. No issues have been found so far.
💻 Catch issues earlier using the plugins for VS Code, JetBrains IDEs, Visual Studio, and Eclipse. |
valeriy42
changed the title
valeriy42
marked this pull request as draft
October 28, 2025 14:00
- Added new tests for Sandbox2 functionality, including privilege validation, filesystem isolation, and syscall filtering. - Introduced a TestCleanup class for managing temporary files during tests. - Updated the CDetachedProcessSpawner_Linux.cc to support new command line arguments for log handling and model path. - Renamed modelDir to modelPath for clarity and adjusted policy building to accommodate file access. - Improved overall test coverage for Sandbox2 features and ensured graceful degradation when Sandbox2 is not available.
- Improved error handling for cases when Sandbox2 is disabled or unavailable for pytorch_inference processes. - Enhanced logging to provide clearer feedback on spawning failures with Sandbox2. - Updated comments for clarity regarding the fallback to standard posix_spawn for non-pytorch_inference processes.
- Added a new function to apply standard ML syscall restrictions using Sandbox2's PolicyBuilder, ensuring consistent security across ML processes. - Updated the CDetachedProcessSpawner_Linux to utilize the new syscall policy for pytorch_inference, eliminating the need for seccomp filtering in this context. - Enhanced comments and documentation to outline future migration plans for other ML processes to Sandbox2. - Noted the gradual transition from seccomp filters to Sandbox2 policies in the CSystemCallFilter_Linux implementation.
- Removed SetUserAndGroup from PolicyBuilder due to updates in the sandboxed-api. - Updated AddTmpfs to include a size parameter for better resource management. - Refactored Sandbox2 instantiation to use unique_ptr for the executor, improving memory management. - Enhanced comments to clarify changes and provide context for future updates.
- Removed outdated syscall number definitions and replaced them with fallback definitions for newer syscalls, ensuring compatibility with RHEL8 headers. - Updated the handling of input pipes in the sandbox policy to allow read and write access, improving functionality. - Enhanced comments for better clarity on syscall handling and future maintenance.
…l-cpp into enhancement/sandbox2
…for Linux - Deleted Sandbox2SecurityTest.cc as it is no longer needed. - Updated CMakeLists.txt to remove references to the deleted test file. - Introduced CDetachedProcessSpawnerTest_Linux.cc, which includes tests for process spawning and integration with Sandbox2. - Enhanced CMakeLists.txt to link against Sandbox2 libraries for the new tests.
…pawner spawn() detached a monitor thread that captured a raw this and, once the sandboxee exited, took the spawner's mutex and erased from its PID set. The destructor was defaulted and synchronised nothing, so tearing the spawner down while a sandboxed pytorch_inference was still running left the monitor writing through a dangling pointer whenever the sandboxee later exited. That window is wide in practice: the monitor exists precisely because a sandboxee outlives the spawn call, and it is a long-lived daemon. Joining in the destructor is not an option - it would block controller shutdown until pytorch_inference exits. Instead the mutex and PID set move into an SPidRegistry held by shared_ptr, and the monitor co-owns it. The registry then outlives the spawner exactly as long as some monitor still needs it, and the destructor stays trivial. Also drops the now-unnecessary raw this capture from the lambda.
extractArgDirs treated every key=value argument as a pipe path candidate and reported anything whose value did not start with '/' as rejected. Real pytorch_inference invocations always carry scalar options - --namedPipeConnectTimeout=1, --numAllocations=1, --validElasticLicenseKeyConfirmed=true, --modelid=... - so each of them was listed as "(path not absolute)" on every spawn. The failure-modes doc tells operators to check that rejectedPipeArgs is empty when triaging FIFO and mount problems, and that could never be true, which made the primary triage signal useless. The pipe options cannot be told apart by name: --input, --output and --restore carry paths while --inputIsPipe and friends are bare flags. A value containing no '/' cannot be a pipe path, so those are now skipped silently, and a value that does contain one but is relative is still reported - which is the case the list was meant to catch.
…cation spawnSandboxedWithTimeout captured everything by reference and, on timeout, detached the spawn thread and returned. The thread then kept writing spawnResult and spawnFinished in a frame that had gone away, plus childPid and failureReason in the caller's, and kept using a spawner reference that the test was about to destroy. Detaching is the right way to avoid the hang that ~std::future would cause, but everything the thread touches has to outlive the call: the results now live in a heap-allocated SSpawnAttempt the thread co-owns, the spawner is passed as a shared_ptr, and the path and arguments are copied in. The caller reads the results out only on the success path. Also adds the first tests for extractArgDirs, which had none: that scalar options are ignored rather than reported as rejected, that absolute pipe paths contribute their directory, and that relative and root-level paths are still rejected with the right reason.
extractArgDirs now mounts only input/output/restore/logPipe values and rejects non-absolute paths on those options instead of inferring from value shape, so scalars like --modelid=/etc/passwd never grant mounts.
Replace PID set with generation-tagged child map holding shared_ptr<Sandbox2> and pidfd. Monitor erases before logging with generation check; thread-start failure kills sandbox and cleans registry. terminateChild uses pidfd_send_signal (SIGTERM) with ::kill fallback on kernels without pidfd.
Move the "Spawned" INFO to after successful monitor detach and set childPid to 0 in the std::thread constructor catch path so false returns match the out-param contract of other spawn() failures.
… catch Before assigning a new s_Children entry, close any existing pidfd for the same PID to avoid fd leaks on reuse. Catch std::exception (not only std::system_error) around monitor thread creation so bad_alloc takes the same Kill/AwaitResult cleanup path.
…write The paragraph still described the earlier heuristic - treat a value containing a '/' as a path candidate - which e624f1b replaced with an explicit list of the four path-bearing options. The distinction is visible to an operator: --input=foo is now reported as "(path not absolute)" rather than skipped, and a scalar option is skipped by name whatever its value looks like.
- Wrap the pidfd in an RAII owner and reap the sandboxee if registry insertion throws, so a bad_alloc cannot leak the fd or orphan the process. - Retain the monitor thread in a named variable and join it during cleanup so a detach() failure cannot destroy a joinable thread and std::terminate. - Hardcode ML_SANDBOX2_REQUIRE=enforced in the host re-run; drop the ML_SANDBOX2_HOST_REQUIRE override that let CI pass without enforced coverage. - Add an abandon/terminate handshake to spawnSandboxedWithTimeout so a spawn that succeeds after the test's timeout terminates its sandboxee.
After the policy file merge, comments still pointed at the deleted _Linux.cc path. Update allowlist/sync pointers to CPytorchInferenceSandboxPolicy.cc and attribute ML_SANDBOXED=1 to CSandboxedProcessSpawner_Linux.cc.
valeriy42
marked this pull request as draft
September 8, 2026 19:05
Contributor
Author
|
Frozen at Last green integration evidence: Buildkite #3081. This branch will not be rebased, force-pushed, or merged, and its history is preserved. The Sandbox2 integration is being reconstructed from current
Why replace rather than repair: the recurring defects in this branch share one cause — there was no explicit capability graph and no resource lifecycle state machine, while tests frequently asserted an outcome without proving the mechanism that produced it. Slicing the work lets each boundary be reviewed against a stated contract instead of inferred from a large diff. Security logic is rebuilt from that contract rather than bulk-cherry-picked; only audited vendoring and build material is transplanted mechanically. The established tab-separated This PR stays open and will be closed as superseded once every replacement PR is linked here. |
valeriy42
added a commit
to valeriy42/ml-cpp
that referenced
this pull request
Sep 9, 2026
M2: replace the implicit "seccomp precedes initIo" attestation with an explicit structured degradedModeAttestationMarker(), logged over the existing log pipe and asserted by a dedicated test, instead of a controller/ES observer inferring readiness from the absence of a fatal log line. M4: carry forward, into the shared declaration, syscall fixes frozen PR elastic#2873 discovered the hard way - clone3-by-number (57f00ed, already present via ML_NR_clone3) and __NR_prlimit64 (03b1ee4, newly added) - each with a named regression test (testCarryForwardSyscallsPresent) plus the x86_64 legacy-fs set (ec7d3ed, already present, now also asserted). d9a856d (Sandbox2 AllowFutexOp arg-filtering) and 730933d/f8b0a534 (Buildkite run_tests.sh/build.sh packaging) are out of this file's scope; see the bd note on elastic-workspace-3b59.3 for the explicit classification and deferral to PR C/D/E rather than silent coverage here. Verified on valeriy-devbox-26-04: ml_test_seccomp now 7/7 passing (2 new cases); real pytorch_inference run confirms both "ml.seccomp.installed" and the new structured marker {"ml_sandbox2_route":"legacy","event":"seccomp_installed"} are emitted.
valeriy42
added a commit
to valeriy42/ml-cpp
that referenced
this pull request
Sep 9, 2026
PR A of the Sandbox2 clean rebuild plan (see docs/projects/mlcpp-sandbox2-pr2873 in elastic/elastic-workspace, and the frozen elastic#2873 for prior art). Reconstructs the dependency/build layer from current main, replacing elastic#2873's inline file(WRITE)/string(REGEX REPLACE) source rewrites with checked-in, version-pinned patches that fail configure loudly on upstream drift. - 3rd_party/CMakeLists.txt: FetchContent sandboxed-api v20241008 on Linux, apply 4 patches via `git apply` with FATAL_ERROR on --check failure (idempotent across reconfigure via a reverse-check pre-test) - 3rd_party/patches/sandboxed-api/: the 4 patches (disable vendored gtest, stop -fno-exceptions propagating into ml-cpp targets, make Python3 optional, link zlib + static libstdc++/libgcc into the forkserver binary), each verified against a real shallow clone of the pinned tag - 3rd_party/licenses/{abseil,sandbox2}-*: license/attribution files - lib/sandbox/: dormant MlSandbox target (CMlSandboxAvailability query only - no policy/spawner/diagnostics, no controller wiring; those land in later PRs) plus a Linux-only forkserver runtime smoke test against a statically-linked, dependency-free payload binary Verified on Linux x86_64 (GCP devbox): configure reaches "Sandbox2 enabled: using sandbox2::sandbox2"; ml_test_sandbox passes under both a normal and a -DCMAKE_UNITY_BUILD=ON configure, including the forkserver smoke test actually forking/execing/reaping the payload. Not yet verified: aarch64, macOS, Windows non-Sandbox2 builds, license/Snyk scan.
valeriy42
added a commit
to valeriy42/ml-cpp
that referenced
this pull request
Sep 9, 2026
…aded seccomp Replace the hand-maintained BPF jump-offset table in CSystemCallFilter_Linux.cc with a program builder that derives every jump from the allowlist vector's own size/index. The applied program is generated from CPytorchInferenceSyscallAllowlist.h, a single machine-readable declaration, instead of a parallel hardcoded list (design.md V3/MG6). CSystemCallFilter::installSystemCallFilter() returns a typed ESystemCallFilterInstallOutcome across all three platform implementations instead of void, and logs the ml.seccomp.installed readiness marker on success. pytorch_inference/Main.cc gains a decideDegradedModeAction() decision that would terminate before CIoManager::initIo() on any degraded-mode seccomp failure; that termination stays behind an internal switch defaulting to false until ml-cpp PR E's typed controller routing can guarantee a degraded-mode launch was deliberate (design.md MG8, "activation is deliberately split across two slices"). The four non-PyTorch callers now make their unchanged log-and-continue policy explicit instead of silently discarding the result. Adds CSeccompFilterBuilderTest.cc: decodes the actually-built BPF program to prove it matches the declaration and that jump offsets are derived, not hand-maintained; fault-injected coverage of decideDegradedModeAction() for every install-failure class (V13); a named regression test per design.md M4 carry-forward syscall (clone3 by number, prlimit64, x86_64 legacy filesystem syscalls) so a blank-slate reconstruction cannot silently drop a hard-won compatibility fix from frozen PR elastic#2873; and an explicit structured degradedModeAttestationMarker() (design.md M2) so a controller/ES observer can assert seccomp installation directly instead of inferring it from the absence of a fatal log line. The Sandbox2-grants half of V3 (CPytorchInferenceSandboxPolicy.cc) does not exist yet in this clean rebuild lineage (ml-cpp PR C); this change establishes the single declaration for that PR to consume. d9a856d (Sandbox2 AllowFutexOp arg-filtering) and 730933d/f8b0a534 (Buildkite run_tests.sh/build.sh packaging) are likewise out of this file's scope - see the bd note on elastic-workspace-3b59.3 for the explicit deferral to PR C/D/E. Verified on a real x86_64 devbox: ml_test_seccomp (7 test cases) and the real pytorch_inference/autodetect/categorize/normalize/ data_frame_analyzer binaries all build, link, and run correctly against the new typed API; the generated BPF program installs via a real prctl(PR_SET_SECCOMP) call.
valeriy42
added a commit
that referenced
this pull request
Sep 9, 2026
…aded seccomp Replace the hand-maintained BPF jump-offset table in CSystemCallFilter_Linux.cc with a program builder that derives every jump from the allowlist vector's own size/index. The applied program is generated from CPytorchInferenceSyscallAllowlist.h, a single machine-readable declaration, instead of a parallel hardcoded list (design.md V3/MG6). CSystemCallFilter::installSystemCallFilter() returns a typed ESystemCallFilterInstallOutcome across all three platform implementations instead of void, and logs the ml.seccomp.installed readiness marker on success. pytorch_inference/Main.cc gains a decideDegradedModeAction() decision that would terminate before CIoManager::initIo() on any degraded-mode seccomp failure; that termination stays behind an internal switch defaulting to false until ml-cpp PR E's typed controller routing can guarantee a degraded-mode launch was deliberate (design.md MG8, "activation is deliberately split across two slices"). The four non-PyTorch callers now make their unchanged log-and-continue policy explicit instead of silently discarding the result. Adds CSeccompFilterBuilderTest.cc: decodes the actually-built BPF program to prove it matches the declaration and that jump offsets are derived, not hand-maintained; fault-injected coverage of decideDegradedModeAction() for every install-failure class (V13); a named regression test per design.md M4 carry-forward syscall (clone3 by number, prlimit64, x86_64 legacy filesystem syscalls) so a blank-slate reconstruction cannot silently drop a hard-won compatibility fix from frozen PR #2873; and an explicit structured degradedModeAttestationMarker() (design.md M2) so a controller/ES observer can assert seccomp installation directly instead of inferring it from the absence of a fatal log line. The Sandbox2-grants half of V3 (CPytorchInferenceSandboxPolicy.cc) does not exist yet in this clean rebuild lineage (ml-cpp PR C); this change establishes the single declaration for that PR to consume. d9a856d (Sandbox2 AllowFutexOp arg-filtering) and 730933d/f8b0a534 (Buildkite run_tests.sh/build.sh packaging) are likewise out of this file's scope - see the bd note on elastic-workspace-3b59.3 for the explicit deferral to PR C/D/E. Verified on a real x86_64 devbox: ml_test_seccomp (7 test cases) and the real pytorch_inference/autodetect/categorize/normalize/ data_frame_analyzer binaries all build, link, and run correctly against the new typed API; the generated BPF program installs via a real prctl(PR_SET_SECCOMP) call.
valeriy42
added a commit
that referenced
this pull request
Sep 9, 2026
…aded seccomp Replace the hand-maintained BPF jump-offset table in CSystemCallFilter_Linux.cc with a program builder that derives every jump from the allowlist vector's own size/index. The applied program is generated from CPytorchInferenceSyscallAllowlist.h, a single machine-readable declaration, instead of a parallel hardcoded list (design.md V3/MG6). CSystemCallFilter::installSystemCallFilter() returns a typed ESystemCallFilterInstallOutcome across all three platform implementations instead of void, and logs the ml.seccomp.installed readiness marker on success. pytorch_inference/Main.cc gains a decideDegradedModeAction() decision that would terminate before CIoManager::initIo() on any degraded-mode seccomp failure; that termination stays behind an internal switch defaulting to false until ml-cpp PR E's typed controller routing can guarantee a degraded-mode launch was deliberate (design.md MG8, "activation is deliberately split across two slices"). The four non-PyTorch callers now make their unchanged log-and-continue policy explicit instead of silently discarding the result. Adds CSeccompFilterBuilderTest.cc: decodes the actually-built BPF program to prove it matches the declaration and that jump offsets are derived, not hand-maintained; fault-injected coverage of decideDegradedModeAction() for every install-failure class (V13); a named regression test per design.md M4 carry-forward syscall (clone3 by number, prlimit64, x86_64 legacy filesystem syscalls) so a blank-slate reconstruction cannot silently drop a hard-won compatibility fix from frozen PR #2873; and an explicit structured degradedModeAttestationMarker() (design.md M2) so a controller/ES observer can assert seccomp installation directly instead of inferring it from the absence of a fatal log line. The Sandbox2-grants half of V3 (CPytorchInferenceSandboxPolicy.cc) does not exist yet in this clean rebuild lineage (ml-cpp PR C); this change establishes the single declaration for that PR to consume. d9a856d (Sandbox2 AllowFutexOp arg-filtering) and 730933d/f8b0a534 (Buildkite run_tests.sh/build.sh packaging) are likewise out of this file's scope - see the bd note on elastic-workspace-3b59.3 for the explicit deferral to PR C/D/E. Verified on a real x86_64 devbox: ml_test_seccomp (7 test cases) and the real pytorch_inference/autodetect/categorize/normalize/ data_frame_analyzer binaries all build, link, and run correctly against the new typed API; the generated BPF program installs via a real prctl(PR_SET_SECCOMP) call.
valeriy42
added a commit
that referenced
this pull request
Sep 9, 2026
…aded seccomp Replace the hand-maintained BPF jump-offset table in CSystemCallFilter_Linux.cc with a program builder that derives every jump from the allowlist vector's own size/index. The applied program is generated from CPytorchInferenceSyscallAllowlist.h, a single machine-readable declaration, instead of a parallel hardcoded list (design.md V3/MG6). CSystemCallFilter::installSystemCallFilter() returns a typed ESystemCallFilterInstallOutcome across all three platform implementations instead of void, and logs the ml.seccomp.installed readiness marker on success. pytorch_inference/Main.cc gains a decideDegradedModeAction() decision that would terminate before CIoManager::initIo() on any degraded-mode seccomp failure; that termination stays behind an internal switch defaulting to false until ml-cpp PR E's typed controller routing can guarantee a degraded-mode launch was deliberate (design.md MG8, "activation is deliberately split across two slices"). The four non-PyTorch callers now make their unchanged log-and-continue policy explicit instead of silently discarding the result. Adds CSeccompFilterBuilderTest.cc: decodes the actually-built BPF program to prove it matches the declaration and that jump offsets are derived, not hand-maintained; fault-injected coverage of decideDegradedModeAction() for every install-failure class (V13); a named regression test per design.md M4 carry-forward syscall (clone3 by number, prlimit64, x86_64 legacy filesystem syscalls) so a blank-slate reconstruction cannot silently drop a hard-won compatibility fix from frozen PR #2873; and an explicit structured degradedModeAttestationMarker() (design.md M2) so a controller/ES observer can assert seccomp installation directly instead of inferring it from the absence of a fatal log line. The Sandbox2-grants half of V3 (CPytorchInferenceSandboxPolicy.cc) does not exist yet in this clean rebuild lineage (ml-cpp PR C); this change establishes the single declaration for that PR to consume. d9a856d (Sandbox2 AllowFutexOp arg-filtering) and 730933d/f8b0a534 (Buildkite run_tests.sh/build.sh packaging) are likewise out of this file's scope - see the bd note on elastic-workspace-3b59.3 for the explicit deferral to PR C/D/E. Verified on a real x86_64 devbox: ml_test_seccomp (7 test cases) and the real pytorch_inference/autodetect/categorize/normalize/ data_frame_analyzer binaries all build, link, and run correctly against the new typed API; the generated BPF program installs via a real prctl(PR_SET_SECCOMP) call.
valeriy42
added a commit
that referenced
this pull request
Sep 14, 2026
The clean rebuild's Sandbox2 policy builder grants syscalls to pytorch_inference by looping over legacyBpfAllowedSyscalls() alone - the same list used to build the legacy in-process BPF filter. That is not sufficient: Sandbox2's namespace/threading setup makes pytorch_inference exercise syscalls (scheduling, epoll, pipes, directory/file management for forecast temp storage) that the simpler legacy filter never needed a grant for, since it never sets up namespaces or Sandbox2's own thread/monitor machinery. #2873's enhancement/sandbox2 branch already solved this with a dedicated sandbox2ExplicitSyscalls() list, granted in addition to what Sandbox2's PolicyBuilder helpers (AllowRead/AllowWrite/AllowOpen/etc.) cover implicitly. That list was never carried into this rebuild's CPytorchInferenceSyscallAllowlist.h, so any syscall in it (confirmed via a real run: sched_getaffinity) got denied and killed the sandboxed process. Ports sandbox2ExplicitSyscalls() and sandbox2HelperCoveredSyscalls() from the original branch, wires the explicit list into buildPytorchInferencePolicy() alongside the existing legacy-list loop, and adds sandbox2AllowsAllLegacySyscalls() plus a regression test so a future change to legacyBpfAllowedSyscalls() without a corresponding Sandbox2 grant fails loudly instead of silently regressing.
valeriy42
added a commit
that referenced
this pull request
Sep 18, 2026
PR A of the Sandbox2 clean rebuild plan (see docs/projects/mlcpp-sandbox2-pr2873 in elastic/elastic-workspace, and the frozen #2873 for prior art). Reconstructs the dependency/build layer from current main, replacing #2873's inline file(WRITE)/string(REGEX REPLACE) source rewrites with checked-in, version-pinned patches that fail configure loudly on upstream drift. - 3rd_party/CMakeLists.txt: FetchContent sandboxed-api v20241008 on Linux, apply 4 patches via `git apply` with FATAL_ERROR on --check failure (idempotent across reconfigure via a reverse-check pre-test) - 3rd_party/patches/sandboxed-api/: the 4 patches (disable vendored gtest, stop -fno-exceptions propagating into ml-cpp targets, make Python3 optional, link zlib + static libstdc++/libgcc into the forkserver binary), each verified against a real shallow clone of the pinned tag - 3rd_party/licenses/{abseil,sandbox2}-*: license/attribution files - lib/sandbox/: dormant MlSandbox target (CMlSandboxAvailability query only - no policy/spawner/diagnostics, no controller wiring; those land in later PRs) plus a Linux-only forkserver runtime smoke test against a statically-linked, dependency-free payload binary Verified on Linux x86_64 (GCP devbox): configure reaches "Sandbox2 enabled: using sandbox2::sandbox2"; ml_test_sandbox passes under both a normal and a -DCMAKE_UNITY_BUILD=ON configure, including the forkserver smoke test actually forking/execing/reaping the payload. Not yet verified: aarch64, macOS, Windows non-Sandbox2 builds, license/Snyk scan.
valeriy42
added a commit
that referenced
this pull request
Sep 18, 2026
The clean rebuild's Sandbox2 policy builder grants syscalls to pytorch_inference by looping over legacyBpfAllowedSyscalls() alone - the same list used to build the legacy in-process BPF filter. That is not sufficient: Sandbox2's namespace/threading setup makes pytorch_inference exercise syscalls (scheduling, epoll, pipes, directory/file management for forecast temp storage) that the simpler legacy filter never needed a grant for, since it never sets up namespaces or Sandbox2's own thread/monitor machinery. #2873's enhancement/sandbox2 branch already solved this with a dedicated sandbox2ExplicitSyscalls() list, granted in addition to what Sandbox2's PolicyBuilder helpers (AllowRead/AllowWrite/AllowOpen/etc.) cover implicitly. That list was never carried into this rebuild's CPytorchInferenceSyscallAllowlist.h, so any syscall in it (confirmed via a real run: sched_getaffinity) got denied and killed the sandboxed process. Ports sandbox2ExplicitSyscalls() and sandbox2HelperCoveredSyscalls() from the original branch, wires the explicit list into buildPytorchInferencePolicy() alongside the existing legacy-list loop, and adds sandbox2AllowsAllLegacySyscalls() plus a regression test so a future change to legacyBpfAllowedSyscalls() without a corresponding Sandbox2 grant fails loudly instead of silently regressing.
valeriy42
added a commit
that referenced
this pull request
Sep 18, 2026
## Summary Reconstructs the dormant dependency/build foundation for Sandbox2 from current `main`, as a clean first slice ahead of the sandbox policy, spawner, and controller-routing changes that land in follow-up PRs. No controller or `pytorch_inference` routing changes in this PR — Sandbox2 is not selectable from any production code path yet. Frozen PR #2873 attempted this feature in one large branch; this PR takes just its dependency/build layer and replaces its inline `file(WRITE)`/`string(REGEX REPLACE)` source rewrites with checked-in, version-pinned patches that fail the configure step loudly instead of silently no-op'ing on upstream drift. - `3rd_party/CMakeLists.txt`: FetchContent `sandboxed-api` `v20241008` on Linux, applying 4 checked-in patches via `git apply` (fails configure loudly on upstream drift, idempotent across reconfigure). - `3rd_party/patches/sandboxed-api/`: the 4 patches (disable vendored gtest, stop `-fno-exceptions` propagating into ml-cpp targets, make Python3 optional, link zlib + static libstdc++/libgcc into the forkserver binary). - `3rd_party/licenses/{abseil,sandbox2}-*`: license/attribution files. - `lib/sandbox/`: dormant `MlSandbox` target (`CMlSandboxAvailability` query only — no policy/spawner/diagnostics) plus a Linux-only forkserver runtime smoke test that forks/execs/reaps a dynamically-linked payload via `PolicyBuilder::AddLibrariesForBinary()`. Verified locally (Linux x86_64, both a plain configure and `-DCMAKE_UNITY_BUILD=ON`) before pushing, and green on this repo's own Linux/macOS/Windows CI, license/security scans, and Java integration suites. <sub>Stack created with <a href="https://github.com/github/gh-stack">GitHub Stacks CLI</a> • <a href="https://gh.io/stacks-feedback">Give Feedback 💬</a></sub>
valeriy42
added a commit
that referenced
this pull request
Sep 18, 2026
The clean rebuild's Sandbox2 policy builder grants syscalls to pytorch_inference by looping over legacyBpfAllowedSyscalls() alone - the same list used to build the legacy in-process BPF filter. That is not sufficient: Sandbox2's namespace/threading setup makes pytorch_inference exercise syscalls (scheduling, epoll, pipes, directory/file management for forecast temp storage) that the simpler legacy filter never needed a grant for, since it never sets up namespaces or Sandbox2's own thread/monitor machinery. #2873's enhancement/sandbox2 branch already solved this with a dedicated sandbox2ExplicitSyscalls() list, granted in addition to what Sandbox2's PolicyBuilder helpers (AllowRead/AllowWrite/AllowOpen/etc.) cover implicitly. That list was never carried into this rebuild's CPytorchInferenceSyscallAllowlist.h, so any syscall in it (confirmed via a real run: sched_getaffinity) got denied and killed the sandboxed process. Ports sandbox2ExplicitSyscalls() and sandbox2HelperCoveredSyscalls() from the original branch, wires the explicit list into buildPytorchInferencePolicy() alongside the existing legacy-list loop, and adds sandbox2AllowsAllLegacySyscalls() plus a regression test so a future change to legacyBpfAllowedSyscalls() without a corresponding Sandbox2 grant fails loudly instead of silently regressing.
valeriy42
added a commit
to valeriy42/ml-cpp
that referenced
this pull request
Sep 18, 2026
…aded seccomp Replace the hand-maintained BPF jump-offset table in CSystemCallFilter_Linux.cc with a program builder that derives every jump from the allowlist vector's own size/index. The applied program is generated from CPytorchInferenceSyscallAllowlist.h, a single machine-readable declaration, instead of a parallel hardcoded list (design.md V3/MG6). CSystemCallFilter::installSystemCallFilter() returns a typed ESystemCallFilterInstallOutcome across all three platform implementations instead of void, and logs the ml.seccomp.installed readiness marker on success. pytorch_inference/Main.cc gains a decideDegradedModeAction() decision that would terminate before CIoManager::initIo() on any degraded-mode seccomp failure; that termination stays behind an internal switch defaulting to false until ml-cpp PR E's typed controller routing can guarantee a degraded-mode launch was deliberate (design.md MG8, "activation is deliberately split across two slices"). The four non-PyTorch callers now make their unchanged log-and-continue policy explicit instead of silently discarding the result. Adds CSeccompFilterBuilderTest.cc: decodes the actually-built BPF program to prove it matches the declaration and that jump offsets are derived, not hand-maintained; fault-injected coverage of decideDegradedModeAction() for every install-failure class (V13); a named regression test per design.md M4 carry-forward syscall (clone3 by number, prlimit64, x86_64 legacy filesystem syscalls) so a blank-slate reconstruction cannot silently drop a hard-won compatibility fix from frozen PR elastic#2873; and an explicit structured degradedModeAttestationMarker() (design.md M2) so a controller/ES observer can assert seccomp installation directly instead of inferring it from the absence of a fatal log line. The Sandbox2-grants half of V3 (CPytorchInferenceSandboxPolicy.cc) does not exist yet in this clean rebuild lineage (ml-cpp PR C); this change establishes the single declaration for that PR to consume. d9a856d (Sandbox2 AllowFutexOp arg-filtering) and 730933d/f8b0a534 (Buildkite run_tests.sh/build.sh packaging) are likewise out of this file's scope - see the bd note on elastic-workspace-3b59.3 for the explicit deferral to PR C/D/E. Verified on a real x86_64 devbox: ml_test_seccomp (7 test cases) and the real pytorch_inference/autodetect/categorize/normalize/ data_frame_analyzer binaries all build, link, and run correctly against the new typed API; the generated BPF program installs via a real prctl(PR_SET_SECCOMP) call.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
5 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
PyTorch inference runs untrusted TorchScript models supplied by Elasticsearch users, so the native process must be treated as hostile. Today
pytorch_inferencerelies largely on in-process seccomp and graph validation, but that still leaves a large syscall and filesystem surface exposed inside the same address space as libtorch. This PR hardens the Linux production path by spawningpytorch_inferenceinside Google Sandbox2 from the ML controller, so isolation, syscall policy, and filesystem access are enforced before the model binary starts executing.Architecture
MlSandbox(lib/sandbox/): ownsCSandboxedProcessSpawner,CPytorchInferenceSandboxPolicy, andCSandbox2Diagnostics.SANDBOX2_AVAILABLEis scoped to this library, notMlCore.CProcessSpawnerRouter(bin/controller/): controller-owned routing layer. Nominates./pytorch_inferenceexplicitly, strips--disableSandbox, and chooses sandboxed vs legacy spawn based on the ES kill switch (xpack.ml.trained_models.sandbox_enabled).CDetachedProcessSpawner(lib/core/): generic POSIXposix_spawnspawner again — nopytorch_inference-specific dispatch or Sandbox2 knowledge.pytorch_inference: runtime-only seccomp gate on Linux whenML_SANDBOXED=1; skips in-process BPF when running under Sandbox2.Build support vendors Abseil and the Sandboxed API on Linux, wires them into CMake, and adds the required license files.
Kill switch
When sandboxing cannot be enforced (e.g. user namespaces unavailable) or the operator disables it, the controller falls back to the legacy
posix_spawn+ in-process seccomp path via--disableSandbox.Testing
lib/sandbox/unittest: spawn, allowlist drift guard, fail-closed behaviour, and policy-violation differential tests.enforced(real sandboxed spawn/terminate plus the filesystem-policy differential) orfail_closed(spawn refusal plus kill-switch hint) accordingly. The allowlist drift guard is mode independent and always runs.ML_SANDBOX2_REQUIRE=enforcedpins the aarch64 host runner so enforced coverage cannot silently degrade; the container run is left unpinned and supplies the fail-closed half..buildkite/scripts/steps/diagnose_userns.sh: reports which user-namespace stage a runner denies, on the host and across container configurations, without needing a build. Kept as a regression canary for the pin above.test/test_sandbox2_attack_defense.py: end-to-end attack-defense harness.CPytorchInferenceSyscallAllowlist.h.Review sequence (post-restructure)
lib/sandbox/CSandboxedProcessSpawner_Linux.cc+CPytorchInferenceSandboxPolicy_Linux.cc— spawn and policy.bin/controller/CProcessSpawnerRouter.cc— kill-switch routing.lib/core/CDetachedProcessSpawner.cc— generic legacy spawn baseline.lib/seccomp/CSystemCallFilter_Linux.cc+include/seccomp/CPytorchInferenceSyscallAllowlist.h— legacy vs Sandbox2 syscall coherence.bin/pytorch_inference/Main.cc— runtime seccomp gate.3rd_party/CMakeLists.txt,lib/sandbox/CMakeLists.txt,lib/core/CMakeLists.txt— build wiring.lib/sandbox/unittest/CSandboxedProcessSpawnerTest_Linux.cc— the user-namespace probe and the enforced/fail-closed dispatch it drives;test/test_sandbox2_attack_defense.py..buildkite/scripts/steps/run_tests.sh(where each mode is covered),diagnose_userns.sh+userns_probe.sh.docs/changelog/2873.yaml,docs/sandbox2_production_failure_modes.md.ES companion PRs