Revert NODE_RED_CLI_DEFAULT_USERDIR; replace with a validated /data auto-probe + explicit --docker-userdir override

[tbrandenburg]

Summary

#31/#32 introduced NODE_RED_CLI_DEFAULT_USERDIR, a bespoke, node-red-cli-specific env var that community image authors would need to explicitly opt into for their image's pre-installed node packages to be auto-discovered by --docker. On reflection this isn't realistic: no community image maintainer is going to adopt a convention invented by and specific to one consumer CLI. Verified this isn't just theoretical — ghcr.io/tbrandenburg/agentic-workflow-dev-env (the concrete motivating image for #31) still only exports its own pre-existing NODE_RED_HOME=/data, not the new variable, so the mechanism is dead on arrival for the very image that motivated it.

This issue proposes reverting NODE_RED_CLI_DEFAULT_USERDIR and replacing it with two complementary mechanisms that don't require any image-author cooperation for the common case, plus a fully explicit escape hatch for everything else.

Background research

The real, already-adopted community convention

The official nodered/node-red Docker image — by far the most widely used/derived-from Node-RED image in the ecosystem — already uses /data as its userDir mount point, and sets:

NODE_PATH=/usr/src/node-red/node_modules:/data/node_modules

Confirmed via docker inspect nodered/node-red:latest. Any community image built FROM nodered/node-red, or deliberately replicating its convention (very likely why agentic-workflow-dev-env also uses /data), gets this for free — no bespoke opt-in needed.

Node-RED's own CLI (node_modules/node-red/red.js:141) additionally has a hardcoded built-in default of $HOME/.node-red when no --userDir is configured at all — confirmed by reading the actual source in this repo's own node-red dependency.

Why auto-probing these conventions is not fully waterproof (must be documented, not silently assumed)

1. False positives: an image could have an unrelated /data directory containing an incidental node_modules folder for a completely different tool. Must validate more strictly than "directory exists" — require finding at least one <candidate>/node_modules/*/package.json with a "node-red" key before trusting the candidate.
2. False negatives: any image customizing userDir to something other than exactly /data or $HOME/.node-red (e.g. /opt/nodered-data) is invisible to this heuristic. This is a best-effort improvement, not a universal guarantee — hence the explicit override flag below.
3. Self-inflicted conflict with issue --docker: hardened read-only rootfs breaks tools needing writable $HOME (e.g. opencode agent node) #17: --docker already forces HOME=/tmp inside the container (the fix for --docker: hardened read-only rootfs breaks tools needing writable $HOME (e.g. opencode agent node) #17's read-only-rootfs $HOME problem). That means probing $HOME/.node-red at container runtime would always resolve to /tmp/.node-red — our own override, never the image's real original $HOME. The $HOME/.node-red probe is effectively unusable in --docker mode given --docker: hardened read-only rootfs breaks tools needing writable $HOME (e.g. opencode agent node) #17's fix and should likely be dropped from the auto-probe entirely (host-mode-only, if anywhere); /data is the one convention worth auto-probing for --docker specifically.
4. --node-modules on top of an auto-detected path isn't automatically safe: confirmed via docker inspect that even the official nodered/node-red image does not declare /data as a Docker VOLUME in its image metadata (Config.Volumes is null) — it's plain rootfs, writable only when a caller mounts a volume there themselves. Under --docker's --read-only sandbox, if --node-modules is combined with an auto-detected default and no volume is actually mounted at that path, the install will fail with EROFS. Consuming pre-installed packages read-only is safe (confirmed empirically: --docker always converts flows to in-memory storage before sending the envelope into the container — even <flows.json> from-file mode — so nothing is ever written to userDir unless --node-modules is explicitly requested too); but installing more packages on top of an auto-detected, non-volume-backed path is not.

Proposed solution

1. Revert NODE_RED_CLI_DEFAULT_USERDIR entirely

Remove:

• src/run-envelope.js: the env-var read/validation logic (resolveDefaultUserDir-style function, its doc comments referencing --docker: pre-installed node packages in an image's own default userDir are never discovered (ephemeral/mounted userDir never points at the image's own default) #31)
• bin/node-red-cli-sandbox-entry.js: the doc comment referencing the env-var convention
• bin/node-red-cli.js: the help-text line documenting NODE_RED_CLI_DEFAULT_USERDIR=<path>
• test/unit/run-envelope.unit.test.js: the NODE_RED_CLI_DEFAULT_USERDIR unit tests
• test/integration/docker.integration.test.js: the two integration tests using a baked ENV NODE_RED_CLI_DEFAULT_USERDIR=... test image
• test/integration/run-envelope.integration.test.js: the two integration tests exercising the env var directly
• README.md: the paragraph documenting the env var for image authors

2. Add a validated /data auto-probe (best-effort, --docker only, read-only use)

• In runFlowInvocation (src/run-envelope.js) or a new small helper, when --docker mode is active (i.e. running inside the sandbox entrypoint) and no explicit userDir was given: check for /data existing, and validate it by scanning /data/node_modules/*/package.json (one level of scoped-package awareness, e.g. @scope/name) for at least one entry with a "node-red" key. Only use /data as the resolved userDir if this validation passes; otherwise fall through to the existing ephemeral-tmpdir behavior unchanged.
• Do not probe $HOME/.node-red given the --docker: hardened read-only rootfs breaks tools needing writable $HOME (e.g. opencode agent node) #17 conflict explained above — keep this scoped to the one convention that's actually usable given this project's existing sandboxing choices.
• This path is treated as effectively read-only: --node-modules combined with the auto-probed /data should either (a) be rejected with a clear error telling the caller to use --user-dir explicitly instead (simplest, matches the existing "--node-modules requires --user-dir" precedent), or (b) copy /data into a fresh writable location first — recommend (a) for this issue's scope; (b) can be a separate future enhancement if there's real demand.

3. Add an explicit --docker-userdir <path> override (fully waterproof escape hatch)

• New CLI flag, e.g. --docker-userdir <path>: tells --docker mode to use <path> (an absolute path inside the target container) as the resolved userDir directly, no auto-detection/validation needed — the caller is explicitly asserting "this is where the pre-populated userDir is in this specific image."
• Works for any image/convention, including ones that don't match /data (e.g. a hypothetical image using /opt/nodered-data), with zero guessing.
• Precedence, most to least specific: --user-dir (host-managed, mounted/ephemeral, unchanged) > --docker-userdir <path> (explicit passthrough) > auto-probed /data (best-effort) > ephemeral fallback (today's default, unchanged for everyone not using any of the above).
• Same read-only consideration as the auto-probe: combining --docker-userdir with --node-modules should be rejected with a clear error unless the caller also passes --user-dir (in which case --user-dir's own mounted volume wins per the precedence above, so this combination is naturally moot — worth a unit test asserting the precedence order explicitly).

Implementation plan

1. Revert commit: remove all NODE_RED_CLI_DEFAULT_USERDIR code/tests/docs listed in section 1 above. Verify make test still passes with the old --docker: pre-installed node packages in an image's own default userDir are never discovered (ephemeral/mounted userDir never points at the image's own default) #31/fix(docker): fall back to image's default userDir via NODE_RED_CLI_DEFAULT_USERDIR #32 tests removed (no leftover references).
2. /data auto-probe: add a small, testable helper (e.g. resolveContainerDefaultUserDir() in src/run-envelope.js or a new src/docker-userdir-probe.js if it grows non-trivial) that:
   • Checks /data exists and is a directory.
   • Scans /data/node_modules/* (and /data/node_modules/@*/* for scoped packages) for any package.json with a "node-red" key.
   • Returns /data if validated, undefined otherwise.
   • Called only from the sandbox entrypoint path (never on the host, since /data has no special meaning outside a container).
3. --docker-userdir <path> flag: add to bin/node-red-cli.js's CLI option parsing, threaded through the envelope's options (e.g. options.dockerUserDir), consumed in src/run-envelope.js's userDir resolution alongside the existing fixedUserDir (--user-dir) and the new auto-probe, respecting the precedence order above.
4. --node-modules guard: add a clear preflight error (in bin/node-red-cli.js, alongside the existing "--node-modules requires --user-dir" check) when --node-modules is combined with either --docker-userdir or a to-be-auto-probed default, without an explicit --user-dir also given.
5. Update README: replace the NODE_RED_CLI_DEFAULT_USERDIR paragraph with documentation of the /data auto-probe (best-effort, mention the false-positive/negative caveats briefly) and the --docker-userdir flag (the reliable option), and update the agentic-workflow-dev-env example to use whichever now actually works against that real image (it should, since it uses /data).

Testing strategy (per this repo's 50/30/20 pyramid, no mocking in integration/e2e)

• Unit tests (test/unit/run-envelope.unit.test.js or new file): the /data-validation helper in isolation — validates true positive (well-formed node-red package present), true negative (/data exists but no valid node-red package inside, e.g. only unrelated files), and absent-/data cases, using a real temp directory on disk (no mocking of fs).
• Integration tests (test/integration/docker.integration.test.js, replacing the removed env-var-based ones): build a small throwaway test image (FROM <existing test base> + bake a fake /data/node_modules/@test/pkg with a valid "node-red" package.json) and confirm --docker <image> (no --user-dir/--node-modules) discovers it; a second test with /data present but not a valid node-red package (e.g. random files) confirming it correctly falls through to the ephemeral default instead of misfiring; a third test for --docker-userdir <path> explicitly pointing at a non-/data path in a throwaway image, confirming the override works for an arbitrary convention.
• E2E: the real agentic-workflow-dev-env full round-trip test (already exists for the --node-modules/--user-dir case) should gain a variant using the /data auto-probe alone (no --node-modules/--user-dir at all), since that's the actual real-world image this was all motivated by.

Risks / edge cases to keep in mind during implementation

• Backward compatibility: anyone who already started relying on NODE_RED_CLI_DEFAULT_USERDIR in the (very short) time it's existed will break — acceptable given it was just released and is being corrected quickly, but worth a one-line release-notes callout.
• Precedence must be unit-tested explicitly (all four levels: --user-dir > --docker-userdir > auto-probed /data > ephemeral).
• The /data validation helper must handle scoped (@scope/name) and unscoped package names, and must not throw on a missing/unreadable /data (should resolve to "not usable", not crash the whole invocation).
• Keep the auto-probe strictly opt-out-free/zero-config for the common case — no new flag required to benefit from it, only --docker-userdir is an explicit flag (for the uncommon case).