Skip to content

0.5: extras, GPU and offline env install; packaging pipeline; env-setup/updater/report UI - #15

Merged
CSSFrancis merged 7 commits into
mainfrom
feat/packaged-env-extras-offline
Oct 13, 2026
Merged

CSSFrancis merged 7 commits into
mainfrom
feat/packaged-env-extras-offline

Conversation

@CSSFrancis

Copy link
Copy Markdown
Member

The de-shell half of directelectron/de_ground_crew#410: shipping an app as an installer, online or offline. What SpyDE does today moves into the shell, so Ground Crew and Autopilot can ship the same way.

What's in it

pythonEnv: extras, GPU, offline (main/pythonEnv.ts, main/config.ts)

  • ShellConfig.pythonExtras are always installed. gpuExtras are installed only when nvidia-smi finds a device. EnsureOptions.gpu or <APPID>_GPU=0|1 overrides the probe.
  • Without this, a packaged GC would install CPU-only, because CuPy sits behind its gpu extra and the sync never passed --extra.
  • The lock stamp includes the extras. With no extras it stays the bare hash, so existing installs don't re-sync.
  • Offline wheelhouse. A wheelhouse is a folder with wheelhouse.json, every locked wheel and a python-build-standalone mirror. The install runs:
    1. uv python install from the file:// mirror;
    2. uv venv;
    3. uv export --frozen;
    4. uv pip install --offline --no-index --require-hashes.
  • A frozen uv sync --offline --find-links can't do this: uv looks for registry-locked wheels only in its cache. uv venv also refuses the mirror under UV_OFFLINE, which is why installing the interpreter is a separate step.
  • A probed GPU whose extras the wheelhouse lacks falls back to a CPU install and stamps +gpu=unavailable. Without that stamp, an air-gapped GPU machine would retry online, and fail, on every launch. A GPU the user explicitly requested still fails if its extras are missing.
  • Managed interpreters now go to <userData>/python instead of %APPDATA%\uv\python, so nothing is left behind on uninstall.

First-run, updater and report UI: one call each (main/envSetup*.ts, main/shellIpc*.ts, preload, renderer)

  • runEnvSetup(win, opts) streams throttled env_setup messages. On failure it waits for the user (Retry, Install from a folder…, Quit) instead of leaving a dead window.
  • registerUpdaterIpc() and registerProblemReportIpc() register the IPC channels.
  • The preload bridge gains updater, report and envSetup, and exports a ShellBridge type.
  • EnvSetupOverlay, UpdateCard, UpdateDialog and ReportProblemDialog come from SpyDE, with every string a prop.

Packaging (de_shell/js/packaging/, reached as electron/shell/packaging/)

  • python -m de_shell.js link <electronDir> [--soft], plus shell-link.mjs. These replace the copy all three apps carry, which is identical in each.
  • bundle-python.mjs stages the <resources>/python payload. It is configured by flags or a deShell block in electron/package.json.
  • build-wheelhouse.mjs (new) builds the offline wheelhouse. It evaluates markers for the target platform, not the build host.
  • installer.nsh:
    • It tree-kills the app and its sidecar, limited to this app's own userData. SpyDE's version swept any app's python-env.
    • On uninstall it offers to remove the managed environment. The default is No, and silent runs and updates always answer No.
  • electron-builder.base.yml is a shared base for an app's builder config to extends.
  • .github/workflows/electron-release.yml is a reusable workflow (workflow_call) taken from SpyDE's release pipeline. It also attaches the Windows wheelhouse zip and signs with Azure Trusted Signing when the secrets are present.

The README has a new Shipping an app section.

Testing

  • npm run typecheck is clean.
  • npm run test:unit: 171 pass.
  • uv run pytest: 127 pass, including 10 new tests for link.
  • End-to-end, network blocked. I bundled a tiny app with bundle-python.mjs, built GPU and CPU wheelhouses with build-wheelhouse.mjs, then called resolvePythonEnv from Node:
    • GPU wheelhouse: installs, and a relaunch skips the sync.
    • CPU wheelhouse on a GPU machine: falls back to CPU, and a relaunch doesn't retry even with the wheelhouse gone.
    • Explicit GPU with a CPU wheelhouse: fails clearly.
    • Wheelhouse in userData/wheelhouse: found automatically.
    • No wheelhouse: fails cleanly.
    • The interpreter lands under userData/python.

Not yet exercised:

  • installer.nsh has never been through makensis.
  • The release workflow has never run.
  • bundle-python hasn't been run with MinGit or with a multi-package workspace.
  • The new React components typecheck but haven't been rendered.

The Ground Crew PR that adopts this will be the first real installer build.

ShellConfig gains pythonExtras (always) and gpuExtras (when nvidia-smi finds a
device; EnsureOptions.gpu and <APPID>_GPU override the probe). The lock stamp
carries the extras, but stays the bare hash when there are none so existing
installs do not re-sync.

A wheelhouse (wheelhouse.json + every locked wheel + a python-build-standalone
mirror) installs with no network: uv python install from the file:// mirror,
uv venv, uv export --frozen, then uv pip install --offline --no-index
--require-hashes. A frozen uv sync --offline --find-links cannot do it: uv only
looks for registry-locked wheels in its cache. And uv venv refuses the mirror
under UV_OFFLINE, hence the separate python install step.

A probed GPU whose extras the wheelhouse lacks degrades to CPU and stamps
+gpu=unavailable, so an air-gapped GPU machine does not retry online on every
launch. Managed interpreters now live under <userData>/python.

Refs directelectron/de_ground_crew#410
runEnvSetup streams throttled env_setup messages and, on failure, waits for
the user (retry / install from a folder / quit) instead of leaving a dead
window. registerUpdaterIpc and registerProblemReportIpc register the channels
every app hand-wired. The bridge gains updater/report/envSetup and exports a
ShellBridge type. EnvSetupOverlay, UpdateCard, UpdateDialog and
ReportProblemDialog come from SpyDE with every string a prop.

Refs directelectron/de_ground_crew#410
…, release workflow

What SpyDE ships with, made app-agnostic, plus the shell-link script all three
apps carried identically. python -m de_shell.js link makes the electron/shell
link without the chicken-and-egg of a script living inside it.
build-wheelhouse.mjs is new: the offline install's input, with markers
evaluated for the target platform rather than the build host. installer.nsh
scopes its sidecar sweep to this app's own userData and offers (default No,
always No when silent) to remove the managed environment on uninstall.

Refs directelectron/de_ground_crew#410
…our own interpreter

The Ground Crew packaged run showed the venv linked to a python3.13 launcher in
~/.local/bin: uv python install drops one there (and a PEP 514 registry key) by
default, and uv then discovers it ahead of the userData install dir. Both
outlive an uninstall and point at a folder it deletes. UV_PYTHON_INSTALL_BIN=0,
UV_PYTHON_INSTALL_REGISTRY=0 and UV_MANAGED_PYTHON=1 on every packaged uv call.

build-wheelhouse: --no-extras for a CPU-only wheelhouse when the config names
extras, and a clear error for git/path/URL requirements, which pip otherwise
reports as being unable to hash version-control repositories.
Keeps both sides of pythonEnv.test.ts: this branch's extras/GPU/wheelhouse tests
and #4's packaged-cwd tests, whose layout helper now uses the file's tempDir so
it is cleaned up. #4's cwd: envDir merged cleanly into resolvePythonEnv.
@CSSFrancis
CSSFrancis merged commit 60be29e into main Oct 13, 2026
8 checks passed
@CSSFrancis
CSSFrancis deleted the feat/packaged-env-extras-offline branch October 13, 2026 20:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant