diff --git a/.github/workflows/live-shutdown-windows.yml b/.github/workflows/live-shutdown-windows.yml index 5fee6e0..62f7de5 100644 --- a/.github/workflows/live-shutdown-windows.yml +++ b/.github/workflows/live-shutdown-windows.yml @@ -1,4 +1,4 @@ -name: Windows live-session shutdown +name: Windows CLI and live-session checks on: pull_request: @@ -7,10 +7,14 @@ on: - CMakeLists.txt - cpp/** - python/** + - docs/tutorials/** + - docs/development/tutorial-command-validation.md + - environments/**/README.md + - viewer/README.md - pyproject.toml - uv.lock push: - branches: [master, marpaia/17] + branches: [master, marpaia/17, marpaia/24] workflow_dispatch: permissions: @@ -31,6 +35,7 @@ jobs: CMAKE_BUILD_PARALLEL_LEVEL: "2" # Exercise the host reference engine without requiring a GPU toolkit. CMAKE_ARGS: -DCM_ENABLE_METAL=OFF -DCM_ENABLE_CUDA=OFF -DCM_BUILD_TESTS=OFF + MICROSIMULATOR_COMMAND_REPORT: build/shutdown-evidence/tutorial-commands steps: - name: Check out source uses: actions/checkout@v6 @@ -60,6 +65,11 @@ jobs: python/tests/test_viewer_server.py python/tests/test_viewer_shutdown.py -v --tb=short --junitxml=build/shutdown-evidence/tests.xml + - name: Execute the documented PowerShell CLI commands + run: >- + uv run --no-sync python -m pytest python/tests/test_tutorial_commands.py + -v --tb=short --junitxml=build/shutdown-evidence/tutorial-commands.xml + - name: Upload shutdown evidence if: ${{ always() }} uses: actions/upload-artifact@v7 diff --git a/docs/README.md b/docs/README.md index fa08956..fdab531 100644 --- a/docs/README.md +++ b/docs/README.md @@ -8,6 +8,7 @@ MicroSimulator models microbial populations in microfluidic environments, connec | --- | --- | | Understand how devices, flow, and cells fit together | [Microfluidics modeling guide](microfluidics.md) | | Run a first simulation | [Getting started](tutorials/getting-started.md) | +| Select a backend or use PowerShell | [Copyable tutorial commands](tutorials/commands.md) | | Build a trap or channel with growth and washout | [Microfluidic devices](tutorials/microfluidics.md) | | Choose a flow solver and assess its numerical behavior | [Flow models](microfluidics.md#choosing-a-flow-model) and [flow benchmarks](tutorials/flow-solvers.md#numerical-evidence) | | Measure nutrient penetration and growth | [Controlled nutrient study](tutorials/nutrient-validation.md) | diff --git a/docs/development/tutorial-command-validation.md b/docs/development/tutorial-command-validation.md new file mode 100644 index 0000000..737de9e --- /dev/null +++ b/docs/development/tutorial-command-validation.md @@ -0,0 +1,34 @@ +# Tutorial command verification + +The [command guide](../tutorials/commands.md) separates shell argument handling, backend availability, and scientific backend validation. The executable examples are read directly from its marked Markdown blocks by [`test_tutorial_commands.py`](../../python/tests/test_tutorial_commands.py). + +## Coverage and limits + +The command tests exercise human-readable and JSON device discovery; the identical 100-step trap command on every available backend; explicit unavailable-backend failures; JSON string arguments and line continuation; repository, model, checkpoint, and output paths containing spaces; restored seed/parameters/source digest; and uninterrupted versus resumed controller/native state. They launch the documented trap, growth, and resumed live sessions in sequence on port 8765, save a checkpoint, send authenticated Stop, observe Stopped, and verify each process returns successfully before the next launch. A cleanup regression also verifies that a failed test terminates its own subprocess tree and drains inherited pipes. The CLI tests supply minimal static assets and do not claim browser rendering coverage. + +PowerShell tests set `$PSNativeCommandArgumentPassing = 'Standard'` exactly as documented. Windows uses `pwsh` 7.3+; Windows PowerShell 5.1 and `cmd.exe` are not covered. POSIX examples are executed in `sh`, Bash, and Zsh when those shells are installed. A missing shell is explicitly skipped. Reports include the exact script text, exit status, stdout/stderr, platform, Python version, shell version, and enumerated devices. + +The [Windows CLI and live-session workflow](../../.github/workflows/live-shutdown-windows.yml) builds a native CPU extension on `windows-2025`, executes these command tests and the existing live-session tests, and uploads reports. Its GPU backends are deliberately disabled. The shutdown tests additionally generate an actual Windows `CTRL_C_EVENT` in an isolated console; they do not simulate a human keyboard press in every terminal application. The viewer guide retains a [manual terminal procedure](../../viewer/README.md#stop-one-model-and-start-another). + +## Execution record + +The local command run on 2026-09-24 passed all ten tests with Python 3.12.8, Bash 3.2.57, Zsh 5.9, and PowerShell 7.6.4. Windows verification also passed on 2026-09-24 at commit `215e87dbf58c9964aef5a7821cdb3cb494c91490`: [workflow run 36068841367](https://github.com/DRAGGON-Lab/MicroSimulator/actions/runs/36068841367) recorded four command/cleanup tests passing in 17.934 seconds and all 28 live-session shutdown tests passing. Reports are retained as CI artifacts. + +| Platform and shell | Backend execution | Command coverage | +| --- | --- | --- | +| macOS, POSIX `sh`, Bash, Zsh | Native CPU and Apple M4 Max Metal | Passed: discovery, 100-step CPU/Metal trap, JSON/space paths, headless CPU resume, live CPU checkpoint/Stop/restart | +| macOS, PowerShell 7.6.4 with Standard arguments | Native CPU and Apple M4 Max Metal | Passed: same commands, PowerShell backtick continuation and string quoting | +| Windows Server 2025 build 26100, PowerShell 7.6.6, Python 3.12.10 | Native CPU | Passed: discovery, 100-step trap, PowerShell JSON/space paths, resume, three live Stop/restarts, cleanup regression; Metal/CUDA unavailable as expected | +| NVIDIA CUDA hardware | Not available in the command-verification campaign | Unavailable-backend error checked; no CUDA runtime or numerical claim | + +The test runner reports actual Metal availability for each macOS run. This run executed Metal headless trap commands; its JSON scenario, resume, and live-session examples selected CPU. A skipped/unavailable Metal result does not count as GPU execution. Backend support still requires the independent [native and application conformance gates](validation.md), even when a tutorial smoke command succeeds. + +## Reproduce + +After installing the development environment, run: + +```console +uv run --no-sync python -m pytest python/tests/test_tutorial_commands.py -v +``` + +Set `MICROSIMULATOR_COMMAND_REPORT` to a new output directory to retain per-shell JSON reports; the Windows workflow supplies this environment variable. Run from an environment with loopback networking enabled and port 8765 free. The tests use their own temporary working directories and never overwrite tutorial outputs in the source checkout. diff --git a/docs/tutorials/README.md b/docs/tutorials/README.md index 1399f97..cefcc0d 100644 --- a/docs/tutorials/README.md +++ b/docs/tutorials/README.md @@ -2,6 +2,8 @@ These tutorials explore cells in microfluidic devices through runnable models: geometry and flow supply the environment, while growth, mechanics, and circuits determine how populations respond. The [modeling guide](../microfluidics.md) introduces the full workflow and its assumptions. Each example can also be used independently. +For backend selection, PowerShell syntax, quoted JSON parameters, and paths with spaces, see [tutorial commands by backend and shell](commands.md#choose-a-shell). Multiline commands on this page use POSIX shell backslashes; the guide provides the PowerShell equivalents and [explicit CPU, Metal, and CUDA trap launches](commands.md#run-the-same-trap-on-cpu-metal-or-cuda). + ## Start here Follow [getting started](getting-started.md) to run a nutrient-fed trap, configure the viewer, and resume a checkpoint. Then choose a path below. diff --git a/docs/tutorials/analysis.md b/docs/tutorials/analysis.md index 120bdc5..4fc6564 100644 --- a/docs/tutorials/analysis.md +++ b/docs/tutorials/analysis.md @@ -2,6 +2,8 @@ MicroSimulator separates simulation output into three artifacts: +For backend selection, PowerShell syntax, quoted JSON parameters, and paths with spaces, see [tutorial commands by backend and shell](commands.md#choose-a-shell). Multiline commands on this page use POSIX shell backslashes; the guide provides the PowerShell equivalents and [explicit CPU, Metal, and CUDA trap launches](commands.md#run-the-same-trap-on-cpu-metal-or-cuda). + - a checkpoint is an exact, integrity-checked restart artifact; - a scene is an immutable presentation snapshot; and - an analysis dataset is an immutable Parquet/Zarr projection with schemas and provenance. @@ -13,6 +15,7 @@ Use checkpoints for resuming, scenes for viewing, and datasets for statistics. ```console uv run microsimulator run \ --model examples/tutorials/biophysics.py \ + --backend cpu \ --parameter scenario='"basics"' \ --seed 42 \ --steps 200 \ diff --git a/docs/tutorials/biophysics-and-growth.md b/docs/tutorials/biophysics-and-growth.md index acac890..55b4647 100644 --- a/docs/tutorials/biophysics-and-growth.md +++ b/docs/tutorials/biophysics-and-growth.md @@ -4,6 +4,8 @@ This tutorial introduces cell geometry, growth, division, lineage, cell types, m The `basics`, `two_types`, and `competition` scenarios add XY-only division jitter but retain unrestricted 3D mechanics. `short_cells` adds XYZ jitter. `box` also adds XYZ jitter and has a floor and four lateral walls, with no ceiling. None guarantees a planar colony; see [division jitter and out-of-plane motion](planarity.md) for the full contract and reproducible diagnostics. +For backend selection, PowerShell syntax, quoted JSON parameters, and paths with spaces, see [tutorial commands by backend and shell](commands.md#choose-a-shell). Multiline commands on this page use POSIX shell backslashes; the guide provides the PowerShell equivalents and [explicit CPU, Metal, and CUDA trap launches](commands.md#run-the-same-trap-on-cpu-metal-or-cuda). + ## 1. A founder that grows and divides Run the basic model: @@ -11,6 +13,7 @@ Run the basic model: ```console uv run microsimulator view \ --model examples/tutorials/biophysics.py \ + --backend cpu \ --parameter scenario='"basics"' \ --seed 42 \ --dt 0.05 \ @@ -51,6 +54,7 @@ During new tutorial construction, each founder target is sampled exactly once fr ```console uv run microsimulator view \ --model examples/tutorials/biophysics.py \ + --backend cpu \ --parameter scenario='"two_types"' \ --seed 42 \ --dt 0.02 \ @@ -64,6 +68,7 @@ The model places type 0 at `x = -10` and type 1 at `x = 10`. Both use the same g ```console uv run microsimulator view \ --model examples/tutorials/biophysics.py \ + --backend cpu \ --parameter scenario='"short_cells"' \ --seed 42 \ --dt 0.01 \ @@ -77,6 +82,7 @@ This scenario lowers the post-founder division length to produce short spherocyl ```console uv run microsimulator view \ --model examples/tutorials/biophysics.py \ + --backend cpu \ --parameter scenario='"competition"' \ --seed 7 \ --dt 0.01 \ @@ -100,6 +106,7 @@ Use `Growth rate` coloring to see the active zone and `Cell type` coloring to se ```console uv run microsimulator view \ --model examples/tutorials/biophysics.py \ + --backend cpu \ --parameter scenario='"box"' \ --seed 42 \ --dt 0.01 \ diff --git a/docs/tutorials/commands.md b/docs/tutorials/commands.md new file mode 100644 index 0000000..72e3497 --- /dev/null +++ b/docs/tutorials/commands.md @@ -0,0 +1,179 @@ +# Tutorial commands by backend and shell + +Run these commands from the repository root. They use the same Python models on every backend. `--backend` selects `cpu`, `metal`, or `cuda`; `--device-index` selects the zero-based device within that backend. Model paths, JSON parameters, seeds, and timesteps do not acquire backend-specific syntax. + +## Prepare and discover devices + +Install Python 3.12, `uv`, CMake, Ninja, and a C++23 compiler. Windows CPU builds need the Visual Studio C++ build tools and Windows SDK available to the build process; the tested Windows route uses PowerShell 7 on `windows-2025`. Metal needs macOS and an accessible Apple GPU. CUDA additionally needs a CUDA-enabled build, an NVIDIA GPU, and a compatible toolkit/driver; see the [Metal](../../environments/metal/README.md) and [CUDA](../../environments/cuda/README.md) environment guides. + +```console +uv sync --locked --group dev +``` + +The commands below use `uv run --no-sync` after this installation so running a tutorial does not change the environment. Run `uv sync --locked --group dev` again after changing dependencies or checking out another version. No virtual-environment activation is required. + + +```console +uv run --no-sync microsimulator devices +``` + + +```console +uv run --no-sync microsimulator devices --json +``` + +An entry such as `metal:0 ...` identifies backend `metal`, device index `0`. CPU also uses index `0`. For a second enumerated GPU, change only `--device-index 0` to `--device-index 1`. `devices` reports what this installation can construct; it does not certify scientific conformance. CPU is the reference backend, Metal is the supported Apple backend, and CUDA remains under development pending the required NVIDIA runtime and application gates. A CUDA compile check alone does not establish runtime support. See the [validation policy](../development/validation.md). + +Requesting an unavailable backend/device fails with an error such as `backend cuda device 0 is unavailable (0 device(s) found)` and a nonzero exit status. There is no automatic CPU fallback. Choose an available backend explicitly or install/configure the requested backend. Do not change a backend flag merely to label a CPU result as a GPU result. + +## Choose a shell + +Single-line `console` commands on this page work in POSIX `sh`, Bash, Zsh, and PowerShell 7.3 or newer configured as below. Multiline `sh` blocks use a trailing backslash; multiline `powershell` blocks use a trailing backtick. The continuation character must be the last character on the line, with no trailing spaces or comment. Do not paste the backslashes from a `sh` block into PowerShell; use the PowerShell block or join the command onto one line. + +For PowerShell, open `pwsh` and set its native argument-passing mode once in that session: + + +```powershell +$PSNativeCommandArgumentPassing = 'Standard' +``` + +The tested Windows route is PowerShell 7.3+ (`pwsh`), not Windows PowerShell 5.1 (`powershell.exe`) or Command Prompt (`cmd.exe`). Their native JSON quoting differs; install/use `pwsh` for these examples. A shell passing the arguments correctly does not establish GPU availability on that operating system. CUDA's current hardware conformance scripts target Linux. + +## Run the same trap on CPU, Metal, or CUDA + +Choose one available backend. These commands all run [`examples/microfluidic_trap.py`](../../examples/microfluidic_trap.py), with seed `42`, no model parameter overrides, 100 steps, `dt=0.02`, device index `0`, and checkpoints every 20 steps. Only the backend and output filename differ. Keeping separate output names prevents one backend's result from replacing another's. + +CPU: + + +```console +uv run --no-sync microsimulator run --model examples/microfluidic_trap.py --backend cpu --device-index 0 --seed 42 --steps 100 --dt 0.02 --checkpoint-every 20 --output "results/tutorial runs/trap-cpu.json" +``` + +Metal: + + +```console +uv run --no-sync microsimulator run --model examples/microfluidic_trap.py --backend metal --device-index 0 --seed 42 --steps 100 --dt 0.02 --checkpoint-every 20 --output "results/tutorial runs/trap-metal.json" +``` + +CUDA: + + +```console +uv run --no-sync microsimulator run --model examples/microfluidic_trap.py --backend cuda --device-index 0 --seed 42 --steps 100 --dt 0.02 --checkpoint-every 20 --output "results/tutorial runs/trap-cuda.json" +``` + +The quoted output paths contain spaces. Parent directories are created automatically. Final and periodic outputs must be new: choose another output name on a second run. `--overwrite` is an explicit replacement option, not a prerequisite for running a tutorial. Identical seeds define the same experiment; floating-point results across backends must be compared using the project's numerical tolerances, not an assumption of byte-identical checkpoints. + +## JSON parameters and model paths with spaces + +`--parameter` takes one `NAME=JSON` argument. For a JSON string, the shell must preserve the inner double quotes: `'scenario="basics"'` reaches Python as `scenario="basics"`. A bare `scenario=basics` is invalid JSON. Numeric and Boolean examples are `--parameter copies_per_cell=6` and `--parameter enabled=true`, when the selected model defines those parameters. Repeat `--parameter` for additional names. + +This example copies the self-contained growth model to a path containing spaces, then runs its `basics` scenario. Choose the block for your shell. Both blocks describe the same run and output, so run only one. + +POSIX `sh`, Bash, or Zsh: + + +```sh +mkdir -p "results/tutorial models" +cp examples/tutorials/biophysics.py "results/tutorial models/biophysics.py" +``` + + +```sh +uv run --no-sync microsimulator run \ + --model "results/tutorial models/biophysics.py" \ + --parameter 'scenario="basics"' \ + --backend cpu --device-index 0 --seed 42 \ + --steps 10 --dt 0.02 \ + --output "results/tutorial runs/basics.json" +``` + +PowerShell 7.3+ with `Standard` argument passing: + + +```powershell +New-Item -ItemType Directory -Force "results/tutorial models" | Out-Null +Copy-Item examples/tutorials/biophysics.py "results/tutorial models/biophysics.py" +``` + + +```powershell +uv run --no-sync microsimulator run ` + --model "results/tutorial models/biophysics.py" ` + --parameter 'scenario="basics"' ` + --backend cpu --device-index 0 --seed 42 ` + --steps 10 --dt 0.02 ` + --output "results/tutorial runs/basics.json" +``` + +To run this scenario on a GPU, replace `--backend cpu` with an enumerated `metal` or `cuda` backend and choose a new output filename. Keep the model, `scenario`, seed, step count, and timestep unchanged for a comparison. Quotes also protect an absolute repository or model path containing spaces; on Windows, forward slashes in these Python CLI paths are accepted. + +## Resume saved parameters and state + +Resume the preceding scenario for ten additional steps: + + +```console +uv run --no-sync microsimulator run --model "results/tutorial models/biophysics.py" --resume "results/tutorial runs/basics.json" --backend cpu --device-index 0 --steps 10 --dt 0.02 --output "results/tutorial runs/basics-resumed.json" +``` + +For native `--model ... --resume ...`, the CLI obtains the seed and parameters from the checkpoint, restores controller/random/native state, and checks the model file's SHA-256 before executing it. Do not pass `--parameter` on resume: the CLI rejects it. `--seed` is a construction option and does not override the saved resume seed; omit it here. The model file may move, but its bytes must match the saved digest. Keep the original model source when updating a checkout. `--steps` is an additional step count, and `--dt` remains an explicit choice; retain the original timestep when continuing the same experiment. An available backend/device can be selected explicitly for resume; that does not guarantee bitwise equality across backends. + +To continue the CPU trap from above: + + +```console +uv run --no-sync microsimulator run --model examples/microfluidic_trap.py --resume "results/tutorial runs/trap-cpu.json" --backend cpu --device-index 0 --steps 100 --dt 0.02 --output "results/tutorial runs/trap-cpu-resumed.json" +``` + +For a Metal or CUDA trap checkpoint, change both `--resume` and `--output` to that run's filenames and select the intended available backend. Controller-backed checkpoints need their original `--model`; a checkpoint is data, not a substitute for model behavior. + +## Live view, Stop, and restart + +Build the browser assets once from the repository root: + +```console +pnpm --dir viewer install +pnpm --dir viewer build +``` + +Start a CPU trap session. Open the tokenized loopback URL printed in the terminal, or append `--open` to open it automatically. The explicit `--viewer-dist` resolves from the current directory. + + +```console +uv run --no-sync microsimulator view --model examples/microfluidic_trap.py --backend cpu --device-index 0 --seed 42 --dt 0.02 --viewer-dist viewer/dist --checkpoint-output "results/tutorial runs/live-trap.json" +``` + +To use a GPU, the only simulation selection changes are `--backend metal` or `--backend cuda` and, if needed, `--device-index`. `--port` defaults to `8765`; choose another free port explicitly if it is occupied. Use the new tokenized URL after each launch. + +| Control or action | Effect | +| --- | --- | +| Pause | Stops continuous playback; keeps the model process and current state available. | +| Reset | Rebuilds this session's original model. For a resumed session, reloads its starting checkpoint. Does not select a different model. | +| Close the browser | Disconnects and pauses the session; the server remains available for reconnection. | +| Checkpoint | Saves to `--checkpoint-output`; use this before Stop when restartable state is needed. | +| Stop session or one terminal Ctrl+C | Finishes the current individual step or checkpoint write, drains the worker, releases the port, and returns to the prompt. No automatic checkpoint is written. | + +Wait for **Stopped** and the terminal prompt before starting another command. A long `--frame-steps` batch is interrupted between individual steps; Stop does not forcibly interrupt a single model callback or solver. After stopping the trap, launch another model on the same default port: + + +```console +uv run --no-sync microsimulator view --model examples/tutorials/biophysics.py --parameter 'scenario="basics"' --backend cpu --device-index 0 --seed 42 --dt 0.02 --viewer-dist viewer/dist +``` + +After stopping that session, the saved headless scenario can also be opened live: + + +```console +uv run --no-sync microsimulator view --model "results/tutorial models/biophysics.py" --resume "results/tutorial runs/basics.json" --backend cpu --device-index 0 --dt 0.02 --viewer-dist viewer/dist +``` + +The [viewer guide](../../viewer/README.md#stop-one-model-and-start-another) describes shutdown checks, including the distinction between a generated Windows console Ctrl+C event and a human keyboard press in a particular terminal application. + +## Executed-command coverage + +[`test_tutorial_commands.py`](../../python/tests/test_tutorial_commands.py) executes the marked command blocks above through the actual shell, in a temporary repository path containing spaces. It checks device discovery, each available backend's trap checkpoint, unavailable-backend errors, JSON string parameters, quoted model/output/resume paths, saved provenance, continuation syntax, resume equivalence, and real live-session Stop/restart on the same port. Live CLI tests use minimal static assets; browser control behavior is covered separately by the [viewer shutdown tests](../../python/tests/test_viewer_shutdown.py). + +The [Windows CLI and live-session workflow](../../.github/workflows/live-shutdown-windows.yml) runs PowerShell commands against a freshly built native CPU extension and uploads shell/platform/backend/command evidence. The [command verification record](../development/tutorial-command-validation.md) states which platforms, shells, and hardware were actually exercised. Missing GPU hardware is recorded as unavailable, never counted as a passing GPU execution. These smoke checks establish command behavior, not complete backend conformance. diff --git a/docs/tutorials/discrete-state-and-contacts.md b/docs/tutorials/discrete-state-and-contacts.md index 722b2f0..27c9282 100644 --- a/docs/tutorials/discrete-state-and-contacts.md +++ b/docs/tutorials/discrete-state-and-contacts.md @@ -4,6 +4,8 @@ This tutorial uses plasmid segregation and conjugation to show how discrete biol Both models start with centers at Z=0 and axes in XY, and division adds no orientation jitter. Neither has mechanical Z confinement: daughters inherit the parent axis and contact relaxation remains three-dimensional. See [division jitter and out-of-plane motion](planarity.md). +For backend selection, PowerShell syntax, quoted JSON parameters, and paths with spaces, see [tutorial commands by backend and shell](commands.md#choose-a-shell). Multiline commands on this page use POSIX shell backslashes; the guide provides the PowerShell equivalents and [explicit CPU, Metal, and CUDA trap launches](commands.md#run-the-same-trap-on-cpu-metal-or-cuda). + New founders preserve their requested length unless it exceeds the single sampled division target. This also handles the rare short Gaussian target in the conjugation model without rejection sampling. Checkpoint restoration keeps stored lengths and targets. See [founder initialization](biophysics-and-growth.md#length-and-volume). ## 1. Incompatible plasmid segregation @@ -11,6 +13,7 @@ New founders preserve their requested length unless it exceeds the single sample ```console uv run microsimulator view \ --model examples/tutorials/plasmid_segregation.py \ + --backend cpu \ --parameter copies_per_cell=10 \ --seed 42 \ --dt 0.02 \ @@ -73,6 +76,7 @@ Contact graphs are derived on demand and have no fixed scientific contact cap; a ```console uv run microsimulator view \ --model examples/tutorials/conjugation.py \ + --backend cpu \ --parameter transfer_probability=0.1 \ --seed 42 \ --dt 0.02 \ diff --git a/docs/tutorials/flow-solvers.md b/docs/tutorials/flow-solvers.md index 0f04742..c800b0b 100644 --- a/docs/tutorials/flow-solvers.md +++ b/docs/tutorials/flow-solvers.md @@ -4,8 +4,10 @@ The [pillar-channel model](../../examples/tutorials/pillar_channel.py) combines Cells retain three-dimensional mechanics inside the channel walls at Z=±3. XY-only division jitter and depth-integrated flow do not impose a planar cell constraint; fixed founders remain attached while released daughters can move and tilt within the finite-height chamber. See the [dimensionality audit](planarity.md). +For backend selection, PowerShell syntax, quoted JSON parameters, and paths with spaces, see [tutorial commands by backend and shell](commands.md#choose-a-shell). Multiline commands on this page use POSIX shell backslashes; the guide provides the PowerShell equivalents and [explicit CPU, Metal, and CUDA trap launches](commands.md#run-the-same-trap-on-cpu-metal-or-cuda). + ```console -uv run microsimulator view --model examples/tutorials/pillar_channel.py --seed 7 --dt 0.01 --backend metal --open +uv run microsimulator view --model examples/tutorials/pillar_channel.py --seed 7 --dt 0.01 --backend cpu --open ``` ## Geometry and flow diff --git a/docs/tutorials/getting-started.md b/docs/tutorials/getting-started.md index b475f7f..f057599 100644 --- a/docs/tutorials/getting-started.md +++ b/docs/tutorials/getting-started.md @@ -2,6 +2,8 @@ Start with a cell trap supplied by a flowing nutrient channel. This model combines device walls, a steady flow solve, solute transport, nutrient-dependent growth, and cell motion. The [microfluidics tutorial](microfluidics.md) explains the model, and the [modeling guide](../microfluidics.md) introduces the broader workflow. +For backend selection, PowerShell syntax, quoted JSON parameters, and paths with spaces, see [tutorial commands by backend and shell](commands.md#choose-a-shell). Multiline commands on this page use POSIX shell backslashes; the guide provides the PowerShell equivalents and [explicit CPU, Metal, and CUDA trap launches](commands.md#run-the-same-trap-on-cpu-metal-or-cuda). + ## Prepare the workspace MicroSimulator requires Python 3.12, CMake, Ninja, a C++23 compiler, and `uv`. From the repository root: @@ -66,20 +68,19 @@ uv run microsimulator view \ --open ``` -The viewer can play, pause, step, reset, and request a checkpoint. For the trap, enable a nutrient signal slice and choose `Growth rate` coloring to inspect the population alongside its environment. Other models can use `Species` coloring for intracellular channels or `Cell type` for strain or discrete-state categories. Selecting a cell shows its stable ID, lineage parent, geometry, type, growth rate, and ordered species values. +The viewer can play, pause, step, reset, request a checkpoint, and stop the session. Pause retains the current process/state; Reset rebuilds this session's starting model; closing the browser pauses for reconnection. Stop session or terminal Ctrl+C releases the server after current work completes, without automatically saving. See [Stop and restart](commands.md#live-view-stop-and-restart) before launching another model. For the trap, enable a nutrient signal slice and choose `Growth rate` coloring to inspect the population alongside its environment. Other models can use `Species` coloring for intracellular channels or `Cell type` for strain or discrete-state categories. Selecting a cell shows its stable ID, lineage parent, geometry, type, growth rate, and ordered species values. The browser owns only presentation state. Python owns the clock, model, backend, checkpoint path, and random state. ## Resume exactly -Controller-backed checkpoints must be resumed with the same model source, seed, and parameters. MicroSimulator verifies the source digest before running the file: +Controller-backed checkpoints require the same model source bytes. The CLI restores the saved seed and parameters automatically and verifies the source digest before running the file. Do not pass new `--parameter` values; they are rejected. Omit `--seed` because it does not override the saved seed during resume. The step count below is additional, and the timestep is retained explicitly: ```console uv run microsimulator run \ --model examples/microfluidic_trap.py \ --resume results/tutorial-trap.json \ --backend cpu \ - --seed 42 \ --steps 100 \ --dt 0.02 \ --output results/trap-resumed.json diff --git a/docs/tutorials/intracellular-dynamics.md b/docs/tutorials/intracellular-dynamics.md index 619f2d8..48b84cd 100644 --- a/docs/tutorials/intracellular-dynamics.md +++ b/docs/tutorials/intracellular-dynamics.md @@ -4,6 +4,8 @@ This tutorial introduces intracellular concentrations, growth dilution, typed ra All five scenarios use XY-only division jitter and start with centers at Z=0, but add no mechanical walls. Their cells retain three-dimensional translations and rotations. See [division jitter and out-of-plane motion](planarity.md) before treating a planar-looking trajectory as a strict 2D model. +For backend selection, PowerShell syntax, quoted JSON parameters, and paths with spaces, see [tutorial commands by backend and shell](commands.md#choose-a-shell). Multiline commands on this page use POSIX shell backslashes; the guide provides the PowerShell equivalents and [explicit CPU, Metal, and CUDA trap launches](commands.md#run-the-same-trap-on-cpu-metal-or-cuda). + All five gene-expression scenarios request a founder centerline length of 3.5 and cap it at the one sampled division target. Initial concentrations are unchanged; the smaller biomass can change total initial amount. See [founder initialization and volume conventions](biophysics-and-growth.md#length-and-volume). ## The native species contract @@ -26,6 +28,7 @@ Equal division copies concentrations to both daughters. Since their effective vo ```console uv run microsimulator view \ --model examples/tutorials/gene_expression.py \ + --backend cpu \ --parameter scenario='"constitutive"' \ --seed 42 \ --dt 0.01 \ @@ -56,6 +59,7 @@ The `legacy_constitutive` scenario provides an alternative parameterization with ```console uv run microsimulator view \ --model examples/tutorials/gene_expression.py \ + --backend cpu \ --parameter scenario='"dilution"' \ --seed 42 \ --dt 0.01 \ @@ -69,6 +73,7 @@ The founder starts at `x = 10` and the explicit chemical rate is zero. Any decli ```console uv run microsimulator view \ --model examples/tutorials/gene_expression.py \ + --backend cpu \ --parameter scenario='"derepression"' \ --seed 42 \ --dt 0.01 \ @@ -89,6 +94,7 @@ As growth dilutes `x0`, reporter production approaches one. Inspect both channel ```console uv run microsimulator view \ --model examples/tutorials/gene_expression.py \ + --backend cpu \ --parameter scenario='"oscillator"' \ --seed 42 \ --dt 0.005 \ @@ -113,6 +119,7 @@ Create periodic checkpoints, export them, and plot or inspect one stable cell li ```console uv run microsimulator run \ --model examples/tutorials/gene_expression.py \ + --backend cpu \ --parameter scenario='"oscillator"' \ --seed 42 \ --steps 400 \ diff --git a/docs/tutorials/microfluidics.md b/docs/tutorials/microfluidics.md index bea083d..62d7018 100644 --- a/docs/tutorials/microfluidics.md +++ b/docs/tutorials/microfluidics.md @@ -4,6 +4,8 @@ This tutorial connects device geometry, flowing media, and cell biology in runna These models use XY-only division jitter and finite-height 3D confinement. A thin cavity can encourage a monolayer, but it does not force a common center Z or eliminate tilt; walls are soft constraints whose residual depends on relaxation tolerance and passes. The [dimensionality audit](planarity.md) lists each device and reproduces these distinctions. +For backend selection, PowerShell syntax, quoted JSON parameters, and paths with spaces, see [tutorial commands by backend and shell](commands.md#choose-a-shell). Multiline commands on this page use POSIX shell backslashes; the guide provides the PowerShell equivalents and [explicit CPU, Metal, and CUDA trap launches](commands.md#run-the-same-trap-on-cpu-metal-or-cuda). + The microfluidic-trap, Danino, biopixel, and pillar tutorial founders request centerline length 3.5, capped at their single sampled target in [3.2, 3.8]. Attachment, position, radius, and concentrations are preserved. This affects new construction only; saved geometry is restored unchanged. See [founder initialization and volume conventions](biophysics-and-growth.md#length-and-volume). | Model | Device | Demonstrates | @@ -16,7 +18,7 @@ The microfluidic-trap, Danino, biopixel, and pillar tutorial founders request ce Run any of them live: ```console -uv run microsimulator view --model examples/microfluidic_trap.py --seed 42 --dt 0.02 --backend metal --open +uv run microsimulator view --model examples/microfluidic_trap.py --seed 42 --dt 0.02 --backend cpu --open ``` ## Walls that cells and chemistry both respect @@ -145,7 +147,7 @@ With `include_blocks=True`, the reader also exposes geometry in unplaced block d The executable example loads and checks this layout, then simulates one cavity using the independently published dimensions. That single-trap reduction assumes one selected local inlet condition; it does not assert uniform flow across the array, reproduce the array manifold, or include inter-trap coupling. Run it live: ```console -uv run microsimulator view --model examples/tutorials/biopixel_trap.py --seed 5 --dt 0.02 --backend metal --open +uv run microsimulator view --model examples/tutorials/biopixel_trap.py --seed 5 --dt 0.02 --backend cpu --open ``` ## Units and timescales diff --git a/docs/tutorials/nutrient-validation.md b/docs/tutorials/nutrient-validation.md index bc0b197..838bcc5 100644 --- a/docs/tutorials/nutrient-validation.md +++ b/docs/tutorials/nutrient-validation.md @@ -2,6 +2,8 @@ Run the controlled numerical study with: +For backend selection, PowerShell syntax, quoted JSON parameters, and paths with spaces, see [tutorial commands by backend and shell](commands.md#choose-a-shell). Multiline commands on this page use POSIX shell backslashes; the guide provides the PowerShell equivalents and [explicit CPU, Metal, and CUDA trap launches](commands.md#run-the-same-trap-on-cpu-metal-or-cuda). + ```console uv run python scripts/run_nutrient_benchmarks.py --backend cpu --output build/nutrient-cpu.json uv run python scripts/run_nutrient_benchmarks.py --backend metal --output build/nutrient-metal.json diff --git a/docs/tutorials/planarity.md b/docs/tutorials/planarity.md index e1366ca..2935868 100644 --- a/docs/tutorials/planarity.md +++ b/docs/tutorials/planarity.md @@ -27,6 +27,8 @@ The following contracts describe the checked-in models after the founder-initial ## Reproduce and locate a departure +For backend discovery and shell-specific JSON quoting, see the [shared command guide](commands.md#choose-a-shell). The commands below run the dedicated diagnostic script; its diagnostic flags are described here. + Run the diagnostic from the repository root after installing the development environment: ```console diff --git a/docs/tutorials/signaling.md b/docs/tutorials/signaling.md index f8e42ef..e54171c 100644 --- a/docs/tutorials/signaling.md +++ b/docs/tutorials/signaling.md @@ -4,6 +4,8 @@ This tutorial introduces extracellular grids, diffusion, cell-grid exchange, sen All scenarios use XY-only division jitter with three-dimensional mechanics. `single_gene` and `communication` have lateral Y walls only; `mutualism` has no mechanical walls. Signal-grid depth does not constrain cell Z or tilt. See [division jitter and out-of-plane motion](planarity.md). +For backend selection, PowerShell syntax, quoted JSON parameters, and paths with spaces, see [tutorial commands by backend and shell](commands.md#choose-a-shell). Multiline commands on this page use POSIX shell backslashes; the guide provides the PowerShell equivalents and [explicit CPU, Metal, and CUDA trap launches](commands.md#run-the-same-trap-on-cpu-metal-or-cuda). + Each new founder requests centerline length 3.5 and is capped at its sampled division threshold. Its radius, position, cell type, and initial concentrations are preserved. See [founder initialization](biophysics-and-growth.md#length-and-volume). ## Grid geometry and units @@ -32,6 +34,7 @@ Both coefficient arrays must be finite and non-negative. A relaxation toward a n ```console uv run microsimulator view \ --model examples/tutorials/signaling.py \ + --backend cpu \ --parameter scenario='"single_gene"' \ --seed 42 \ --dt 0.01 \ @@ -60,6 +63,7 @@ Two inward-facing planes at `y = -16` and `y = 16` confine cells. The signal gri ```console uv run microsimulator view \ --model examples/tutorials/signaling.py \ + --backend cpu \ --parameter scenario='"communication"' \ --seed 42 \ --dt 0.01 \ @@ -88,6 +92,7 @@ Use cell-type coloring to identify sender and receiver lineages, species channel ```console uv run microsimulator view \ --model examples/tutorials/signaling.py \ + --backend cpu \ --parameter scenario='"mutualism"' \ --seed 42 \ --dt 0.01 \ diff --git a/docs/tutorials/simbol.md b/docs/tutorials/simbol.md index 6bffc75..2d28543 100644 --- a/docs/tutorials/simbol.md +++ b/docs/tutorials/simbol.md @@ -4,6 +4,8 @@ SimBOL connects an SBOL 3 design to simulator-specific code through a summarized The six circuit models start in XY and use XY-only division jitter without mechanical walls. The Danino clock uses a finite-height trap. Both retain three-dimensional mechanics; see [division jitter, confinement, and out-of-plane motion](planarity.md). +For backend selection, PowerShell syntax, quoted JSON parameters, and paths with spaces, see [tutorial commands by backend and shell](commands.md#choose-a-shell). Multiline commands on this page use POSIX shell backslashes; the guide provides the PowerShell equivalents and [explicit CPU, Metal, and CUDA trap launches](commands.md#run-the-same-trap-on-cpu-metal-or-cuda). + These are explicit example models, not a general SBOL-to-rate-plan import path. The [source reference](../compatibility/tutorial-source-provenance.md#simbol-source-workflow) describes how they relate to the SimBOL notebook, generated Python, and JSON fixtures. ## Run the six circuits @@ -13,6 +15,7 @@ Use one model and select a circuit: ```console uv run microsimulator view \ --model examples/tutorials/simbol_circuits.py \ + --backend cpu \ --parameter circuit='"bba_0001"' \ --seed 42 \ --dt 0.01 \ @@ -35,6 +38,7 @@ Parameters are JSON numbers: ```console uv run microsimulator run \ --model examples/tutorials/simbol_circuits.py \ + --backend cpu \ --parameter circuit='"bba_0004"' \ --parameter inducer_concentration=4.0 \ --seed 42 \ @@ -117,6 +121,7 @@ These choices change trajectories relative to the generated callback scripts. A ```console uv run microsimulator view \ --model examples/tutorials/danino_clock.py \ + --backend cpu \ --seed 42 \ --dt 0.005 \ --open diff --git a/environments/cuda/README.md b/environments/cuda/README.md index cb16efe..603dbb5 100644 --- a/environments/cuda/README.md +++ b/environments/cuda/README.md @@ -2,6 +2,8 @@ CUDA is the NVIDIA backend under active development. It is implemented directly in CUDA C++ with the CUDA Runtime API: no portability layer, translated Metal source, or CPU computational fallback is used. +For application commands, use the [shared CPU, Metal, and CUDA tutorial examples](../../docs/tutorials/commands.md#run-the-same-trap-on-cpu-metal-or-cuda), [device discovery](../../docs/tutorials/commands.md#prepare-and-discover-devices), and [shell quoting guide](../../docs/tutorials/commands.md#choose-a-shell). Select this backend with `--backend cuda` and an enumerated `--device-index`; the model syntax is unchanged. + `CM_ENABLE_CUDA` is off by default so ordinary CPU builds do not acquire a CUDA toolchain dependency. ## Compile check diff --git a/environments/metal/README.md b/environments/metal/README.md index 53500ea..d5214f7 100644 --- a/environments/metal/README.md +++ b/environments/metal/README.md @@ -2,6 +2,8 @@ Metal is the feature-complete Apple GPU backend. It is implemented directly with the Metal API and independent Metal Shading Language kernels, and it is validated against both the CPU reference and recorded behavior from the original CellModeller OpenCL runtime. +For application commands, use the [shared CPU, Metal, and CUDA tutorial examples](../../docs/tutorials/commands.md#run-the-same-trap-on-cpu-metal-or-cuda), [device discovery](../../docs/tutorials/commands.md#prepare-and-discover-devices), and [shell quoting guide](../../docs/tutorials/commands.md#choose-a-shell). Select this backend with `--backend metal` and an enumerated `--device-index`; the model syntax is unchanged. + Metal is enabled by default on Apple platforms. It compiles embedded MSL source at runtime through `MTLDevice`, making kernel compilation part of device construction and validation. ## Native conformance diff --git a/python/tests/test_tutorial_commands.py b/python/tests/test_tutorial_commands.py new file mode 100644 index 0000000..738df00 --- /dev/null +++ b/python/tests/test_tutorial_commands.py @@ -0,0 +1,426 @@ +"""Execute the published CLI examples through real shells, without rewriting them.""" + +from __future__ import annotations + +import asyncio +import hashlib +import json +import math +import os +import platform +import re +import shlex +import shutil +import signal +import subprocess +import sys +from collections.abc import Iterator +from contextlib import suppress +from dataclasses import dataclass, field +from pathlib import Path +from typing import Any, cast +from urllib.parse import urlsplit + +import pytest +from aiohttp import ClientSession, WSMsgType +from microsimulator import load_checkpoint_bundle +from microsimulator.cli import _parser # pyright: ignore[reportPrivateUsage] + +ROOT = Path(__file__).resolve().parents[2] +GUIDE = ROOT / "docs/tutorials/commands.md" +BLOCKS = dict( + re.findall( + r"\s*```\w+\n(.*?)\n```", + GUIDE.read_text(encoding="utf-8"), + re.DOTALL, + ) +) +SHELLS = ("pwsh",) if sys.platform == "win32" else ("sh", "bash", "zsh", "pwsh") + + +def _same_time(actual: float, expected: float) -> bool: + # dt is stored in native float32; retain pytest.approx's intended tolerance. + return math.isclose(actual, expected, rel_tol=1e-6, abs_tol=1e-8) + + +def _kill_tree(pid: int) -> None: + # Only called for a subprocess created by this test. Killing just its shell + # leaves uv/the live server holding the inherited stdout pipe and port. + if sys.platform == "win32": + subprocess.run( + ["taskkill", "/PID", str(pid), "/T", "/F"], + capture_output=True, + check=False, + timeout=10, + ) + else: + with suppress(ProcessLookupError): + os.killpg(pid, signal.SIGKILL) + + +def _run_script( + arguments: list[str], + environment: dict[str, str], + cwd: Path, +) -> subprocess.CompletedProcess[str]: + with subprocess.Popen( + arguments, + cwd=cwd, + env=environment, + text=True, + stdout=subprocess.PIPE, + stderr=subprocess.PIPE, + start_new_session=sys.platform != "win32", + ) as process: + try: + stdout, stderr = process.communicate(timeout=120) + except subprocess.TimeoutExpired as error: + _kill_tree(process.pid) + stdout, stderr = process.communicate(timeout=10) + raise AssertionError(f"command timed out: {stdout} {stderr}") from error + return subprocess.CompletedProcess(arguments, process.returncode, stdout, stderr) + + +@dataclass +class CommandShell: + name: str + executable: str + cwd: Path + records: list[dict[str, Any]] = field(default_factory=lambda: list[dict[str, Any]]()) + + def prepare(self, command: str) -> tuple[list[str], dict[str, str]]: + environment = dict(os.environ) + environment.pop("VIRTUAL_ENV", None) + environment["UV_PROJECT_ENVIRONMENT"] = sys.prefix + environment["PYTHONUNBUFFERED"] = "1" + environment["UV_OFFLINE"] = "1" + if self.name == "pwsh": + script = self.cwd / "command.ps1" + script.write_text( + "$ErrorActionPreference = 'Stop'\n" + "$global:LASTEXITCODE = 0\n" + + BLOCKS["powershell-mode"] + + "\n" + + command + + "\nexit $LASTEXITCODE\n", + encoding="utf-8", + ) + arguments = [self.executable, "-NoLogo", "-NoProfile", "-File", str(script)] + else: + script = self.cwd / "command.sh" + script.write_text("set -e\n" + command + "\n", encoding="utf-8") + arguments = [self.executable, str(script)] + return arguments, environment + + def run(self, identifier: str, *, success: bool = True) -> subprocess.CompletedProcess[str]: + command = BLOCKS[identifier] + arguments, environment = self.prepare(command) + print(f"Executing {self.name}: {identifier}", flush=True) + result = _run_script(arguments, environment, self.cwd) + self.records.append( + { + "id": identifier, + "command": command, + "exit_code": result.returncode, + "stdout": result.stdout, + "stderr": result.stderr, + } + ) + assert (result.returncode == 0) is success, result.stdout + result.stderr + return result + + +@pytest.fixture(params=SHELLS) +def shell(request: pytest.FixtureRequest, tmp_path: Path) -> Iterator[CommandShell]: + name = cast(str, request.param) + executable = shutil.which(name) + if executable is None: + pytest.skip(f"{name} is not installed; no {name} coverage claimed") + cwd = tmp_path / "repository with spaces" + cwd.mkdir() + shutil.copytree(ROOT / "examples", cwd / "examples") + (cwd / "pyproject.toml").write_text( + '[project]\nname="tutorial-command-test"\nversion="0.0.0"\n' + 'requires-python=">=3.12,<3.13"\n', + encoding="utf-8", + ) + # CLI lifecycle checks deliberately do not claim to test the browser bundle. + (cwd / "viewer/dist/assets").mkdir(parents=True) + (cwd / "viewer/dist/index.html").write_text("CLI test") + instance = CommandShell(name, executable, cwd) + version_args = ( + ["-NoLogo", "-NoProfile", "-Command", "$PSVersionTable.PSVersion.ToString()"] + if name == "pwsh" + else ["--version"] + if name != "sh" + else ["-c", "echo POSIX-sh"] + ) + version = subprocess.run( + [executable, *version_args], capture_output=True, text=True, timeout=60, check=True + ).stdout.strip() + if name == "pwsh": + assert tuple(int(part) for part in version.split(".")[:2]) >= (7, 3) + try: + yield instance + finally: + if destination := os.environ.get("MICROSIMULATOR_COMMAND_REPORT"): + directory = Path(destination) + directory.mkdir(parents=True, exist_ok=True) + node = cast(Any, request).node # pytest's request.node is untyped. + (directory / f"{node.name}.json").write_text( + json.dumps( + { + "platform": platform.platform(), + "python": sys.version, + "shell": name, + "shell_version": version, + "cwd_contains_spaces": True, + "commands": instance.records, + }, + indent=2, + ), + encoding="utf-8", + ) + + +def _document(path: Path) -> dict[str, Any]: + # Authenticate checkpoints as well as inspecting their human-readable fields. + load_checkpoint_bundle(path) + return cast(dict[str, Any], json.loads(path.read_text(encoding="utf-8"))) + + +def test_documented_headless_commands(shell: CommandShell) -> None: + shell.run("devices") + devices = cast(list[dict[str, Any]], json.loads(shell.run("devices-json").stdout)) + assert {record["backend"] for record in devices} == {"cpu", "metal", "cuda"} + for record in devices: + backend = record["backend"] + result = shell.run(f"trap-{backend}", success=record["available"]) + output = shell.cwd / f"results/tutorial runs/trap-{backend}.json" + if not record["available"]: + assert f"backend {backend} device 0 is unavailable" in result.stderr + assert not output.exists() + continue + document = _document(output) + assert document["source_backend"]["kind"] == backend + assert document["provenance"]["model"]["seed"] == 42 + assert document["provenance"]["model"]["parameters"] == {} + assert document["provenance"]["run"]["completed_steps"] == 100 + assert _same_time(document["simulation"]["time"], 2.0) + assert len(list(output.parent.glob(f"trap-{backend}.step-*.json"))) == 5 + shell.run("resume-trap") + assert _same_time( + _document(shell.cwd / "results/tutorial runs/trap-cpu-resumed.json")["simulation"]["time"], + 4.0, + ) + + suffix = "powershell" if shell.name == "pwsh" else "posix" + shell.run(f"copy-{suffix}") + shell.run(f"basics-{suffix}") + shell.run("resume-basics") + initial = _document(shell.cwd / "results/tutorial runs/basics.json") + resumed = _document(shell.cwd / "results/tutorial runs/basics-resumed.json") + for document in (initial, resumed): + provenance = document["provenance"]["model"] + assert provenance["seed"] == 42 + assert provenance["parameters"] == {"scenario": "basics"} + assert ( + provenance["sha256"] + == hashlib.sha256( + (shell.cwd / "results/tutorial models/biophysics.py").read_bytes() + ).hexdigest() + ) + assert _same_time(initial["simulation"]["time"], 0.2) + assert _same_time(resumed["simulation"]["time"], 0.4) + assert ( + resumed["provenance"]["resume"]["sha256"] + == hashlib.sha256( + (shell.cwd / "results/tutorial runs/basics.json").read_bytes() + ).hexdigest() + ) + + # Independently establish the claimed continuation semantics, using the same + # installed CLI for a 20-step uninterrupted run. + subprocess.run( + [ + sys.executable, + "-m", + "microsimulator", + "run", + "--model", + str(shell.cwd / "results/tutorial models/biophysics.py"), + "--parameter", + 'scenario="basics"', + "--seed", + "42", + "--steps", + "20", + "--dt", + "0.02", + "--output", + str(shell.cwd / "uninterrupted.json"), + ], + check=True, + capture_output=True, + text=True, + timeout=60, + ) + uninterrupted = _document(shell.cwd / "uninterrupted.json") + assert resumed["simulation"] == uninterrupted["simulation"] + assert resumed["controller"] == uninterrupted["controller"] + + # The documented guard rejects overrides and edited source before executing + # it; neither failed resume should create an output or run injected code. + arguments, environment = shell.prepare(BLOCKS["resume-basics"] + " --parameter 'scenario=1'") + rejected = subprocess.run( + arguments, + env=environment, + cwd=shell.cwd, + capture_output=True, + text=True, + timeout=60, + check=False, + ) + assert rejected.returncode != 0 + assert "do not pass --parameter" in rejected.stderr + model = shell.cwd / "results/tutorial models/biophysics.py" + model.write_text("raise RuntimeError('model executed before digest check')\n", encoding="utf-8") + rejected = shell.run("resume-basics", success=False) + assert "model digest does not match checkpoint" in rejected.stderr + assert "model executed before digest check" not in rejected.stderr + + +def test_documented_live_commands(shell: CommandShell) -> None: + suffix = "powershell" if shell.name == "pwsh" else "posix" + shell.run(f"copy-{suffix}") + shell.run(f"basics-{suffix}") + + async def exercise() -> None: + for identifier in ("live-trap", "live-basics", "live-resume"): + arguments, environment = shell.prepare(BLOCKS[identifier]) + process = await asyncio.create_subprocess_exec( + *arguments, + cwd=shell.cwd, + env=environment, + stdout=asyncio.subprocess.PIPE, + stderr=asyncio.subprocess.PIPE, + start_new_session=sys.platform != "win32", + ) + stopped = False + try: + assert process.stdout is not None + line = (await asyncio.wait_for(process.stdout.readline(), 60)).decode().strip() + assert line.startswith("MicroSimulator live viewer: "), line + url = urlsplit(line.split(": ", 1)[1]) + assert url.port == 8765 + async with ( + ClientSession() as client, + client.ws_connect( + f"http://{url.netloc}/api/v1/session?{url.query}", + headers={"Origin": f"http://{url.netloc}"}, + ) as ws, + ): + frame = await ws.receive_json(timeout=10) + assert frame["playing"] is False + assert _same_time( + frame["scene"]["frame"]["time"], + 0.2 if identifier == "live-resume" else 0.0, + ) + if identifier == "live-trap": + await ws.send_json({"type": "checkpoint"}) + saved = await ws.receive_json(timeout=10) + assert saved["type"] == "checkpoint" + _document(shell.cwd / "results/tutorial runs/live-trap.json") + await ws.send_json({"type": "stop"}) + while True: + message = await ws.receive(timeout=15) + if message.type in {WSMsgType.CLOSE, WSMsgType.CLOSED, WSMsgType.ERROR}: + break + if message.type == WSMsgType.TEXT: + stopped |= message.json() == {"type": "session", "state": "stopped"} + stdout, stderr = await asyncio.wait_for(process.communicate(), 15) + shell.records.append( + { + "id": identifier, + "command": BLOCKS[identifier], + "exit_code": process.returncode, + "stopped": stopped, + "stdout": stdout.decode(), + "stderr": stderr.decode(), + } + ) + assert stopped + assert process.returncode == 0, stderr.decode() + finally: + if process.returncode is None: + await asyncio.to_thread(_kill_tree, process.pid) + await asyncio.wait_for(process.communicate(), 10) + + asyncio.run(exercise()) + + +def test_tutorial_flags_paths_and_links() -> None: + documents = [ + *sorted((ROOT / "docs/tutorials").glob("*.md")), + ROOT / "environments/metal/README.md", + ROOT / "environments/cuda/README.md", + ROOT / "viewer/README.md", + ROOT / "docs/development/tutorial-command-validation.md", + ] + parser = _parser() + checked = 0 + for path in documents: + text = path.read_text(encoding="utf-8") + for target in re.findall(r"\[[^\]]+\]\(([^)]+)\)", text): + if re.match(r"[a-z]+://", target): + continue + filename, _, anchor = target.partition("#") + destination = (path.parent / filename).resolve() if filename else path + assert destination.exists(), (path, target) + if anchor and destination.suffix == ".md": + headings = re.findall( + r"^#+\s+(.+)$", destination.read_text(encoding="utf-8"), re.MULTILINE + ) + anchors = {re.sub(r"[^\w -]", "", h.lower()).replace(" ", "-") for h in headings} + assert anchor in anchors, (path, target) + if path.parent == ROOT / "docs/tutorials" and path != GUIDE: + assert "commands.md#" in text, path + for language, block in re.findall(r"```(console|sh)\n(.*?)\n```", text, re.DOTALL): + del language + for line in block.replace("\\\n", " ").splitlines(): + parts = shlex.split(line) + if "microsimulator" not in parts or "--help" in parts: + continue + # Exclude prose or output; only parse literal CLI invocations. + if parts[:2] != ["uv", "run"]: + continue + arguments = parser.parse_args(parts[parts.index("microsimulator") + 1 :]) + if (model := getattr(arguments, "model", None)) is not None: + model = cast(Path, model) + if model != Path("results/tutorial models/biophysics.py"): + assert (ROOT / model).is_file(), (path, model) + checked += 1 + assert checked >= 35 + + +def test_failed_command_cleanup_drains_descendant_pipes() -> None: + # Regression for a failed live assertion hanging after only its shell died. + code = ( + "import subprocess, sys, time; " + "subprocess.Popen([sys.executable, '-c', 'import time; time.sleep(60)']); " + "print('ready', flush=True); time.sleep(60)" + ) + with subprocess.Popen( + [sys.executable, "-c", code], + stdout=subprocess.PIPE, + stderr=subprocess.PIPE, + text=True, + start_new_session=sys.platform != "win32", + ) as process: + try: + assert process.stdout is not None + assert process.stdout.readline().strip() == "ready" + finally: + _kill_tree(process.pid) + process.communicate(timeout=10) + assert process.returncode is not None diff --git a/viewer/README.md b/viewer/README.md index 42022f5..948fc59 100644 --- a/viewer/README.md +++ b/viewer/README.md @@ -2,6 +2,8 @@ The MicroSimulator viewer displays cells, device walls, and signal fields so you can inspect a population in its microfluidic environment. Use it to explore saved scenes or follow a live simulation with growth-rate coloring, nutrient slices, and individual-cell inspection. +See [tutorial commands by backend and shell](../docs/tutorials/commands.md#live-view-stop-and-restart) for copyable CPU/Metal/CUDA selection, PowerShell quoting, paths with spaces, checkpoint resume, and Stop/restart. Multiline commands below use POSIX shell backslashes. + The viewer is a TypeScript and Three.js client for `microsimulator-scene` documents. Standalone mode reads scene files; live mode sends typed controls to a Python-owned engine session and verifies every returned scene document. Python owns the model, simulation clock, backend, and checkpoint writer. ## Run locally @@ -45,7 +47,7 @@ For example, after stopping the trap model above, launch a different model on th uv run microsimulator view --model examples/tutorials/biophysics.py --backend cpu --seed 42 --dt 0.02 --port 8765 --open ``` -The command is a single line and also works in PowerShell where the Python/native build is available. To distinguish Windows console behavior from browser behavior, use this manual verification procedure in an attached PowerShell or Command Prompt console: +The command is a single line and also works in PowerShell 7.3+ where the Python/native build is available; configure [Standard argument passing](../docs/tutorials/commands.md#choose-a-shell) for JSON-valued parameters. To distinguish Windows console behavior from browser behavior, use this manual verification procedure in an attached PowerShell or Command Prompt console: 1. Record the Windows version, terminal application/version, Python version, and exact launch command. Start the command above and click Stop while paused. Confirm the prompt returns, then start the second model on port 8765. 2. Repeat with Play active and `--frame-steps 10000`. Confirm Stopping transitions to Stopped without finishing the entire batch. @@ -54,7 +56,7 @@ The command is a single line and also works in PowerShell where the Python/nativ The automated `python/tests/test_viewer_shutdown.py` suite covers same-socket Stop, checkpoint completion, worker cleanup, and repeated real subprocess restarts. It sends SIGINT on POSIX. On Windows it starts each viewer in an isolated console and uses a separate attached sender to deliver a real [Windows CTRL_C_EVENT](https://learn.microsoft.com/en-us/windows/console/generateconsolectrlevent), leaving the test runner unaffected. Both paths verify orderly browser notifications, clean process exit, and three different models reusing the same port. This exercises the operating-system interruption path; use the manual procedure above to check a particular interactive terminal application and keyboard configuration. -The `Windows live-session shutdown` GitHub Actions job builds the CPU extension on `windows-2025` and runs the server and shutdown tests, including isolated-console Ctrl+C, with dependencies from `uv.lock`. Its uploaded report records Windows, PowerShell, Python, backend availability, and individual test results. +The `Windows CLI and live-session checks` GitHub Actions job builds the CPU extension on `windows-2025` and runs the server and shutdown tests, including isolated-console Ctrl+C, with dependencies from `uv.lock`. Its uploaded report records Windows, PowerShell, Python, backend availability, and individual test results. ## Capabilities