Skip to content
Open
Original file line number Diff line number Diff line change
@@ -0,0 +1,127 @@
# 0087. The unit tests run on Windows in CI, and a list of files still failing there only shrinks

- **Status:** Accepted — not yet implemented
- **Date:** 2026-11-18
- **Issue:** [#1693](https://github.com/callstackincubator/simlock/issues/1693) (request [#1070](https://github.com/callstackincubator/simlock/issues/1070))
- **Narrows:** testing rule 5 in `docs/internal/agent-rules/testing.md`
("never skip, disable, or quarantine a test to get to green"): for good,
a test of a fact Windows does not have is skipped there (§3); and while
this feature is delivered, the Windows job leaves out the files on its
list (§4).

## Context

CI runs only on Linux. The Windows code paths are tested by passing
`platform: "win32"` to a function on Linux, so nothing that only Windows
does (a named pipe, a batch file through `cmd.exe`, a file that cannot be
deleted while it is open) ever runs. On a Windows 11 machine at
`2df19f881`, 1,664 of 9,544 unit tests fail, in 138 of 2,184 files. Most
of them fail for a few shared reasons, not for scattered fixture paths:

- The in-memory fake filesystem keys its entries on `/` paths, while the
code under test builds paths with the host's `node:path`, which gives
`D:\home\…` on Windows (about 710 tests).
- Android tests leave the platform to its default, so on Windows the code
looks for `emulator.exe` in a fake SDK that holds `emulator` (260).
- Tests listen on `<folder>/daemon.sock`, which Windows refuses; the daemon
there listens on a named pipe (176).
- The repository's own tooling tests call `sh` and `python3` (201).
- A test deletes its folder while a file in it is still open (96).
- A test checks a fact Windows does not have, such as mode `0600` (~20).

The fast e2e lane cannot run on Windows yet: its harness finds a daemon by
its socket file and `lsof`.

## Decision

### 1. A Windows job runs the unit and DOM tests, and Quality needs it

CI gets one job on a pinned Windows runner image that runs
`pnpm test:unit` on every push and pull request. The required `Quality`
check needs it, as it needs the Linux unit job. The e2e lane, the timing
tests, the console lane and the static checks stay on Linux.

```mermaid
flowchart LR
PR[push or PR] --> L[Linux jobs, as today]
PR --> W[Unit tests on Windows]
L --> Q[Quality, required]
W --> Q
```

### 2. The in-memory filesystem reads both path styles

The fake reads `D:\a\b`, `\a\b` and `/a/b` as one entry: it drops a drive
letter and reads `\` as `/` in an absolute path it is given and in every
link target it stores. A relative path is read as today. What it returns, such as `realpath`, is the `/` form.
Production code keeps the host's `node:path`, and no source file is given
a path flavour or a platform option to make a test pass; a test's fixtures
follow the host instead (a fake Android SDK holds `emulator.exe` on
Windows). A test that compares a path the code built builds the expected
one with `node:path`, the way the code does; one that compares a path the
fake returned expects the `/` form.

### 3. A test of a fact Windows does not have runs off Windows only

A test that checks a POSIX mode, a umask, a launchd or systemd file, or a
call Windows does not have (such as Python's `os.fork`) is skipped on `win32`, the way the e2e files
already skip by platform. That is not skipping a test to get to green:
the fact it checks does not exist there. Every other test runs on both.

### 4. A list of files still failing on Windows only shrinks

Until every file passes, the Windows job leaves out the files on one list,
read only on a Windows host. The first task adds the job with every file
that fails in any of three runs of it; each later task fixes a group of files and takes
them off. A file is never added once the job exists: a change in the
feature's stack that breaks a file on Windows is fixed before the stack
merges. The feature merges as one stack, as every feature does, so `main`
gets the job with the last task. That task also fixes every file that
another change, merged to `main` in the meantime, broke on Windows; then it
deletes the list and this exception.

### 5. A product defect a Windows test finds is fixed in the same task

When a test fails on Windows because Simlock is wrong there, not the test,
the task that owns the file fixes the code with it.

### 6. The repository's tooling runs on Windows too

The agent scripts and their tests are made to run on a Windows host from
Git Bash with Python 3 installed, as the CI job runs them, so a
contributor on Windows can run every unit test. Running them from
PowerShell or `cmd.exe` is not supported.

## Alternatives rejected

- **A path flavour injected into the code**, the way `platform` is. About
67 source files would take `posix` or `win32` path functions from the
composition root, the tests would pass `posix`, and the unit tests would
never meet a real Windows path. It changes production code to suit a fake.
- **Every fixture built with `node:path`.** About 120 test files rewritten,
and the fake still has to read `\`; more conflicts with open work.
- **Add the job only once every file passes.** About twenty open PRs edit
these tests; each that merges in the meantime can break a file already
fixed, and the last task would chase them all. With the job first in the
stack, each task's own changes are checked on Windows, and the last task
chases only what other features merged meanwhile.
- **Merge the job to `main` ahead of the rest of the feature.** It would
guard `main` sooner, but `main` would hold part of a feature; the
maintainer kept the one-stack rule.
- **Only the tests that touch the OS.** A logic test that breaks only on
Windows, as most of the 1,664 do, would stay unseen.
- **The e2e lane in this feature.** Its harness needs a Windows way to find
and stop a daemon first; that is its own change.

## Consequences

- Every pull request waits for a Windows runner as well, once the feature
has merged. Until then, other features' changes reach `main` with no
Windows check, and the last task fixes what they break.
- While the list exists, a file on it is not run on Windows, and a defect
only it would show stays unseen until its task.
- Windows contributors can run the whole unit suite, from Git Bash.
- An agent on a Mac reproduces a Windows-only failure by rerunning the
Windows job, not by running it locally.
- Shapes, task split and the list's file are in
[#1693](https://github.com/callstackincubator/simlock/issues/1693).
1 change: 1 addition & 0 deletions docs/internal/adr/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -120,4 +120,5 @@ the status is stale.
| [0083](0083-a-worker-may-turn-single-slim-services-on-or-off-beside-its-categories.md) | A worker may turn single slim services on or off beside its categories | Accepted; narrows 0060 §1, §2 and 0066 §2 |
| [0084](0084-a-worker-may-serve-a-platforms-enrolled-phones-only.md) | A worker may serve a platform's enrolled phones only | Accepted; narrows 0022 and 0009 §4 |
| [0085](0085-an-ios-simulator-takes-keys-on-every-runtime-and-the-app-switcher-and-lock.md) | An iOS simulator takes keys on every runtime, and the app switcher and Lock | Accepted; narrows 0049 §4 |
| [0087](0087-the-unit-tests-run-on-windows-in-ci-and-a-list-of-files-still-failing-there-only-shrinks.md) | The unit tests run on Windows in CI, and a list of files still failing there only shrinks | Accepted — not yet implemented; narrows testing rule 5 |
| [0088](0088-a-macos-worker-checks-fseventsd-when-doctor-runs.md) | A macOS worker checks fseventsd's memory when doctor runs, and an optional module may be on by default on a macOS worker | Accepted; narrows 0030 §2, §3 and §7, and 0046 §1 |
Loading