From d4ac2a684131803954a4d0155c77ead5acbc21a2 Mon Sep 17 00:00:00 2001 From: Szymon Chmal Date: Wed, 18 Nov 2026 10:33:28 +0100 Subject: [PATCH 1/6] docs(adr): propose 0087, the unit tests run on Windows in CI and a list of files still failing there only shrinks (#1693) --- ...-files-still-failing-there-only-shrinks.md | 106 ++++++++++++++++++ docs/internal/adr/README.md | 1 + 2 files changed, 107 insertions(+) create mode 100644 docs/internal/adr/0087-the-unit-tests-run-on-windows-in-ci-and-a-list-of-files-still-failing-there-only-shrinks.md diff --git a/docs/internal/adr/0087-the-unit-tests-run-on-windows-in-ci-and-a-list-of-files-still-failing-there-only-shrinks.md b/docs/internal/adr/0087-the-unit-tests-run-on-windows-in-ci-and-a-list-of-files-still-failing-there-only-shrinks.md new file mode 100644 index 000000000..98b0e0337 --- /dev/null +++ b/docs/internal/adr/0087-the-unit-tests-run-on-windows-in-ci-and-a-list-of-files-still-failing-there-only-shrinks.md @@ -0,0 +1,106 @@ +# 0087. The unit tests run on Windows in CI, and a list of files still failing there only shrinks + +- **Status:** Proposed +- **Date:** 2026-11-18 +- **Issue:** [#1693](https://github.com/callstackincubator/simlock/issues/1693) (request [#1070](https://github.com/callstackincubator/simlock/issues/1070)) +- **Narrows:** testing rule 5 in `docs/internal/agent-rules/testing.md` + ("never skip, disable, or quarantine a test to get to green"), for the + Windows job's list only, while this feature is delivered. + +## Context + +CI runs only on Linux. The Windows code paths are tested by passing +`platform: "win32"` to a function on Linux, so nothing that only Windows +does (a named pipe, a batch file through `cmd.exe`, a file that cannot be +deleted while it is open) ever runs. On a Windows 11 machine at +`2df19f881`, 1,664 of 9,544 unit tests fail, in 138 of 2,184 files. Most +of them fail for a few shared reasons, not for scattered fixture paths: + +- The in-memory fake filesystem keys its entries on `/` paths, while the + code under test builds paths with the host's `node:path`, which gives + `D:\home\…` on Windows (about 710 tests). +- Android tests leave the platform to its default, so on Windows the code + looks for `emulator.exe` in a fake SDK that holds `emulator` (260). +- Tests listen on `/daemon.sock`, which Windows refuses; the daemon + there listens on a named pipe (176). +- The repository's own tooling tests call `sh` and `python3` (201). +- A test deletes its folder while a file in it is still open (96). +- A test checks a fact Windows does not have, such as mode `0600` (~20). + +The fast e2e lane cannot run on Windows yet: its harness finds a daemon by +its socket file and `lsof`. + +## Decision + +### 1. A Windows job runs the unit and DOM tests, and Quality needs it + +CI gets one job on a pinned Windows runner image that runs +`pnpm test:unit` on every push and pull request. The required `Quality` +check needs it, as it needs the Linux unit job. The e2e lane, the timing +tests, the console lane and the static checks stay on Linux. + +```mermaid +flowchart LR + PR[push or PR] --> L[Linux jobs, as today] + PR --> W[Unit tests on Windows] + L --> Q[Quality, required] + W --> Q +``` + +### 2. The in-memory filesystem reads both path styles + +The fake reads `D:\a\b`, `\a\b` and `/a/b` as one entry: it drops a drive +letter and reads `\` as `/`. Production code keeps the host's `node:path`, +and no source file is given a path flavour to make a test pass. A test +that compares a path the code returned builds the expected path with +`node:path`, the way the code does. + +### 3. A test of a fact Windows does not have runs off Windows only + +A test that checks a POSIX mode, a umask, a launchd or systemd file, or a +tool Windows does not ship is skipped on `win32`, the way the e2e files +already skip by platform. Every other test runs on both. + +### 4. A list of files still failing on Windows only shrinks + +Until every file passes, the Windows job leaves out the files on one list, +read only on a Windows host. The first task adds the job with every file +that still fails on it; each later task fixes a group of files and takes +them off. A file is never added once the job exists: a change that breaks a +file on Windows is fixed before it merges. The last task deletes the list +and this exception. + +### 5. A product defect a Windows test finds is fixed in the same task + +When a test fails on Windows because Simlock is wrong there, not the test, +the task that owns the file fixes the code with it. + +### 6. The repository's tooling runs on Windows too + +The agent scripts and their tests are made to run on a Windows host, so a +contributor on Windows can run every unit test. + +## Alternatives rejected + +- **A path flavour injected into the code**, the way `platform` is. About + 67 source files would take `posix` or `win32` path functions from the + composition root, the tests would pass `posix`, and the unit tests would + never meet a real Windows path. It changes production code to suit a fake. +- **Every fixture built with `node:path`.** About 120 test files rewritten, + and the fake still has to read `\`; more conflicts with open work. +- **Add the job only once every file passes.** About twenty open PRs edit + these tests; each that merges in the meantime can break a file already + fixed, and the last task would chase them. +- **Only the tests that touch the OS.** A logic test that breaks only on + Windows, as most of the 1,664 do, would stay unseen. +- **The e2e lane in this feature.** Its harness needs a Windows way to find + and stop a daemon first; that is its own change. + +## Consequences + +- Every pull request waits for a Windows runner as well. +- While the list exists, a file on it is not run on Windows, and a defect + only it would show stays unseen until its task. +- Windows contributors can run the whole unit suite. +- Shapes, task split and the list's file are in + [#1693](https://github.com/callstackincubator/simlock/issues/1693). diff --git a/docs/internal/adr/README.md b/docs/internal/adr/README.md index ffa8104f9..63bd22c20 100644 --- a/docs/internal/adr/README.md +++ b/docs/internal/adr/README.md @@ -119,3 +119,4 @@ the status is stale. | [0083](0083-a-worker-may-turn-single-slim-services-on-or-off-beside-its-categories.md) | A worker may turn single slim services on or off beside its categories | Accepted; narrows 0060 §1, §2 and 0066 §2 | | [0084](0084-a-worker-may-serve-a-platforms-enrolled-phones-only.md) | A worker may serve a platform's enrolled phones only | Accepted; narrows 0022 and 0009 §4 | | [0085](0085-an-ios-simulator-takes-keys-on-every-runtime-and-the-app-switcher-and-lock.md) | An iOS simulator takes keys on every runtime, and the app switcher and Lock | Accepted; narrows 0049 §4 | +| [0087](0087-the-unit-tests-run-on-windows-in-ci-and-a-list-of-files-still-failing-there-only-shrinks.md) | The unit tests run on Windows in CI, and a list of files still failing there only shrinks | Proposed; narrows testing rule 5 | From 7fcd250e63f3349109fd4c87c4a0b24d23a751b0 Mon Sep 17 00:00:00 2001 From: Szymon Chmal Date: Wed, 18 Nov 2026 11:02:09 +0100 Subject: [PATCH 2/6] docs(adr): 0087 says what the fake returns, that POSIX-only skips stay, and that tooling runs from Git Bash (#1693) --- ...-files-still-failing-there-only-shrinks.md | 33 ++++++++++++------- 1 file changed, 22 insertions(+), 11 deletions(-) diff --git a/docs/internal/adr/0087-the-unit-tests-run-on-windows-in-ci-and-a-list-of-files-still-failing-there-only-shrinks.md b/docs/internal/adr/0087-the-unit-tests-run-on-windows-in-ci-and-a-list-of-files-still-failing-there-only-shrinks.md index 98b0e0337..23caa60f9 100644 --- a/docs/internal/adr/0087-the-unit-tests-run-on-windows-in-ci-and-a-list-of-files-still-failing-there-only-shrinks.md +++ b/docs/internal/adr/0087-the-unit-tests-run-on-windows-in-ci-and-a-list-of-files-still-failing-there-only-shrinks.md @@ -4,8 +4,10 @@ - **Date:** 2026-11-18 - **Issue:** [#1693](https://github.com/callstackincubator/simlock/issues/1693) (request [#1070](https://github.com/callstackincubator/simlock/issues/1070)) - **Narrows:** testing rule 5 in `docs/internal/agent-rules/testing.md` - ("never skip, disable, or quarantine a test to get to green"), for the - Windows job's list only, while this feature is delivered. + ("never skip, disable, or quarantine a test to get to green"): for good, + a test of a fact Windows does not have is skipped there (§3); and while + this feature is delivered, the Windows job leaves out the files on its + list (§4). ## Context @@ -50,22 +52,27 @@ flowchart LR ### 2. The in-memory filesystem reads both path styles The fake reads `D:\a\b`, `\a\b` and `/a/b` as one entry: it drops a drive -letter and reads `\` as `/`. Production code keeps the host's `node:path`, -and no source file is given a path flavour to make a test pass. A test -that compares a path the code returned builds the expected path with -`node:path`, the way the code does. +letter and reads `\` as `/`, for every path it is given and every link +target it stores. What it returns, such as `realpath`, is the `/` form. +Production code keeps the host's `node:path`, and no source file is given +a path flavour or a platform option to make a test pass; a test's fixtures +follow the host instead (a fake Android SDK holds `emulator.exe` on +Windows). A test that compares a path the code built builds the expected +one with `node:path`, the way the code does; one that compares a path the +fake returned expects the `/` form. ### 3. A test of a fact Windows does not have runs off Windows only A test that checks a POSIX mode, a umask, a launchd or systemd file, or a tool Windows does not ship is skipped on `win32`, the way the e2e files -already skip by platform. Every other test runs on both. +already skip by platform. That is not skipping a test to get to green: +the fact it checks does not exist there. Every other test runs on both. ### 4. A list of files still failing on Windows only shrinks Until every file passes, the Windows job leaves out the files on one list, read only on a Windows host. The first task adds the job with every file -that still fails on it; each later task fixes a group of files and takes +that fails in any of three runs of it; each later task fixes a group of files and takes them off. A file is never added once the job exists: a change that breaks a file on Windows is fixed before it merges. The last task deletes the list and this exception. @@ -77,8 +84,10 @@ the task that owns the file fixes the code with it. ### 6. The repository's tooling runs on Windows too -The agent scripts and their tests are made to run on a Windows host, so a -contributor on Windows can run every unit test. +The agent scripts and their tests are made to run on a Windows host from +Git Bash with Python 3 installed, as the CI job runs them, so a +contributor on Windows can run every unit test. Running them from +PowerShell or `cmd.exe` is not supported. ## Alternatives rejected @@ -101,6 +110,8 @@ contributor on Windows can run every unit test. - Every pull request waits for a Windows runner as well. - While the list exists, a file on it is not run on Windows, and a defect only it would show stays unseen until its task. -- Windows contributors can run the whole unit suite. +- Windows contributors can run the whole unit suite, from Git Bash. +- An agent on a Mac reproduces a Windows-only failure by rerunning the + Windows job, not by running it locally. - Shapes, task split and the list's file are in [#1693](https://github.com/callstackincubator/simlock/issues/1693). From 3be74dd19bc480037f050336a87f004c1f751b98 Mon Sep 17 00:00:00 2001 From: Szymon Chmal Date: Wed, 18 Nov 2026 11:03:53 +0100 Subject: [PATCH 3/6] =?UTF-8?q?docs(adr):=200087=20=C2=A73=20names=20a=20c?= =?UTF-8?q?all=20Windows=20lacks,=20not=20a=20tool=20it=20does=20not=20shi?= =?UTF-8?q?p=20(#1693)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- ...n-ci-and-a-list-of-files-still-failing-there-only-shrinks.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/internal/adr/0087-the-unit-tests-run-on-windows-in-ci-and-a-list-of-files-still-failing-there-only-shrinks.md b/docs/internal/adr/0087-the-unit-tests-run-on-windows-in-ci-and-a-list-of-files-still-failing-there-only-shrinks.md index 23caa60f9..662ef48a5 100644 --- a/docs/internal/adr/0087-the-unit-tests-run-on-windows-in-ci-and-a-list-of-files-still-failing-there-only-shrinks.md +++ b/docs/internal/adr/0087-the-unit-tests-run-on-windows-in-ci-and-a-list-of-files-still-failing-there-only-shrinks.md @@ -64,7 +64,7 @@ fake returned expects the `/` form. ### 3. A test of a fact Windows does not have runs off Windows only A test that checks a POSIX mode, a umask, a launchd or systemd file, or a -tool Windows does not ship is skipped on `win32`, the way the e2e files +call Windows does not have (such as Python's `os.fork`) is skipped on `win32`, the way the e2e files already skip by platform. That is not skipping a test to get to green: the fact it checks does not exist there. Every other test runs on both. From 30d423c9a68c0931402e09c77511d65a2a3472af Mon Sep 17 00:00:00 2001 From: Szymon Chmal Date: Wed, 18 Nov 2026 11:12:13 +0100 Subject: [PATCH 4/6] docs(adr): accept 0087, not yet implemented (#1693) --- ...n-ci-and-a-list-of-files-still-failing-there-only-shrinks.md | 2 +- docs/internal/adr/README.md | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/internal/adr/0087-the-unit-tests-run-on-windows-in-ci-and-a-list-of-files-still-failing-there-only-shrinks.md b/docs/internal/adr/0087-the-unit-tests-run-on-windows-in-ci-and-a-list-of-files-still-failing-there-only-shrinks.md index 662ef48a5..0a3820b6f 100644 --- a/docs/internal/adr/0087-the-unit-tests-run-on-windows-in-ci-and-a-list-of-files-still-failing-there-only-shrinks.md +++ b/docs/internal/adr/0087-the-unit-tests-run-on-windows-in-ci-and-a-list-of-files-still-failing-there-only-shrinks.md @@ -1,6 +1,6 @@ # 0087. The unit tests run on Windows in CI, and a list of files still failing there only shrinks -- **Status:** Proposed +- **Status:** Accepted — not yet implemented - **Date:** 2026-11-18 - **Issue:** [#1693](https://github.com/callstackincubator/simlock/issues/1693) (request [#1070](https://github.com/callstackincubator/simlock/issues/1070)) - **Narrows:** testing rule 5 in `docs/internal/agent-rules/testing.md` diff --git a/docs/internal/adr/README.md b/docs/internal/adr/README.md index 63bd22c20..1cfdf7c72 100644 --- a/docs/internal/adr/README.md +++ b/docs/internal/adr/README.md @@ -119,4 +119,4 @@ the status is stale. | [0083](0083-a-worker-may-turn-single-slim-services-on-or-off-beside-its-categories.md) | A worker may turn single slim services on or off beside its categories | Accepted; narrows 0060 §1, §2 and 0066 §2 | | [0084](0084-a-worker-may-serve-a-platforms-enrolled-phones-only.md) | A worker may serve a platform's enrolled phones only | Accepted; narrows 0022 and 0009 §4 | | [0085](0085-an-ios-simulator-takes-keys-on-every-runtime-and-the-app-switcher-and-lock.md) | An iOS simulator takes keys on every runtime, and the app switcher and Lock | Accepted; narrows 0049 §4 | -| [0087](0087-the-unit-tests-run-on-windows-in-ci-and-a-list-of-files-still-failing-there-only-shrinks.md) | The unit tests run on Windows in CI, and a list of files still failing there only shrinks | Proposed; narrows testing rule 5 | +| [0087](0087-the-unit-tests-run-on-windows-in-ci-and-a-list-of-files-still-failing-there-only-shrinks.md) | The unit tests run on Windows in CI, and a list of files still failing there only shrinks | Accepted — not yet implemented; narrows testing rule 5 | From 0baa63e5090cc01332bfb769713d8454621f1b68 Mon Sep 17 00:00:00 2001 From: Szymon Chmal Date: Wed, 18 Nov 2026 11:30:17 +0100 Subject: [PATCH 5/6] docs(adr): 0087 keeps one stack; the last task fixes what other changes broke on Windows (#1693) --- ...-files-still-failing-there-only-shrinks.md | 20 ++++++++++++++----- 1 file changed, 15 insertions(+), 5 deletions(-) diff --git a/docs/internal/adr/0087-the-unit-tests-run-on-windows-in-ci-and-a-list-of-files-still-failing-there-only-shrinks.md b/docs/internal/adr/0087-the-unit-tests-run-on-windows-in-ci-and-a-list-of-files-still-failing-there-only-shrinks.md index 0a3820b6f..f92c7760f 100644 --- a/docs/internal/adr/0087-the-unit-tests-run-on-windows-in-ci-and-a-list-of-files-still-failing-there-only-shrinks.md +++ b/docs/internal/adr/0087-the-unit-tests-run-on-windows-in-ci-and-a-list-of-files-still-failing-there-only-shrinks.md @@ -73,9 +73,12 @@ the fact it checks does not exist there. Every other test runs on both. Until every file passes, the Windows job leaves out the files on one list, read only on a Windows host. The first task adds the job with every file that fails in any of three runs of it; each later task fixes a group of files and takes -them off. A file is never added once the job exists: a change that breaks a -file on Windows is fixed before it merges. The last task deletes the list -and this exception. +them off. A file is never added once the job exists: a change in the +feature's stack that breaks a file on Windows is fixed before the stack +merges. The feature merges as one stack, as every feature does, so `main` +gets the job with the last task. That task also fixes every file that +another change, merged to `main` in the meantime, broke on Windows; then it +deletes the list and this exception. ### 5. A product defect a Windows test finds is fixed in the same task @@ -99,7 +102,12 @@ PowerShell or `cmd.exe` is not supported. and the fake still has to read `\`; more conflicts with open work. - **Add the job only once every file passes.** About twenty open PRs edit these tests; each that merges in the meantime can break a file already - fixed, and the last task would chase them. + fixed, and the last task would chase them all. With the job first in the + stack, each task's own changes are checked on Windows, and the last task + chases only what other features merged meanwhile. +- **Merge the job to `main` ahead of the rest of the feature.** It would + guard `main` sooner, but `main` would hold part of a feature; the + maintainer kept the one-stack rule. - **Only the tests that touch the OS.** A logic test that breaks only on Windows, as most of the 1,664 do, would stay unseen. - **The e2e lane in this feature.** Its harness needs a Windows way to find @@ -107,7 +115,9 @@ PowerShell or `cmd.exe` is not supported. ## Consequences -- Every pull request waits for a Windows runner as well. +- Every pull request waits for a Windows runner as well, once the feature + has merged. Until then, other features' changes reach `main` with no + Windows check, and the last task fixes what they break. - While the list exists, a file on it is not run on Windows, and a defect only it would show stays unseen until its task. - Windows contributors can run the whole unit suite, from Git Bash. From fad067173ea249fe0509e1cee608fbbecd983c62 Mon Sep 17 00:00:00 2001 From: Szymon Chmal Date: Wed, 18 Nov 2026 14:19:23 +0100 Subject: [PATCH 6/6] =?UTF-8?q?docs(adr):=200087=20=C2=A72=20applies=20the?= =?UTF-8?q?=20drive=20and=20slash=20reading=20to=20absolute=20paths=20only?= =?UTF-8?q?=20(#1693)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- ...ci-and-a-list-of-files-still-failing-there-only-shrinks.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/internal/adr/0087-the-unit-tests-run-on-windows-in-ci-and-a-list-of-files-still-failing-there-only-shrinks.md b/docs/internal/adr/0087-the-unit-tests-run-on-windows-in-ci-and-a-list-of-files-still-failing-there-only-shrinks.md index f92c7760f..76a379ba0 100644 --- a/docs/internal/adr/0087-the-unit-tests-run-on-windows-in-ci-and-a-list-of-files-still-failing-there-only-shrinks.md +++ b/docs/internal/adr/0087-the-unit-tests-run-on-windows-in-ci-and-a-list-of-files-still-failing-there-only-shrinks.md @@ -52,8 +52,8 @@ flowchart LR ### 2. The in-memory filesystem reads both path styles The fake reads `D:\a\b`, `\a\b` and `/a/b` as one entry: it drops a drive -letter and reads `\` as `/`, for every path it is given and every link -target it stores. What it returns, such as `realpath`, is the `/` form. +letter and reads `\` as `/` in an absolute path it is given and in every +link target it stores. A relative path is read as today. What it returns, such as `realpath`, is the `/` form. Production code keeps the host's `node:path`, and no source file is given a path flavour or a platform option to make a test pass; a test's fixtures follow the host instead (a fake Android SDK holds `emulator.exe` on