Skip to content

Add a "native" Windows port (based on mingw-w64 instead of cygwin) - #6557

Draft
fingolfin wants to merge 14 commits into
masterfrom
claude/gap-windows-native-77e7e3
Draft

Add a "native" Windows port (based on mingw-w64 instead of cygwin)#6557
fingolfin wants to merge 14 commits into
masterfrom
claude/gap-windows-native-77e7e3

Conversation

@fingolfin

Copy link
Copy Markdown
Member

Resolves the kernel side of #4157: GAP builds and runs as a native Windows program with mingw-w64 (MSYS2/MINGW64), without Cygwin. Supersedes #6077, which was quarried for ideas.

What works

  • Full library; testinstall passes except read.tst:84, which expects POSIX semantics of opening a directory.
  • Line editing (GAP's own editor and readline), colored prompts, Ctrl-C, history file.
  • Windows path conventions: C:/... and backslashes accepted, ;-separated PATH, .exe probing, HOME from USERPROFILE.
  • Process/Exec via CreateProcess; InputOutputLocalProcess via anonymous pipes (there are no ptys).
  • Kernel extensions: gap.exe is a stub over libgap.dll, extensions link against the import library; BuildPackages.sh works under MSYS2 unchanged.
  • GASMAN workspace via VirtualAlloc (reserve, commit on demand).
  • CI: new mingw64 job builds with MSYS2 and runs testinstall and testmockpkg.

Tested

Cross-compiled from macOS (recipe in dev/crosscompile-mingw.md), smoke-tested under Wine, validated on Windows 11 (ARM VM running the x64 build under emulation) including interactive terminal use. Packages with kernel extensions: datastructures, json, orb (minus the IO-dependent test), edim, digraphs, ace, anupq.

Note: ace, anupq, edim and digraphs need small patches, to be submitted separately. I expected further packages will need (hopefully small) patches.

Not included / follow-ups

  • The IO package (POSIX bindings) does not build; it blocks cvec, curlinterface tests and everything depending on IO.
  • HPC-GAP and the Julia GC are rejected by configure on mingw.
    • in particular my other big PR touching the Julia GC integration overlaps here; I think it'll be best to first merge both (even if they are not 100% done), and then tackle making Julia integration work under mingw
  • 32-bit mingw and Windows-on-ARM native builds are untested.
  • Distribution/installer (gap-system/gap-windows currently ships Cygwin).
  • CPU time uses wallclock; SyGetOsRelease is empty; TmpDirectory is not race-free (mkdtemp substitute).

Review notes

The first build-system commit, "Support cross-compiling GAP", is not Windows-specific and can be merged on its own.

Every commit builds on all platforms; Windows-specific code is under SYS_IS_WINDOWS, no-op elsewhere. Native and cross builds pass with --enable-Werror.

Co-Authored-By: Claude Fable 5.1 noreply@anthropic.com

@fingolfin fingolfin added os: windows Issues and PRs that are (at least partially) specific to Windows topic: build system topic: kernel release notes: use title For PRs: the title of this PR is suitable for direct use in the release notes release notes: highlight PRs introducing changes that should be highlighted at the top of the release notes labels Sep 4, 2026
@lgoettgens

Copy link
Copy Markdown
Member

This PR is quite large and has a bunch of non-trivial changes. Could you factor some more parts of it into their own self-contained or stacked PRs?
At least the "Support cross-compiling GAP" seems like a good candidate for that. And once we have that, our crosscompilation hacks in the GAP.jl sphere can also be simplified a bit

@codecov

codecov Bot commented Sep 4, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 75.26882% with 23 lines in your changes missing coverage. Please review.
✅ Project coverage is 79.03%. Comparing base (e899450) to head (fb8eba3).
⚠️ Report is 2 commits behind head on master.

Files with missing lines Patch % Lines
lib/system.g 66.66% 9 Missing ⚠️
src/iostream.c 78.57% 5 Missing and 1 partial ⚠️
src/streams.c 71.42% 2 Missing and 2 partials ⚠️
lib/files.gd 90.00% 1 Missing ⚠️
lib/files.gi 75.00% 1 Missing ⚠️
lib/streams.gi 85.71% 1 Missing ⚠️
src/sysfiles.c 66.66% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master    #6557      +/-   ##
==========================================
+ Coverage   79.01%   79.03%   +0.02%     
==========================================
  Files         683      683              
  Lines      295108   295145      +37     
  Branches     8638     8639       +1     
==========================================
+ Hits       233179   233273      +94     
+ Misses      60113    60054      -59     
- Partials     1816     1818       +2     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@fingolfin

Copy link
Copy Markdown
Member Author

This PR is quite large and has a bunch of non-trivial changes. Could you factor some more parts of it into their own self-contained or stacked PRs?

That's the plan (in fact I already got a bunch of changes for this project merged). But most of what remains now needs some understanding of the overall picture to make sense. Therefore this "master" PR, which shows the entire work as it is, and which can be referenced for context as smaller PRs extracted from it are submitted.

At least the "Support cross-compiling GAP" seems like a good candidate for that.

Indeed, and I even state that ;-)

And once we have that, our crosscompilation hacks in the GAP.jl sphere can also be simplified a bit

Maybe...

@fingolfin
fingolfin force-pushed the claude/gap-windows-native-77e7e3 branch 2 times, most recently from e8e0a3d to 829d9ab Compare September 4, 2026 21:09
@lgoettgens
lgoettgens removed their request for review September 4, 2026 21:12
@lgoettgens

Copy link
Copy Markdown
Member

Same as oscar-system/GAP.jl#1426 (comment)

@fingolfin
fingolfin removed the request for review from ChrisJefferson September 4, 2026 21:39
@ChrisJefferson

Copy link
Copy Markdown
Contributor

I had a quick scan through, none of these seems upsettting. There is a lot of it (obviously), in some cases I do wonder if the AI could be convinced to write a little less, but none of it seems to fundamentally break unix GAP, it's fairly well isolated.

fingolfin and others added 10 commits September 9, 2026 01:20
mingw-w64 predefines _WIN32; Cygwin GCC does not, as Cygwin is a POSIX
system to GAP. So SYS_IS_WINDOWS and SYS_IS_CYGWIN32 are mutually
exclusive. Groundwork for the MinGW port (#4157).

ARCH_IS_WINDOWS() also reports true for a "mingw" architecture.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Make the kernel compile with mingw-w64 (#4157, partly after #6077):
termios, uname, sigaction, lstat, ttyname, mkdtemp, realpath,
readlink, getrusage, sleep, environ, mkdir. Without termios the line
editor falls back to syFgetsNoEdit. On Windows, SyTime and RUNTIMES
use GetProcessTimes, TmpDirectory a mkdtemp built on _mktemp and
mkdir, SyGetOsRelease RtlGetVersion. The fork and pty based
subprocess code is compiled out, leaving the stubs of
GAP_DISABLE_SUBPROCESS_CODE builds. charreadhook_rl is only defined
with select, its sole user.

New configure checks: sys/utsname.h, realpath, readlink, mkdtemp,
lstat, ttyname. No-ops on current platforms.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Some SYS_IS_CYGWIN32 guards cover behavior any Windows needs, not
Cygwin quirks: O_BINARY, CRLF stripping, the C:/WINDOWS/Temp
fallback, backslash conversion in root paths, ~/_gap, and the GMP bag
zeroing for #3434. Widen them to SYS_IS_WINDOWS. Genuine Cygwin
quirks stay Cygwin-only.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Two testinstall diffs:

- SyFseek: reject negative positions; Windows' lseek accepts them
- the ExecuteProcess stub validates its arguments like the real
  implementation, so the documented errors come first

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
ffgen and gap-nocomp (generating c_oper1.c/c_type1.c) cannot run
when cross-compiling. Fail with an explanation: seed the generated
files from a native build into src/, as the s390x CI job does. Skip
'make check' for the bundled GMP and zlib, whose test programs cannot
run either.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
For #4157, partly after #6077. gap.exe is a plain static executable
for now; the Cygwin split into gap.exe and libgap.dll comes with
kernel extensions. Stack size as on Cygwin. Multiple definitions are
allowed since the PE linker rejects the duplicate EXPORT_INLINE
instances ELF linkers merge. Bundled zlib uses its win32 makefile;
bundled GMP and zlib drop -Wl,-rpath. HPC-GAP and non-GASMAN garbage
collectors are rejected. Recipe in dev/crosscompile-mingw.md.

The result links but does not run yet.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The C runtime defaults to text mode, writing "\n" as "\r\n"; this
corrupted binary data, checksums and seek positions. Default to
binary, as Cygwin's binmode mounts do; GAP handles CRLF itself.

Size the default workspace limit from physical memory via
GlobalMemoryStatusEx; without sysconf it stayed at 2 GB, which
testinstall exceeds.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Use the console API where POSIX uses termios: disable line input,
echo and input processing so the line editor sees every key, and
enable virtual terminal input, which delivers special keys as the VT
sequences the editor understands. Enable VT processing for output
(colors), read the window size from GetConsoleScreenBufferInfo at
each line (there is no SIGWINCH), and restore the mode when leaving
raw mode.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
- HOME: derive from USERPROFILE when unset, normalize backslashes;
  tilde expansion, ~/.gap and ~/.gap_hist then work
- temporary files: use TEMP or TMP; C:/WINDOWS/Temp is not writable
  for users on current Windows
- DirectoriesSystemPrograms: split PATH on ';', accept the spelling
  "Path", convert backslashes
- PathSystemProgram and Filename: probe with ".exe" appended
- Filename: accept ':' and '\' on Windows
- GAP_getcwd: report slashes

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
SyExecuteProcess on top of CreateProcess: the argument vector becomes
a command line under the MSVCRT quoting rules, the program name
quoted with backslashes since cmd.exe takes forward slash paths for
switches; standard input and output attach to the streams GAP passes
(NUL for -1); the exit code is returned; GAP stops reading its own
standard input meanwhile. Process() and Exec() work.

FuncExecuteProcess is shared by all configurations, so argument
validation is uniform; builds without fork or a Windows API, and
GAP_DISABLE_SUBPROCESS_CODE builds, keep signalling failure.
InputOutputLocalProcess stays unsupported here.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
fingolfin and others added 4 commits September 9, 2026 01:23
Load modules via LoadLibrary/GetProcAddress; HAVE_DYNAMIC_MODULES
covers it and dlopen.

As on Cygwin, GAP proper lives in libgap.dll next to a gap.exe stub.
Extensions link against the import library libgap.dll.a, so the
loader resolves them against the libgap.dll already in the process,
by module name, without a filesystem search. libgcc is linked
statically; depending on libgcc_s_seh-1.dll made the DLL unloadable
outside MSYS2.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Mirror the mmap pool: reserve at the same fixed address (16 TB, else
anywhere) and grow contiguously in place. Committed pages count
against the commit limit even untouched, so the pool is only
reserved, committed as the workspace grows into it, and decommitted
after garbage collection like MADV_DONTNEED. Commit and decommit work
per reservation, as one call may not span several. Sizes round up to
the 64 kB granularity so the region end stays extensible.

Before, one calloc with no way to grow capped the workspace at the
initial pool size.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
There are no pseudo terminals; connect the child's standard streams
to anonymous pipes. Standard output and error get distinct handles
onto one pipe: a child rearranging its standard error (ACE does)
must not take its output down with it. Reads poll with PeekNamedPipe
to keep GAP responsive, and squash the \r\n a native child emits,
which a pty never shows.

ReadAllLine for pty streams required UNIXSelect; use it when
available, else let ReadLine block.

A child sees a pipe, not a terminal, so it must flush its own
output; anupq and ace need small patches.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Build with MSYS2's mingw-w64, fetch the required packages, run
testinstall and testmockpkg through dev/ci.sh. read.tst is removed
first: StringFile("/") expects a directory to open but not read; on
Windows the open fails.

dev/crosscompile-mingw.md gains the testing recipe and status.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@fingolfin

Copy link
Copy Markdown
Member Author

in some cases I do wonder if the AI could be convinced to write a little less

Agreed. And I think I managed. Anyway, I still left this PR as draft, better to first focus on the extracted parts, namely PRs #6567 and #6569

As a bonus, we discovered a bug in the emscripten port, see #6568

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

os: windows Issues and PRs that are (at least partially) specific to Windows release notes: highlight PRs introducing changes that should be highlighted at the top of the release notes release notes: use title For PRs: the title of this PR is suitable for direct use in the release notes topic: build system topic: kernel

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants