Skip to content

Add opt-in incremental GC with automatic pacing - #380

Merged
dimensionscape merged 2 commits into
mainfrom
feature/incremental-gc
Oct 27, 2026
Merged

dimensionscape merged 2 commits into
mainfrom
feature/incremental-gc

Conversation

@dimensionscape

@dimensionscape dimensionscape commented Oct 27, 2026 •

Copy link
Copy Markdown
Member

Major collections currently trace the live heap in one stop. This adds an experimental HXCPP_GC_INCREMENTAL mode that retains marking progress across shorter stops and advances automatically through allocation slow paths. The existing collector remains the default.

The mode uses Haxe 4.3's existing generated setter barriers without a compiler or standard library patch. It adds mutation logs, resumable array and map scans, library-owned scheduling hooks, an independent reachability verifier and synchronous recovery. Native bulk mutations retain their owners, detached threads transfer pending entries, and partial collections preserve allocator ranges. Recovery rewinds the partial epoch before a full mark so the alternating-bit fast path cannot mistake unvisited objects for survivors.

Enable with -lib hxcpp -D HXCPP_GC_INCREMENTAL and regenerate C++. Optional hooks are in hxcpp.gc.Incremental. Emscripten, cppia, nursery, moving and unsupported Haxe API layouts are rejected. Weak processing, finalizers, root capture, rendezvous, reclamation and forced completion can exceed the slice target.

Validation:

  • Nine phase and lifecycle fixtures passed with verification and deterministic controls at both normal and big block sizes. Both sizes passed three 30-second, eight-thread stress runs.
  • The runtime suite passed all 1,100 assertions with incremental verification and big blocks. Default collector controls and single-threaded incremental phase tests passed.
  • Linux ASan and UBSan passed slice, container and finishing fixtures and 30-second, eight-thread stress runs. GCC passed all 79 define configurations.
  • The isolated haxelib install, setup, update and off checks passed on Windows and Linux after including the macro dependency in their synthetic package.
  • Local TSan could not produce a usable run under the installed WSL/runtime configuration. Incremental sanitizer and native CI jobs are included.

Matched Windows x64 measurements use three processes, two warmups and five measured samples per process, with verification disabled and identical checksums:

Workload Existing max, ms Incremental max, ms Elapsed time increase
gc_frames 9.728 1.858 22.8%
gc_pauses 8.289 5.232 76.6%

The largest per-process frame p99.9 fell from 7.742 ms to 0.969 ms. The throughput target is still unmet, and cumulative explicit collections still exceed 2 ms. This remains a draft pending CI, broader integration coverage and the performance gates in #372. The protocol, source hashes and raw samples are in bench/results/incremental-gc; API and mutation details are in docs/IncrementalGc.md.

Refs #372. Concurrent marking in #327 remains deferred.

Large live heaps currently require one uninterrupted major mark. Retain mark queues between stopped-world slices, record mutations with the existing generated barriers, and resume array and map scans through managed owners.

Keep the current collector as the default. Add library scheduling hooks, independent reachability verification, epoch recovery, allocator range preservation, conventional regressions and CI stress coverage.

Record matched frame and allocation benchmarks. Tail latency improves on the measured workloads, but throughput and finishing-pause gates remain open in #372. Concurrent marking in #327 stays deferred.
The installation fixture copied extraParams.hxml without its new macro dependency, so compiler probes failed with Type not found before reading the version defines. Include the actual macro and archive nested fixture files.

Reproduced the missing type in the Linux installation check. All installation, setup, update and off checks pass on Windows and Linux after the fixture correction.
@dimensionscape
dimensionscape merged commit 1afcf1e into main Oct 27, 2026
75 of 76 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant