Activity

1. dimensionscape commented on Oct 25, 2026

   

   dimensionscape

   on Oct 25, 2026

   MemberAuthor

   More actions

   Open issues to deal with before this starts, from the GC issues open today:

   Understand first

   • Haxe nightly (API level 500): GC stress segfaults and suite failures #164: the GC stress test segfaults in every mode at API level 500. Concurrent marking should not go on top of a collector that crashes under Haxe 5, so the cause needs to be known before the mode is offered for Haxe 5 builds.

   Land first, since they change what the barrier records

   • Track dirty pointer-array ranges to reduce minor GC work after sparse writes #301 (dirty ranges in old pointer arrays) and Avoid scanning entire old maps after sparse writes during minor GC #305 (sparse writes to old maps). Both move the generational barrier from remembering the whole object to remembering what changed. This mode uses the same barrier, so it should build on that version rather than rework it twice.

   Stop-the-world work that the start and final pauses would still pay for

   • Allow GC progress while other threads run long native regular expression matches #330: a long EReg match keeps its thread from reaching a safe point, and pcre2_match_8 and pcre2_match_16 in RegExp.cpp run without leaving the managed zone. Both pauses of a concurrent cycle would wait for it.
   • Avoid blocking file finalizer I/O while all mutators are stopped #331: the file finalizer calls fclose, which can block on I/O while every thread is stopped. Finalizers run in the final pause.
   • Move expired large-buffer cache release out of the global GC pause #332: the large-buffer cache is freed inside the global pause.
   • Skip ephemeron value scans for weak maps with primitive storage #333: weak maps with primitive values still have their values walked, twice per mark, and that happens in the final pause.
   • Avoid scanning every old finalizer during minor collections #302 and Avoid rescanning stable old weak references during every minor GC #314: old finalizers and weak references are rescanned on every minor collection.

   Helpful, not blocking

   • Track benchmark results in CI and report regressions #258: benchmark tracking in CI, for the measurements in step 3.
   • Large-buffer release scans unrelated live allocations while holding the shared allocator lock #298 and Index recycled large buffers by size to avoid repeated scans under the allocator lock #300: the allocator's shared lock is held while scanning large buffers. The program keeps allocating while marking runs, so that contention matters more here.
   • Account for PCRE2 native allocations in GC memory pressure #303, Account for SQLite native database and statement memory in GC pressure #319 and Account for native TLS context memory in GC pressure #325: native memory counted towards GC pressure, which helps decide when a cycle starts.

   The single-threaded Emscripten issues do not apply, since this mode needs threads.
2. dimensionscape commented on Oct 25, 2026

   

   dimensionscape

   on Oct 25, 2026

   MemberAuthor

   More actions

   Proposed design for the opt-in mode, leaving the prerequisite fixes in the previous comment as separate work.

   The main change from the original proposal is to have background workers trace immutable edge records captured during stop-the-world slices. Directly running __Mark() against mutable C++ fields would introduce data races, even with a write barrier.

   For the first version:

   • Keep HXCPP_GC_CONCURRENT off by default. Use major collections, with no nursery, moving or concurrent sweeping.
   • Capture roots and object edges while stopped. Let workers trace those records while application threads run.
   • Log changed holders independently of the generational remembered set. Recapture dirty objects and finish strong closure before accepting the result.
   • Defer large-buffer recycling and other early releases until snapshot records can no longer refer to that storage.
   • Keep weak-map closure, weak references, finalizers and reclamation in the final pause.
   • Fall back to a complete synchronous collection on log overflow, metadata exhaustion or insufficient progress. Never sweep from incomplete tracing.
   • Add conventional tests first, then deterministic phase tests and an independent synchronous reachability verifier.

   This needs no compiler or standard library patch, provided the generated-store audit confirms complete coverage through existing barriers and runtime methods. Snapshot copying adds CPU and memory overhead, and the final pause can still depend on heap size. The performance case needs measured pause, throughput and memory results before enabling this beyond an experiment.

   Full design, correctness conditions, test plan and rollout

   Opt-in concurrent GC design

   Status: proposed architecture, not an implemented or validated collector.

   Based on #327, against revision 290638e71351c4e67f8e10cf33c0b187d46c6e22.
   The prerequisite fixes in the issue comment remain separate work.

   Decision

   Implement an experimental concurrent snapshot marker behind -D HXCPP_GC_CONCURRENT. Application threads run while GC workers traverse immutable records of heap edges. Capture those records during cooperative stop-the-world slices. Finish strong closure, weak processing and reclamation while stopped.

   This is concurrent tracing with incremental, paused capture. It is not direct background traversal of live C++ objects. That distinction is part of the public documentation and benchmark reporting.

   The first implementation uses fresh records every cycle. It does not maintain a persistent shadow heap, run minor collections during a cycle, move objects during a cycle, or sweep concurrently. These choices keep the lifetime and synchronization rules small enough to verify. All are candidates for later designs, rather than assumptions hidden in this one.

   The performance objective is a smaller longest GC pause on substantial, mostly stable heaps. Snapshot copying adds work and memory. Whether this is a useful improvement must be measured before shipping it as anything beyond an experiment.

   Why the direct marking proposal needs a different access protocol

   ObjectPtr::mPtr, array elements and container links are ordinary C++ memory. A background __Mark() or __Visit() reading a field at the same time a mutator writes it introduces a C++ data race. Recording the new value after a store does not legalize the field read. Making only the reader atomic does not fix mixed ordinary and atomic accesses either. Resizing arrays and hashes also changes their storage and traversal boundaries.

   The current header-byte barriers and marker queues have additional stopped-mutator assumptions. HX_OBJ_WB_CTX checks an old holder and a nursery value, then modifies an ordinary mark byte. Its filtering is unsuitable for a concurrent major collection.

   A future direct marker would need a compatible access protocol for every participating pointer slot and mutable container layout, including generated code and native integrations. This design avoids that requirement: workers cannot access application payload, allocation headers, mutable allocation indexes or running thread stacks.

   Supported first version

   Configuration or operation	Proposed behavior
   Default build	Existing collector and barriers unchanged
   Concurrent build	Major collection only, nursery disabled, fresh snapshots per cycle
   Windows, Linux, macOS with threads	Enable individually after the platform gates pass
   Single-threaded builds and non-threaded Wasm	Reject the concurrent option with an explanation
   Threaded Wasm	Defer support until its own thread and safepoint validation passes
   Moving collector	Do not offer the combination in the first version
   Cppia interpreter and JIT	Initially reject scriptable concurrent builds; enable only after both store paths and module lifetimes pass the same gates
   Haxe 5	Keep disabled until the separately tracked GC stress failure is understood
   Arbitrary native tracing code	Require the participation rules below; otherwise use synchronous collection for that process

   An unsupported build combination fails at build time. A supported build may fall back to a synchronous cycle for a documented runtime reason. It must report that reason in diagnostics, rather than silently presenting every collection as concurrent.

   Build policy without a compiler or standard library patch

   The existing Haxe generator emits relevant write barriers when HXCPP_GC_GENERATIONAL is defined. A library-owned configuration macro, loaded conditionally from extraParams.hxml, must arrange that definition before C++ generation for a concurrent build. A late C++ compiler flag is insufficient.

   Separate three concepts in the runtime:

   1. Compiler barrier emission, using the existing Haxe define as a compatibility mechanism.
   2. Actual generational collection and nursery allocation.
   3. Concurrent-cycle logging.

   Keep the compatibility definition available wherever generated C++ guards depend on it. Introduce an internal effective generational-mode definition for runtime nursery behavior, and audit every existing generational conditional when making that separation. In particular, include/hxcpp.h currently derives HXCPP_GC_NURSERY directly from HXCPP_GC_GENERATIONAL. Merely implying the latter would enable the wrong allocation behavior.

   Concurrent builds must have ordinary non-nursery allocation headers and full-major accounting. Reject an explicitly requested nursery or moving combination for version one. Treat the mode and barrier policy as part of the runtime and generated-code build identity. Rebuild all generated C++, native objects and extensions that participate in heap mutation; reject a mismatched linked runtime at startup where detectable.

   Before accepting this route, compile small examples with the installed supported Haxe versions and inspect emitted stores for fields, constructors, anonymous objects, closures, arrays and reflection. If a necessary store has no usable generated barrier and cannot be covered inside libhxcpp, leave that configuration unsupported. A build define is not evidence of complete barrier coverage.

   A generation-only probe with Haxe 4.3.7 confirmed that a dynamic instance field uses an _hx_set_edge helper containing HX_OBJ_WB in its constructor, ordinary setter and reflective setter. Anonymous construction uses runtime setFixed, and a mutable captured variable uses a runtime virtual-array cell. These are useful runtime interception points, not a completed store audit. The probe did not compile or run a concurrent collector.

   Cycle data and ownership

   Each cycle owns native metadata, allocated outside the GC heap:

   • A monotonically numbered cycle and a table of cycle-local allocation handles.
   • Immutable edge-record batches. An object record lists immediate strong object and allocation edges, without recursively visiting its children.
   • Side reachability bits and worker queues containing handles only.
   • Requests to capture objects whose outgoing edges are not yet known.
   • Per-thread dirty-holder logs, plus a log of allocations made during the cycle.
   • A sticky failure flag and a reserved metadata budget.

   Assign handles only while stopped, after validating and normalizing pointers through the collector's existing allocation membership rules. Preserve distinctions between object pointers, backing allocations, constant memory and registered interior/interface pointers. Do not dereference a pointer merely because it appeared in a conservative root scan. The worker-visible graph contains integer handles, never pointers it may follow into the heap.

   Handles are valid only for their cycle. Start no reclamation, moving operation, memory reuse or other collection after activating a cycle and before its workers have parked and its result has been accepted or abandoned. Allocations already dead before the cycle still cannot be reused during it. Finish any previous asynchronous reclamation that could violate this rule before activation. New allocations can use genuinely free space.

   This removes address-reuse ambiguity without requiring permanent per-allocation IDs. Destroy the graph before allowing storage reuse. Do not cache its edges across cycles.

   Deferral also applies to explicit runtime release paths, not just collection. InternalRealloc calls InternalReleaseMem, which can pass large backing storage to FreeLarge for recycling immediately. During a cycle, enqueue such releases in a coordinator-owned retirement list instead. Do not clear their headers, remove their allocation identity or recycle them while a handle can still refer to them. Charge retained bytes to reserve pressure. After retiring the graph, consume each deferred release exactly once under allocator ownership, coordinated with sweeping so it cannot free the same allocation twice. Audit every other early release and storage-reuse path for this rule.

   One coordinator owns the phase state. Each published batch has a single immutable lifetime, released only after worker acknowledgement. Use release publication and acquire consumption; use atomic side reachability claims and correctly synchronized queues. Workers write only cycle metadata. The ordinary allocator mark epoch stays unchanged during background tracing.

   Mutation protocol

   During an active cycle, every reference-changing heap operation records its owning object or an equivalent traced owner before its thread acknowledges another safepoint. Logging applies regardless of the holder's age, the target's age, or whether the holder is currently marked. It is independent of the generational remembered set.

   The concurrent branch of HX_OBJ_WB_CTX, its pessimistic variants and runtime array/map barriers must record the holder without accessing or modifying its mark byte. Record deletion and null stores too. They can conservatively retain old snapshot edges, but must not be treated as evidence that a cached capture is current.

   Current call sites include barriers both before and after assignment. A pause must never split an unlogged store from the log record required for that store. A pre-store barrier may not itself allocate, collect or enter a safepoint before the store completes. A post-store path may not acknowledge a safepoint until its log is committed. Bulk mutations that can allocate or block need an explicit mutation scope or smaller completed, logged steps. Such a scope cannot cover a potentially unbounded native operation.

   Use fixed-capacity thread-local log storage on the fast path. Duplicate records are acceptable. On overflow, atomically set mustFinishSynchronously before losing an entry. A thread in that condition must reach a safepoint before participating in another collection. No dropped record may be followed by sweeping from the concurrent result.

   Audit these paths explicitly:

   • Generated field writes, initialization, anonymous objects and captured variables.
   • Pointer arrays, dynamic arrays, growth, copy, slice, splice, insertion and bulk movement.
   • Hash insertion, replacement, rehash, removal and backing-storage replacement.
   • Reflection, dynamic field maps, CFFI stores, native wrappers and resource registration.
   • All pointer-bearing backing allocations that have a tracing owner.

   Dirty a stable tracing owner, not an untracked backing-buffer address. Primitive byte writes do not need an edge barrier, but replacement of their GC-managed storage does. New allocations are retained for the entire cycle and recorded even if allocation happens before publication to another object. They must also have their outgoing edges captured before acceptance, since they can be the only path to an older allocation.

   Native code that writes traced storage directly must use the supported barrier and safepoint contract. Native code that may change traced storage while in a GC-free zone does not satisfy that contract. A registered native component without supported capture and mutation behavior disables concurrent cycles process-wide in version one. An unknown extension cannot be made safe by waiting for verification to notice a lost edge.

   Phases

   Idle -> StartPause -> Trace <-> CapturePause -> FinalPause -> Reclaim -> Idle
                          |           |              |
                          +-----------+--------------+-> SynchronousFallback -> Idle
   

   Start pause

   Acquire collector ownership independently of the world-stop flag. The existing collection ownership and world-stop mechanism must be separated: mutators need to run during Trace, while competing collectors remain excluded.

   Stop all participating threads using the existing register/stack handshake. Finish incompatible reclaim jobs, allocate the minimum cycle metadata and establish the no-reclaim lifetime interval. Publish the active logging epoch to every context before resuming any of them.

   Capture roots into handles: class statics, registered roots and offset roots, stacks/registers and zombie roots. Scan stacks only while their owners are stopped. Use the same conservative validation and pinning rules as the synchronous collector. Do not save a pointer to a stack location for a worker to inspect later.

   Root capture seeds the graph but does not synchronously trace all descendants. Resume mutators and start workers over the published records. Missing object records produce capture requests.

   Root-set changes and new thread attachment are serialized with cycle ownership and the handshake. Attaching threads must inherit the active logging epoch before accessing traced memory. Detaching threads must transfer their allocation and dirty logs before destroying their context.

   Background tracing and capture pauses

   Workers claim reachable handles and traverse available immutable edges. A reachable handle with no complete record requests capture. Workers never call application __Mark() or __Visit().

   The coordinator requests a capture pause when outstanding requests or dirty work justify it. Park workers at a batch boundary and stop mutators. Drain completed logs, merge allocation records, validate pointers and capture requested objects and selected dirty holders.

   A read-only VisitContext adapter can collect immediate edges from generated __Visit() methods while paused. It must not rewrite slots, invoke user code, run finalizers or recursively trace the graph. Runtime types need an explicit semantic adapter audit. An inherited no-op visitor does not prove that a native type has no edges.

   Keep every previously captured strong edge for the remainder of the cycle and append edges from recaptures. Removing an edge never removes reachability already discovered in this cycle. Dirty objects whose handle is already reachable must have newly captured edges scheduled again; a visited bit cannot suppress that work. This is an incremental-update overapproximation, not a single instantaneous heap snapshot.

   Publish completed batches, acknowledge drained logs while still paused, then resume threads and workers. A subsequent mutation creates a new log entry. Do not clear a dirty flag concurrently with a mutator setting it; version one can simply retain duplicate per-thread records instead of using shared header flags.

   The pause budget is advisory. Generic __Visit() is indivisible and may traverse a huge array or hash. Bounded range adapters can later slice those types, with layout inspected afresh while stopped and mutation generations checked between slices. Never resume a saved raw iterator into mutable storage. Without such adapters, a single large object can exceed the requested pause budget.

   Final pause and strong closure

   Enter finalization when outstanding work is small, collection age reaches its limit, or allocation reserve is low. Park workers and stop all mutators. No subsequent mutator execution is allowed until closure and reclamation finish.

   1. Transfer all thread logs and allocation records. Acknowledge the stopped epoch from every thread, including newly attached contexts.
   2. Capture current roots again and seed them into the side graph.
   3. Capture every pending new allocation with the existing collector's construction-safety rules. Its allocation record alone is not proof that its outgoing fields have been scanned.
   4. Recapture dirty holders that are or become reachable. Keep unresolved dirty entries until closure finishes, since an initially white holder may become reachable later in the same pause.
   5. Drain every reachable missing capture and every newly appended edge to a fixed point. Run graph work synchronously if necessary. A queue becoming empty before all pending captures and logs are accounted for is not completion.

   At acceptance every currently reachable strong object has a complete current outgoing-edge capture, or a retained older capture plus all subsequent mutations represented by a final recapture. New allocations are retained and their strong edges participate in closure. Final root contents are included. This can require tracing a large newly published graph while stopped.

   Only after this check should the coordinator rotate the ordinary mark epoch, clear and reconstruct row/large-allocation liveness, and transfer the side reachable set to the existing collector's representation. Preserve allocation categories, container flags, pinned objects and finalizable-allocation rules. This needs a dedicated mark-publication routine; assigning a header byte alone does not establish valid row accounting.

   Weak references, finalizers and reclaim

   Weak references and weak-key maps require semantic adapters. The generic hash visitor visits keys and values and cannot supply a strong-edge snapshot for a weak-key map. Its internal storage remains live as appropriate, but weak keys and conditional values must not be inserted as ordinary strong edges.

   For version one, defer ephemeron processing until the final pause and run it against the current heap. After publishing strong liveness, use the existing stopped-world marker to resolve conditional values and their descendants. Preserve the current ordering: weak-map closure, zombie discovery, a second weak-map closure, weak-reference/finalizer processing and script lifetime cleanup where supported. A newly reached ordinary object is scanned by the synchronous marker during this tail.

   Keep current weak semantics, including any conservative retention. Do not allow a background worker to clear a weak slot, finalize an object or release a resource. Do not resurrect dead weak keys by treating the snapshot metadata itself as a GC root.

   Perform current row/large-allocation sweeping and accounting while stopped. Any asynchronous zeroing or reclaim scheduled afterward must obey its existing ownership rules and must not race destruction of cycle records. Park and release all cycle workers before permitting address reuse.

   Correctness argument and limits

   Workers cannot race on application payload because only stopped threads capture it. Graph records are immutable and object storage cannot disappear during the cycle.

   For a strong path reachable from a root at the final pause, its root is reseeded. For each successive live holder, either its captured edges are current or it was changed after capture and remains pending for recapture. Therefore closure discovers the next object on the path. Repeating this argument reaches every currently strongly reachable allocation. New allocations are retained explicitly and scanned for older descendants. Weak reachability is resolved afterward under the existing stopped-world rules.

   This argument depends on complete mutation coverage, complete visitors, correct pointer normalization, the safepoint handshake and the absence of reclaim during the cycle. Each is a release gate, not an assumption to waive when a benchmark looks good. Overretention is permitted for a cycle; collecting a reachable allocation is not.

   Total pause time still includes root capture, object-edge capture, the final dirty closure, mark publication, weak closure, finalizers and sweeping. Graph capture is proportional to the edges copied, and fresh records repeat that work each cycle. There is no heap-independent final-pause guarantee. Allocation throughput, retained garbage, additional CPU use and metadata memory can all regress.

   Progress, reserve and fallback

   Start a cycle before exhausting allocation reserve. Define soft thresholds for elapsed cycle time, metadata bytes, outstanding captures and dirty rate. Expose their effective values in diagnostics; choose defaults from measurements rather than embedding an untested millisecond promise.

   On persistent mutation, insufficient reserve, metadata allocation failure, log overflow, invalid capture capability or an internal consistency failure: stop mutators, park workers, discard the side result and perform a complete ordinary major collection from current roots. Reset the ordinary collector's marking state independently; partial concurrent marks are not an authoritative starting point. The fallback must not require allocating another shadow graph.

   If the ordinary collector cannot obtain sufficient memory, retain its established allocation-failure behavior. Concurrent mode does not guarantee recovery from process memory exhaustion.

   API or event	Required behavior
   Automatic allocation-triggered collection	Start or advance one cycle; request final pause under pressure
   Explicit Gc.run	Synchronize with the coordinator and complete a collection before returning, preserving the existing API contract
   Gc.compact	Unsupported moving combination in version one; retain synchronous behavior in ordinary builds
   Gc.enable(false)	Finish or abandon an active cycle safely before disabling future automatic collection
   Census and trace diagnostics	Serialize with collection ownership; finish or abandon the cycle, then run synchronously with request-local state
   Shutdown	Stop new cycle admission, park/join workers and retire all cycle metadata before runtime teardown
   Finalizer or diagnostic failure	Restore world-stop, worker and collector ownership through structured cleanup; never resume with a partially accepted cycle

   The coordinator must not wait for a worker that is waiting for the coordinator's world-stop lock. Test all cancellation and exception exits explicitly. The independent fixes to current diagnostics and finalizer behavior remain prerequisites rather than changes included in this proposal.

   Verification

   Add HXCPP_GC_CONCURRENT_VERIFY. Before reclaim, while all threads and workers are stopped, compute an independent synchronous reachability oracle. It needs separate visited state or a deliberately saved and restored mark epoch. Reusing concurrent marks would hide omissions.

   Compare strong reachability before weak processing, then verify weak/zombie closure using the same semantic rules. Every oracle-live allocation must survive the concurrent result. Extra survivors are counted as retention, not mistaken for a correctness failure. The oracle must not invoke finalizers, clear weak references or free modules; observable lifecycle processing happens once, after verification succeeds.

   On mismatch, print the allocation identity, tracing category, phase and available predecessor/capture history, then fail the test before reclaim. Verification is a test tool and cannot substitute for audited barriers in a release build.

   Tests before runtime implementation

   First add conventional tests of existing collector behavior. Use the existing test/regression/Name/{Main.hx,build.hxml,stdout.txt} layout, plus native helpers where necessary. Reuse relevant existing coverage instead of duplicating it. Tests should establish payload survival, eventual release and weak semantics independently of the new scheduling hooks.

   Then add a test-only phase controller with explicit acknowledgements for start, capture publication, final closure and fallback. Native tests can call it without a standard library API change. Synchronization must use latches or semaphores with timeouts, not sleeps that hope to hit an interleaving.

   Proposed case	Controlled operation and expected result
   ConcurrentGcPublishAfterCapture	Capture a holder, publish a previously white graph into it, drop other roots and verify every payload survives
   ConcurrentGcLateReachableDirty	Dirty an initially white holder, make it reachable later in final closure and verify its new descendants survive
   ConcurrentGcNewToOld	Keep an older graph only through allocations made during tracing; retain its contents
   ConcurrentGcBulkMutation	Copy, splice, resize and rehash pointer-bearing containers between captures; verify moved edges and backing lifetimes
   ConcurrentGcRootsAndThreads	Add/remove roots, replace static roots and attach/detach threads during tracing; verify current roots and transferred logs
   ConcurrentGcMutationBoundary	Force pause requests around pre-store and post-store barriers; observe only completed, logged mutation states
   ConcurrentGcWeakClosure	Exercise weak references, ephemeron chains, cycles, zombies and finalizer counts across several complete cycles
   ConcurrentGcFallback	Inject each log/metadata/reserve failure; require a complete synchronous collection with live payload intact
   ConcurrentGcNoAddressReuse	Exercise allocation pressure and prior reclaim jobs; assert that no cycle handle aliases reused storage
   ConcurrentGcApiLifecycle	Overlap explicit collection, diagnostics, disabling collection and teardown; require completion without deadlock or double lifecycle processing
   ConcurrentGcNativeParticipation	Reject or disable concurrency for a registered unsupported native tracer; exercise a supported native writer

   Finalization tests must clear incidental stack roots and allow multiple collections. Do not require exact prompt reclamation in one cycle. Verify that the next ordinary cycle or completed subsequent concurrent cycles release floating garbage.

   Add the define checks, stress configuration, documentation and CI jobs required by CONTRIBUTING.md before enabling the option. Run default collection as a control, concurrent plus verification, forced fallback, big blocks and supported architecture combinations. Keep generation-only and moving-only regression jobs as controls for the build-policy refactor. Use ASan/UBSan and TSan on supported toolchains, with allocator and metadata races treated separately from application-level races.

   Performance gate

   Extend the benchmark suite with stable large object graphs, large pointer arrays, sparse maps, weak-map chains, burst allocation, new-to-old references, pointer churn and many-thread server-style request workloads. Run both fresh and warmed heaps, and include heaps that approach the configured memory reserve. Primitive byte buffers alone do not represent tracing cost.

   Record identical payload work and heap limits across the existing collector, its existing parallel marker and the concurrent candidate. Include compiler, OS, CPU/core count, Haxe version, runtime revision, defines, worker count, warmup, repetitions and raw samples.

   Report longest and percentile world-stop durations, total stopped time, safepoint waiting, capture bytes/time, graph tracing CPU, final strong closure, weak processing, sweep time, fallback reasons/count, throughput, request latency, peak committed memory and metadata high-water usage. Distinguish GC CPU time from elapsed cycle duration. Fallback pauses remain part of the reported distribution.

   Agree workload-specific throughput and memory regression budgets before comparing results. Require repeatable pause improvement beyond run-to-run noise on the target server workloads. If copied edges, mark publication or the synchronous tail dominate, keep this experimental or stop this route. Do not describe it as a successful low-pause collector merely because worker threads run alongside mutators.

   Delivery sequence

   1. Add baseline semantic tests and inspect generated barrier coverage. Complete the separate prerequisite work without bundling it into this change.
   2. Separate emission policy from nursery behavior, add build identity checks, and run the existing collectors unchanged.
   3. Add cycle metadata, snapshot adapters, mutation logs and the independent verifier, with mutators still stopped. Validate equivalence and inject failure paths.
   4. Run graph workers concurrently with mutators, preserving paused payload capture. Add deterministic phase tests before enabling automatic scheduling.
   5. Add scheduling, reserve policy and diagnostics. Run soak, sanitizer and benchmark gates on each supported platform.
   6. Publish the flag as experimental only after those gates pass. Consider persistent snapshots, generational integration, direct atomic-slot tracing, concurrent sweep or moving integration as separate proposals with their own lifetime proofs.

   Source integration map

   Source	Design responsibility
   extraParams.hxml and a library-owned configuration macro	Early compiler barrier-emission policy
   toolchain/common-defines.xml, include/hxcpp.h	Mode validation, effective nursery policy and build identity
   include/hx/GC.h	Concurrent logging branch and native mutation contract
   include/hx/Object.h, include/Array.h, src/hx/Hash.h	Confirm payload access assumptions, capture adapters and dirty-owner coverage
   src/hx/gc/Immix.cpp	Ownership, phased cycle, capture handshake, mark publication, fallback and reclaim integration
   src/hx/Class.cpp and root registries	Paused static/root capture and registration ordering
   src/hx/cppia/Cppia.cpp and runtime types	Later interpreter/JIT barrier and lifetime support
   test/regression, test/gc_stress, test/gc-defines, CI and benchmarks	Semantic, scheduling, configuration and performance gates

   No runtime implementation, benchmark result or claim of proven pause improvement is included in this design.
3. dimensionscape commented on Oct 26, 2026

   

   dimensionscape

   on Oct 26, 2026

   MemberAuthor

   More actions

   Putting concurrent marking on hold while #372 is developed.

   The local snapshot prototype did not meet the pause or throughput goals. It adds paused reference capture and metadata work, and the measured fallbacks make the longest interruptions worse than the ordinary collector. It should remain an experiment rather than the next collector rollout.

   The immediate priority is incremental marking of the actual heap in short pauses, with automatic allocation-based pacing and optional library-owned scheduling hooks. This needs no frame-loop knowledge in hxcpp and is intended to avoid a compiler or standard library change.

   The concurrent prototype and its test and benchmark results are being preserved separately. This issue stays open for later research. A future concurrent marker still needs a complete field, header and container-storage access protocol, along with its own correctness and performance gates.
4. dimensionscape commented on Oct 27, 2026

   

   dimensionscape

   on Oct 27, 2026

   MemberAuthor

   More actions

   The incremental collector is now pushed on feature/incremental-gc, commit 67208e6b, with draft PR #380. Implementation status, tests, raw measurements and remaining gates are recorded in #372.

   This implementation traces the actual heap during stopped-world slices and retains progress between them. It does not run marker workers while application threads mutate fields. Concurrent marking remains deferred, and the separate concurrent prototype worktree is preserved.

   The measured game workload has shorter tail frames, but incremental throughput and finishing-pause targets are still open. The existing collector remains the default. There is no claim that the concurrent design's goals have been completed by this PR.
5. HomuHomu833

   added a commit that references this issue on Oct 28, 2026

   Add opt-in incremental GC with automatic pacing

   ...

   67208e6