Skip to content

[Core, GC] Owned Unmanaged Memory Allocations must report to GC byte size #605

@Nucs

Description

@Nucs

Overview

All unmanaged memory allocations where NumSharp takes ownership must call GC.AddMemoryPressure(bytes) on allocation and GC.RemoveMemoryPressure(bytes) on deallocation. This informs the GC about memory it cannot see, enabling proper collection scheduling.

Problem

When NumSharp allocates unmanaged memory via NativeMemory.Alloc, the GC only sees small managed wrapper objects (~100-200 bytes) but is unaware of the actual unmanaged data (potentially megabytes). Without this information, the GC delays collection, causing memory to grow unbounded.

Example from #501:

for (int i = 0; i < 1_000_000; i++)
{
    NDArray array2 = np.array(new double[110]); // 880 bytes each
}
// Without fix: peaks at 10+ GB
// With fix: stable at ~54 MB

Solution

Track memory pressure in UnmanagedMemoryBlock<T>.Disposer:

Allocation Type Tracks Pressure? Reason
Native (NativeMemory.Alloc) ✅ YES NumSharp allocates → NumSharp tracks
External with dispose ❌ No Caller allocates → Caller's responsibility
GCHandle (pinned managed) ❌ No GC already knows about managed arrays
Wrap (no ownership) ❌ No Not our memory

Implementation

// Disposer constructor for Native allocations
public Disposer(IntPtr address, long bytesCount)
{
    Address = address;
    _bytesCount = bytesCount;
    _type = AllocationType.Native;
    
    if (bytesCount > 0)
        GC.AddMemoryPressure(bytesCount);
}

// On cleanup
private void ReleaseUnmanagedResources()
{
    // ...
    case AllocationType.Native:
        NativeMemory.Free((void*)Address);
        if (_bytesCount > 0)
            GC.RemoveMemoryPressure(_bytesCount);
        return;
}

Checklist

  • Native allocations (UnmanagedMemoryBlock(count)) track pressure
  • External Disposer has optional bytesCount parameter (default 0)
  • Document in CLAUDE.md / architecture docs
  • Add unit test verifying memory stays bounded in allocation loop

Related

Metadata

Metadata

Assignees

Labels

architectureCross-cutting structural changes affecting multiple componentscoreInternal engine: Shape, Storage, TensorEngine, iterators

Type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions