Committed memory growing over time

[petrsm]

Hi !
We are using snmalloc in game engine, where we want both performance and memory efficiency. In our game, as
player moves through environment, lot of content is generated procedurally on worker threads. It is common,
that we allocate (quite big) chunks of memory on one thread, and then release it after while from different thread.
I observe following behavior: even if number of allocated bytes stays roughly same (around 6 GB) memory used by
snmalloc is constantly growing and is significantly higher : usually about 3 GB more and this value is increasing.
Now, question is : is this expected behavior ? Are there any configuration settings which can help us reduce committed
memory size ?

Thanks !
Petr

[mjp41]

This is not expected behaviour the memory usage should not grow constantly.

What platform are you on? Is there a way I can reproduce this behaviour. Have you attempted to minimise the example. What size is "quite big"? Some people view 4KiB as big, some 4MiB, and some 4GiB.

[petrsm]

Sorry, I should have provide more detail upfront:

1. We are on Windows
2. Quite big is in range: 512 kB - 16 MB

I am not sure if I can minimize the repro, but for start, I will minimize number of worker threads to see, if it makes any difference.

[mjp41]

What kind of time frame are you looking at? What allocation rate of other sizes?

[petrsm]

I tried to disable worker threads completely (there were 28 of them on my machine) and forced synchronous execution of jobs on main thread. Quite surprisingly, I still observe same behavior. After I spawn into game, I see that total allocated memory is 4.4 GB, yet snmalloc committed memory is around 6.5 GB (I added tracking of committed memory to PAL, but it gives me pretty similar numbers as snmalloc::Alloc::Config::Backend::get_current_usage()). After flying over map for about 30 seconds, which forces engine to generate and cache additional tiles of data I see that application allocated 6.4 GB and snmalloc committed memory size is 9.8 GB, so the difference between allocated and actually committed memory widens.

To answer your questions:

1. I am looking in time range of about 30 seconds
2. Allocation rate for other (smaller) sizes can be pretty high, but mostly on during the initialization phase. Later during the game we make effort to minimize number of dynamic allocations.

[mjp41]

I have been experimenting with a benchmark that is showing some growth:
#815

I'll see if I can find what is causing the behaviour.

[petrsm]

Thanks a lot Matt !

[mjp41]

I tried to disable worker threads completely (there were 28 of them on my machine) and forced synchronous execution of jobs on main thread. Quite surprisingly, I still observe same behavior. After I spawn into game, I see that total allocated memory is 4.4 GB, yet snmalloc committed memory is around 6.5 GB (I added tracking of committed memory to PAL, but it gives me pretty similar numbers as snmalloc::Alloc::Config::Backend::get_current_usage()). After flying over map for about 30 seconds, which forces engine to generate and cache additional tiles of data I see that application allocated 6.4 GB and snmalloc committed memory size is 9.8 GB, so the difference between allocated and actually committed memory widens.

So those are both about 50% overhead. For large allocations, snmalloc just uses power of two sizes. I wonder if that could be an issue for your use case.

Running my test case locally seems to tend to a limit, which is what you would expect for a random benchmark, as time progresses you are more likely to see a worse case, but eventually you have seen them all.

If you run for longer do the ratio stay the same? Do you know what the size profile of allocations is?

[petrsm]

As per large allocations - does the power of two sizes apply to any 'large' size ? For example, if block of 8193 kB is used, will it occupy 16MB ? If yes, this may be indeed the case, as it is possible that growth will stabilize after longer time than what I was measuring. I will give it try.

[mjp41]

Yeah, this is not an ideal part of the design, but means we don't need as complex handling of large objects, and can get very efficient bounds checking on memcpy. Everything above 16KiB is going to be in this range.

[petrsm]

Do you think that this pow2 scheme can be replaced by 'Bins with 8 bit floating point distribution' as described here ? :
https://github.com/sebbbi/OffsetAllocator

It seems like it may be possible without too much hassle and it would elevate snmalloc real world utility significantly ! Their mapping is O(1) and very fast while they claim 12.5 % waste (6.25% on average).

[mjp41]

That is what the front end uses. It is just the backend is using a buddy allocator to consolidate the of address range to different sizes. The large allocations defer to the backend. This has been an area I wanted to work on, but it has never reached a high enough priority. Your issue and the issues raised by @akrieger are making consider adding more priority to changing this.

[petrsm]

Cool ! Thanks !