MDEV-38918 Make large pages an explicit per-caller opt-in

[vaintroub]

Summary

my_large_malloc() attempted large pages whenever --large-pages was enabled, silently rounding the size up and reporting it back via an in/out parameter. ut_malloc_dontdump() never passed that adjusted size on to its own callers (the InnoDB redo log buffer and recv_sys_t::tmp_buf), so freeing later used the original, smaller size, causing the reported "faux memory leak".

Only the buffer pool and the MyISAM/Aria key caches are documented to benefit from large pages. Everything else that ended up calling my_large_malloc() only wanted its "do not dump to core" property and picked up large pages as an undocumented side effect; those buffers are also small and sequentially accessed, so they would have gained little from large pages anyway.

• Add MY_TRY_LARGE_PAGES: my_large_malloc() and my_large_virtual_alloc() now only attempt large pages when a caller passes this flag, instead of always trying whenever the global option is set. Only the buffer pool and the key caches pass it.
• The redo log buffer, tmp_buf, and row0log.cc's crypt buffers no longer request large pages at all, removing the size-rounding bug for them without touching that code.
• Fix a broken mtr suppression regex in main.large_pages that would fail the test on Windows.

Test plan

• main.large_pages passes
• Manual repro: mariadbd.exe --large-pages with an oversized buffer pool, confirmed graceful fallback and shutdown reports no leak

[dr-m]

The test main.large_pages is crashing across the board, apparently on the very first access to buf_pool.memory in buf_pool_t::create().
I would like to note that by default, large pages are unavailable on Linux. They have to be reserved separately:

echo 96|sudo tee /sys/kernel/mm/hugepages/hugepages-1048576kB/nr_hugepages
echo 1024|sudo tee /sys/kernel/mm/hugepages/hugepages-2048kB/nr_hugepages

Because such pages would be unavailable for normal allocation, I assume that we have no Linux environment in our CI where the large_pages option would actually work.

[vaintroub]

Thank you for catching this, and for the testing tip — reserving huge pages via /sys/kernel/mm/hugepages/.../nr_hugepages let me verify both the fallback and the real hugetlb-success paths locally. Addressed in the follow-up commit.

re-requested already

[Copilot]

Pull request overview

This PR makes large-page allocation an explicit per-caller opt-in by introducing MY_TRY_LARGE_PAGES (and helper my_large_pages_flag()), so only documented beneficiaries (buffer pool and key caches) attempt large pages, avoiding unintended size rounding and mismatched free sizes elsewhere.

Changes:

• Introduce MY_TRY_LARGE_PAGES and my_large_pages_flag() to explicitly request large pages per allocation/caller.
• Plumb myf my_flags through my_virtual_mem_{reserve,commit,decommit,release}() and my_large_virtual_alloc() and update key call sites (InnoDB buffer pool, Maria/MyISAM key caches/page cache).
• Fix the main.large_pages suppression regex to cover Windows MEM_LARGE_PAGES warnings.

Reviewed changes

Copilot reviewed 10 out of 10 changed files in this pull request and generated 1 comment.

Show a summary per file

File	Description
storage/maria/ma_pagecache.c	Page cache allocation now explicitly opts into large pages only when enabled.
storage/innobase/buf/buf0buf.cc	Buffer pool virtual-memory reserve/commit/decommit/release now consistently passes the large-page opt-in flag.
mysys/my_virtual_mem.c	Virtual memory API now takes my_flags and uses MY_TRY_LARGE_PAGES to control large-page behavior.
mysys/my_largepage.c	Large-page allocators now attempt large pages only when MY_TRY_LARGE_PAGES is provided; reserve fallback protection behavior clarified.
mysys/my_alloc.c	MEM_ROOT vmem allocations updated to new my_virtual_mem_* signatures.
mysys/mf_keycache.c	Key cache allocations now explicitly opt into large pages when enabled.
mysql-test/main/large_pages.test	Suppression regex updated for Windows MEM_LARGE_PAGES wording.
mysql-test/main/large_pages.result	Expected output updated to match the new suppression regex.
include/my_virtual_mem.h	Public header updated for new my_virtual_mem_* signatures and myf type.
include/my_sys.h	Adds MY_TRY_LARGE_PAGES and my_large_pages_flag() helper; updates my_large_virtual_alloc() signature.

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.