MDEV-39795: Assertion `n_reserved > 0' failed#5356
Open
mariadb-TafzeelShams wants to merge 1 commit into
Open
Conversation
Contributor
|
Warning Gemini encountered an error creating the review. You can try again by commenting |
mariadb-TafzeelShams
marked this pull request as ready for review
July 8, 2026 00:49
Problem: ======== 1. Assertion `n_reserved > 0` failed in fseg_create(): fsp_reserve_free_extents() has a special condition for small tablespaces where it reserves individual pages instead of full extents. In such cases, n_reserved can be 0 even when the reservation succeeds, causing the assertion ut_ad(n_reserved > 0) to fail incorrectly. The code was checking n_reserved to determine whether a reservation had already been attempted, but this logic breaks for small tablespaces where pages, rather than extents, are reserved. 2. Encryption metadata not cleared for compressed-only pages: buf_page_encrypt() only cleared encryption-related metadata fields (key-version and crypt-checksum) when the page was neither encrypted nor compressed. However, these fields should also be cleared when page_compressed is true but encrypted is false, to avoid leaving stale encryption metadata in compressed-only pages. Solution: ========= buf_page_encrypt(): Refactored the early-return logic. Encryption metadata fields are now cleared whenever encrypted is false, regardless of page_compressed. The function returns early only when both !encrypted and !page_compressed. fseg_create(): Reintroduced a boolean variable `reserved` to track whether fsp_reserve_free_extents() has been attempted (removed as part of MDEV-38419 | c7313da), replacing assertion `n_reserved > 0`. Added an early return when DB_DECRYPTION_FAILED is encountered during inode allocation. my_error_innodb(): Added handling for DB_DECRYPTION_FAILED to report decryption errors to the user through ER_GET_ERRMSG.
mariadb-TafzeelShams
force-pushed
the
10.6-MDEV-39795
branch
from
July 8, 2026 14:09
f26e78a to
99e18bb
Compare
| reserved successfully. | ||
| See fsp_reserve_free_extents() | ||
| So try allocating the page or inode */ | ||
| reserved = true; |
Member
There was a problem hiding this comment.
reserved = (*err == DB_SUCCESS);
Thirunarayanan
requested changes
Jul 9, 2026
| case DB_CORRUPTION: | ||
| my_error(ER_NOT_KEYFILE, MYF(0), table); | ||
| break; | ||
| case DB_DECRYPTION_FAILED: |
Member
There was a problem hiding this comment.
Please test it without fixing page_compressed + encryption test case.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Problem:
n_reserved > 0failed in fseg_create():fsp_reserve_free_extents() has a special condition for small tablespaces where it reserves individual pages instead of full extents. In such cases, n_reserved can be 0 even when the reservation succeeds, causing the assertion ut_ad(n_reserved > 0) to fail incorrectly.
The code was checking n_reserved to determine whether a reservation had already been attempted, but this logic breaks for small tablespaces where pages, rather than extents, are reserved.
buf_page_encrypt() only cleared encryption-related metadata fields (key-version and crypt-checksum) when the page was neither encrypted nor compressed. However, these fields should also be cleared when page_compressed is true but encrypted is false, to avoid leaving stale encryption metadata in compressed-only pages.
Solution:
buf_page_encrypt(): Refactored the early-return logic. Encryption metadata fields are now cleared whenever encrypted is false, regardless of page_compressed. The function returns early only when both !encrypted and !page_compressed.
fseg_create(): Introduced a boolean variable
reservedto track whether fsp_reserve_free_extents() has been attempted, replacing the flawed check ofn_reserved > 0. Added an early return when DB_DECRYPTION_FAILED is encountered during inode allocation.my_error_innodb(): Added handling for DB_DECRYPTION_FAILED to report decryption errors to the user through ER_GET_ERRMSG.
Release Notes
Fixed assertion failure
n_reserved > 0How can this PR be tested?
Added MTR test case innodb_encrypt_compression_algorithm.test
Basing the PR against the correct MariaDB version
mainbranch.PR quality check