Conversation
…ding old metadata (apache#61342) ### What problem does this PR solve? When FE starts with -r (metadata_failure_recovery) or without a checkpoint image, it replays all journal logs from the beginning. Old journal entries may contain PrimaryKeyConstraint objects that were serialized before the `foreignTables` field was added to the class. Gson uses Unsafe to instantiate objects during deserialization, bypassing the constructor and its field initializers. If a field is absent from the JSON, Gson leaves it as null rather than using the declared initializer (e.g. `= new HashSet<>()`). Calling .isEmpty() or any method on a null collection then throws a NullPointerException. Fix: remove `final` from the `foreignTables` collection field (so it can be reassigned in gsonPostProcess) and add a null-guard at the top of gsonPostProcess() to initialize it to an empty collection when absent. Note: on branch-4.2 the class only has `foreignTables`; the newer `foreignTableNameStrs` / `foreignTableInfos` fields come from a later refactor that is not part of this branch, so only that field is guarded. Impact on -r mode: when FE is started with --metadata_failure_recovery it replays every journal from journal #1, encountering the oldest serialized PrimaryKeyConstraint objects. Without this fix, FE always crashes during -r recovery if any PrimaryKeyConstraint was persisted before the field existed. ### Release note None
Contributor
|
Thank you for your contribution to Apache Doris. Please clearly describe your PR:
|
Contributor
Author
|
run buildall |
Contributor
FE UT Coverage ReportIncrement line coverage |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Cherry-pick of #61342 to branch-4.2
fix Fix NPE in PrimaryKeyConstraint when loading old metadata
On branch-4.2 the class only has the
foreignTablescollection (the newerforeignTableNameStrs/foreignTableInfosfields come from a later refactor that isnot part of this branch), so only that field is guarded. A unit test covering
deserialization of metadata without the
ftfield was added.