Skip to content

[HUDI-19878] Initialize Spark context for record index deletion - #19969

Open
txwyy123 wants to merge 1 commit into
apache:masterfrom
txwyy123:codex/hudi-19878-metadata-delete-record-index
Open

txwyy123 wants to merge 1 commit into
apache:masterfrom
txwyy123:codex/hudi-19878-metadata-delete-record-index

Conversation

@txwyy123

Copy link
Copy Markdown

What is the problem?

metadata delete-record-index constructs a HoodieSparkEngineContext from the command's Spark context, but does not initialize that context first. When this is the first metadata command executed in a CLI session, the context is null and the command fails with a NullPointerException.

What does this change do?

  • Add the standard --sparkMaster option used by the other Spark-backed metadata commands.
  • Initialize the command's Spark context before deleting the Record Index partition.
  • Add a regression test that invokes the command on a fresh MetadataCommand instance and verifies the Record Index partition is removed.

This is the smallest targeted fix: it reuses the existing lazy initialization helper and leaves the metadata deletion behavior unchanged.

Fixes #19878

Testing

  • mvn -P spark3.5 -Dtest=TestMetadataCommand#testMetadataDeleteRecordIndexInitializesSparkContext -Dsurefire.failIfNoSpecifiedTests=false -Dcheckstyle.skip=true -Dspotless.check.skip=true -Drat.skip=true test
  • mvn -P spark3.5 -Dtest=TestMetadataCommand -Dsurefire.failIfNoSpecifiedTests=false -Dcheckstyle.skip=true -Dspotless.check.skip=true -Drat.skip=true test
  • mvn -P spark3.5 -DskipTests -Dspotless.check.skip=true -Drat.skip=true checkstyle:check

@github-actions github-actions Bot added the size:S PR with lines of changes in (10, 100] label Sep 16, 2026
@hudi-bot

Copy link
Copy Markdown
Collaborator

CI report:

Bot commands @hudi-bot supports the following commands:
  • @hudi-bot run azure re-run the last Azure build

@hudi-agent hudi-agent left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ 🤖 This review was generated by an AI agent and may contain mistakes. Please verify any suggestions before applying.

Thanks for the contribution! This PR fixes a NullPointerException in metadata delete-record-index by adding the --sparkMaster option and lazily initializing the CLI's Spark context before the deletion, matching the pattern already used by the other Spark-backed metadata commands, and adds a regression test on a fresh command instance. No issues flagged from this automated pass — a Hudi committer or PMC member can take it from here for a final review.

cc @yihua

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size:S PR with lines of changes in (10, 100]

Projects

None yet

Development

Successfully merging this pull request may close these issues.

hudi-cli: metadata delete-record-index NPEs unless another metadata command ran first

3 participants