Skip to content

[MINOR][ML] Simplify SummarizerBuffer branching - #58236

Draft
zhengruifeng wants to merge 2 commits into
apache:masterfrom
zhengruifeng:simplify-summarizer-dev-5
Draft

[MINOR][ML] Simplify SummarizerBuffer branching#58236
zhengruifeng wants to merge 2 commits into
apache:masterfrom
zhengruifeng:simplify-summarizer-dev-5

Conversation

@zhengruifeng

Copy link
Copy Markdown
Contributor

What changes were proposed in this pull request?

This PR simplifies SummarizerBuffer by precomputing requested metric and compute metric flags once, moving one-time array initialization into a helper, and splitting merge handling into empty-summary guard clauses plus focused non-empty merge/copy helpers.

The change preserves the existing aggregate state layout and public behavior.

Why are the changes needed?

SummarizerBuffer has several hot paths with repeated Seq.contains checks and nested null/empty branches. The refactor makes the control flow easier to read and removes repeated metric membership checks while keeping the same metric computations.

Does this PR introduce any user-facing change?

No.

How was this patch tested?

Ran:

git diff --check
awk 'length>100 && $0 !~ /^[[:space:]]*(import|package) / && $0 !~ /https?:\\/\\// {print FILENAME":"FNR": "length" chars"}' mllib/src/main/scala/org/apache/spark/ml/stat/Summarizer.scala\ngrep -rn -P "[^\\x00-\\x7F]" mllib/src/main/scala/org/apache/spark/ml/stat/Summarizer.scala\n```\n\nDid not run the full SBT test suite.\n\n### Was this patch authored or co-authored using generative AI tooling?\n\nGenerated-by: OpenAI Codex (GPT-5)\n

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants