[SPARK-48701][SQL] Make PandasMode collation-aware

[vinodkc]

What changes were proposed in this pull request?

Make the internal PandasMode aggregate (pandas_mode) collation-aware.
This PR extracts the folding logic into a shared ModeCollationAware trait mixed into both Mode and PandasMode, and calls getCollationAwareBuffer in PandasMode.eval (with a null-key guard, since PandasMode stores a null key when ignoreNA = false). Eval-only change; Mode's behavior is unchanged.

Why are the changes needed?

pandas_mode (behind Pandas API on Spark Series.mode() / DataFrame.mode()) silently returns wrong results on non-binary collated strings: collation-equal values (e.g. 'b' and 'B' under UTF8_LCASE) are counted as separate keys.

Example : a UTF8_LCASE column with values a, a, b, B, B (so 'b' and 'B' are equal: counts a=2, b/B=3, mode should be the single b/B group):

sdf = spark.sql("""
  SELECT CAST(c AS STRING COLLATE UTF8_LCASE) AS s
  FROM VALUES ('a'), ('a'), ('b'), ('B'), ('B') AS t(c)
""")
print(sorted(sdf.pandas_api()["s"].mode().to_numpy().tolist()))

Before: ['B', 'a'] # wrong: a=2 ties B=2, b counted separately (2 modes)
After: ['b'] # correct: b/B fold to one group of count 3, winning (1 mode)

Does this PR introduce any user-facing change?

Yes. pandas_mode (Pandas API on Spark Series.mode() / DataFrame.mode()) now returns the correct mode(s) on columns with a non-binary collation, instead of counting collation-equal strings as distinct values. Results on default (UTF8_BINARY) collation and non-string types are unchanged.

How was this patch tested?

Added unit tests in CollationAggregationSuite

Was this patch authored or co-authored using generative AI tooling?

Yes. Generated-by: Claude code.

[uros-b]

Thank you @vinodkc! cc @HyukjinKwon for Pandas