[SPARK-58951][SQL] Support non-binary collations in collect_set

[vinodkc]

What changes were proposed in this pull request?

This PR makes collect_set support non-binary collated string inputs . Previously such inputs were rejected with an UNSUPPORTED_INPUT_TYPE error; now collect_set deduplicates them collation-aware, returning one representative per collation-equal group. Binary-stable inputs are unchanged, and MapType is still rejected.

Why are the changes needed?

collect_set on a non-binary collated column throws instead of working:

CREATE OR REPLACE TEMP VIEW v AS
SELECT CAST(s AS STRING COLLATE UTF8_LCASE) AS s
FROM VALUES ('foo'), ('FOO'), ('bar') AS raw(s);

SELECT collect_set(s) FROM v;
-- [DATATYPE_MISMATCH.UNSUPPORTED_INPUT_TYPE] ... collect_set ...

Under UTF8_LCASE, 'foo' and 'FOO' are equal, so collect_set should treat them as one group rather than error out. This closes a gap in collation support for aggregate functions.

Actual result after this PR:

SELECT collect_set(s) FROM v;
+----------------+
|collect_set(s)  |
+----------------+
|[foo, bar]      |
+----------------+

Does this PR introduce any user-facing change?

Yes. collect_set over a non-binary collated string (top-level or nested in a struct/array) now succeeds instead of raising UNSUPPORTED_INPUT_TYPE, returning one representative per collation-equal group. For the view above, the result has 2 elements ({foo-group, bar}). As with any set aggregate, which representative of a group appears is unspecified.
No change for default (UTF8_BINARY) or non-string inputs; MapType is still rejected.

How was this patch tested?

Added new tests in CollationAggregationSuite

Was this patch authored or co-authored using generative AI tooling?

Yes. Generated-by: Claude Code

[uros-b]

Looking into this

[uros-b]

Please note same gaps in test coverage: the new path is exercised only at nesting depth 1 and only for case-folding collations (UTF8_LCASE, UNICODE_CI). Two cheap additions would close the confidence gaps in the injectCollationKey recursion this PR now depends on: (1) a depth≥2 case, e.g. collect_set(array(named_struct('s', ))); and (2) a space-trimming collation like UTF8_LCASE_RTRIM, where 'foo ' and 'foo' must fold to one key.

[uros-b]

Other than the above, the changes seem correct and cleanly scoped - thank you @vinodkc!

[vinodkc]

Thanks for the review, @uros-b.
Added both tests in CollationAggregationSuite