fix: enforce null-key rejection and mapKeyDedupPolicy in native map construction

[peterxcli]

Which issue does this PR close?

• Closes [Bug] map_from_arrays / map_from_entries do not enforce null-key rejection or spark.sql.mapKeyDedupPolicy #4680.
• Closes str_to_map does not honour mapKeyDedupPolicy=LAST_WIN #5857.

Rationale for this change

Spark builds every map through ArrayBasedMapBuilder, which refuses a NULL key and resolves duplicate keys according to spark.sql.mapKeyDedupPolicy. Comet's map_from_arrays and map_from_entries did neither. A NULL inside the keys array produced a map with a NULL key instead of an error, and setting the policy to LAST_WIN pushed the whole expression back to Spark.

DataFusion 55 supplies what was missing. Its datafusion.spark.map_key_dedup_policy option takes the same EXCEPTION and LAST_WIN values as the Spark config, and the datafusion-spark map kernels already follow it. Once Comet passes the setting through, LAST_WIN runs natively, and the remaining checks that ArrayBasedMapBuilder performs cost only a few lines on top.

What changes are included in this PR?

spark.sql.mapKeyDedupPolicy now crosses JNI. CometExecIterator.serializeCometSQLConfs sends it explicitly, since cometSqlConfs carries only keys under spark.comet., and prepare_datafusion_session_context applies it to the session as datafusion.spark.map_key_dedup_policy.

A second change was needed before that setting could reach a kernel at all. create_scalar_function_expr handed every ScalarFunctionExpr a fresh ConfigOptions::default(), so any kernel reading a session option saw DataFusion's defaults. It now passes the session's own ConfigOptions.

native/spark-expr/src/map_funcs/map_builders.rs adds three wrappers, SparkMapFromArrays, SparkMapFromEntries and SparkStrToMap. Each calls the matching datafusion-spark kernel, adds the checks that kernel skips, and translates its errors into the Spark error classes that SparkErrorConverter converts back into QueryExecutionErrors:

• a NULL key raises NULL_MAP_KEY, before any check for duplicates, matching the order Spark applies them;
• key and value arrays of unequal length raise MAP_KEY_VALUE_DIFF_SIZES;
• a duplicate key under EXCEPTION raises DUPLICATED_MAP_KEY and names the key.

str_to_map needs only the last of these, because splitting a string never yields a NULL key. Passing the config through also fixed its LAST_WIN case, which used to raise an error where Spark returns a map.

CometMapFromArrays now emits map_from_arrays. It used to emit the generic map wrapped in CaseWhen(IsNotNull(left) AND IsNotNull(right), ...) so that a NULL input array yielded a NULL map; the Spark kernel already behaves that way, so the wrapper came out. Both serdes also drop their LAST_WIN Incompatible branch.

One difference with Spark remains. ArrayBasedMapBuilder normalizes a floating point key before storing it, so -0.0 becomes +0.0 and every NaN collapses into one. The native builders compare the raw Arrow values, so a map built from both -0.0 and +0.0 keeps two entries where Spark reports a duplicate key. The compatibility notes record this, and spark.comet.exec.strictFloatingPoint makes Comet decline a floating point key type for anyone who needs the guarantee.

How are these changes tested?

The 21 native unit tests cover the wrappers. Two of them pin the exact wording DataFusion uses when it reports a duplicate key, because the wrapper reads that message to recover the key it should name. If DataFusion rewords the message, those tests fail rather than the error quietly degrading into a generic execution failure.

Seven new tests in CometMapExpressionSuite run each case through both engines and compare the exception type, error class and SQLSTATE, along with the answers each engine returns under LAST_WIN.

Among the SQL fixtures, the two *_dedup_policy.sql files used to assert the LAST_WIN fallback and now assert native execution. map_from_arrays.sql, map_from_entries.sql and str_to_map.sql gained the EXCEPTION error cases, and str_to_map_dedup_policy.sql is new. That also retires the TODO: Add LAST_WIN policy tests when spark.sql.mapKeyDedupPolicy config is supported note in str_to_map.sql.

The test for mismatched array lengths compares the two engines against each other instead of naming an error condition. Spark still reports that case through a _LEGACY_ERROR_TEMP_* condition whose number moves between Spark versions, so CometTestBase.checkSparkError now builds on a new checkSparkErrorParity helper.

The ConfigOptions change affects every scalar function, so the full 487-fixture suite ran green as well.