[SPARK-58945][SQL] Fix mismatched messageParameters keys that cause INTERNAL_ERROR

[subhramit]

Closes SPARK-58945

Note that the following description was initially generated using GPT 5.4, and then edited by me.

What changes were proposed in this pull request?

This fixes several broken Spark error-reporting paths caused by mismatches between messageParameters keys and the placeholders declared in error-conditions.json.

The changes fall into two groups:

1. Fix Scala-side parameter key mismatches

   • sql/catalyst/src/main/scala/org/apache/spark/sql/errors/QueryCompilationErrors.scala
     • _LEGACY_ERROR_TEMP_2450: use clazz instead of invalidClass
   • sql/catalyst/src/main/scala/org/apache/spark/sql/errors/QueryExecutionErrors.scala
     • INVALID_PARAMETER_VALUE.EXTENSION: use invalidValue instead of fileExtension / acceptable
     • INVALID_WRITER_COMMIT_MESSAGE: use detail instead of details
   • sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/state/StateStoreErrors.scala
     • STATE_STORE_COLUMN_FAMILY_SCHEMA_INCOMPATIBLE: use colFamilyName / oldSchema / newSchema
   • sql/core/src/main/scala/org/apache/spark/sql/jdbc/H2Dialect.scala
     • TABLE_OR_VIEW_NOT_FOUND: add the missing searchPath parameter as "not available". classifyException receives only pre-rendered strings from JDBCTableCatalog.scala:104-106, so no Spark-side search path exists at that point.

2. Fix one duplicated JSON message template

   • common/utils/src/main/resources/error/error-conditions.json
   • _LEGACY_ERROR_TEMP_3069 and _LEGACY_ERROR_TEMP_3070 had byte-identical message templates.
   • _LEGACY_ERROR_TEMP_3069 is the reserved-column-name collision case and its message is correct.
   • _LEGACY_ERROR_TEMP_3070 is the unrecognized file metadata field fallback, and its Scala call site (Map("field" -> field.toString)) was already correct.

This PR fixes only the template for _LEGACY_ERROR_TEMP_3070, restoring the pre-error class-migration wording: Unrecognized file metadata field: <field>. This is the text the branch threw before the error-class migration in SPARK-46351, still visible at the v3.5.0 tag.

Also adds regression coverage for the fixed paths:

• sql/core/src/test/scala/org/apache/spark/sql/errors/ErrorMessageParametersSuite.scala
• sql/core/src/test/scala/org/apache/spark/sql/errors/QueryExecutionErrorsSuite.scala
• sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/v2/jdbc/JDBCTableCatalogSuite.scala
• sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/FileSourceCustomMetadataStructSuite.scala

A minimal repro for one affected path is:

spark.range(1).write.option("extension", "12").csv(path)

Before this change, that path fails during message rendering and surfaces INTERNAL_ERROR instead of INVALID_PARAMETER_VALUE.EXTENSION.

Why are the changes needed?

Spark's StringSubstitutor defaults to enableUndefinedVariableException = true, so an unresolved placeholder throws IllegalArgumentException, which ErrorClassesJSONReader converts into SparkException.internalError.

Error messages are rendered eagerly when constructing many exceptions. The relevant exception constructors call SparkThrowableHelper.getMessage during construction (example). When substitution fails, the intended exception instance is never created, so the affected paths return INTERNAL_ERROR in place of the actual diagnosis.

These are user-visible diagnostics bugs rather than behavior changes in the main execution path.

Two of the six sites are not demonstrated by an end-to-end repro today. stateStoreColumnFamilyMismatch currently has no callers, so that path is latent. It would have failed during message construction before this fix, so the constructor is covered directly by a unit test.

The H2Dialect.scala:230 change is also covered directly rather than through loadTable(...): missing-table loadTable is intercepted earlier by JDBCRDD.resolveTable(...), so that path does not exercise the dialect branch. The dialect classification path itself is fixed here by supplying the missing searchPath parameter.

For _LEGACY_ERROR_TEMP_3070 the failure mode differs: the Scala call site was already correct, but the JSON template was duplicated from _LEGACY_ERROR_TEMP_3069 during the error-class migration in SPARK-46351, so the fallback reports a reserved-column-name collision that never occurred.

Does this PR introduce any user-facing change?

Yes.

This PR fixes user-facing error reporting for the currently reachable affected paths on master, and also fixes two latent error paths.

Examples:

• Invalid CSV writer extension values now report INVALID_PARAMETER_VALUE.EXTENSION instead of failing during message construction.
• Invalid writer commit message counts now report INVALID_WRITER_COMMIT_MESSAGE with the intended detail text.
• _LEGACY_ERROR_TEMP_3070 now reports Unrecognized file metadata field: <field> instead of the unrelated reserved-column-name message.

The H2 dialect change is a correctness fix in the JDBC classification path, but it is not demonstrated here by an end-to-end query repro. The state store fix is latent today because stateStoreColumnFamilyMismatch currently has no callers.

This does not change the semantics of the underlying operations. It fixes the diagnostics that Spark surfaces when those error paths are hit.

How was this patch tested?

Added/updated regression tests:

• ErrorMessageParametersSuite
• QueryExecutionErrorsSuite
• JDBCTableCatalogSuite
• FileSourceCustomMetadataStructSuite

Executed:

build/sbt "core/testOnly *SparkThrowableSuite"
build/sbt "sql/testOnly *ErrorMessageParametersSuite *QueryExecutionErrorsSuite"
build/sbt "sql/testOnly *JDBCTableCatalogSuite *FileSourceCustomMetadataStructSuite"
./dev/scalastyle
./dev/run-tests

and all of them passed.

Also performed manual repro -

• On main:fa6f713:

• On this branch:

Was this patch authored or co-authored using generative AI tooling?

Generated-by: Claude Opus 5 + GPT-5.4 (Zed coding agent) + manually touched up in IntelliJ IDEA

[uros-b]

Thank you @subhramit!

[subhramit]

Thank you @subhramit!

Thank you for the quick review :)

I will be filing a small follow-up after this for SPARK-58946 (currently this doesn't surface because due to INTERNAL_ERROR that was fixed here).

Update - filed as #58226.