Add nested struct schema evolution support for Map types

[kosiew]

Which issue does this PR close?

Closes #20835
This is the last of a series of PR to close #20835

Rationale for this change

Map types were not handled by DataFusion's recursive nested schema adaptation logic, leaving map key/value struct evolution undefined and unsupported. This change adds centralized validation and casting semantics for Map columns so planner-time validation and runtime casting behave consistently for supported schema evolution scenarios while rejecting unsafe key evolution.

What changes are included in this PR?

• Add recursive Map support to the nested schema adaptation path used by planning and execution.

• Introduce Map compatibility validation covering:

  • positional matching of map entry key/value children,
  • name-based evolution of nested Struct fields,
  • matching sorted flags,
  • validation of map entry invariants,
  • conservative rules for map key evolution to preserve key identity.

• Implement Map casting that:

  • recursively adapts key and value children,
  • preserves map offsets, nulls, and sortedness,
  • compacts unreachable or null-hidden entries before recursive casting to avoid invalid child data affecting visible rows.

• Extend nested-cast detection so compatible Map types are routed through the specialized nested adaptation path.

• Document supported map evolution semantics alongside the validation logic.

• Add comprehensive unit tests covering:

  • map key and value struct evolution,
  • positional entry matching with renamed entry fields,
  • planner/runtime validation parity,
  • sorted vs. unsorted map behavior,
  • key evolution rejection cases,
  • null and sliced map handling,
  • all-null map behavior, and
  • map entry invariant validation.

• Add end-to-end Parquet tests covering successful map value struct evolution and rejection of incompatible evolution.

Are these changes tested?

Yes.

New test coverage includes unit tests in nested_struct.rs for:

• Map key/value struct schema evolution
• Nested map-entry struct adaptation
• Planner/runtime validation parity
• Sorted map restrictions
• Invalid key evolution rejection
• Null parent and sliced map handling
• All-null map behavior
• Map entry invariant validation

It also adds end-to-end Parquet tests:

• test_map_value_struct_schema_evolution_end_to_end
• test_map_value_struct_incompatible_schema_evolution_rejected

Are there any user-facing changes?

Yes.

DataFusion now supports recursive schema adaptation for compatible Map types whose keys and/or values recursively contain Structs. Compatible schema evolution is applied during planning and execution using consistent validation rules, while unsupported or unsafe map key evolution, sorted-flag changes, and incompatible non-nullable schema changes are rejected. This extends schema evolution behavior but does not introduce a public API change.

LLM-generated code disclosure

This PR includes LLM-generated code and comments. All LLM-generated content has been manually reviewed.

[codecov-commenter]

Codecov Report

❌ Patch coverage is 97.48322% with 15 lines in your changes missing coverage. Please review.
✅ Project coverage is 80.69%. Comparing base (f1ab86d) to head (46560db).
⚠️ Report is 15 commits behind head on main.

Files with missing lines	Patch %	Lines
datafusion/common/src/nested_struct.rs	97.48%	12 Missing and 3 partials ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main   #23914      +/-   ##
==========================================
+ Coverage   80.65%   80.69%   +0.03%     
==========================================
  Files        1091     1094       +3     
  Lines      371031   372366    +1335     
  Branches   371031   372366    +1335     
==========================================
+ Hits       299256   300478    +1222     
- Misses      53935    53981      +46     
- Partials    17840    17907      +67     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.