[SPARK-56927][SDP] Implement SCD1 Batch Processor; Merge Microbatch onto Target

[AnishMahto]

Approved AutoCDC SPIP: https://lists.apache.org/thread/j6sj9wo9odgdpgzlxtvhoy7szs0jplf7

---

This is a stacked PR. Review incremental diff here: AnishMahto/spark@SPARK-56923-SCD1-merge-microbatch-onto-auxiliary...SPARK-56927-SCD1-merge-microbatch-onto-target

---

Preamble:

The SCD type 1 flow is a foreachBatch streaming query on an input change-data-feed, and is responsible for reconciling the incoming change data onto some target table that follows SCD1 replication semantics.

SCD1 flows also maintain an "auxiliary" table to keep track of early-arriving out-of-order received events state. Each microbatch will need to reconcile against this auxiliary table as well, and update the auxiliary table's state appropriately for future microbatches.

Merge Microbatch onto Target:

The final part of an SCD1 reconciliation - merge the reconciled dataframe (deduplicated, target columns and operational metadata projected, and tombstone applied) onto the target table.

The target table's contents represent the SCD1 transformation of the change data feed, for the specified target column selection in the ChangeArgs.

Merging onto the target under SCD1 semantics means deleting existing rows that match against the reconciled microbatch's delete events, and inserting or updating rows as appropriate for the reconciled microbatch's upsert events.