refactor: Make shard spec collector an interface for extensibility

[abhishekrb19]

This is a follow-up to #19571. This patch addresses this review comment #19571 (comment) by refactoring the logic into an interface so it's extensible and maintainable. No other functional change.

New abstractions (2 interfaces + 2 impls)

• StreamingPartitionsSpec — was a concrete class, now a polymorphic Jackson interface (@JsonTypeInfo with type property, defaultImpl = DimensionValueSetPartitionsSpec). Its only method is createCollector() for now.
• DimensionValueSetPartitionsSpec (type: "dim_value_set") — the built-in impl holding the old fields (partitionDimensions, maxValuesPerDimension, validation, equals/hashCode/toString). createCollector() returns null when there are no
  dimensions (disabled by default), otherwise a DimensionValueSetCollector.
• StreamingShardSpecCollector — new interface defining the per-task lifecycle required for stamping the shard specs: collect(segmentId, row) (run-loop, per row), onSegmentsRestored(segmentIds) (startup, disables pruning for restart-spanned segments), annotate(segment) (publish-time stamping), onSegmentPublished(segmentId).
• DimensionValueSetCollector — the concrete collector holding the state that used to live in the runner (observedPartitionDimValuesBySegment ConcurrentHashMap, restartSpannedSegments set) and all the value-recording, capping, sorting, null-handling, and stamping logic.

This PR has:

• been self-reviewed.
• added documentation for new or modified features or behaviors.
• a release note entry in the PR description.
• added Javadocs for most classes and all non-trivial methods. Linked related entities via Javadoc links.
• added comments explaining the "why" and the intent of the code wherever would not be obvious for an unfamiliar reader.
• added unit tests or modified existing tests to cover new code paths, ensuring the threshold for code coverage is met.
• been tested in a test Druid cluster.

[jtuglu1]

Should this go in v38?

[jtuglu1]

future: I wonder if it's worth somehow making this exclusion condition dependent on the frequency of the column values? For example, keeping the best – those which have highest chance of reducing read amplification – pruning column values in the shard spec?

[abhishekrb19]

I wonder if it's worth somehow making this exclusion condition dependent on the frequency of the column values?

Yes, frequency-based should be possible. I think it'd either require introducing a mode into this existing shard type or adding a new shard spec altogether (that resembles a bloom value set use case).

[FrankChen021]

Severity	Findings
P0	0
P1	0
P2	1
P3	0
Total	1

Severity	Findings
P0	0
P1	0
P2	1
P3	0
Total	1

Reviewed 10 of 10 changed files.

Found one polymorphic-deserialization issue: an explicit unknown streaming partition type silently falls back to the built-in strategy instead of failing.

---

This is an automated review by Codex GPT-5.6-Sol

[FrankChen021]

I have reviewed the updated code for correctness, edge cases, concurrency, and integration risks; no issues found. The incremental fix rejects explicit unknown type IDs while preserving the omitted-type default.

Reviewed 10 of 10 changed files.

---

This is an automated review by Codex GPT-5.6-Sol