[Pipe] Support region-level realtime downgrading

[Caideyipi]

Description

Per-Pipe region-level downgrade ordering

• Add source.realtime.region-level-downgrading and its legacy extractor.* alias, disabled by default. Each Hybrid source reads the option from its own Pipe parameters.
• When enabled, serialize downgraded TsFiles at DataRegion level. A later TsFile cannot pass until the previous TsFile and all tablets generated from it have been committed.
• Leave downgrade only after all downgraded TsFiles complete, and stop forwarding realtime tablets through USING_BOTH while region-level downgrade is active. This preserves ordering for equal-timestamp data with serialize-by-region and realtime-first=false.

Bounded latest-TsFile tail

• Retain a bounded realtime tail only for the latest TsFile so flushes cannot continuously chase and prolong downgrade.
• Promote the previous tail to TsFile transfer when a newer epoch arrives, while preserving events queued before downgrade.

TsFile lifecycle

• Track generated-tablet completion for both processor conversion paths.
• Keep the TsFile progress index until the final generated tablet commits.
• Restrict raw-tablet commit hooks to progress-reporting or TsFile-derived events.

Verification

• DataNode test compilation.
• Node Commons and DataNode compilation with both English and Chinese locales.
• DataNode and Node Commons Spotless checks.
• PipeRealtimeExtractTest: 10 tests passed, including enabled/disabled Pipe isolation.
• git diff --check.

---

This PR has:

• been self-reviewed.
  • concurrent read and write
• added comments explaining the intent of non-obvious logic.
• added or updated unit tests to cover the new code paths.

---

Key changed/added classes (or packages if there are too many classes) in this PR

• PipeRealtimeDataRegionHybridSource
• PipeTsFileInsertionEvent
• PipeRawTabletInsertionEvent
• PipeEventCollector
• PipeSourceConstant
• PipeRealtimeExtractTest

[luoluoyuyu]

1. Synchronous tablet generation assumption for custom processors. The current mechanism relies on process(TsFileInsertionEvent, ...) generating and collecting all tablets synchronously before the TsFile event itself is committed. The built-in DoNothingProcessor (passes the TsFile event through as a whole) and the default PipeProcessor.process(TsFileInsertionEvent) (iterates toTabletInsertionEvents() synchronously) both satisfy this. But if a third-party processor stores the toTabletInsertionEvents() iterable and consumes it later, the TsFile event may already be committed while the generated-tablet count is still 0, causing it to be marked transferred immediately and letting region-level downgrading exit early. Please document this constraint explicitly, or add a safe boundary when generatedTabletInsertionEventCount == 0.

2. Transferred hook may not fire if a single event is discarded without closing the pipe. Today close() resets inFlightTsFileCount, so dropping the pipe is a safe fallback. However, if an already-supplied TsFile event, or one of its generated tablet events, is directly cleared/released while the pipe is still alive, clearTsFileEpochAfterCommit may never run and region-level downgrading recovery can stay stuck with inFlightTsFileCount > 0. Please verify that all single-event cleanup paths either reach close() or explicitly compensate the in-flight counter.

[Caideyipi]

Applied in 6f468b5.

1. toTabletInsertionEvents() now marks generation start and returns an iterator wrapper that closes the generation boundary only when the iterable is exhausted (or parsing fails). The transfer hook requires the TsFile commit, the generation boundary, and all registered generated tablets to be transferred or discarded. The API Javadoc documents that asynchronous processors must eventually exhaust the iterable.

2. Added discard accounting/hooks for supplied TsFiles and registered generated tablets. clearReferenceCount() now compensates the realtime source's inFlightTsFileCount; generated-tablet outcomes are idempotent, and registration happens before queue publication to avoid a cleanup race.

Added regression tests for deferred generation and direct TsFile/generated-tablet discard paths. Spotless, checkstyle, diff checks, and pipe-api test-compile pass. Full datanode test-compile is currently blocked by pre-existing generated-source/dependency errors (commons-collections4, IFill/LinearFill) in this workspace.