[rionmonster]

What is the purpose of the change

LateralSnapshotJoinOperator stored build rows whose join key has a NULL in a field that isn't null-safe. Such rows can never match. Unlike in the window and interval joins, nothing bounds this state except state TTL: a row stays until the build side retracts it, or until TTL removes it if TTL is configured. All such rows share one key, so they pile up in a single subtask, and every probe with a NULL key scans all of them. This is the follow-up to FLINK-41034 for this operator.

Brief change log

• processElement2 doesn't buffer an INSERT or UPDATE_AFTER build change when JoinConditionWithNullFilters#currentKeyCannotMatch is true. The build side is never preserved, so the output doesn't change. The idle-flip timer is still armed first, because any build record counts as a build-side signal.
• Retractions are still buffered and applied, so a row that an earlier version stored is removed.
• applyBuildChanges doesn't count a retraction of such a row in numUnmatchedBuildRetractions when it finds nothing. Otherwise the metric would count every NULL-key delete. The metric's description in the docs (English and Chinese) says so. Non-NULL keys are counted as before.
• The probe side doesn't change. A NULL-key probe now scans an empty build table for its key.

Verifying this change

This change added tests:

• LateralSnapshotJoinOperatorTest#joinRespectsNullKeysFilter now also checks that a NULL-key build row isn't stored, for INNER and LEFT joins. With a null-safe key the row is still stored, which is the positive control.
• LateralSnapshotJoinOperatorTest#retractsNullKeyBuildRowStoredByEarlierVersion restores state with a stored NULL-key row, on the heap and RocksDB backends. It checks that the row's retraction still removes it, and that a retraction of a row that was never stored isn't counted in the metric.

The new checks fail if NULL-key rows are stored, if their retractions are dropped too, or if those retractions are counted. The existing LATERAL SNAPSHOT tests pass: the runtime join tests, and the planner plan, semantic, restore and ITCase tests.

No new restore test is needed, because the state layout doesn't change. NULL-key rows stored by an earlier version stay until they are retracted or TTL removes them, as before.

Does this pull request potentially affect one of the following parts:

• Dependencies (does it add or upgrade a dependency): no
• The public API, i.e., is any changed class annotated with @Public(Evolving): no
• The serializers: no
• The runtime per-record code paths (performance sensitive): yes. It adds one null check per build-side accumulation, and in return it skips buffering and storing rows that can never match.
• Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn, ZooKeeper: no
• The S3 file system connector: no

Documentation

• Does this pull request introduce a new feature? no
• If yes, how is the feature documented? not applicable. The description of numUnmatchedBuildRetractions is updated.

---

Was generative AI tooling used to co-author this PR?

• Yes (please specify the tool below)

Generated-by: Claude Code (Claude Opus 5.5)

[flinkbot]

CI report:

• e09288c Azure: SUCCESS
• da6ebc5 Azure: PENDING

Bot commands

The @flinkbot bot supports the following commands:

• @flinkbot run azure re-run the last Azure build

[gustavodemorais]

Thanks for the PR, @rionmonster!

[gustavodemorais]

The NULL-key build tests only use +I and -D. Could we also add a +U NULL-key row to joinRespectsNullKeysFilter and use an UPDATE_BEFORE for the stored row in retractsNullKeyBuildRowStoredByEarlierVersion? Right now narrowing the guard to RowKind.INSERT wouldn't fail any test.

[rionmonster]

Thanks for the review, much appreciated! Good catch here, I've gone ahead and updated this and verified that the guard handles it as expected! 👍

[gustavodemorais]

Thanks @rionmonster! I don't see the changes on the PR yet - the head is still e09288c. Could you push them?

[rionmonster]

Doh! That's what I get for trying to juggle 4-5 branches at a time. The changes should be pushed up now. Thanks again for the review @gustavodemorais!