Skip to content

[FLINK-40148][common] Fix TableIdRouter producing wrong sink table for multi-digit source suffix#4473

Open
sd4324530 wants to merge 1 commit into
apache:masterfrom
sd4324530:fix/table-id-router-find-vs-matches
Open

[FLINK-40148][common] Fix TableIdRouter producing wrong sink table for multi-digit source suffix#4473
sd4324530 wants to merge 1 commit into
apache:masterfrom
sd4324530:fix/table-id-router-find-vs-matches

Conversation

@sd4324530

@sd4324530 sd4324530 commented Jul 14, 2026

Copy link
Copy Markdown
Contributor

What is the purpose of this pull request?

Fix TableIdRouter.resolveReplacement so it routes tables correctly when the source-table regex contains a multi-character capturing group (e.g. a multi-digit suffix like 13, 14, 16) regardless of whether the sink-table uses a $N back-reference.

Previously, with a rule such as:

route:
  - source-table: db_6.table_([1-9]|1[0-6])
    sink-table:   new_db_6.table_merged

db_6.table_13 was wrongly routed to new_db_6.table_merged3 (and similarly _14...merged4, _16...merged6). The bug was caused by Matcher.find() + Matcher.replaceAll in resolveReplacement: find() only matched the prefix db_6.table_1 (because the alternation [1-9] wins over 1[0-6] for input 13), and replaceAll left the trailing 3 appended to the configured sink-table. Tables 1-9 were unaffected because the regex consumed the entire single-digit suffix.

Brief change log

  • Replace Matcher.find() + Matcher.replaceAll(sinkTable) with Matcher.matches() + Matcher.appendReplacement / Matcher.appendTail in TableIdRouter.resolveReplacement. This guarantees the whole source table id is consumed before the substitution is applied, so trailing characters can no longer leak into the sink-table name.
  • Add testRouteWithoutBackReferenceWithMultiDigitSuffix and testRouteWithBackReferenceAndMultiDigitCapture to TableIdRouterTest as regression coverage for this bug.

Verifying this change

This change added tests and can be verified as follows:

  • Added unit tests in flink-cdc-runtime/src/test/java/org/apache/flink/cdc/common/route/TableIdRouterTest.java:
    • testRouteWithoutBackReferenceWithMultiDigitSuffix — reproduces the originally reported scenario (sink-table without $1, multi-digit source suffix); fails on master, passes with the fix.
    • testRouteWithBackReferenceAndMultiDigitCapture — guards the same root cause for the $1 back-reference case to prevent future regressions.
  • All 12 tests in TableIdRouterTest pass, together with the 7 tests in TableIdRouterMatchModeTest and the 8 tests in SchemaDerivatorTest (27 tests total, no regressions).
  • Manually verified by running the failing test against unmodified master: the new test reproduces wrong sink-table names such as new_db_6.table_merged0, ...merged1, ...merged3, ...merged4, ...merged5, ...merged6, which match the issue's report.

Was generative AI tooling used to co-author this PR?
  • Yes

Generated-by: cluade code with minimax m3

…r multi-digit source suffix

TableIdRouter.resolveReplacement used Matcher.find() followed by Matcher.replaceAll
to compute the sink table id. find() only matches a prefix of the source table id,
and replaceAll internally calls reset() + find(), so:

  - When the source-table regex has a multi-character capturing group (e.g.
    db_6.table_([1-9]|1[0-6])), find() matched the prefix db_6.table_1 and left
    the trailing digit (3, 4, 5, 6, ...) appended to the configured sink-table,
    producing wrong sinks like new_db_6.table_merged3 for source db_6.table_13.
  - When the sink-table uses a $1 back-reference, the captured value was only
    the first matched digit for the same reason.

Switch to Matcher.matches() + Matcher.appendReplacement / Matcher.appendTail so
the entire source table id is consumed before the substitution, eliminating the
trailing-characters-leak. Add regression tests in TableIdRouterTest.

Signed-off-by: Pei Yu <125331682@qq.com>
@sd4324530
sd4324530 force-pushed the fix/table-id-router-find-vs-matches branch from adb78b7 to 543bb57 Compare July 15, 2026 00:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant