[FLINK-40416][runtime] Add built-in text AI functions

[haruki-830]

What is the purpose of this pull request?

This PR implements FLINK-40416 by adding six built-in text AI functions to Pipeline transforms:

• AI_CLASSIFY
• AI_TRANSLATE
• AI_SUMMARIZE
• AI_SENTIMENT
• AI_EXTRACT
• AI_MASK

Brief change log

• Add Calcite definitions and Janino/runtime dispatch for the six text AI functions.
• Use English built-in prompt templates and return structured responses as VARIANT.
• Define NULL semantics: NULL input or a NULL model response returns NULL.
• Raise a function-specific error for a non-NULL invalid JSON response.
• Validate model references and text-generation/embedding capabilities.
• Preserve UDF-first behavior when a UDF has the same name as a built-in AI function.
• Extend the Dummy model, E2E coverage, and English/Chinese documentation.

---

Verifying this change

This change added tests and was verified as follows:

• Runtime parser, Janino, and function tests passed on both Flink 1.20 and Flink 2 profiles: 42 tests per profile.
• Composer MiniCluster tests with the Dummy model passed on both profiles: 4 tests per profile.
• Added regression coverage for same-name UDF precedence.
• Updated AiFunctionE2eITCase; the Docker-based Pipeline E2E test was not run locally.
• Spotless, Checkstyle, and diff checks passed.

Documentation

• Does this pull request introduce a new feature? yes
• If yes, how is the feature documented? English and Chinese docs

---

Was generative AI tooling used to co-author this PR?

• Yes (OpenAI Codex)

Generated-by: OpenAI Codex

[yuxiqian]

Looks good! One minor question: why do we need to modify JaninoParser after #4505? Shouldn't the UDF-precedence logic have been implemented there?

[haruki-830]

The 2 PRs apply to different validation paths. #4505 makes Calcite validation follow the existing UDF-first Janino behavior, while this PR adds AI model reference and capability validation that runs earlier.
Without this check, a same-named UDF such as AI_SENTIMENT(id) would be treated as the built-in AI function and rejected before #4505’s resolution logic is reached. So I think it is necessary to keep this handling.

One minor question: why do we need to modify JaninoParser after #4505? Shouldn't the UDF-precedence logic have been implemented there?

[yuxiqian]

Got it, thanks for the clarification!