Skip to content

[FLINK-40687][build] Relocate the packages the model uber jars actually bundle - #29227

Open
MartijnVisser wants to merge 7 commits into
apache:masterfrom
MartijnVisser:FLINK-40687
Open

[FLINK-40687][build] Relocate the packages the model uber jars actually bundle#29227
MartijnVisser wants to merge 7 commits into
apache:masterfrom
MartijnVisser:FLINK-40687

Conversation

@MartijnVisser

@MartijnVisser MartijnVisser commented Sep 17, 2026

Copy link
Copy Markdown
Contributor

What is the purpose of the change

The shade relocations in both flink-model poms name Maven groupIds, com.squareup and
org.apache.httpcomponents, not Java packages. They never match, so jackson is the only thing
relocated and both jars publish the rest in the default namespace: 1405 unrelocated classes in
flink-model-triton, 13250 in flink-model-openai.

Brief change log

  • Relocations name the real packages, with a trailing dot, since shade matches a pattern as a
    bare prefix against string constants too
  • No org.apache.hc: openai-java 4.63.1 no longer pulls httpclient5. io.swagger,
    com.github.victools and com.fasterxml:classmate are added, because it now pulls those
  • The target gains a shaded segment, which is what stops ArchUnit importing relocated
    classes as production code. It imports 1336 of them today
  • com.openai and kotlin stay unrelocated in flink-model-openai; the pom records why
  • flink-model-triton pins Kotlin to 1.9.10, the version okio-jvm 3.6.0 is built against, and
    no longer bundles empty or metadata-only Kotlin artifacts
  • check_shaded_artifacts_model fails the build on any unrelocated dependency

Verifying this change

Nothing in the build inspected these jars. The new check is red on both modules before this
change and green after.

The relocation set was measured by running each module's tests against the shaded jar instead of
target/classes. flink-model-openai is 22/22 on master and as proposed, but 10/22 with
com.openai relocated, and relocating kotlin breaks kotlin-reflect once no other
kotlin-stdlib is on the classpath. flink-model-triton bundles no kotlin-reflect and relocates
Kotlin safely.

Both NOTICE files now match the shade log exactly. Aggregate javadoc, flink-docs and
flink-architecture-tests-production were run as well.

Does this pull request potentially affect one of the following parts:

  • Dependencies (does it add or upgrade a dependency): no, but four artifacts that contribute
    no classes are no longer bundled and flink-model-triton's Kotlin is pinned to 1.9.10
  • The public API, i.e., is any changed class annotated with @Public(Evolving): no. Relocated
    types do appear in public signatures of the shaded jars, as jackson already does
  • The serializers: no
  • The runtime per-record code paths (performance sensitive): no
  • Anything that affects deployment or recovery: no
  • The S3 file system connector: no

Documentation

  • Does this pull request introduce a new feature? no
Was generative AI tooling used to co-author this PR?
  • Yes (please specify the tool below)

Generated-by: Claude Code (Claude Opus 5)

@MartijnVisser
[hotfix][build] Append rather than replace the shade filters in the m…
9dc3db8 
…odel uber jars

Both model poms declared <filters> without combine.children="append", which the
root pom explicitly warns against ("Filters MUST be appended; merging filters does
not work properly, see MSHADE-305"). The inherited excludes do survive today, because
Maven's merge happens to line up the two single <artifact>*</artifact> filters, so
this changes nothing in the built jars. It stops the result depending on that.

Generated-by: Claude Code (Claude Opus 5)
@MartijnVisser
[FLINK-40687][build] Relocate the model uber jar dependencies under a…
e2727d0 
… shaded package

Every other relocating module in the repo puts a "shaded" segment in the target
package, and it is load-bearing here: flink-architecture-tests-production depends on
the shaded flink-model-openai jar, and ExcludeShadedImportOption only skips paths
matching .*/shaded/.*. Without the segment the relocated jackson classes are imported
as if they were Flink production code, which also puts every package relocated from
here on into that scan.

Generated-by: Claude Code (Claude Opus 5)
@MartijnVisser
[FLINK-40687][build] Relocate the packages the model uber jars actual…
6c67b26 
…ly bundle

The relocations named Maven groupIds, not Java packages, so they never matched and only
jackson was relocated: 1405 unrelocated classes in triton, 13250 in openai.

No org.apache.hc, which openai-java 4.63.1 no longer pulls. Patterns carry a trailing dot
because shade matches them as a bare prefix against string constants too. com.openai and
kotlin stay unrelocated in openai; the pom says why.

Generated-by: Claude Code (Claude Opus 5)
@MartijnVisser
[FLINK-40687][build] Pin the Kotlin version bundled in flink-model-tr…
784031d 
…iton

okhttp 4.12.0 asks for kotlin-stdlib-jdk8 1.8.21 and wins the mediation, while the
okio-jvm 3.6.0 it pulls in is built against 1.9.10 and contributes kotlin-stdlib-common
at that version. The jar therefore bundled a 1.8.21 runtime with 1.9.10 metadata.
Pin both to 1.9.10, the version okio-jvm expects, the way flink-model-openai already
does. The dependencyConvergence enforcer does not catch this: Maven's mediation
resolves the split before the rule sees the tree, and it passes either way.

Generated-by: Claude Code (Claude Opus 5)
@MartijnVisser
[hotfix][NOTICE] Drop flink-model-openai entries left stale by the op…
4264746 
…enai-java bump

openai-java 4.63.1 dropped httpclient5, httpcore5, httpcore5-h2 and the okhttp3
logging-interceptor, but FLINK-40680 only added the new entries. The shade log lists
25 bundled dependencies against 29 in the NOTICE. NoticeFileChecker rates "listed but
not bundled" as TOLERATED and only fails on the reverse, so the licence gate stayed
green.

Generated-by: Claude Code (Claude Opus 5)
@MartijnVisser
[FLINK-40687][build] Do not bundle empty and metadata-only Kotlin art…
0b0f982 
…ifacts

kotlin-stdlib-jdk7 and -jdk8 have been empty since Kotlin 1.8. kotlin-stdlib-common and
com.squareup.okio:okio are multiplatform metadata artifacts, holding 402 .kotlin_metadata
resources and no classes. okio-jvm carries the actual okio code and stays.

Both NOTICE files now match the shade log exactly, 21 entries for openai and 7 for triton.

Generated-by: Claude Code (Claude Opus 5)
@MartijnVisser
[FLINK-40687][build] Verify the model uber jars do not ship unrelocat…
679a478 
…ed dependencies

Nothing in the build looked at these jars, which is how patterns that never matched
survived two releases. The check is an allowlist, so a new dependency cannot ship
unrelocated unnoticed. Sources and javadoc jars are skipped; they sort before the shaded
jar and would make it pass without inspecting anything.

Generated-by: Claude Code (Claude Opus 5)
@flinkbot

flinkbot commented Sep 17, 2026 •
edited
Loading

Copy link
Copy Markdown
Collaborator

CI report:

Bot commands The @flinkbot bot supports the following commands:
  • @flinkbot run azure re-run the last Azure build

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@lihaosky lihaosky Awaiting requested review from lihaosky
@snuyanzin snuyanzin Awaiting requested review from snuyanzin

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@MartijnVisser @flinkbot