[sheinbergon]

What is the purpose of the change

Since JDK 24 the Java Security Manager is permanently disabled (JEP 486): System#setSecurityManager always throws UnsupportedOperationException, even when passed null. FlinkSecurityManager#forceProcessExit called it unconditionally, so on JDK 24+ the fatal-error paths (FatalExitExceptionHandler, ClusterEntrypoint#onFatalError, TaskManagerRunner#terminateJVM) threw instead of terminating the JVM and left the process running. This is also why ClusterUncaughtExceptionHandlerITCase had to be tagged FailsOnJava25.

This PR makes the exit helpers independent of an installed Security Manager and reports a clear error when the Security Manager based options are enabled on a JVM that cannot install one. Behaviour on JDK 11-23 is unchanged. Full removal of FlinkSecurityManager and cluster.intercept-user-system-exit (as attempted in #26491) is intentionally out of scope and needs a dev-list discussion first.

Brief change log

• FlinkSecurityManager#forceProcessExit / #checkExit only unset the Security Manager when one is actually installed, which can never be the case on JDK 24+.
• Enabling cluster.intercept-user-system-exit or cluster.processes.halt-on-fatal-error without Security Manager support (JDK 24+, or JDK 18-23 without -Djava.security.manager=allow) fails with a message naming JEP 486 and the options to disable, instead of a misleading "no permission" error.
• ExecutorThreadFactory no longer consults System#getSecurityManager for the thread group (the lookup was a no-op).
• Option descriptions document the JDK 24+ limitation; configuration docs regenerated.
• ClusterUncaughtExceptionHandlerITCase untagged from FailsOnJava25; TestJvmProcess gained a getJvmArgs() hook.

Verifying this change

This change added tests and can be verified as follows:

• FlinkSecurityManagerITCase#testForcedJVMExitWithoutSecurityManager: a child JVM calls forceProcessExit without any Security Manager configured and must exit with the requested code. Fails on JDK 25 without the fix (UnsupportedOperationException: Setting a Security Manager is not supported).
• FlinkSecurityManagerITCase#testUnavailableSecurityManagerIsReported: a child JVM started with -Djava.security.manager=disallow must fail with the descriptive IllegalConfigurationException. Runs on JDK 12+.
• ClusterUncaughtExceptionHandlerITCase passes on JDK 25 again (hung before the fix).
• Existing FlinkSecurityManagerTest and the FailsOnJava25-tagged tests are unchanged and pass on JDK 21.

Verified locally on JDK 21 and JDK 25 (Zulu).

Does this pull request potentially affect one of the following parts:

• Dependencies (does it add or upgrade a dependency): no
• The public API, i.e., is any changed class annotated with @Public(Evolving): no
• The serializers: no
• The runtime per-record code paths (performance sensitive): no
• Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn, ZooKeeper: yes - JobManager/TaskManager fatal-error exit on JDK 24+ now terminates the process as intended
• The S3 file system connector: no

Documentation

• Does this pull request introduce a new feature? no
• If yes, how is the feature documented? not applicable (option descriptions updated for the JDK 24+ limitation)

---

Was generative AI tooling used to co-author this PR?

• Yes (please specify the tool below)

Generated-by: Claude Code (Claude Fable 5.1)

[flinkbot]

CI report:

• 7fe07e7 Azure: FAILURE

Bot commands

The @flinkbot bot supports the following commands:

• @flinkbot run azure re-run the last Azure build

[sheinbergon]

@snuyanzin can you please see if this fix is worthy?

[snuyanzin]

thank you for the contribution

why do we need it working in jdk25 if in previous versions it was caught?
Ideally we should have own implementation rather than make it working in complete opposite way
see appendix https://openjdk.org/jeps/486

[sheinbergon]

thank you for the contribution

why do we need it working in jdk25 if in previous versions it was caught? Ideally we should have own implementation rather than make it working in complete opposite way see appendix https://openjdk.org/jeps/486

Thanks for the review. To make sure I read you right: this PR does not touch System.exit interception. On JDK <= 23 cluster.intercept-user-system-exit behaves exactly as before.

It only fixes FlinkSecurityManager#forceProcessExit, Flink's own exit on fatal errors. Since JDK 24 System.setSecurityManager(null) throws, so on JDK 24+ a fatal error currently leaves the JobManager/TaskManager process hanging instead of exiting (that is why ClusterUncaughtExceptionHandlerITCase hangs on JDK 25).

Does this hang fix work for you on its own, with an agent-based replacement for interception (JEP 486 appendix) handled as a separate ticket?

@snuyanzin