[FLINK-39969] [runtime] Apply configured SSL protocols and algorithms

[qiuyanjun888]

What is the purpose of the change

This pull request fixes FLINK-39969 so configured SSL protocols and cipher suites are consistently applied to Flink REST SSL services and internal SSL client sockets. Previously, REST server Netty SSL contexts skipped configured protocols/ciphers when no trust manager was installed, and createSSLClientSocketFactory returned the raw socket factory without applying Flink's configured SSL protocol and algorithms.

Brief change log

• Apply configured REST SSL protocols, cipher suites, and client-auth mode directly on the Netty SslContextBuilder regardless of whether a trust manager is present.
• Wrap internal SSL client socket factories so each created SSLSocket receives the configured protocols and cipher suites, matching the existing server-side socket factory behavior.
• Add regression coverage for REST server Netty SSL context configuration and internal SSL client socket configuration.

Verifying this change

This change added tests and can be verified as follows:

• ./mvnw -pl flink-runtime -Dtest=SSLUtilsTest#testRestServerAppliesConfiguredProtocolsAndCipherSuites+testSetSSLVersionAndCipherSuitesForSSLClientSocket -DfailIfNoTests=false -DskipITs -Dfast -Drat.skip=true -Dcheckstyle.skip=true -Dspotless.check.skip=true test
  • Verified RED before the fix: the new focused tests failed because configured protocols were not applied.
  • Verified GREEN after the fix: Tests run: 2, Failures: 0, Errors: 0, Skipped: 0.
• ./mvnw -pl flink-runtime -Dtest=SSLUtilsTest -DfailIfNoTests=false -DskipITs -Dfast -Drat.skip=true -Dspotless.check.skip=true test
  • Tests run: 28, Failures: 0, Errors: 0, Skipped: 0.
• ./mvnw -pl flink-runtime -DskipTests -DskipITs -Drat.skip=true spotless:check
  • BUILD SUCCESS.

Does this pull request potentially affect one of the following parts:

• Dependencies (does it add or upgrade a dependency): no
• The public API, i.e., is any changed class annotated with @Public(Evolving): no
• The serializers: no
• The runtime per-record code paths (performance sensitive): no
• Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn, ZooKeeper: yes, SSL setup for REST and internal runtime communication
• The S3 file system connector: no

Documentation

• Does this pull request introduce a new feature? no
• If yes, how is the feature documented? not applicable

---

Was generative AI tooling used to co-author this PR?

• Yes (Hermes Agent, OpenAI GPT-5.5)

Generated-by: Hermes Agent (OpenAI GPT-5.5)

[flinkbot]

CI report:

• ac07a10 Azure: SUCCESS

Bot commands

The @flinkbot bot supports the following commands:

• @flinkbot run azure re-run the last Azure build

[qiuyanjun888]

This PR is focused on runtime / SSLUtils protocol and algorithm handling.

@tweise @1996fanrui @RocMarshal could you please take a look when you have time and advise whether this approach is acceptable for this area and can move forward?

[JozsefKuti]

I would also add debug logs for SSL settings (even for PekkoUtils to see the RPC settings), because when error happens we are blind.
I know config is there, but log is 100% - increasing the debug level and we can see where the proper configs are applied... even when we are developing its more imporant to see where we missed to update the jar-s...

[JozsefKuti]

Move .clientAuth(clientAuth) a few lines before where clientAuth is enabled - since it is set even for service side as well at the moment.