[SPARK-54293][SQL] Make ThriftHttpCLIService SNI host check configurable by yadavay-amzn · Pull Request #55308 · apache/spark

yadavay-amzn · 2026-04-11T03:50:20Z

What changes were proposed in this pull request?

Make the SNI host check in ThriftHttpCLIService configurable via spark.sql.hive.thriftServer.http.sniHostCheckEnabled, following the same pattern as SPARK-56528 which made it configurable for the Spark UI.

Why are the changes needed?

SPARK-45522 (Jetty 10+ upgrade) silently enabled SNI host checking, which causes HTTPS connection failures when the certificate CN does not match the hostname. This was made configurable in the Spark UI server (SPARK-56528) but ThriftHttpCLIService had no way to disable it.

This PR adds the same configurability: default true (preserving the current SPARK-45522 behavior), with the option to set false to restore pre-SPARK-45522 behavior.

Does this PR introduce any user-facing change?

A new internal configuration spark.sql.hive.thriftServer.http.sniHostCheckEnabled (default: true) is added. Users who need to disable SNI host checking can set it to false.

How was this patch tested?

Added ThriftHttpCLIServiceSuite with 3 tests:

SNI host check enabled by default
SNI host check disabled when configured
Jetty 10+ default behavior verification

Was this patch authored or co-authored using generative AI tooling?

Yes

yadavay-amzn · 2026-04-30T17:59:17Z

@LuciferYang Could you take a look when you get a chance? Thanks!

LuciferYang · 2026-05-05T16:59:22Z

          Arrays.toString(sslContextFactoryServer.getExcludeProtocols()));
        sslContextFactoryServer.setKeyStorePath(keyStorePath);
        sslContextFactoryServer.setKeyStorePassword(keyStorePassword);
+        // SPARK-54293: Disable SNI host check, which defaults to true since Jetty 10.


also cc @pan3793

pan3793 · 2026-05-05T18:43:14Z

can we follow SPARK-56528 (#55396) to make it configurable?

@pan3793

Address review feedback from @pan3793 (PR apache#55308): follow the pattern established in SPARK-56528 and make the SNI host check configurable rather than unconditionally disabled. Introduces spark.sql.hive.thriftServer.http.sniHostCheckEnabled (default: false, preserving the behavior from the first commit on this branch). Operators who want stricter host checking for security can opt in by setting the flag to true.

@pan3793

Address review feedback from @pan3793 (PR apache#55308): follow the pattern established in SPARK-56528 and make the SNI host check configurable rather than unconditionally disabled. Introduces spark.sql.hive.thriftServer.http.sniHostCheckEnabled (default: false, preserving the behavior from the first commit on this branch). Operators who want stricter host checking for security can opt in by setting the flag to true.

yadavay-amzn · 2026-05-11T07:12:07Z

Done — updated to follow the SPARK-56528 pattern. Added spark.sql.hive.thriftServer.http.sniHostCheckEnabled (default: false) so it's configurable rather than unconditionally disabled.

pan3793 · 2026-05-11T07:45:45Z

+        "connector. Since SPARK-45522 (Jetty 10+), Spark has disabled SNI host check to " +
+        "preserve backward compatibility. Set to true to enforce SNI host checking for " +
+        "stricter security. See SPARK-54293.")
+      .version("4.2.0")


code change lgtm, this requires a PMC member to justify whether to include this in 4.2 as a late-arriving feature.

@pan3793

Address review feedback from @pan3793 (PR apache#55308): follow the pattern established in SPARK-56528 and make the SNI host check configurable rather than unconditionally disabled. Introduces spark.sql.hive.thriftServer.http.sniHostCheckEnabled (default: false, preserving the behavior from the first commit on this branch). Operators who want stricter host checking for security can opt in by setting the flag to true.

yadavay-amzn · 2026-05-11T18:05:02Z

Updated .version("4.2.0") to .version("5.0.0") — happy for this to land in the next release rather than 4.2.

yadavay-amzn · 2026-05-18T16:49:43Z

@pan3793 Gentle reminder, could you please take a look when you get a chance and see if this PR is ready to merge?
Previous CI failure was unrelated, I've rebased to re-trigger it. Please let me know if there is anything else needed.
Thanks!

Disable SNI host check in ThriftHttpCLIService's Jetty SSL connector, consistent with the fix applied to JettyUtils.scala in SPARK-45522. Since Jetty 10, SniHostCheck defaults to true. This was fixed in the Spark UI server but not in ThriftHttpCLIService, which also creates a Jetty server with SSL support. Note: RestSubmissionServer (also mentioned in the JIRA) does not have SSL support, so it does not need this fix.

@pan3793

Address review feedback from @pan3793 (PR apache#55308): follow the pattern established in SPARK-56528 and make the SNI host check configurable rather than unconditionally disabled. Introduces spark.sql.hive.thriftServer.http.sniHostCheckEnabled (default: false, preserving the behavior from the first commit on this branch). Operators who want stricter host checking for security can opt in by setting the flag to true.

pan3793 · 2026-05-18T17:06:06Z

+        "stricter security. See SPARK-54293.")
+      .version("5.0.0")


as default value does not change the current behavior, we can deliver this in 4.3.0. and it's not necessary to mention which JIRA ticket added the config in doc.

Suggested change

"stricter security. See SPARK-54293.")

.version("5.0.0")

"stricter security.")

.version("4.3.0")

@pan3793

Address review feedback from @pan3793 (PR apache#55308): follow the pattern established in SPARK-56528 and make the SNI host check configurable rather than unconditionally disabled. Introduces spark.sql.hive.thriftServer.http.sniHostCheckEnabled (default: false, preserving the behavior from the first commit on this branch). Operators who want stricter host checking for security can opt in by setting the flag to true.

yadavay-amzn · 2026-05-18T17:25:46Z

Done, removed JIRA references from the doc string and updated version to 4.3.0. Thanks @pan3793!

pan3793 · 2026-05-18T18:16:24Z

PR title needs to be updated.

Co-authored-by: Cheng Pan <pan3793@gmail.com>

yadavay-amzn · 2026-05-18T21:06:01Z

@pan3793 Sorry for not applying the recommended change as is earlier, I used the apply suggestion button this time to do so.
Also changed the code comments to reflect the default behavior, updated PR title as well. Thanks!

pan3793

LGTM, will leave this open for another one day before merging.

### What changes were proposed in this pull request? Make the SNI host check in `ThriftHttpCLIService` configurable via `spark.sql.hive.thriftServer.http.sniHostCheckEnabled`, following the same pattern as SPARK-56528 which made it configurable for the Spark UI. ### Why are the changes needed? SPARK-45522 (Jetty 10+ upgrade) silently enabled SNI host checking, which causes HTTPS connection failures when the certificate CN does not match the hostname. This was made configurable in the Spark UI server (SPARK-56528) but `ThriftHttpCLIService` had no way to disable it. This PR adds the same configurability: default `true` (preserving the current SPARK-45522 behavior), with the option to set `false` to restore pre-SPARK-45522 behavior. ### Does this PR introduce _any_ user-facing change? A new internal configuration `spark.sql.hive.thriftServer.http.sniHostCheckEnabled` (default: `true`) is added. Users who need to disable SNI host checking can set it to `false`. ### How was this patch tested? Added `ThriftHttpCLIServiceSuite` with 3 tests: - SNI host check enabled by default - SNI host check disabled when configured - Jetty 10+ default behavior verification ### Was this patch authored or co-authored using generative AI tooling? Yes Closes #55308 from yadavay-amzn/fix/SPARK-54293-sni-host-check. Authored-by: Anupam Yadav <anupamya@amazon.com> Signed-off-by: Cheng Pan <chengpan@apache.org> (cherry picked from commit fa1373e) Signed-off-by: Cheng Pan <chengpan@apache.org>

pan3793 · 2026-05-21T13:28:32Z

thanks, merged to master/4.x

yadavay-amzn force-pushed the fix/SPARK-54293-sni-host-check branch 2 times, most recently from 1a3a880 to 1f7e24c Compare April 27, 2026 22:35

LuciferYang reviewed May 5, 2026

View reviewed changes

yadavay-amzn force-pushed the fix/SPARK-54293-sni-host-check branch 2 times, most recently from 63c708a to 2a05403 Compare May 11, 2026 07:05

pan3793 approved these changes May 11, 2026

View reviewed changes

pan3793 reviewed May 11, 2026

View reviewed changes

yadavay-amzn force-pushed the fix/SPARK-54293-sni-host-check branch from 2a05403 to b648511 Compare May 11, 2026 18:04

yadavay-amzn force-pushed the fix/SPARK-54293-sni-host-check branch from b648511 to 8b88fe0 Compare May 18, 2026 16:52

pan3793 reviewed May 18, 2026

View reviewed changes

yadavay-amzn force-pushed the fix/SPARK-54293-sni-host-check branch from 8b88fe0 to 49892b4 Compare May 18, 2026 17:25

pan3793 reviewed May 18, 2026

View reviewed changes

Comment thread sql/catalyst/src/main/scala/org/apache/spark/sql/internal/StaticSQLConf.scala Outdated

yadavay-amzn and others added 2 commits May 18, 2026 13:40

Apply suggestion from @pan3793

16d0837

Co-authored-by: Cheng Pan <pan3793@gmail.com>

Update comments and test names to reflect default=true

c3378db

yadavay-amzn changed the title ~~[SPARK-54293][SQL] Disable SNI host check in ThriftHttpCLIService~~ [SPARK-54293][SQL] Make ThriftHttpCLIService SNI host check configurable May 18, 2026

pan3793 approved these changes May 19, 2026

View reviewed changes

pan3793 closed this in fa1373e May 21, 2026

dongjoon-hyun mentioned this pull request Jun 10, 2026

[SPARK-57377][INFRA] Add CI check to prevent new entries in the config binding policy exceptions file #56437

Closed

Uh oh!

Conversation

yadavay-amzn commented Apr 11, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What changes were proposed in this pull request?

Why are the changes needed?

Does this PR introduce any user-facing change?

How was this patch tested?

Was this patch authored or co-authored using generative AI tooling?

Uh oh!

yadavay-amzn commented Apr 30, 2026

Uh oh!

LuciferYang May 5, 2026

Choose a reason for hiding this comment

Uh oh!

pan3793 commented May 5, 2026

Uh oh!

yadavay-amzn commented May 11, 2026

Uh oh!

pan3793 May 11, 2026

Choose a reason for hiding this comment

Uh oh!

yadavay-amzn commented May 11, 2026

Uh oh!

yadavay-amzn commented May 18, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

pan3793 May 18, 2026

Choose a reason for hiding this comment

Uh oh!

yadavay-amzn commented May 18, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

pan3793 commented May 18, 2026

Uh oh!

yadavay-amzn commented May 18, 2026

Uh oh!

pan3793 left a comment

Choose a reason for hiding this comment

Uh oh!

pan3793 commented May 21, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

yadavay-amzn commented Apr 11, 2026 •

edited

Loading

yadavay-amzn commented May 18, 2026 •

edited

Loading

yadavay-amzn commented May 18, 2026 •

edited

Loading