Skip to content

feat(kernel): richer TLS Go options — CA bundle + independent hostname skip#400

Draft
mani-mathur-arch wants to merge 1 commit into
mani/sea-kernel-consolidatedfrom
mani/sea-kernel-richer-tls
Draft

feat(kernel): richer TLS Go options — CA bundle + independent hostname skip#400
mani-mathur-arch wants to merge 1 commit into
mani/sea-kernel-consolidatedfrom
mani/sea-kernel-richer-tls

Conversation

@mani-mathur-arch

@mani-mathur-arch mani-mathur-arch commented Jul 14, 2026

Copy link
Copy Markdown
Collaborator

Context for reviewers

Part of the SEA-via-kernel backend for the Go driver: an alternative to the Thrift backend that talks to Databricks SQL through the Rust databricks-sql-kernel static library over a C ABI (built into the driver behind the databricks_kernel build tag; the default pure-Go build is unaffected). This PR adds two TLS knobs to that backend.

Stack position (each stacked on the one below):

#399  OAuth / namespace / metric-view / params / types / query-id   (base)
  └─ #400  richer TLS options            ← THIS PR
       └─ #401  kernel logging → driver log level
            └─ #402  nightly E2E workflow
                 └─ #403  CI dispatch label + CLOSE_STATEMENT telemetry

How to read the diff: the new public surface is the two With… options in connector.go + their config fields; the interesting logic is (a) the Thrift-path reject and (b) the OpenSession forwarding via the new cBytes cgo helper. Everything else is wiring + tests.


What

Exposes two kernel-only TLS options via an experimental-option idiom — no kernel change (both C-ABI setters already exist on kernel main, verified against the origin/main kernel header):

  • WithKernelTrustedCerts(pem []byte)kernel_session_config_set_tls_trusted_certs: adds a PEM CA bundle to the kernel's trust store on top of the system roots (corporate re-signing proxy / on-prem CA). Required rather than relying on SSL_CERT_FILE, which the kernel's rustls stack ignores.
  • WithKernelSkipHostnameVerify()kernel_session_config_set_tls_skip_hostname_verification: skips only the certificate hostname check while keeping chain validation (finer-grained than the blanket WithSkipTLSHostVerify).

Closes PECOBLR-3651.

Design

  • Knobs live on a non-exported config.KernelExperimentalConfig off config.Config (NOT UserConfig), so they stay off the stable DSN/exported surface — same treatment as TLSConfig, mirroring Node's non-exported InternalConnectionOptions. DeepCopy copies the CA byte slice.
  • The default (Thrift) backend rejects a non-nil block loudly at connect, so a caller who sets one and forgets WithUseKernel learns the option had no effect rather than connecting with a weaker-than-intended trust store.
  • OpenSession forwards each to the kernel C ABI via a new byte-buffer helper (cBytes, mirroring cStr). A reflective guard (TestKernelExperimentalFieldsClassified) fails if a new experimental field isn't classified forwarded/rejected — so a future option can't silently be neither forwarded nor rejected.

Scope boundary (deliberately excluded, deferred)

Bundle 1b only (the no-kernel-dependency part of the richer-TLS work). The following were confirmed absent from kernel main, so they need a kernel C-ABI setter first and are correctly deferred:

  • mTLS client cert/key (set_tls_client_certificate) → PECOBLR-3652 (K5).
  • CloudFetch enable/disable (set_cloudfetch_enabled) → PECOBLR-3653 (K3).
  • Private-key auth → PECOBLR-3605 (crate-blocked).

Testing

  • Default CGO_ENABLED=0: build + vet + go test ./... — 24 pkg ok, 0 fail.
  • Tagged CGO_ENABLED=1 -tags databricks_kernel (linked against a locally-built kernel .a): build + vet + test — 24 pkg ok, 0 fail. New TestSetKernelTLS drives the real cgo setters; untagged kernel_experimental_test.go covers the option→config wiring, the classification guard, and DeepCopy.
  • Isaac Review: clean (0 critical / 0 major / 0 minor). Two info-level items were reviewed and judged harmless: the empty→nil []byte normalization, and the idempotent double hostname-skip call when both WithSkipTLSHostVerify + WithKernelSkipHostnameVerify are set.

Stacking / merge order

Stacked on #399 (mani/sea-kernel-consolidated). No KERNEL_REV change here (inherits #399's pin). #401 (kernel logging) is stacked on top of this.

This pull request and its description were written by Isaac.

…e skip

Expose the two kernel-only TLS knobs whose C-ABI setters already exist on
kernel main (no kernel change needed), via an experimental-option idiom:

  - WithKernelTrustedCerts(pem) -> kernel_session_config_set_tls_trusted_certs,
    adding a PEM CA bundle on top of the system roots. Required because the
    kernel's rustls stack ignores SSL_CERT_FILE, so a custom CA (corporate
    re-signing proxy / on-prem CA) must be handed over explicitly.
  - WithKernelSkipHostnameVerify() -> set_tls_skip_hostname_verification,
    skipping only the hostname check while keeping chain validation
    (finer-grained than the blanket WithSkipTLSHostVerify).

The knobs live on a non-exported config.KernelExperimentalConfig off
config.Config (not UserConfig), so they stay off the stable DSN surface
(mirroring Node's InternalConnectionOptions / Python's underscore kwargs).
The Thrift path rejects a non-nil block loudly at connect rather than
silently ignoring it, so a caller who forgets WithUseKernel learns the
option had no effect. OpenSession forwards each to the kernel C ABI via a
byte-buffer helper (cBytes); a reflective guard
(TestKernelExperimentalFieldsClassified) keeps a new field from slipping
either path unclassified.

mTLS client cert/key (needs an absent kernel C-ABI setter, K5) and the
CloudFetch on/off toggle (K3) are deliberately out of scope here — they are
tracked separately (PECOBLR-3652 / PECOBLR-3653).

Closes PECOBLR-3651.

Co-authored-by: Isaac
Signed-off-by: Mani Kaustubh Mathur <mani.mathur@databricks.com>
@mani-mathur-arch mani-mathur-arch force-pushed the mani/sea-kernel-consolidated branch from d0c999e to 2caa354 Compare July 14, 2026 18:53
@mani-mathur-arch mani-mathur-arch force-pushed the mani/sea-kernel-richer-tls branch from 89c9b36 to b147d0e Compare July 14, 2026 18:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant