feat(kernel): richer TLS Go options — CA bundle + independent hostname skip#400
Draft
mani-mathur-arch wants to merge 1 commit into
Draft
feat(kernel): richer TLS Go options — CA bundle + independent hostname skip#400mani-mathur-arch wants to merge 1 commit into
mani-mathur-arch wants to merge 1 commit into
Conversation
543c7a2 to
ba466d3
Compare
96ba29c to
ab90eff
Compare
ba466d3 to
f35b387
Compare
ab90eff to
0abfb38
Compare
f35b387 to
d0c999e
Compare
0abfb38 to
89c9b36
Compare
…e skip
Expose the two kernel-only TLS knobs whose C-ABI setters already exist on
kernel main (no kernel change needed), via an experimental-option idiom:
- WithKernelTrustedCerts(pem) -> kernel_session_config_set_tls_trusted_certs,
adding a PEM CA bundle on top of the system roots. Required because the
kernel's rustls stack ignores SSL_CERT_FILE, so a custom CA (corporate
re-signing proxy / on-prem CA) must be handed over explicitly.
- WithKernelSkipHostnameVerify() -> set_tls_skip_hostname_verification,
skipping only the hostname check while keeping chain validation
(finer-grained than the blanket WithSkipTLSHostVerify).
The knobs live on a non-exported config.KernelExperimentalConfig off
config.Config (not UserConfig), so they stay off the stable DSN surface
(mirroring Node's InternalConnectionOptions / Python's underscore kwargs).
The Thrift path rejects a non-nil block loudly at connect rather than
silently ignoring it, so a caller who forgets WithUseKernel learns the
option had no effect. OpenSession forwards each to the kernel C ABI via a
byte-buffer helper (cBytes); a reflective guard
(TestKernelExperimentalFieldsClassified) keeps a new field from slipping
either path unclassified.
mTLS client cert/key (needs an absent kernel C-ABI setter, K5) and the
CloudFetch on/off toggle (K3) are deliberately out of scope here — they are
tracked separately (PECOBLR-3652 / PECOBLR-3653).
Closes PECOBLR-3651.
Co-authored-by: Isaac
Signed-off-by: Mani Kaustubh Mathur <mani.mathur@databricks.com>
d0c999e to
2caa354
Compare
89c9b36 to
b147d0e
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Context for reviewers
Part of the SEA-via-kernel backend for the Go driver: an alternative to the Thrift backend that talks to Databricks SQL through the Rust
databricks-sql-kernelstatic library over a C ABI (built into the driver behind thedatabricks_kernelbuild tag; the default pure-Go build is unaffected). This PR adds two TLS knobs to that backend.Stack position (each stacked on the one below):
How to read the diff: the new public surface is the two
With…options inconnector.go+ theirconfigfields; the interesting logic is (a) the Thrift-path reject and (b) theOpenSessionforwarding via the newcBytescgo helper. Everything else is wiring + tests.What
Exposes two kernel-only TLS options via an experimental-option idiom — no kernel change (both C-ABI setters already exist on kernel main, verified against the
origin/mainkernel header):WithKernelTrustedCerts(pem []byte)→kernel_session_config_set_tls_trusted_certs: adds a PEM CA bundle to the kernel's trust store on top of the system roots (corporate re-signing proxy / on-prem CA). Required rather than relying onSSL_CERT_FILE, which the kernel's rustls stack ignores.WithKernelSkipHostnameVerify()→kernel_session_config_set_tls_skip_hostname_verification: skips only the certificate hostname check while keeping chain validation (finer-grained than the blanketWithSkipTLSHostVerify).Closes PECOBLR-3651.
Design
config.KernelExperimentalConfigoffconfig.Config(NOTUserConfig), so they stay off the stable DSN/exported surface — same treatment asTLSConfig, mirroring Node's non-exportedInternalConnectionOptions.DeepCopycopies the CA byte slice.WithUseKernellearns the option had no effect rather than connecting with a weaker-than-intended trust store.OpenSessionforwards each to the kernel C ABI via a new byte-buffer helper (cBytes, mirroringcStr). A reflective guard (TestKernelExperimentalFieldsClassified) fails if a new experimental field isn't classified forwarded/rejected — so a future option can't silently be neither forwarded nor rejected.Scope boundary (deliberately excluded, deferred)
Bundle 1b only (the no-kernel-dependency part of the richer-TLS work). The following were confirmed absent from kernel main, so they need a kernel C-ABI setter first and are correctly deferred:
set_tls_client_certificate) → PECOBLR-3652 (K5).set_cloudfetch_enabled) → PECOBLR-3653 (K3).Testing
CGO_ENABLED=0: build + vet +go test ./...— 24 pkg ok, 0 fail.CGO_ENABLED=1 -tags databricks_kernel(linked against a locally-built kernel.a): build + vet + test — 24 pkg ok, 0 fail. NewTestSetKernelTLSdrives the real cgo setters; untaggedkernel_experimental_test.gocovers the option→config wiring, the classification guard, andDeepCopy.[]bytenormalization, and the idempotent double hostname-skip call when bothWithSkipTLSHostVerify+WithKernelSkipHostnameVerifyare set.Stacking / merge order
Stacked on #399 (
mani/sea-kernel-consolidated). NoKERNEL_REVchange here (inherits #399's pin). #401 (kernel logging) is stacked on top of this.This pull request and its description were written by Isaac.