Gate thin-client (Gateway V2) routing behind an endpoint connectivity probe when COSMOS.THINCLIENT_ENABLED is unset

[jeet1995]

Motivation

Gateway V2 (ThinClient) is the new Compute-hosted data plane for Cosmos DB. To roll it out safely the Java SDK needs a user-transparent way to route eligible data-plane and QueryPlan requests to ThinClient only once the proxy is proven reachable (or the user explicitly opts in) — without user-visible errors or per-request overhead. Today the SDK has no client-side reachability signal which this PR aims to solve:

• Making COSMOS.THINCLIENT_ENABLED an explicit tri-state: true = hard opt-in (probe bypassed), false = hard opt-out, unset = probe-gated (route to Gateway V2 only once the connectivity probe greenlights the proxy, else Gateway V1). Unset does not default routing on.
• Adding an out-of-band connectivity probe (EndpointProbeClient) that POSTs to /connectivity-probe over the thin-client HTTP/2 transport every topology refresh. The verdict feeds the routing gate via GlobalEndpointManager.getProxyProbeDecision().

Key invariants:

• No per-request probe I/O — probing piggybacks on the topology-refresh cadence; the data-plane gate is a single in-memory read.
• Conservative implicit path — with the flag unset, before any region is proven the gate is false (empty endpoint set), so a slow/failing first probe never pushes traffic onto an unproven proxy.
• Explicit opt-in bypasses the probe — THINCLIENT_ENABLED=true skips the probe gate for routing; the probe governs only the implicit path.
• Never trip client init — every probe failure is swallowed and logged; client construction and topology refresh are protected by onErrorResume.
• Tri-state gate, no oscillation counters — getProxyProbeDecision() is true only when every known region has a proven (HTTP 200) probe, false when a known region is unproven/non-200, null before any verdict. On the implicit path both false and null pin to Gateway V1; a region flips healthy on its first 200 — no thresholds.
• Add-only proven-healthy cache — a 200 records a region proven and is never pruned. Each cycle probes only the delta; the gate is provenHealthyEndpoints.containsAll(currentEndpoints) against a volatile snapshot.

Call sequence

sequenceDiagram
    autonumber
    participant App as User App
    participant Client as CosmosClient<br/>(RxDocumentClientImpl)
    participant GEM as GlobalEndpointManager
    participant Probe as EndpointProbeClient
    participant Proxy as ThinClient<br/>(Gateway V2)
    participant V1 as Gateway V1<br/>(fallback)

    rect rgba(200,220,255,0.25)
    Note over App,Probe: Bootstrap (one-time)
    App->>Client: new CosmosClient(...)
    Client->>Client: canThinClientBeUsed() (HTTP/2 + Gateway mode, not opted out)
    alt thin-client usable
    Client->>GEM: setThinClientHttpClient(...)
    GEM->>Probe: new EndpointProbeClient(httpClient)
    Note over Probe: wired even for explicit true — routing bypasses it on that path
    else not usable
    Note over Client,Probe: probe not wired
    end
    end

    rect rgba(200,255,210,0.25)
    Note over GEM,Proxy: Topology refresh (every tick)
    GEM->>GEM: refreshLocationPrivateAsync() → runThinClientProbeCycleMono()
    GEM->>Probe: runProbeCycle(delta endpoints)
    par Per new endpoint
        Probe->>Proxy: POST /connectivity-probe (HTTP/2)
        Proxy-->>Probe: 200 (or error/timeout)
    end
    Probe->>Probe: applyCycleResult() — cache proven regions
    Note right of Probe: all regions 200 → true — any unproven/non-200 → false
    end

    rect rgba(255,235,200,0.25)
    Note over App,V1: Data-plane / QueryPlan request
    App->>Client: readItem(...) / queryItems(...)
    Client->>GEM: getProxyProbeDecision()
    GEM-->>Client: true / false / null
    alt usable && hasThinClientReadLocations && eligible && (explicitOptIn || decision==true)
        Client->>Proxy: route through Gateway V2
    else gate failed (false/null, implicit path)
        Client->>V1: route through Gateway V1
    end
    Client-->>App: response (transparent fallback)
    end

Loading

Configuration

Property	Default	Purpose
COSMOS.THINCLIENT_ENABLED	unset (tri-state)	Master switch (getter: Configs.isThinClientEnabled()). Unset ⇒ probe-gated; true ⇒ opt-in bypassing the probe; false ⇒ never route thin-client.

This PR only changes the semantics of the master switch; HTTP/2 (COSMOS.HTTP2_ENABLED) + GATEWAY mode remain the transport prerequisite. /connectivity-probe is a fixed proxy contract — no separate kill-switch, threshold, or probe-path property.

[xinlian12]

✅ Review complete (46:59)

Posted 6 inline comment(s).

_{Steps: ✓ context, correctness, cross-sdk, design, history, past-prs, synthesis, test-coverage}

[jeet1995]

/azp run java - cosmos - kafka

[jeet1995]

/azp run java - cosmos - spark

[jeet1995]

/azp run java - cosmos - tests

[azure-pipelines]

Azure Pipelines successfully started running 1 pipeline(s).

[azure-pipelines]

Azure Pipelines successfully started running 1 pipeline(s).

[azure-pipelines]

Azure Pipelines successfully started running 1 pipeline(s).

[jeet1995]

/azp run java - cosmos - tests

[jeet1995]

/azp run java - cosmos - spark

[jeet1995]

/azp run java - cosmos - kafka

[azure-pipelines]

Azure Pipelines successfully started running 1 pipeline(s).

[azure-pipelines]

Azure Pipelines successfully started running 1 pipeline(s).

[azure-pipelines]

Azure Pipelines successfully started running 1 pipeline(s).

[jeet1995]

/azp run java - cosmos - tests

[jeet1995]

/azp run java - cosmos - spark

[jeet1995]

/azp run java - cosmos - kafka

[azure-pipelines]

Azure Pipelines successfully started running 1 pipeline(s).

[azure-pipelines]

Azure Pipelines successfully started running 1 pipeline(s).

[azure-pipelines]

Azure Pipelines successfully started running 1 pipeline(s).