perf: replace SELECT * with explicit projections in topology monitor queries#858
Merged
dkropachev merged 1 commit intoApr 3, 2026
Conversation
nikagra
force-pushed
the
driver-367-optimize-system-local-query
branch
2 times, most recently
from
5ae1d6d to
2cfced8
Compare
There was a problem hiding this comment.
Pull request overview
This PR aims to reduce topology-monitor overhead by changing system table queries away from SELECT * and introducing mechanisms to control/refresh the chosen projections across reconnects.
Changes:
- Added column-name caching and a
resetColumnCaches()hook to influence howDefaultTopologyMonitorbuilds system tableSELECTstatements. - Added
AdminResult#getColumnNames()(plus tests/helpers) to expose response metadata for the caching logic. - Updated unit/integration tests to account for the new query strings / caching behavior.
Reviewed changes
Copilot reviewed 9 out of 9 changed files in this pull request and generated 4 comments.
Show a summary per file
| File | Description |
|---|---|
| integration-tests/src/test/java/com/datastax/oss/driver/core/ProtocolVersionMixedClusterIT.java | Adjusts expected query sequence and adds a projected system.local query constant. |
| integration-tests/src/test/java/com/datastax/oss/driver/core/PeersV2NodeRefreshIT.java | Relaxes query matching to tolerate different SELECT projections. |
| core/src/test/java/com/datastax/oss/driver/internal/core/metadata/DefaultTopologyMonitorTest.java | Adds tests for projection caching behavior and cache reset semantics. |
| core/src/test/java/com/datastax/oss/driver/internal/core/metadata/AdminResultTestHelper.java | Adds helper to stub AdminResult#getColumnNames() in tests. |
| core/src/test/java/com/datastax/oss/driver/internal/core/adminrequest/AdminResultTest.java | New tests validating AdminResult#getColumnNames() behavior. |
| core/src/main/java/com/datastax/oss/driver/internal/core/metadata/TopologyMonitor.java | Adds default resetColumnCaches() API. |
| core/src/main/java/com/datastax/oss/driver/internal/core/metadata/DefaultTopologyMonitor.java | Implements column-name caches and uses them to build projected SELECT queries. |
| core/src/main/java/com/datastax/oss/driver/internal/core/control/ControlConnection.java | Resets topology monitor column caches on successful reconnect before refresh. |
| core/src/main/java/com/datastax/oss/driver/internal/core/adminrequest/AdminResult.java | Exposes result column names via getColumnNames(). |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
nikagra
force-pushed
the
driver-367-optimize-system-local-query
branch
from
2cfced8 to
e5f65a3
Compare
nikagra
force-pushed
the
driver-367-optimize-system-local-query
branch
from
e5f65a3 to
086dad9
Compare
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 9 out of 9 changed files in this pull request and generated 3 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
…R-367) Replace SELECT * with explicit column projections in system.local and system.peers/peers_v2 queries issued by DefaultTopologyMonitor. The column list is learned dynamically at runtime: the first SELECT * response populates a volatile cache per table; all subsequent queries use the cached column names to build projected SELECTs. The caches are reset to null on control-connection reconnect, which causes the next call to re-issue SELECT * and re-learn whatever columns are available. This avoids sending unnecessary data (e.g. large tokens columns) on every topology refresh while remaining compatible with any server schema version without a hardcoded column list. Implementation details: - AdminResult.getColumnNames(): new method returning the set of column names from the result metadata, used to populate the caches. - DefaultTopologyMonitor: three volatile caches (localColumns, peersColumns, peersV2Columns), buildQuery() helpers, and resetColumnCaches() called by ControlConnection.onSuccessfulReconnect. - Cache-population sites guard against caching an empty column set (e.g. when the server returns no rows) so the next call retries with SELECT * instead of projecting zero columns. - Narrow single-node refreshNode() WHERE-clause queries always use SELECT * because projecting a one-row result gives negligible benefit and the fixed form is required by test infrastructure (Simulacron primes only SELECT * for that query shape). Tests: - DefaultTopologyMonitorTest: 30 unit tests (2 new warm-cache/reset tests, 1 new empty-column-set guard test). - AdminResultTest: 3 new tests for getColumnNames(). - ProtocolVersionMixedClusterIT: updated to expect projected query on the second system.local call after the cache is warmed. - PeersV2NodeRefreshIT: updated hasNodeRefreshQuery() to match on the stable WHERE-clause suffix rather than the SELECT * literal.
nikagra
force-pushed
the
driver-367-optimize-system-local-query
branch
from
086dad9 to
4a63624
Compare
dkropachev
approved these changes
Apr 3, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
SELECT *queries inDefaultTopologyMonitorwith dynamic column projections that only fetch the columns the driver actually reads.SELECT *is still issued to discover which columns the server actually has. The response column list is then intersected withLOCAL_COLUMNS_OF_INTEREST/PEERS_COLUMNS_OF_INTEREST/PEERS_V2_COLUMNS_OF_INTEREST— threeImmutableSet<String>constants listing every column the driver reads from each table. Subsequent queries use the resulting projected column list.intersectWithNeeded(List<String> serverColumns, ImmutableSet<String> needed)helper performs the intersection, preserving server-response order and silently dropping columns absent from the server (e.g. DSE-specific columns on non-DSE clusters, or version-specific columns).dse_version,graph,workload,workloads,server_id,storage_port,storage_port_ssl,jmx_port) are included in all*_COLUMNS_OF_INTERESTsets and are silently dropped by the intersection when the server does not expose them.resetColumnCaches()resets all three caches tonull, causing the next query to re-issueSELECT *and re-learn the projection (used on schema-change events).Motivation
SELECT * FROM system.local WHERE key='local'andSELECT * FROM system.peers*fetch 30–40 columns per row, many of which are never accessed by the topology monitor (e.g.key,bootstrapped,cluster_name,cql_version,supported_features). This causes unnecessary serialization on the server, wire transfer, and deserialization on the client — contributing to the session initialization slowdown reported in #282.Note:
tokensis still fetched — it is required for token-aware routing and is read bynodeInfoBuilder(). The optimization eliminates columns the driver never reads, not all large columns.SchemaAgreementCheckeralready follows the explicit-projection pattern. This PR applies the equivalent approach toDefaultTopologyMonitor, using a dynamic intersection rather than hard-coded query strings so that the projection adapts to whatever columns the server exposes.Testing
DefaultTopologyMonitorTesttests pass.ProtocolVersionMixedClusterITupdated to expect the 13-column projected query string that results from intersecting Simulacron's mock column list withLOCAL_COLUMNS_OF_INTEREST.Closes: DRIVER-367
Parent epic: DRIVER-274