Expand connection pool benchmark coverage for pool implementation A/B testing#4459
Open
mdaigle wants to merge 2 commits into
Open
Expand connection pool benchmark coverage for pool implementation A/B testing#4459mdaigle wants to merge 2 commits into
mdaigle wants to merge 2 commits into
Conversation
… testing Adds tooling and benchmarks to evaluate the new ChannelDbConnectionPool (UseConnectionPoolV2) against the legacy WaitHandleDbConnectionPool, aligned with the design goals in issue #3356 (parallel connection opening, async best practices / low threadpool pressure, minimal lock contention). - Add a process-level `UseConnectionPoolV2` config flag (runnerconfig.jsonc -> Config -> Program) that sets the AppContext switch once at startup. The pool implementation switch is read and cached when the first pool is created, so it cannot be toggled per iteration; comparing the two pools requires two runs (flag false, then true). This applies to every pool benchmark in the suite. - Add ConnectionPoolChurnRunner: single-threaded, no-contention open/close on a warm pool. Isolates raw per-checkout CPU and allocation cost (the low-noise counterpart to the parallel/contention runners). - Add ConnectionPoolContentionRunner: steady-state open/query/close with pure sync vs pure async workers and a large (no-wait) vs small (back-pressure) pool axis, focused on the waiter wake path and threadpool pressure. Leaves the existing ConnectionPoolStressRunner and ParallelAsyncConnectionRunner untouched; the new flag lets them all be run against either pool implementation. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Contributor
There was a problem hiding this comment.
Pull request overview
Expands the Microsoft.Data.SqlClient.PerformanceTests benchmark suite to better compare the legacy WaitHandleDbConnectionPool vs the new ChannelDbConnectionPool (via Switch.Microsoft.Data.SqlClient.UseConnectionPoolV2) by adding a process-level config flag and two additional pool-focused runners.
Changes:
- Adds
UseConnectionPoolV2torunnerconfig.jsonc+Configand wires it inProgramto set the AppContext switch at process startup. - Introduces
ConnectionPoolChurnRunnerfor single-thread warm-pool checkout/return cost measurement. - Introduces
ConnectionPoolContentionRunnerfor concurrent steady-state open/query/close throughput across sync vs async and pool size contention axes.
Reviewed changes
Copilot reviewed 5 out of 5 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| src/Microsoft.Data.SqlClient/tests/PerformanceTests/runnerconfig.jsonc | Adds UseConnectionPoolV2 flag and enables configs for the two new pool runners. |
| src/Microsoft.Data.SqlClient/tests/PerformanceTests/Program.cs | Sets UseConnectionPoolV2 AppContext switch early and registers the new runners. |
| src/Microsoft.Data.SqlClient/tests/PerformanceTests/Config/Config.cs | Adds a documented UseConnectionPoolV2 config field and runner job entries. |
| src/Microsoft.Data.SqlClient/tests/PerformanceTests/BenchmarkRunners/ConnectionPoolContentionRunner.cs | New benchmark runner for steady-state pool contention behavior under sync/async workloads. |
| src/Microsoft.Data.SqlClient/tests/PerformanceTests/BenchmarkRunners/ConnectionPoolChurnRunner.cs | New benchmark runner for low-noise single-thread warm-pool churn cost. |
Comment on lines
+107
to
+133
| var tasks = new Task[Parallelism]; | ||
| for (int i = 0; i < Parallelism; i++) | ||
| { | ||
| tasks[i] = Task.Run(async () => | ||
| { | ||
| for (int op = 0; op < OpsPerWorker; op++) | ||
| { | ||
| using var conn = new SqlConnection(_connectionString); | ||
|
|
||
| if (Async is AsyncBehavior.Async) | ||
| { | ||
| await conn.OpenAsync(); | ||
| using var cmd = conn.CreateCommand(); | ||
| cmd.CommandText = "SELECT 1"; | ||
| _ = await cmd.ExecuteScalarAsync(); | ||
| } | ||
| else | ||
| { | ||
| conn.Open(); | ||
| using var cmd = conn.CreateCommand(); | ||
| cmd.CommandText = "SELECT 1"; | ||
| _ = cmd.ExecuteScalar(); | ||
| } | ||
| // Dispose returns the connection to the pool. | ||
| } | ||
| }); | ||
| } |
Comment on lines
+138
to
+151
| private void WarmPool(int count) | ||
| { | ||
| var conns = new SqlConnection[count]; | ||
| for (int i = 0; i < count; i++) | ||
| { | ||
| conns[i] = new SqlConnection(_connectionString); | ||
| conns[i].Open(); | ||
| } | ||
| // Return them all to the pool so subsequent checkouts are served from idle. | ||
| for (int i = 0; i < count; i++) | ||
| { | ||
| conns[i].Close(); | ||
| } | ||
| } |
- Split SteadyStateOpenQueryClose into separate async/sync Task.Run paths (Task.Run(Func<Task>) vs Task.Run(Action)) so the sync branch does not create an async state machine, removing overhead unrelated to pool behavior. - Wrap WarmPool connections in try/finally and dispose each SqlConnection after closing to prevent retaining objects until GC each iteration, which would add allocation noise to the benchmark measurements. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Contributor
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 5 out of 5 changed files in this pull request and generated 1 comment.
Comments suppressed due to low confidence (1)
src/Microsoft.Data.SqlClient/tests/PerformanceTests/BenchmarkRunners/ConnectionPoolContentionRunner.cs:113
- In the async mode, wrapping each worker in Task.Run forces work onto the thread pool and adds scheduler/queueing overhead that can dominate the measurement and inflate ThreadingDiagnoser metrics (e.g., Completed Work Items). Since this runner is meant to highlight the async waiter/wake differences between pool implementations, it should start the async workers without Task.Run (e.g., an immediately-invoked async lambda) so the only thread-pool usage comes from actual async continuations.
if (Async is AsyncBehavior.Async)
{
tasks[i] = Task.Run(async () =>
{
Comment on lines
+85
to
+102
| [Benchmark] | ||
| public async Task RapidOpenCloseSingleThread() | ||
| { | ||
| for (int i = 0; i < OpsPerInvocation; i++) | ||
| { | ||
| using var conn = new SqlConnection(_connectionString); | ||
|
|
||
| if (Async is AsyncBehavior.Async) | ||
| { | ||
| await conn.OpenAsync(); | ||
| } | ||
| else | ||
| { | ||
| conn.Open(); | ||
| } | ||
| // Dispose returns the connection to the pool. | ||
| } | ||
| } |
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #4459 +/- ##
==========================================
- Coverage 65.83% 65.22% -0.62%
==========================================
Files 287 285 -2
Lines 43763 66887 +23124
==========================================
+ Hits 28812 43627 +14815
- Misses 14951 23260 +8309
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
paulmedynski
added
the
Area\Connection Pooling
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Builds on the connection pool benchmarks added in #4447 (now merged to
main) to make the suite useful for evaluating the newChannelDbConnectionPool(UseConnectionPoolV2) against the legacyWaitHandleDbConnectionPool, aligned with the design goals in #3356: parallel connection opening, async best practices (low threadpool pressure), and minimal lock contention.What's added
UseConnectionPoolV2config flag (runnerconfig.jsonc->Config->Program). It sets theSwitch.Microsoft.Data.SqlClient.UseConnectionPoolV2AppContext switch once at startup.false(legacy) thentrue(V2). This flag applies to every pool benchmark in the suite, including the existingConnectionPoolStressRunnerandParallelAsyncConnectionRunner.ConnectionPoolChurnRunner— single-threaded, no-contention open/close on a warm pool. Isolates the raw per-checkout CPU and allocation cost of the pool's acquire/return path (the low-noise counterpart to the parallel/contention runners; the most sensitive measure of per-op allocations).ConnectionPoolContentionRunner— steady-state open ->SELECT 1-> close with pure-sync vs pure-async workers and a large (no-wait) vs small (back-pressure) pool axis. Focuses on the waiter wake path and threadpool pressure (watchCompleted Work Items/Lock Contentions).The existing
ConnectionPoolStressRunnerandParallelAsyncConnectionRunnerare left untouched — the new flag simply lets them all be run against either pool implementation.Not micro-benchmarked
Rate limiting (#4396), pruning (#4304), shutdown drain (#4302), and connection resiliency (#4415) are time/behavior-based and belong in reliability tests (#3667), not hot-path microbenchmarks.
Validation
PerformanceTests(net8.0, Release) — 0 warnings / 0 errors.falseandtrue; V2 showed ~3.5-4.3x faster parallel burst-open and materially lower asyncCompleted Work Items, consistent with Feature | Redesign the SqlClient Connection Pool to Improve Performance and Async Support #3356's goals.Checklist
Related: #3356, #601, #979, #4447