perf: reduce HashMap/collection allocation overhead in gateway path by xinlian12 · Pull Request #48662 · Azure/azure-sdk-for-java

xinlian12 · 2026-04-01T16:24:59Z

Performance: Reduce HashMap/Collection Allocation Overhead in Gateway Path

Motivation

JFR profiling of the baseline (main) under high-concurrency gateway workloads revealed that HashMap-related allocations (HashMap$Node, HashMap, HashMap$ValueIterator) and HTTP header collections (DefaultHeaders$HeaderEntry, HttpHeader) are responsible for a significant share of total object allocation churn.

Baseline JFR allocation profile (c128 Read HTTP/1, ObjectAllocationSample, 10-min recording):

Class	% of Total Allocation
`HashMap$Node`	6.9%
`DefaultHeaders$HeaderEntry`	6.8%
`HashMap$ValueIterator`	1.3%
`HttpHeader`	0.9%
`HashMap`	0.7%
`HttpHeaders`	0.6%
`HashMap$Node[]`	0.5%
Total targeted	~10.9%

Root causes:

HashMap<>() default initial capacity (16) forces 1-2 resize+rehash cycles for typical gateway responses with 20-30 headers, creating throwaway HashMap$Node[] arrays and re-hashed HashMap$Node entries
StoreResponse constructor converts HttpHeaders to Map via HttpUtils.asMap() on every response, allocating a throwaway HashMap$ValueIterator and rebuilding all HashMap$Node entries
HttpHeaders in RxGatewayStoreModel.getHttpRequestHeaders() is undersized, causing internal HashMap resize
Redundant toLowerCase() calls on header keys that are already normalized

Changes

Right-sized HashMap initial capacity: HashMap<>(32) instead of HashMap<>() in RxDocumentServiceRequest, and mapCapacityForSize() helper in HttpUtils to avoid rehashing
Eliminate HashMap to HttpHeaders to HashMap round-trip: StoreResponse now accepts HttpHeaders directly, removing intermediate asMap() conversion that created throwaway HashMap$ValueIterator and HashMap$Node arrays
Pre-sized HttpHeaders in RxGatewayStoreModel: sized to defaultHeaders.size() + headers.size() to avoid internal HashMap resize
Remove redundant toLowerCase() calls: HttpHeaders.set() already normalizes keys; callers no longer double-normalize creating extra String objects

Benchmark Results

Test matrix: 1-tenant {c1, c8, c32, c128} {HTTP/1, HTTP/2} + 30-tenant c3 {HTTP/1, HTTP/2}, GATEWAY mode, ReadThroughput, 10 min per round, 3 independent JVM rounds per config. All endpoints pinned to the same Cosmos gateway frontend via /etc/hosts to eliminate Azure Traffic Manager routing variance. Both main and hashmap-alloc tested on the same VM per config.

1-Tenant Throughput & Latency

Config	Concurrency	Protocol	main (ops/s)	PR (ops/s)	╬ö ops	main p99 (ms)	PR p99 (ms)	main mean (ms)	PR mean (ms)
Read	c1	HTTP/1	444	445	+0.4%	5.57	5.47	2.24	2.26
Read	c1	HTTP/2	450	448	-0.4%	5.42	5.45	2.21	2.23
Read	c8	HTTP/1	3,714	3,711	-0.1%	5.13	5.04	2.13	2.12
Read	c8	HTTP/2	3,702	3,704	+0.0%	5.15	5.13	2.13	2.12
Read	c32	HTTP/1	15,137	15,190	+0.4%	4.70	4.70	2.07	2.06
Read	c32	HTTP/2	15,537	15,525	-0.1%	4.70	4.70	2.03	2.04
Read	c128	HTTP/1	50,220	50,519	+0.6%	5.39	5.39	2.44	2.43
Read	c128	HTTP/2	50,533	50,553	+0.0%	5.38	5.40	2.44	2.44

Note: p99 is the cumulative operation latency (SDK processing + network + server). The histogram uses fixed bucket boundaries (~0.26ms granularity at this range), so p99 differences < 0.3ms are bucket noise, not real latency changes. The mean latency confirms no change.

30-Tenant Throughput & Latency

Config	Concurrency	Protocol	main (ops/s)	PR (ops/s)	╬ö ops	main p99 (ms)	PR p99 (ms)	main mean (ms)	PR mean (ms)
Read	c330t	HTTP/1	1,250	1,256	+0.5%	5.48	5.52	2.35	2.35
Read	c330t	HTTP/2	1,241	1,255	+1.1%	5.56	5.51	2.38	2.34

JFR Allocation Comparison (c128 Read HTTP/1, ObjectAllocationSample)

Class	main %	PR %	Change
`HashMap$Node`	8.1%	6.8%	-1.3pp (-16%)
`DefaultHeaders$HeaderEntry`	4.2%	4.0%	-0.2pp
`HashMap$Node[]`	3.9%	3.0%	-0.9pp (-22%)
`HttpHeader`	1.7%	1.7%
`HeadersUtils$StringEntry`	1.5%	1.6%
`HashMap`	1.4%	1.4%
`LinkedHashMap$Entry`	1.1%	1.3%
`IdentityHashMap$ValueIterator`	0.1%	0.0%	eliminated
Total targeted	~22.0%	~19.8%	-2.2pp

The primary allocation reductions are in HashMap$Node (-16%) and HashMap$Node[] (-22%), reflecting the elimination of resize/rehash cycles from right-sized initial capacity. The IdentityHashMap$ValueIterator is eliminated by removing the asMap() round-trip in StoreResponse.

Per-Round Detail

Click to expand per-round throughput (ops/s)

VM1 (c1, c8, c32-http1):

Config	main r1	main r2	main r3	PR r1	PR r2	PR r3
c1-http1	446	438	446	444	448	444
c1-http2	448	453	448	444	451	448
c8-http1	3,719	3,730	3,693	3,718	3,701	3,714
c8-http2	3,692	3,691	3,724	3,699	3,702	3,709
c32-http1	15,079	15,214	15,117	15,170	15,200	15,200

VM2 (c32-http2, c128, 30t):

Config	main r1	main r2	main r3	PR r1	PR r2	PR r3
c32-http2	15,505	15,528	15,579	15,422	15,621	15,533
c128-http1	49,994	50,203	50,465	50,085	50,200	51,271
c128-http2	50,069	50,850	50,680	49,858	51,265	50,536
30t-c3-http1	1,255	1,253	1,242	1,252	1,263	1,253
30t-c3-http2	1,233	1,244	1,245	1,254	1,254	1,258

All CVs 1.4% across both refs, confirming stable measurements with pinned endpoints.

Conclusion

Throughput: neutral across all 10 configs deltas range from -0.4% to +1.1%, all within measurement noise
Latency: p99 and mean latency unchanged across all configs (differences < histogram bucket granularity)
Allocation efficiency: HashMap$Node -16%, HashMap$Node[] -22%, IdentityHashMap$ValueIterator eliminated at c128
No regression at any concurrency level (c1 through c128) or protocol (HTTP/1, HTTP/2)
30-tenant multi-account: neutral to slightly positive (+0.5% to +1.1%)
The changes remove unnecessary allocation overhead without throughput regression

Eliminate per-response intermediate HashMap allocation by adding a new StoreResponse constructor that accepts HttpHeaders directly. Header names and values are populated into String[] arrays without materializing an intermediate Map. The JsonNodeStorePayload is updated to accept header arrays and only builds a Map lazily on error paths (extremely rare). Pre-size HashMaps throughout the hot path to avoid resize/rehash: - HttpHeaders request construction: sized to defaultHeaders + request headers - StoreResponse.replicaStatusList: pre-sized to 4 - StoreResponse.withRemappedStatusCode: pre-sized to header count - RxDocumentServiceRequest fallback maps: pre-sized to 32 Fix HttpUtils.asMap() double-allocation by iterating HttpHeaders directly instead of calling toMap() which creates an intermediate HashMap. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

…ve null-guard inconsistency - Fix HashMap<>(4) to HashMap<>(6) for replicaStatusList to avoid rehash at 4 replicas (capacity 4 * 0.75 = threshold 3, resizes on 4th insert) - Refactor JsonNodeStorePayload: extract shared parseJson() method with Supplier<Map<String,String>> to eliminate duplicated error-handling logic - Remove misleading null ternary in getHttpRequestHeaders() since getHeaders() always returns non-null (fallback HashMap<>(32)) - Revert HashMap<>(16) to HashMap<>() in HttpHeaders default constructor (16 is already the default capacity, change was no-op noise) - Add unit tests for StoreResponse HttpHeaders constructor, HttpHeaders populateLowerCaseHeaders, and JsonNodeStorePayload array-header constructor Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Fix HashMap<>(headers.size()) in HttpUtils.asMap() to account for the 0.75 load factor, avoiding resize when all headers are inserted. Extract mapCapacityForSize(int) helper in HttpUtils to consolidate the capacity calculation (n * 4 / 3 + 1) used across HttpUtils.asMap(), StoreResponse.withRemappedStatusCode(), and JsonNodeStorePayload.buildHeaderMap(). Addresses review feedback from alzimmermsft. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

…ize to HttpHeaders, clarify lowercase key guarantee - Add comments explaining why HashMap<>(32) is used as fallback in RxDocumentServiceRequest: capacity 32 gives threshold 24, covering typical 15-20 request headers without resize. - Apply HttpUtils.mapCapacityForSize() in RxGatewayStoreModel.getHttpRequestHeaders() to account for 0.75 load factor when constructing HttpHeaders. - Make mapCapacityForSize() public so it can be used from other packages. - Document in populateLowerCaseHeaders() Javadoc that keys are guaranteed lowercase because HttpHeaders.set() stores them via toLowerCase(Locale.ROOT). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Extract duplicate contentStream/payload handling from both StoreResponse constructors into a shared parseResponsePayload() static helper method. Both constructors now use the array-based JsonNodeStorePayload constructor, eliminating code duplication while preserving identical behavior. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

- Remove both unescape(Set<Entry>) and unescape(Map) overloads from HttpUtils as they are no longer needed - Update ResponseUtils to use the HttpHeaders-based StoreResponse constructor (same optimization as RxGatewayStoreModel) - Remove unescape test from HttpUtilsTest, keep asMap() coverage - Clean up unused imports (AbstractMap, ArrayList, List, Set) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Replace previous benchmark charts with comprehensive V3 analysis: - 20 configs (c1/c8/c16/c32/c128 x Read/Write x HTTP1/HTTP2) - 3 rounds each, 10 min/run, GATEWAY mode - Timeline charts with throughput and P99 latency - JFR allocation breakdown comparison - Detailed per-round analysis of outlier patterns Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

JFR ObjectAllocationSample weight = estimated cumulative bytes allocated over the recording (10 min), not heap residency. Heap was 8 GB committed. The ~271 GB 'targeted' is allocation throughput (~4 GB/s alloc rate), most objects immediately GC'd. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

- Replace confusing cumulative GB with allocation share % - Add GC comparison table (817 vs 813 pauses - identical) - Frame as code efficiency improvement, not GC impact Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

…arison - Remove timeline charts (not needed for review) - Add variance analysis using request-level metrics (transitTime) showing variance is server-side, not SDK-related - Add JFR allocation comparison for all 20 configs - Keep summary bar chart and c128 JFR chart Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

The c16 PR showing HashMap\ is from the request-sending path (ReactorNettyClient.bodySendDelegate iterating request headers), NOT the response-side asMap() iterator we eliminated. Added clarifying note and removed the per-config ValueIterator column (too noisy). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

…a-jackson.version The recent Jackson dependency update (8a671dd) bumped Jackson from 2.18.4 to 2.18.6 in all Cosmos Spark child modules but missed updating the scala-jackson.version property in the parent POM. This caused the maven-enforcer-plugin BannedDependencies rule to reject jackson-module-scala_2.12 and _2.13 at version 2.18.6. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

…m-main

…to upstream-main

…om/xinlian12/azure-sdk-for-java into perf/hashmap-collection-allocation

azure-pipelines · 2026-04-14T22:40:08Z

Azure Pipelines successfully started running 1 pipeline(s).

FabianMeiswinkel

LGTM

Copilot

Pull request overview

Note

Copilot was unable to run its full agentic suite in this review.

Optimizes the Cosmos gateway hot path by reducing temporary HashMap/header-collection allocations and removing redundant header normalization.

Changes:

Adds capacity sizing helper (HttpUtils.mapCapacityForSize) and applies pre-sizing to avoid HashMap rehash/resizes.
Changes StoreResponse to accept HttpHeaders directly and avoids HttpHeaders -> Map -> HttpHeaders round-trips.
Adds/updates unit tests for the new header-array population path and new constructors.

Reviewed changes

Copilot reviewed 11 out of 11 changed files in this pull request and generated 6 comments.

Show a summary per file

File	Description
sdk/cosmos/azure-cosmos/src/main/java/com/azure/cosmos/implementation/http/HttpHeaders.java	Adds `populateLowerCaseHeaders` to extract headers without building an intermediate map.
sdk/cosmos/azure-cosmos/src/main/java/com/azure/cosmos/implementation/directconnectivity/StoreResponse.java	Adds `HttpHeaders` ctor and factors payload parsing/stream close into a helper; tweaks map sizing.
sdk/cosmos/azure-cosmos/src/main/java/com/azure/cosmos/implementation/directconnectivity/ResponseUtils.java	Routes response creation through the new `StoreResponse(HttpHeaders, ...)` ctor.
sdk/cosmos/azure-cosmos/src/main/java/com/azure/cosmos/implementation/directconnectivity/JsonNodeStorePayload.java	Adds ctor that consumes header arrays and lazily builds a map only on error paths.
sdk/cosmos/azure-cosmos/src/main/java/com/azure/cosmos/implementation/directconnectivity/HttpUtils.java	Adds `mapCapacityForSize`, updates `asMap`, and removes `unescape` APIs.
sdk/cosmos/azure-cosmos/src/main/java/com/azure/cosmos/implementation/RxGatewayStoreModel.java	Avoids lowercasing/map conversions; pre-sizes `HttpHeaders` for request headers.
sdk/cosmos/azure-cosmos/src/main/java/com/azure/cosmos/implementation/RxDocumentServiceRequest.java	Pre-sizes request headers map to 32 to reduce resizes.
sdk/cosmos/azure-cosmos-tests/src/test/java/com/azure/cosmos/implementation/http/HttpHeadersTests.java	Adds unit tests for `populateLowerCaseHeaders`.
sdk/cosmos/azure-cosmos-tests/src/test/java/com/azure/cosmos/implementation/directconnectivity/StoreResponseTest.java	Adds unit tests covering new `StoreResponse(HttpHeaders, ...)` behavior.
sdk/cosmos/azure-cosmos-tests/src/test/java/com/azure/cosmos/implementation/directconnectivity/JsonNodeStorePayloadTests.java	Adds unit tests for new header-array `JsonNodeStorePayload` constructor.
sdk/cosmos/azure-cosmos-tests/src/test/java/com/azure/cosmos/implementation/directconnectivity/HttpUtilsTest.java	Removes coverage for deleted `unescape(Map)` API.

…to upstream-main

1. HttpHeaders.populateLowerCaseHeaders: Add argument validation - reject null arrays and enforce minimum length (IllegalArgumentException). 2. StoreResponse OWNER_FULL_NAME comparison: Use equalsIgnoreCase() for defensive case-insensitive matching against the lowercase header key. 3. HttpUtils.mapCapacityForSize: Guard against overflow for large expectedSize values; clamp to Integer.MAX_VALUE and handle <= 0 input. 4. HttpHeadersTests: Use toLowerCase(Locale.ROOT) for locale-stable assertions. 5. StoreResponse: Extract magic number 6 into named constant REPLICA_STATUS_MAP_INITIAL_CAPACITY with documenting comment. 6. StoreResponseTest: Add httpHeadersConstructorDecodesOwnerFullName test verifying URL-encoded OWNER_FULL_NAME header is decoded via the HttpHeaders constructor path.

xinlian12 · 2026-04-21T21:34:46Z

/azp run java - cosmos - tests

azure-pipelines · 2026-04-21T21:35:09Z

Azure Pipelines successfully started running 1 pipeline(s).

…to upstream-main

xinlian12 · 2026-04-24T03:42:33Z

/azp run java - cosmos - tests

azure-pipelines · 2026-04-24T03:42:55Z

Azure Pipelines successfully started running 1 pipeline(s).

xinlian12 · 2026-04-24T04:04:00Z

/azp run java - cosmos - spark

xinlian12 · 2026-04-24T04:04:05Z

/azp run java - cosmos - kafka

azure-pipelines · 2026-04-24T04:04:10Z

Azure Pipelines successfully started running 1 pipeline(s).

azure-pipelines · 2026-04-24T04:04:17Z

Azure Pipelines successfully started running 1 pipeline(s).

xinlian12 · 2026-04-24T04:04:38Z

/azp run java - spring - ci

azure-pipelines · 2026-04-24T04:04:47Z

Azure Pipelines could not run because the pipeline triggers exclude this branch/path.

FabianMeiswinkel

LGTM

xinlian12 · 2026-04-24T22:22:39Z

/check-enforcer override

github-actions Bot added the Cosmos label Apr 1, 2026

Annie Liang and others added 2 commits April 1, 2026 09:53

docs: add benchmark comparison charts for hashmap-collection-allocation

10d1b5e

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

docs: update benchmark charts with corrected per-interval CPU rendering

86ee3c0

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

alzimmermsft reviewed Apr 1, 2026

View reviewed changes

Comment thread ...azure-cosmos/src/main/java/com/azure/cosmos/implementation/directconnectivity/HttpUtils.java Outdated

xinlian12 mentioned this pull request Apr 2, 2026

perf: eliminate UnmodifiableList virtual dispatch overhead in LocationCache #48674

Closed

xinlian12 and others added 3 commits April 4, 2026 22:02

Merge branch 'upstream-main' into perf/hashmap-collection-allocation

5296c02

xinlian12 commented Apr 5, 2026

View reviewed changes

Comment thread ...mos/azure-cosmos/src/main/java/com/azure/cosmos/implementation/RxDocumentServiceRequest.java

xinlian12 commented Apr 5, 2026

View reviewed changes

Comment thread sdk/cosmos/azure-cosmos/src/main/java/com/azure/cosmos/implementation/RxGatewayStoreModel.java Outdated

xinlian12 commented Apr 5, 2026

View reviewed changes

Comment thread ...mos/azure-cosmos/src/main/java/com/azure/cosmos/implementation/RxDocumentServiceRequest.java

xinlian12 commented Apr 5, 2026

View reviewed changes

Comment thread sdk/cosmos/azure-cosmos/src/main/java/com/azure/cosmos/implementation/http/HttpHeaders.java

xinlian12 commented Apr 6, 2026

View reviewed changes

Comment thread ...e-cosmos/src/main/java/com/azure/cosmos/implementation/directconnectivity/StoreResponse.java

xinlian12 and others added 15 commits April 6, 2026 14:14

Merge branch 'upstream-main' into perf/hashmap-collection-allocation

bf3457b

Merge branch 'fix/cosmos-spark-jackson-version-mismatch' into upstrea…

7677a27

…m-main

Merge branch 'main' of https://github.com/Azure/azure-sdk-for-java in…

d205163

…to upstream-main

Merge branch 'main' of https://github.com/Azure/azure-sdk-for-java in…

73308ae

…to upstream-main

Merge branch 'upstream-main' into perf/hashmap-collection-allocation

d6f0130

Merge branch 'perf/hashmap-collection-allocation' of https://github.c…

05ea658

…om/xinlian12/azure-sdk-for-java into perf/hashmap-collection-allocation

change

8743b22

FabianMeiswinkel approved these changes Apr 15, 2026

View reviewed changes

FabianMeiswinkel marked this pull request as ready for review April 15, 2026 10:32

FabianMeiswinkel requested review from a team and kirankumarkolli as code owners April 15, 2026 10:32

Copilot AI review requested due to automatic review settings April 15, 2026 10:32

Copilot AI reviewed Apr 15, 2026

View reviewed changes

Copilot started reviewing on behalf of FabianMeiswinkel April 15, 2026 10:45 View session

xinlian12 added 6 commits April 15, 2026 08:35

Merge branch 'main' of https://github.com/Azure/azure-sdk-for-java in…

0b15550

…to upstream-main

Merge branch 'main' of https://github.com/Azure/azure-sdk-for-java in…

a696f39

…to upstream-main

Merge branch 'main' of https://github.com/Azure/azure-sdk-for-java in…

b46387f

…to upstream-main

Merge branch 'main' of https://github.com/Azure/azure-sdk-for-java in…

9cb2f83

…to upstream-main

Merge branch 'upstream-main' into perf/hashmap-collection-allocation

925ac82

xinlian12 added 2 commits April 23, 2026 13:22

Merge branch 'main' of https://github.com/Azure/azure-sdk-for-java in…

c97d58c

…to upstream-main

Merge branch 'upstream-main' into perf/hashmap-collection-allocation

410e559

FabianMeiswinkel approved these changes Apr 24, 2026

View reviewed changes

xinlian12 merged commit 630c930 into Azure:main Apr 24, 2026
116 of 118 checks passed

Uh oh!

Conversation

xinlian12 commented Apr 1, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Performance: Reduce HashMap/Collection Allocation Overhead in Gateway Path

Motivation

Changes

Benchmark Results

1-Tenant Throughput & Latency

30-Tenant Throughput & Latency

JFR Allocation Comparison (c128 Read HTTP/1, ObjectAllocationSample)

Per-Round Detail

Conclusion

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

azure-pipelines Bot commented Apr 14, 2026

Uh oh!

FabianMeiswinkel left a comment

Choose a reason for hiding this comment

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

xinlian12 commented Apr 21, 2026

Uh oh!

azure-pipelines Bot commented Apr 21, 2026

Uh oh!

xinlian12 commented Apr 24, 2026

Uh oh!

azure-pipelines Bot commented Apr 24, 2026

Uh oh!

xinlian12 commented Apr 24, 2026

Uh oh!

xinlian12 commented Apr 24, 2026

Uh oh!

azure-pipelines Bot commented Apr 24, 2026

Uh oh!

azure-pipelines Bot commented Apr 24, 2026

Uh oh!

xinlian12 commented Apr 24, 2026

Uh oh!

azure-pipelines Bot commented Apr 24, 2026

Uh oh!

FabianMeiswinkel left a comment

Choose a reason for hiding this comment

Uh oh!

xinlian12 commented Apr 24, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

xinlian12 commented Apr 1, 2026 •

edited

Loading