Fix segfault in GC standalone large pages emulation mode (segments path) by cshung · Pull Request #127763 · dotnet/runtime

cshung · 2026-05-04T15:26:20Z

In the non-regions (segments) path, GCLargePages=2 (emulation mode) was not handled by virtual_alloc. The function only had a bool use_large_pages_p parameter, so it could not distinguish between real large pages (VirtualReserveAndCommitLargePages) and emulation mode (which needs VirtualReserve + VirtualCommit). In emulation mode, memory was reserved but never committed, causing a segfault on access.

Fix: Change virtual_alloc to accept an int large_page_config parameter:

0 = no large pages (reserve only)
1 = real large pages (VirtualReserveAndCommitLargePages)
2 = emulation (VirtualReserve + VirtualCommit)

The caller computes large_page_config once and passes it through reserve_initial_memory, eliminating the need for reserve_initial_memory to access the static large_pages_emulation_mode_p field.

The test is updated to use per-object-heap hard limits (GCHeapHardLimitSOH/LOH/POH), which are required for the segments path with large pages since each segment is sized to the full hard limit (soh + loh + poh = 3x limit with a single combined GCHeapHardLimit).

Fixes #127668

dotnet-policy-service · 2026-05-04T15:27:44Z

Tagging subscribers to this area: @JulieLeeMSFT, @dotnet/gc
See info in area-owners.md if you want to be subscribed.

mangod9 · 2026-05-05T21:38:50Z

I am not convinced whether we should do this for segments. In fact we should be looking at not shipping clrgc.dll given that regions have been live for 3+ releases already

cshung · 2026-05-06T17:57:33Z

I am not convinced whether we should do this for segments. In fact we should be looking at not shipping clrgc.dll given that regions have been live for 3+ releases already

I think we should still maintain this path for two reasons:

The segments path (#ifndef USE_REGIONS) is not just about standalone GC — it's the codepath 32-bit systems use in production. Standalone GC is currently the mechanism that exercises this path in CI. If we stop maintaining it, we're effectively letting the 32-bit segments codepath rot without any test coverage.
Even setting aside 32-bit, standalone GC provides a valuable vehicle for flexible testing and experimentation. We may not want to ship it, but that doesn't mean it isn't worth keeping functional.

The fix itself is small and straightforward — it passes the large page config as an integer through virtual_alloc so the emulation mode does Reserve+Commit correctly on both the regions and segments paths.

mangod9 · 2026-05-06T19:03:17Z

is the x86 case broken due to the new changes here?

cshung · 2026-05-06T21:50:28Z

is the x86 case broken due to the new changes here?

No, x86 is good:

x86 is not broken before this PR (large pages are not supported on x86, anything broken because of large page does not impact x86).
x86 is not broken after this PR either (this PR impacts large page only)

janvorli · 2026-06-08T11:46:43Z

@cshung, I'd prefer disabling the large page emulation for segments instead. As Manish mentioned, we only use segments mode on 32 bit systems and as you've also mentioned, for those, the large pages are not enabled and don't make sense.

The emulated large pages are a great tool for testing large pages during GC development and in the PR, but I don't see it as something customers should use. So this mode really seems to have benefit only when regions are enabled.

cshung · 2026-06-12T15:58:29Z

@cshung, I'd prefer disabling the large page emulation for segments instead. As Manish mentioned, we only use segments mode on 32 bit systems and as you've also mentioned, for those, the large pages are not enabled and don't make sense.

The emulated large pages are a great tool for testing large pages during GC development and in the PR, but I don't see it as something customers should use. So this mode really seems to have benefit only when regions are enabled.

I think this is the right fix, but if we wanted to simply disable the test, I am happy with that too.

Without this test though, segments + 64 bits + large page is functional but untested. That being said, it is a niche scenario, and it is just what it was, we didn't make it worse.

janvorli · 2026-06-12T16:45:02Z

I think this is the right fix, but if we wanted to simply disable the test, I am happy with that too.

I didn't mean disabling the test, but rather disabling emulated large pages support for the segments.

janvorli

LGTM, thank you!

Change virtual_alloc to accept an integer large_pages_config parameter (0=none, 1=real large pages, 2=emulation) instead of a boolean. When config=2, use VirtualReserve followed by VirtualCommit to emulate large page behavior without requiring OS large page privileges. Update the test to use per-object-heap hard limits which are required for the segments path with large pages.

This reverts commit 0aaa427.

Large page emulation mode (GCLargePages=2) is not supported on the segments path. When emulation is requested on segments, silently disable large pages instead of crashing with a segfault due to memory being reserved but never committed. Real large pages (GCLargePages=1) continue to work on segments as before. Fixes dotnet#127668 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

janvorli · 2026-06-17T08:19:15Z

/ba-g the failures are timeouts that happen in many if not all PRs, unrelated to this change.

…th) (#127763) In the non-regions (segments) path, `GCLargePages=2` (emulation mode) was not handled by `virtual_alloc`. The function only had a `bool use_large_pages_p` parameter, so it could not distinguish between real large pages (`VirtualReserveAndCommitLargePages`) and emulation mode (which needs `VirtualReserve` + `VirtualCommit`). In emulation mode, memory was reserved but never committed, causing a segfault on access. **Fix:** Change `virtual_alloc` to accept an `int large_page_config` parameter: - `0` = no large pages (reserve only) - `1` = real large pages (`VirtualReserveAndCommitLargePages`) - `2` = emulation (`VirtualReserve` + `VirtualCommit`) The caller computes `large_page_config` once and passes it through `reserve_initial_memory`, eliminating the need for `reserve_initial_memory` to access the static `large_pages_emulation_mode_p` field. The test is updated to use per-object-heap hard limits (`GCHeapHardLimitSOH`/`LOH`/`POH`), which are required for the segments path with large pages since each segment is sized to the full hard limit (soh + loh + poh = 3x limit with a single combined `GCHeapHardLimit`). Fixes #127668 --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

github-actions Bot added the area-GC-coreclr label May 4, 2026

dotnet-policy-service Bot added the community-contribution Indicates that the PR has been added by a community member label May 4, 2026

This was referenced May 4, 2026

Checkout failure: "Git fetch failed with exit code 128" dotnet/arcade#9009

Open

The Operation will be canceled. The next steps may not contain expected logs. dotnet/dnceng#3008

Open

janvorli reviewed May 4, 2026

View reviewed changes

Comment thread src/coreclr/gc/regions_segments.cpp Outdated

janvorli reviewed May 4, 2026

View reviewed changes

Comment thread src/coreclr/gc/init.cpp Outdated

cshung force-pushed the fix/gc-standalone-largepages-segments branch from 86e5612 to 0aaa427 Compare May 5, 2026 21:36

janvorli approved these changes Jun 15, 2026

View reviewed changes

cshung and others added 3 commits June 16, 2026 09:52

Revert "Fix GC standalone large pages emulation in segments path"

3884ff3

This reverts commit 0aaa427.

cshung force-pushed the fix/gc-standalone-largepages-segments branch from c3a2874 to 973587c Compare June 16, 2026 16:52

Merge branch 'main' into fix/gc-standalone-largepages-segments

f014c1f

janvorli merged commit 1af2b9f into dotnet:main Jun 17, 2026
109 of 113 checks passed

cshung deleted the fix/gc-standalone-largepages-segments branch June 17, 2026 14:34

dotnet-maestro Bot mentioned this pull request Jun 18, 2026

[main] Source code updates from dotnet/runtime dotnet/dotnet#7280

Merged

dotnet-milestone-bot Bot added this to the 11.0-preview6 milestone Jun 18, 2026

github-actions Bot locked and limited conversation to collaborators Jul 19, 2026

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Fix segfault in GC standalone large pages emulation mode (segments path) - #127763

Fix segfault in GC standalone large pages emulation mode (segments path)#127763
janvorli merged 4 commits into
dotnet:mainfrom
cshung:fix/gc-standalone-largepages-segments

cshung commented May 4, 2026 •

edited

Loading

Uh oh!

dotnet-policy-service Bot commented May 4, 2026

Uh oh!

Uh oh!

Uh oh!

mangod9 commented May 5, 2026

Uh oh!

cshung commented May 6, 2026

Uh oh!

mangod9 commented May 6, 2026

Uh oh!

cshung commented May 6, 2026

Uh oh!

janvorli commented Jun 8, 2026

Uh oh!

cshung commented Jun 12, 2026

Uh oh!

janvorli commented Jun 12, 2026

Uh oh!

janvorli left a comment

Uh oh!

janvorli commented Jun 17, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Uh oh!

Conversation

cshung commented May 4, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

dotnet-policy-service Bot commented May 4, 2026

Uh oh!

Uh oh!

Uh oh!

mangod9 commented May 5, 2026

Uh oh!

cshung commented May 6, 2026

Uh oh!

mangod9 commented May 6, 2026

Uh oh!

cshung commented May 6, 2026

Uh oh!

janvorli commented Jun 8, 2026

Uh oh!

cshung commented Jun 12, 2026

Uh oh!

janvorli commented Jun 12, 2026

Uh oh!

janvorli left a comment

Choose a reason for hiding this comment

Uh oh!

janvorli commented Jun 17, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

cshung commented May 4, 2026 •

edited

Loading