Skip to content

Cleanup most usages of Windows file mapping#121896

Open
huoyaoyuan wants to merge 43 commits into
dotnet:mainfrom
huoyaoyuan:pal/mmap
Open

Cleanup most usages of Windows file mapping#121896
huoyaoyuan wants to merge 43 commits into
dotnet:mainfrom
huoyaoyuan:pal/mmap

Conversation

@huoyaoyuan

Copy link
Copy Markdown
Member

The remaining cases are superpmi (included in #121367) and PEImage, which involves LoadLibrary and we would want more future-proof abstraction.

All the cases included in this PR are transitively used by ilasm/ildasm. For stgio, I'm doing simple ifdef's on OS api, since it's too complicated around different I/O models, and we would get rid of it with new metadata library.

@github-actions github-actions Bot added the needs-area-label An area label is needed to ensure this gets routed to the appropriate area owners label Nov 22, 2025
@dotnet-policy-service dotnet-policy-service Bot added the community-contribution Indicates that the PR has been added by a community member label Nov 22, 2025
@huoyaoyuan huoyaoyuan added area-PAL-coreclr only for closed issues and removed needs-area-label An area label is needed to ensure this gets routed to the appropriate area owners labels Nov 22, 2025
Comment thread src/coreclr/inc/stresslog.h
Comment thread src/coreclr/minipal/dn-memmap.h Outdated
Comment thread src/coreclr/minipal/dn-memmap.h Outdated
Comment thread src/coreclr/minipal/Windows/memmap.cpp
Comment thread src/coreclr/md/inc/stgio.h Outdated
#include <sys/stat.h>

MemoryMappedFile* MemoryMappedFile::OpenImpl(const WCHAR* path, bool readWrite, uint32_t desiredSize, void* desiredAddress)
{

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there any concern with the path or desiredAddress being null? I'd assume that asserts are warranted. Same for Windows implementation.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Has it been a practice? Shouldn't the OS fs call reliably reject invalid string?

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

path is used below in u16_strlen. That doesn't and shouldn't check for NULL. Unclear on minipal_get_length_utf16_to_utf8.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added assert. For Windows version, it is directly passed to CreateFileW which should reject null.

{
size_t pathLen = u16_strlen(path);
size_t pathU8Len = minipal_get_length_utf16_to_utf8((CHAR16_T*)path, pathLen, 0);
char* pathU8 = new char[pathU8Len + 1];

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder if at all call sites the possibility of badalloc exception is correctly handled. It seems it may make sense to use new (nothrow) here and return nullptr if the allocation failed so that all the error paths are handled in a similar way.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This pattern has been used in various places. If we want to handle badalloc, it should probably be updated altogether.

@huoyaoyuan

Copy link
Copy Markdown
Member Author

All tests are passing now, and I should have responded to all comments.

Comment thread src/coreclr/minipal/Unix/CMakeLists.txt Outdated
dn-u16.cpp
dn-stdio.cpp
memory.cpp
dn-memmap.cpp

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A nit - I am not sure what the dn- filename prefix means. @AaronRobinsonMSFT I can see that you have originally added the dn-u16.cpp file with this naming convention, so you probably know the meaning.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The dn is for "dotnet". It is akin to the PAL_ prefix we've historically used and meant only to be used when needed to disambiguate something. I was hoping it would evolve into a pattern that was obvious. Unclear if that remains the case. I've no concern with dropping it at this point. I do find it useful for headers so I know where they are coming from, but for TUs it is probably unnecessary.

Comment thread src/coreclr/minipal/Windows/dn-memmap.cpp Outdated
Comment thread src/coreclr/tools/metainfo/mdobj.cpp Outdated

#ifdef MEMORY_MAPPED_STRESSLOG
static LPVOID CreateMemoryMappedFile(LPWSTR logFilename, size_t maxBytesTotal)
static LPVOID CreateMemoryMappedFile(LPWSTR logFilename, uint32_t maxBytesTotal)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why was this changed to uint32_t?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The caller already restricts its value to uint32. Doing the cast at caller making it more consistent with what's supported in MemoryMappedFile::Open.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seems to me that the new memory mapping APIs we are adding to minipal would be more future proof if they were not artificially limited to 4GB large files. While it is currently used only for max 4GB mappings, it may get used in the future for something else.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder whether it would be treated as a future-oriented API, or just temporary minimal adapters for migrating legacy components. For the use cases: ilasm/idasm is being migrated away, stgio is complicated from the old days and may go away with new DNMD, and tools like metainfo can simply switch to ReadAllBytes-like approach.

@JulieLeeMSFT
JulieLeeMSFT requested a review from janvorli June 22, 2026 18:43
@github-actions

github-actions Bot commented Jul 19, 2026

Copy link
Copy Markdown
Contributor

Workflow state for the Holistic Review Orchestrator.

{
  "version": 5,
  "last_dispatched_commit": "4260f9f37257066d74ff1ee91bfc1e60ef221b2f",
  "last_dispatched_base_ref": "main",
  "last_dispatched_base_sha": "b88ce03bc28304f2b6cfd27c29923e142a9040c6",
  "last_reviewed_commit": "4260f9f37257066d74ff1ee91bfc1e60ef221b2f",
  "last_reviewed_base_ref": "main",
  "last_reviewed_base_sha": "b88ce03bc28304f2b6cfd27c29923e142a9040c6",
  "last_recorded_worker_run_id": "29677690748",
  "review_attempt_commit": "",
  "review_attempt_base_ref": "",
  "review_attempt_count": 0,
  "max_review_attempts": 5,
  "review_history_format": "holistic-review-disclosure-v1",
  "review_history": [
    {
      "commit": "4260f9f37257066d74ff1ee91bfc1e60ef221b2f",
      "review_id": 4730527939
    }
  ]
}

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Holistic Review

Motivation: Reducing direct dependence on Win32 file-mapping APIs (CreateFileMapping/MapViewOfFile) in the ilasm/ildasm/metadata toolchain is a reasonable, well-scoped cleanup that advances the PAL-removal effort. The problem is real and the direction matches the stated future goal of a portable metadata library.

Approach: Introducing a small cross-platform MemoryMappedFile minipal abstraction plus HOST_WINDOWS/HOST_UNIX #ifdef branches in stgio.cpp is consistent with how this area already handles portability. However, several of the hand-written Unix branches contain correctness bugs (unsigned/ssize_t comparisons that mask errors and drop out-parameters), and one refactor silently drops PID-substitution behavior in the stress log. These need fixing before merge.

Summary: ⚠️ Needs Changes. The abstraction is sound and the mechanical conversions are mostly faithful, but there are concrete regressions: (1) ReadFromDisk can never observe a read failure and never updates *pcbRead; (2) WriteToDisk mishandles write errors/short writes and doesn't propagate the byte count; (3) the stress-log conversion passes the un-substituted filename, breaking per-process log paths. See inline comments and the additional observations below.


Detailed Findings

❌ Correctness — Unix ReadFromDisk/WriteToDisk error handling (inline)

See inline comments on src/coreclr/md/enc/stgio.cpp. Both use ULONG locals to hold the ssize_t result of read/write, so the -1 error sentinel wraps to UINT_MAX and the error/short-transfer paths behave incorrectly, and the actual transferred count is not written back to the caller's *pcbRead/*pcbWritten. The Windows branches propagate both the failure and the count.

❌ Correctness — Stress log drops PID substitution (inline)

See inline comment on src/coreclr/utilcode/stresslog.cpp line 196. logFilenameReplaced is still computed but no longer used; the original logFilename is passed to MemoryMappedFile::OpenRW.

⚠️ Cross-platform semantics — desiredAddress is only a hint on Unix

src/coreclr/minipal/Unix/memmap.cpp calls mmap(desiredAddress, ...) without MAP_FIXED, so desiredAddress is merely a hint and the mapping may land elsewhere with no error. The Windows path uses MapViewOfFileEx, which fails if the requested base address is unavailable. The stress-log caller passes MEMORY_MAPPED_STRESSLOG_BASE_ADDRESS and elsewhere assumes the header lives at that fixed base. If a fixed base is required for correctness on Unix, this needs MAP_FIXED (or an explicit post-check that the returned address matches); if the address is truly advisory, that difference from the Windows contract is worth a comment. Please confirm which behavior is intended.

⚠️ Resource/robustness — Unix memmap.cpp allocation and zero-length map

In OpenImpl the UTF-8 path buffer is allocated with new char[...] and freed with delete[] only on the success path before the goto Fail chain begins, which is fine, but note that if st.st_size == 0 in read mode, mmap with length 0 returns EINVAL and Open fails; the previous Windows code could map a zero-length file. Confirm no caller relies on mapping empty files (the stgio.cpp read path already rejects size 0, so this is likely fine for the metadata cases, but the standalone MemoryMappedFile::Open used by ildasm/ilasm/mdobj does not pre-check).

💡 Style — #else // empty comment

src/coreclr/md/inc/stgio.h: the #else // on the IsMemoryMapped() branch has an empty/typo comment where the sibling branches use // HOST_WINDOWS. Minor consistency nit.

✅ Mechanical conversions in consumers look faithful

The PELoader, MappedFileStream, and mdobj.cpp conversions to the new MemoryMappedFile*/CreateMappedFile abstraction correctly replace the manual CreateFile/CreateFileMapping/MapViewOfFile/UnmapViewOfFile/CloseHandle sequences with construct-and-delete, and the removed getHFile()/open(const char*)/GetMapViewOfFile members appear to have no remaining callers in the diff. The >4GB guards and the getFileSize() replacement for SafeGetFileSize are reasonable given the stated "no files >4GB" constraint.

Note

This review was generated by this repository's Holistic Review agentic workflow to complement the built-in Copilot review.

Generated by Holistic Review · 174.6 AIC · ⌖ 11.9 AIC · ⊞ 10K

Comment thread src/coreclr/md/enc/stgio.cpp Outdated
return (S_OK);
return (MapFileError(GetLastError()));
#else // HOST_WINDOWS
if ((cbRead = read(m_fd, pbBuff, cbBuff)) >= 0)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

read error is never detected, and *pcbRead is not updated.

cbRead is declared as ULONG (unsigned). read() returns ssize_t; on error it returns -1, which becomes UINT_MAX when stored in cbRead, so (cbRead = read(...)) >= 0 is always true and the error path (return HRESULTFromErr(errno);) is unreachable. The Windows branch reports failure via ReadFile, so this is a behavioral regression on Unix.

Additionally, the Windows ReadFile call writes the byte count into *pcbRead, but this branch only assigns the local cbRead and never propagates it to *pcbRead. Callers such as Read() do m_cbOffset += *pcbRead; after ReadFromDisk(..., pcbRead), so they consume a stale/uninitialized value.

Suggest capturing the result in a signed type and writing back, e.g.:

ssize_t cbReadActual = read(m_fd, pbBuff, cbBuff);
if (cbReadActual >= 0)
{
    *pcbRead = (ULONG)cbReadActual;
    return (S_OK);
}
return HRESULTFromErr(errno);

Comment thread src/coreclr/md/enc/stgio.cpp Outdated
hr = MapFileError(GetLastError());
#else // HOST_WINDOWS
_ASSERTE(m_fd != -1);
if ((cbWritten = write(m_fd, pbBuff, cbWrite)) != cbWrite)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Signedness/short-write mismatch in WriteToDisk. cbWritten is ULONG; write() returns ssize_t. On error write returns -1 which becomes UINT_MAX and compares unequal to cbWrite, so hr is set — but errno may not be meaningful in that comparison path, and more importantly a partial write (a legitimate short write) is treated as an error while *pcbWritten is left holding the wrong value. The Windows branch propagates the actual written count via WriteFile's pcbWritten. Consider assigning the actual count back into *pcbWritten and using a signed temporary for the comparison so a genuine error (-1) is distinguished from a short write.

Comment thread src/coreclr/utilcode/stresslog.cpp Outdated

return MapViewOfFileEx(hMap, FILE_MAP_ALL_ACCESS, 0, 0, fileSize, MEMORY_MAPPED_STRESSLOG_BASE_ADDRESS);
// The memory map will be released at process exit
MemoryMappedFile* mmap = MemoryMappedFile::OpenRW(logFilename, maxBytesTotal, MEMORY_MAPPED_STRESSLOG_BASE_ADDRESS);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PID substitution is dropped — the wrong filename is used. The code still computes logFilenameReplaced via ReplacePid(logFilename, logFilenameReplaced, MAX_PATH) on the line above, but this call passes the original logFilename to MemoryMappedFile::OpenRW instead of logFilenameReplaced. The previous Windows code passed logFilenameReplaced to WszCreateFile. As a result the %p/PID token in the configured stress-log path is no longer expanded, so all processes collide on the same unexpanded filename (and logFilenameReplaced becomes a dead variable). Pass logFilenameReplaced here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area-PAL-coreclr only for closed issues community-contribution Indicates that the PR has been added by a community member

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants