Fix intermittent OSX CI failures in StartsWithNoMatch_StringComparison and EndsWithNoMatch_StringComparison by Copilot · Pull Request #125963 · dotnet/runtime

Copilot · 2026-03-23T14:30:27Z

Apple's NSString compare:options:range:locale: is non-deterministic for certain Unicode inputs (C1 control chars \u0080–\u009F), causing StartsWithNoMatch_StringComparison and EndsWithNoMatch_StringComparison to fail intermittently on macOS. The existing [ActiveIssue] only covered IsAppleMobile (iOS/tvOS), leaving macOS unguarded.

Changes

StartsWithNoMatch_StringComparison: Add [ActiveIssue(..., IsOSX)] referencing issue [Apple-mobile] System.Tests.StringTests.[Starts,Ends]WithNoMatch_StringComparison test failure #108832; replace four separate Assert.Equal calls with a foreach loop using Assert.True(expected == actual, message) that captures length, mismatchIndex, comp, char values at the mismatch index, and the expected/actual boolean values for easier debugging when re-enabled.
EndsWithNoMatch_StringComparison: Same fix — same root cause (same Apple native API path), same [ActiveIssue] gap, same diagnostic improvement applied consistently.

The diagnostic message format:

String.StartsWith: length=3, mismatchIndex=1, comp=CurrentCulture, chars=(2,3), expected=True, actual=False
Span.StartsWith:   length=3, mismatchIndex=1, comp=CurrentCulture, chars=(2,3), expected=True, actual=False

Original prompt

This section details on the original issue you should resolve

<issue_title>OSX failure on StringTests.StartsWithNoMatch_StringComparison</issue_title>
<issue_description>## Build Information
Build: https://dev.azure.com/dnceng-public/cbb18261-c48f-4abb-8651-8cdcb5474649/_build/results?buildId=940013
Build error leg or test failing: System.Tests.StringTests.StartsWithNoMatch_StringComparison
Pull request: #111877

Error Message

Fill the error message using step by step known issues guidance.

{
  "ErrorMessage": "[FAIL] System.Tests.StringTests.StartsWithNoMatch_StringComparison",
  "ErrorPattern": "",
  "BuildRetry": false,
  "ExcludeConsoleLog": false
}

Log

[17:27:34] dbug: 2025-02-04 17:27:34.201 Df System.Runtime.Tests[12401:3ab5fc] Failed tests:
[17:27:34] dbug: 2025-02-04 17:27:34.201 Df System.Runtime.Tests[12401:3ab5fc] 1) 	[FAIL] System.Tests.StringTests.StartsWithNoMatch_StringComparison   Test name: System.Tests.StringTests.StartsWithNoMatch_StringComparison
[17:27:34] dbug: Assembly:  [System.Runtime.Tests, Version=10.0.0.0, Culture=neutral, PublicKeyToken=cc7b13ffcd2ddd51]
[17:27:34] dbug: 2025-02-04 17:27:34.201 Df System.Runtime.Tests[12401:3ab5fc]    Exception messages: Assert.Equal() Failure: Values differ
[17:27:34] dbug: Expected: True
[17:27:34] dbug: Actual:   False   Exception stack traces:    at System.Tests.StringTests.StartsWithNoMatch_StringComparison()

Known issue validation

Build: 🔎 https://dev.azure.com/dnceng-public/public/_build/results?buildId=940013
Error message validated: [[FAIL] System.Tests.StringTests.StartsWithNoMatch_StringComparison]
Result validation: ❌ Known issue did not match with the provided build.
Validation performed at: 2/5/2025 4:19:32 PM UTC

Report

Build	Definition	Test	Pull Request
1316114	dotnet/runtime	System.Runtime.Tests.WorkItemExecution	#125056

Summary

24-Hour Hit Count	7-Day Hit Count	1-Month Count
0	0	1

Comments on the Issue (you are @copilot in this section)

@tarekgh @matouskozak please triage. Thanks! @matouskozak Looks like duplicate of https://github.com//issues/108832 but with slightly different error message. fyi: @kotlarmilos @danmoseley > [!NOTE] > This analysis was generated by GitHub Copilot.

Analysis: Root cause identifiable from code inspection

A dump or repro isn't strictly necessary here — this is a wrong-return-value bug, not a crash, so a dump wouldn't add much. The root cause is identifiable from the code.

What the test does

The test (StartsWithNoMatch_StringComparison) creates strings from chars \u0001–\u0096 (control characters + basic Latin), introduces a single-char mismatch at each position, then asserts that culture-aware StartsWith returns consistent results between calls:

Assert.Equal(
    firstSpan.ToString().StartsWith(secondSpan.ToString(), StringComparison.CurrentCulture),
    firstSpan.StartsWith(secondSpan, StringComparison.CurrentCulture));

The failure (Expected: True, Actual: False) means the same underlying comparison returns different results on different invocations for the same input.

Where it goes wrong

On Apple mobile in hybrid globalization mode, culture-aware StartsWith routes through GlobalizationNative_StartsWithNative in pal_collation.m:

NSString *prefixStrComposed = RemoveWeightlessCharacters(
    prefixString.precomposedStringWithCanonicalMapping);
NSString *sourceStrComposed = RemoveWeightlessCharacters(
    sourceString.precomposedStringWithCanonicalMapping);

int32_t result = (int32_t)[sourceStrComposed compare:prefixStrComposed
                                    ...

</details>



<!-- START COPILOT CODING AGENT SUFFIX -->

- Fixes dotnet/runtime#112195

<!-- START COPILOT CODING AGENT TIPS -->
---

✨ Let Copilot coding agent [set things up for you](https://github.com/dotnet/runtime/issues/new?title=✨+Set+up+Copilot+instructions&body=Configure%20instructions%20for%20this%20repository%20as%20documented%20in%20%5BBest%20practices%20for%20Copilot%20coding%20agent%20in%20your%20repository%5D%28https://gh.io/copilot-coding-agent-tips%29%2E%0A%0A%3COnboard%20this%20repo%3E&assignees=copilot) — coding agent works faster and does higher quality work when set up for your repo.

dotnet-policy-service · 2026-03-23T14:45:18Z

Tagging subscribers to this area: @dotnet/area-infrastructure-libraries
See info in area-owners.md if you want to be subscribed.

…put in StartsWithNoMatch_StringComparison Co-authored-by: kotlarmilos <11523312+kotlarmilos@users.noreply.github.com> Agent-Logs-Url: https://github.com/dotnet/runtime/sessions/ebe921f3-5e85-4102-be01-a6bd6fffada3

…improve diagnostic output Co-authored-by: kotlarmilos <11523312+kotlarmilos@users.noreply.github.com> Agent-Logs-Url: https://github.com/dotnet/runtime/sessions/ebe921f3-5e85-4102-be01-a6bd6fffada3

Copilot

Pull request overview

Updates StringTests to avoid intermittent macOS test failures caused by non-deterministic Apple collation behavior for certain Unicode inputs, by expanding existing platform guards and improving failure diagnostics.

Changes:

Add [ActiveIssue(..., IsOSX)] to EndsWithNoMatch_StringComparison and StartsWithNoMatch_StringComparison to disable these tests on macOS (in addition to existing Apple mobile coverage).
Refactor repeated culture-comparison assertions into a loop and add richer diagnostic messages when the assertions fail.

Copilot · 2026-03-28T14:57:02Z

src/libraries/Common/tests/Tests/System/StringTests.cs

+                    foreach (StringComparison comp in new[] { StringComparison.CurrentCulture, StringComparison.CurrentCultureIgnoreCase, StringComparison.InvariantCulture, StringComparison.InvariantCultureIgnoreCase })
+                    {
+                        bool expected = s1.ToString().EndsWith(s2.ToString(), comp);
+                        bool actual = s1.EndsWith(s2, comp);
+                        Assert.True(expected == actual,


Using Assert.True(expected == actual, ...) loses xUnit's default expected/actual diff output. Consider including expected and actual values in the failure message (or otherwise preserving them) so failures are as actionable as before, especially since this PR is focused on improving diagnostics.

Copilot · 2026-03-28T14:57:02Z

src/libraries/Common/tests/Tests/System/StringTests.cs

+                    foreach (StringComparison comp in new[] { StringComparison.CurrentCulture, StringComparison.CurrentCultureIgnoreCase, StringComparison.InvariantCulture, StringComparison.InvariantCultureIgnoreCase })
+                    {
+                        bool expected = firstSpan.ToString().EndsWith(secondSpan.ToString(), comp);
+                        bool actual = firstSpan.EndsWith(secondSpan, comp);
+                        Assert.True(expected == actual,


Using Assert.True(expected == actual, ...) loses xUnit's default expected/actual diff output. Consider including expected and actual values in the failure message (or otherwise preserving them) so failures are as actionable as before, especially since this PR is focused on improving diagnostics.

Copilot · 2026-03-28T14:57:02Z

src/libraries/Common/tests/Tests/System/StringTests.cs

+                    foreach (StringComparison comp in new[] { StringComparison.CurrentCulture, StringComparison.CurrentCultureIgnoreCase, StringComparison.InvariantCulture, StringComparison.InvariantCultureIgnoreCase })
+                    {
+                        bool expected = s1.ToString().StartsWith(s2.ToString(), comp);
+                        bool actual = s1.StartsWith(s2, comp);
+                        Assert.True(expected == actual,


Using Assert.True(expected == actual, ...) loses xUnit's default expected/actual diff output. Consider including expected and actual values in the failure message (or otherwise preserving them) so failures are as actionable as before, especially since this PR is focused on improving diagnostics.

Copilot · 2026-03-28T14:57:02Z

src/libraries/Common/tests/Tests/System/StringTests.cs

+                    foreach (StringComparison comp in new[] { StringComparison.CurrentCulture, StringComparison.CurrentCultureIgnoreCase, StringComparison.InvariantCulture, StringComparison.InvariantCultureIgnoreCase })
+                    {
+                        bool expected = firstSpan.ToString().StartsWith(secondSpan.ToString(), comp);
+                        bool actual = firstSpan.StartsWith(secondSpan, comp);
+                        Assert.True(expected == actual,


Using Assert.True(expected == actual, ...) loses xUnit's default expected/actual diff output. Consider including expected and actual values in the failure message (or otherwise preserving them) so failures are as actionable as before, especially since this PR is focused on improving diagnostics.

danmoseley · 2026-03-28T15:08:51Z

@copilot address this

…EndsWith assertions Agent-Logs-Url: https://github.com/dotnet/runtime/sessions/dd927ad2-980d-4369-895a-0a96fe17f1ca Co-authored-by: danmoseley <6385855+danmoseley@users.noreply.github.com>

Copilot · 2026-03-28T15:32:26Z

@copilot address this

Done in 47fc187. All four Assert.True messages now include , expected={expected}, actual={actual} so the boolean values are visible in the failure output alongside the existing context (length, mismatchIndex, comp, chars).

Initial plan

759c6db

Copilot AI assigned Copilot and kotlarmilos Mar 23, 2026

Copilot started work on behalf of kotlarmilos March 23, 2026 14:30 View session

github-actions bot added the area-Infrastructure-libraries label Mar 23, 2026

Copilot AI and others added 2 commits March 23, 2026 15:04

Also fix EndsWithNoMatch_StringComparison: add IsOSX ActiveIssue and …

ccfaf9b

…improve diagnostic output Co-authored-by: kotlarmilos <11523312+kotlarmilos@users.noreply.github.com> Agent-Logs-Url: https://github.com/dotnet/runtime/sessions/ebe921f3-5e85-4102-be01-a6bd6fffada3

Copilot AI changed the title ~~[WIP] Fix OSX failure on StringTests.StartsWithNoMatch_StringComparison~~ Fix intermittent OSX CI failures in StartsWithNoMatch_StringComparison and EndsWithNoMatch_StringComparison Mar 23, 2026

Copilot AI requested a review from kotlarmilos March 23, 2026 15:10

Copilot finished work on behalf of kotlarmilos March 23, 2026 15:10

tarekgh added the os-mac-os-x macOS aka OSX label Mar 23, 2026

This was referenced Mar 23, 2026

Checkout failure: "Git fetch failed with exit code 128" dotnet/arcade#9009

Open

ProcessTests.SignalHandler_CanDisposeInHandler fails on linux on mono interpreter #125953

Closed

danmoseley requested review from Copilot and removed request for kotlarmilos March 28, 2026 14:53

danmoseley marked this pull request as ready for review March 28, 2026 14:53

Copilot started reviewing on behalf of danmoseley March 28, 2026 14:54 View session

Copilot AI reviewed Mar 28, 2026

View reviewed changes

Copilot started work on behalf of danmoseley March 28, 2026 15:09 View session

Include expected/actual values in diagnostic messages for StartsWith/…

47fc187

…EndsWith assertions Agent-Logs-Url: https://github.com/dotnet/runtime/sessions/dd927ad2-980d-4369-895a-0a96fe17f1ca Co-authored-by: danmoseley <6385855+danmoseley@users.noreply.github.com>

Copilot finished work on behalf of danmoseley March 28, 2026 15:33

Copilot AI requested a review from danmoseley March 28, 2026 15:33

build-analysis bot mentioned this pull request Mar 28, 2026

Unable to pull image from mcr.microsoft.com #117164

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fix intermittent OSX CI failures in StartsWithNoMatch_StringComparison and EndsWithNoMatch_StringComparison#125963

Fix intermittent OSX CI failures in StartsWithNoMatch_StringComparison and EndsWithNoMatch_StringComparison#125963
Copilot wants to merge 4 commits intomainfrom
copilot/fix-osx-failure-string-tests

Copilot AI commented Mar 23, 2026 •

edited

Loading

Uh oh!

dotnet-policy-service bot commented Mar 23, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI Mar 28, 2026

Uh oh!

Copilot AI Mar 28, 2026

Uh oh!

Copilot AI Mar 28, 2026

Uh oh!

Copilot AI Mar 28, 2026

Uh oh!

danmoseley commented Mar 28, 2026

Uh oh!

Copilot AI commented Mar 28, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

Conversation

Copilot AI commented Mar 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Changes

Error Message

Known issue validation

Report

Summary

Comments on the Issue (you are @copilot in this section)

Analysis: Root cause identifiable from code inspection

What the test does

Where it goes wrong

Uh oh!

dotnet-policy-service bot commented Mar 23, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Copilot AI Mar 28, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Mar 28, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Mar 28, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Mar 28, 2026

Choose a reason for hiding this comment

Uh oh!

danmoseley commented Mar 28, 2026

Uh oh!

Copilot AI commented Mar 28, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

Copilot AI commented Mar 23, 2026 •

edited

Loading