`Equals` and `GetHashCode` mismatch with `StringComparer.InvariantCulture`

[VladimirReshetnikov]

Compile and run:

using System;

static class Program
{
    static void Main()
    {
        // U+FFFD is REPLACEMENT CHARACTER
        Console.WriteLine(StringComparer.InvariantCulture.Equals("", "\uFFFD"));
        Console.WriteLine(StringComparer.InvariantCulture.GetHashCode("") ==
                          StringComparer.InvariantCulture.GetHashCode("\uFFFD"));
    }
}

ACTUAL:

True
False

EXPECTED:
The second Boolean value is True if the first one is True.

---

OS ver is Microsoft Windows [Version 10.0.17763.914]

%SystemRoot%\System32\kernel32.dll
File version: 10.0.17763.475
File SHA1: 8de080d3295fb21475715d4051ec9dbc3df972ba

---

The issue occurs both on:

%ProgramFiles%\dotnet\shared\Microsoft.NETCore.App\3.1.0\.version
65f04fb6db7a5e198d05dbebd5c4ad21eb018f89
3.1.0

and

%ProgramFiles%\dotnet\shared\Microsoft.NETCore.App\5.0.0-alpha.1.19564.1\.version
c77948d92a2f950140f09384f057cb893ec3955a
5.0.0-alpha.1.19564.1

[GrabYourPitchforks]

This and #1060 are likely NLS bugs. I'll follow up with the Windows localization team offline.

[GrabYourPitchforks]

Dug into this a bit further, and seems like the NLS functions LCMapStringEx and CompareStringEx are being consistent with respect to each other, though I find it odd that they're ignoring the presence of the U+FFFD character. So that points to the issue being on the managed side, not the NLS side.

Culprit appears to be the quick-exit check at the beginning of the below method.

runtime/src/libraries/System.Private.CoreLib/src/System/Globalization/CompareInfo.Windows.cs

Lines 127 to 135 in 0a29c61

	private unsafe int GetHashCodeOfStringCore(ReadOnlySpan<char> source, CompareOptions options)
	{
	Debug.Assert(!GlobalizationMode.Invariant);
	Debug.Assert((options & (CompareOptions.Ordinal | CompareOptions.OrdinalIgnoreCase)) == 0);
	if (source.Length == 0)
	{
	return 0;
	}

If there are indeed some non-empty strings that would compare as equivalent to the empty string, then we can't perform this shortcut and instead we need to call LCMapStringEx even on the empty string. Since the cchSrc parameter of LCMapStringEx does not allow 0, we can instead pass for lpSrcStr a pointer to a null-terminated zero-length string and pass for cchSrc the constant value -1. In the sample C++ application I wrote this appears to have the correct behavior.

[VladimirReshetnikov]

Yes, there are other strings considered equal to "" by StringComparer.InvariantCulture, e.g, "\u0640" or "\0" (I am not evaluating whether it is reasonable or not). The shortcut is wrong then.