CompareInfo.IndexOf / LastIndexOf early-exit code paths do not account for zero-weight characters

[GrabYourPitchforks]

Repro code:

Console.WriteLine(CultureInfo.InvariantCulture.CompareInfo.IndexOf(string.Empty, "\u200c"));

This should print 0 on all platforms unless we're running under the invariant globalization mode, in which case it should return -1. The reason for this is an incorrect early-exit in CompareInfo.IndexOf, as shown below.

runtime/src/libraries/System.Private.CoreLib/src/System/Globalization/CompareInfo.cs

Lines 940 to 947 in 1ae617b

	if (source.Length == 0)
	{
	if (value.Length == 0)
	{
	return 0;
	}
	return -1;
	}

This early exit does not account for the fact that empty strings and non-empty strings may compare as equal under certain collations if the non-empty strings consist only of zero-weight code points.

Tagging @tarekgh, @safern as an area owner

[tarekgh]

@GrabYourPitchforks thanks for filing it. just wondering if you'll get the fix? or expect someone else to do it?

[GrabYourPitchforks]

The fix is included in #1514. I'm hoping to get a new iteration of that out by the end of today.

[GrabYourPitchforks]

A quick note: there is a difference between NLS and ICU here.

Consider a call to "hello".LastIndexOf("\u200d") (using a linguistic comparer). NLS's FindNLSStringEx will return an index of 5. ICU's usearch_last will return an index of 4. As far as I can tell this only happens with weightless code points. I'm not going to try to work around this in the .NET Core layer, but I'll update the unit tests to account for the target environment and to change the expectation.

[tarekgh]

@GrabYourPitchforks

I'm not going to try to work around this in the .NET Core layer

Any reason why we cannot do that?

[GrabYourPitchforks]

Any reason why we cannot do that?

We're not doing it in other places. For example, on NLS "\u200dhello".IndexOf("hello") returns 1, but logically it should really return 0 (since "\u200dhello".StartsWith("hello") returns true). I think the risk of trying to capture all of these edge cases makes it more likely that we'll introduce bugs in our own code. It'll definitely slow down the common path.

[tarekgh]

ok. thanks!