threadlocaltests.runthreadlocaltest8_values thows taskschedulerexception on SLES

[runfoapp]

Runfo Tracking Issue: threadlocaltests.runthreadlocaltest8_values thows taskschedulerexception on SLES

Build	Definition	Kind	Run Name	Console	Core Dump	Test Results	Run Client
961727	runtime	Rolling	net6.0-Linux-Release-x64-Mono_release-SLES.12.Amd64.Open
961572	runtime	Rolling	net6.0-Linux-Release-x64-Mono_release-SLES.12.Amd64.Open
961065	runtime	Rolling	net6.0-Linux-Release-x64-Mono_release-SLES.12.Amd64.Open
960758	runtime	Rolling	net6.0-Linux-Release-x64-Mono_release-SLES.12.Amd64.Open
959437	runtime	PR 47210	net5.0-Linux-Debug-x64-Mono_release-SLES.12.Amd64.Open	console.log		test results	runclient.py
957661	runtime	PR 47210	net5.0-Linux-Debug-x64-Mono_release-SLES.12.Amd64.Open	console.log		test results	runclient.py
952932	runtime	Rolling	net5.0-Linux-Release-x64-Mono_release-SLES.12.Amd64.Open	console.log		test results	runclient.py
951047	runtime	PR 47002	net5.0-Linux-Debug-x64-Mono_release-SLES.12.Amd64.Open	console.log		test results	runclient.py
949699	runtime	Rolling	net5.0-Linux-Release-x64-Mono_release-SLES.12.Amd64.Open	console.log		test results	runclient.py
946185	runtime	PR 46745	net5.0-Linux-Debug-x64-Mono_release-SLES.12.Amd64.Open	console.log		test results	runclient.py

Build Result Summary

Day Hit Count	Week Hit Count	Month Hit Count
0	0	0

I couldn't figure out the best area label to add to this issue. If you have write-permissions please help me learn by adding exactly one area label.

[hoyosjs]

@kouvel do you know who would be the best to help look at this?

[stephentoub]

The relevant part of the error:
" ---- System.ExecutionEngineException : Couldn't create thread. Error 0x0"

[lambdageek]

SLES is misconfigured by design to limit the number of threads a process can create to an artificially low number. So thread creation can fail sometimes.

#43981 (comment)

Apparently SUSE has some cgroup based limits on the number of threads that can be created. Which for some services is as low as 512. And for user accounts the limit is 12288 threads across all logins.
https://www.suse.com/releasenotes/x86_64/SUSE-SLES/12-SP2/#fate-320358

Unclear why only MonoVM is affected. Perhaps we spawn more threads?

[lambdageek]

That said, there is something very strange going on:

System.ExecutionEngineException : Couldn't create thread. Error 0x0

The only code that I can find that has that string is

runtime/src/mono/mono/metadata/threads.c

Line 1441 in ce4c725

mono_error_set_execution_engine (error, "Couldn't create thread. Error 0x%x", mono_w32error_get_last());

but that code is explicitly for non-netcore Mono. It shouldn't even be compiled - how is it running???

[akoeplinger]

How is this test even running, it's supposed to be disabled on mono:

runtime/src/libraries/System.Threading/tests/ThreadLocalTests.cs

Lines 227 to 228 in 16e6e95

	[ActiveIssue("https://github.com/dotnet/runtime/issues/43981", TestRuntimes.Mono)]
	public static void RunThreadLocalTest8_Values()

[akoeplinger]

Ah, looks like all of these failures come from PRs/builds targetting the release/5.0 branch where this test is still enabled: #43981 (comment)

[akoeplinger]

I've opened a backport PR for 5.0 here: #47276

I don't think there's anything else to do here so closing.