Skip to content

Update SkipOnCoreClrAttributes to new parameters #28875

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
safern wants to merge 1 commit into from
Closed

Update SkipOnCoreClrAttributes to new parameters #28875

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion ...ComponentModel/Composition/Registration/RegistrationBuilderAttributedOverrideUnitTests.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@

namespace System.ComponentModel.Composition.Registration.Tests
{
[SkipOnCoreClr("Test failures on stress tests")]
[SkipOnCoreClr("Test failures on stress tests", RuntimeTestModes.CheckedRuntime)]

@stephentoub stephentoub Feb 1, 2020

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it just checked builds, or any of the stress / non-release configurations?

@safern safern Feb 3, 2020

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This would apply to ALL Checked builds, no matter if it is stress or not.

However, I just realized that the current design is not great as if we added a | RuntimeTestModes.GCStress it would mean, ALL Checked or GCStress (Checked or Release), as I would think what we want is that if I say RuntimeTestModes.Checked it would mean, ALL Checked, but if I add an stress mode flag, then it would mean, ONLY skip when running GCStress when in Checked. What is your expectation/thought for that?

@safern safern Feb 5, 2020

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@stephentoub thoughts on that?

I'll include this change in the arcade update to unblock and can update again based on your thoughts. I think updating it to the latter is the ideal.

@safern safern Feb 5, 2020

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually, I gave it a thought and put up a PR to update it to a better way of doing it: dotnet/arcade#4764

[SkipOnMono("Test failures on stress tests")]
public class RegistrationBuilderAttributedOverrideUnitTests
{
Expand Down
2 changes: 1 addition & 1 deletion src/libraries/System.Data.SqlClient/tests/FunctionalTests/AssemblyInfo.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,5 +4,5 @@

using Xunit;

[assembly: SkipOnCoreClr("Timeout in stress tests on Linux/arm32", TestPlatforms.Linux)]
[assembly: SkipOnCoreClr("Timeout in stress tests on Linux/arm32", TestPlatforms.Linux, RuntimeTestModes.CheckedRuntime)]
[assembly: SkipOnMono("Timeout in stress tests on Linux/arm32", TestPlatforms.Linux)]
2 changes: 1 addition & 1 deletion src/libraries/System.Linq.Expressions/tests/AssemblyInfo.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,4 +4,4 @@

using Xunit;

[assembly: SkipOnCoreClr("Long running tests: https://github.com/dotnet/coreclr/issues/25269", RuntimeStressTestModes.CheckedRuntime)]
[assembly: SkipOnCoreClr("Long running tests: https://github.com/dotnet/coreclr/issues/25269", RuntimeTestModes.CheckedRuntime)]
2 changes: 1 addition & 1 deletion src/libraries/System.Net.Http/tests/FunctionalTests/AssemblyInfo.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,5 +4,5 @@

using Xunit;

[assembly: SkipOnCoreClr("System.Net.Tests are flaky and/or long running: https://github.com/dotnet/runtime/issues/131")]
[assembly: SkipOnCoreClr("System.Net.Tests are flaky and/or long running: https://github.com/dotnet/runtime/issues/131", RuntimeTestModes.CheckedRuntime)]
[assembly: SkipOnMono("System.Net.Tests are flaky and/or long running: https://github.com/dotnet/runtime/issues/131")]
2 changes: 1 addition & 1 deletion src/libraries/System.Net.HttpListener/tests/HttpListenerAuthenticationTests.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@

namespace System.Net.Tests
{
[SkipOnCoreClr("System.Net.Tests may timeout in stress configurations")]
[SkipOnCoreClr("System.Net.Tests may timeout in stress configurations", RuntimeTestModes.CheckedRuntime)]
[ActiveIssue("https://github.com/dotnet/runtime/issues/2391", TargetFrameworkMonikers.Mono)]
[ConditionalClass(typeof(PlatformDetection), nameof(PlatformDetection.IsNotWindowsNanoServer))] // httpsys component missing in Nano.
public class HttpListenerAuthenticationTests : IDisposable
Expand Down
2 changes: 1 addition & 1 deletion src/libraries/System.Net.HttpListener/tests/HttpListenerContextTests.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@

namespace System.Net.Tests
{
[SkipOnCoreClr("System.Net.Tests may timeout in stress configurations")]
[SkipOnCoreClr("System.Net.Tests may timeout in stress configurations", RuntimeTestModes.CheckedRuntime)]
[ActiveIssue("https://github.com/dotnet/runtime/issues/2391", TargetFrameworkMonikers.Mono)]
[ConditionalClass(typeof(PlatformDetection), nameof(PlatformDetection.IsNotWindowsNanoServer))] // httpsys component missing in Nano.
public class HttpListenerContextTests : IDisposable
Expand Down
2 changes: 1 addition & 1 deletion src/libraries/System.Net.HttpListener/tests/HttpListenerResponseTests.Cookies.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@

namespace System.Net.Tests
{
[SkipOnCoreClr("System.Net.Tests may timeout in stress configurations")]
[SkipOnCoreClr("System.Net.Tests may timeout in stress configurations", RuntimeTestModes.CheckedRuntime)]
[ActiveIssue("https://github.com/dotnet/runtime/issues/2391", TargetFrameworkMonikers.Mono)]
[ConditionalClass(typeof(PlatformDetection), nameof(PlatformDetection.IsNotWindowsNanoServer))] // httpsys component missing in Nano.
public class HttpListenerResponseCookiesTests : HttpListenerResponseTestBase
Expand Down
2 changes: 1 addition & 1 deletion src/libraries/System.Net.HttpListener/tests/HttpListenerResponseTests.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -57,7 +57,7 @@ protected async Task<HttpListenerResponse> GetResponse(string httpVersion = "1.1
}
}

[SkipOnCoreClr("System.Net.Tests may timeout in stress configurations")]
[SkipOnCoreClr("System.Net.Tests may timeout in stress configurations", RuntimeTestModes.CheckedRuntime)]
[ActiveIssue("https://github.com/dotnet/runtime/issues/2391", TargetFrameworkMonikers.Mono)]
[ConditionalClass(typeof(PlatformDetection), nameof(PlatformDetection.IsNotWindowsNanoServer))] // httpsys component missing in Nano.
public class HttpListenerResponseTests : HttpListenerResponseTestBase
Expand Down
4 changes: 2 additions & 2 deletions src/libraries/System.Net.Mail/tests/Functional/SmtpClientTest.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -339,7 +339,7 @@ public void TestZeroTimeout()
[InlineData("howdydoo")]
[InlineData("")]
[InlineData(null)]
[SkipOnCoreClr("System.Net.Tests are flaky and/or long running: https://github.com/dotnet/runtime/issues/131")]
[SkipOnCoreClr("System.Net.Tests are flaky and/or long running: https://github.com/dotnet/runtime/issues/131", RuntimeTestModes.CheckedRuntime)]
[SkipOnMono("System.Net.Tests are flaky and/or long running: https://github.com/dotnet/runtime/issues/131")]
public async Task TestMailDeliveryAsync(string body)
{
Expand All @@ -358,7 +358,7 @@ public async Task TestMailDeliveryAsync(string body)

[Fact]
[PlatformSpecific(TestPlatforms.Windows)] // NTLM support required, see https://github.com/dotnet/corefx/issues/28961
[SkipOnCoreClr("System.Net.Tests are flaky and/or long running: https://github.com/dotnet/runtime/issues/131")]
[SkipOnCoreClr("System.Net.Tests are flaky and/or long running: https://github.com/dotnet/runtime/issues/131", RuntimeTestModes.CheckedRuntime)]
[SkipOnMono("System.Net.Tests are flaky and/or long running: https://github.com/dotnet/runtime/issues/131")]
public async Task TestCredentialsCopyInAsyncContext()
{
Expand Down
3 changes: 1 addition & 2 deletions src/libraries/System.Net.Requests/tests/LoggingTest.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,8 +11,7 @@ public class LoggingTest
{
[Fact]
[SkipOnTargetFramework(TargetFrameworkMonikers.Mono, "NetEventSource is only part of .NET Core.")]
[SkipOnCoreClr("System.Net.Tests are flaky")]
[SkipOnMono("System.Net.Tests are flaky")]
[SkipOnCoreClr("System.Net.Tests are flaky", RuntimeTestModes.CheckedRuntime)]
public void EventSource_ExistsWithCorrectId()
{
Type esType = typeof(WebRequest).Assembly.GetType("System.Net.NetEventSource", throwOnError: true, ignoreCase: false);
Expand Down
2 changes: 1 addition & 1 deletion src/libraries/System.Net.Security/tests/FunctionalTests/SslStreamSniTest.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -137,7 +137,7 @@ public async Task SslStream_ServerCallbackNotSet_UsesLocalCertificateSelection(s
}

[Fact]
[SkipOnCoreClr("System.Net.Tests are flaky and/or long running: https://github.com/dotnet/runtime/issues/131")]
[SkipOnCoreClr("System.Net.Tests are flaky and/or long running: https://github.com/dotnet/runtime/issues/131", RuntimeTestModes.CheckedRuntime)]
[SkipOnMono("System.Net.Tests are flaky and/or long running: https://github.com/dotnet/runtime/issues/131")]
public async Task SslStream_NoSniFromClient_CallbackReturnsNull()
{
Expand Down
2 changes: 1 addition & 1 deletion src/libraries/System.Net.Sockets/tests/FunctionalTests/AssemblyInfo.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,5 +4,5 @@

using Xunit;

[assembly: SkipOnCoreClr("System.Net.Tests are flaky and/or long running: https://github.com/dotnet/runtime/issues/131")]
[assembly: SkipOnCoreClr("System.Net.Tests are flaky and/or long running: https://github.com/dotnet/runtime/issues/131", RuntimeTestModes.CheckedRuntime)]
[assembly: SkipOnMono("System.Net.Tests are flaky and/or long running: https://github.com/dotnet/runtime/issues/131")]

@stephentoub stephentoub Feb 1, 2020

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we know if mono actually had problems with these or just copied the coreclr attribution thinking it was about all configs and not just stress?

@safern safern Feb 3, 2020

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My assumption here is that mono did have problems with this, and ActiveIssue isn't usable on an assembly level. @akoeplinger ?

@akoeplinger akoeplinger Feb 3, 2020

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think I just copied from CoreCLR so this might not be needed.

2 changes: 1 addition & 1 deletion src/libraries/System.Runtime.Numerics/tests/BigInteger/BigIntegerToStringTests.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -188,7 +188,7 @@ public static void RunStandardFormatToStringTests()
}

[Fact]
[SkipOnCoreClr("Long running tests: https://github.com/dotnet/coreclr/issues/22414", TestPlatforms.Linux, RuntimeStressTestModes.CheckedRuntime)]
[SkipOnCoreClr("Long running tests: https://github.com/dotnet/coreclr/issues/22414", TestPlatforms.Linux, RuntimeTestModes.CheckedRuntime)]
public static void RunRegionSpecificStandardFormatToStringTests()
{
CultureInfo[] cultures = new CultureInfo[] { new CultureInfo("en-US"), new CultureInfo("en-GB"), new CultureInfo("fr-CA"),
Expand Down
2 changes: 1 addition & 1 deletion src/libraries/System.Runtime.Serialization.Formatters/tests/BinaryFormatterTests.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ public partial class BinaryFormatterTests : FileCleanupTestBase
{
// On 32-bit we can't test these high inputs as they cause OutOfMemoryExceptions.
[ConditionalTheory(typeof(Environment), nameof(Environment.Is64BitProcess))]
[SkipOnCoreClr("Long running tests: https://github.com/dotnet/coreclr/issues/20246", RuntimeStressTestModes.CheckedRuntime)]
[SkipOnCoreClr("Long running tests: https://github.com/dotnet/coreclr/issues/20246", RuntimeTestModes.CheckedRuntime)]
[InlineData(2 * 6_584_983 - 2)] // previous limit
[InlineData(2 * 7_199_369 - 2)] // last pre-computed prime number
public void SerializeHugeObjectGraphs(int limit)
Expand Down
2 changes: 1 addition & 1 deletion src/libraries/System.Text.RegularExpressions/tests/Regex.Match.Tests.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -919,7 +919,7 @@ public void Match_SpecialUnicodeCharacters_Invariant()
[InlineData(RegexOptions.None)]
[InlineData(RegexOptions.Compiled)]
[SkipOnTargetFramework(TargetFrameworkMonikers.NetFramework, ".NET Framework does not have fix for https://github.com/dotnet/corefx/issues/26484")]
[SkipOnCoreClr("Long running tests: https://github.com/dotnet/coreclr/issues/18912", RuntimeStressTestModes.JitMinOpts)]
[SkipOnCoreClr("Long running tests: https://github.com/dotnet/coreclr/issues/18912", RuntimeTestModes.JitMinOpts)]
public void Match_ExcessPrefix(RegexOptions options)
{
RemoteExecutor.Invoke(optionsString =>
Expand Down