[ci-scan] Test failure: System.Text.Json.SourceGeneration.Roslyn4.4.Tests Access Violation crash under NativeAOT

[github-actions]

Build Information

Build: https://dev.azure.com/dnceng-public/public/_build/results?buildId=1422775
Build error leg or test failing: windows-arm64 Release NativeAOT_Libs / windows-x64 Release NativeAOT_Libs / osx-x64 Release NativeAOT_Libs - System.Text.Json.SourceGeneration.Roslyn4.4.Tests

Error Details

System.Text.Json.SourceGeneration.Roslyn4.4.Tests crashes with an Access Violation (exit code -1073740791 / STATUS_STACK_BUFFER_OVERRUN) across multiple NativeAOT legs in the runtime-nativeaot-outerloop pipeline (definition 265).

Failing tests observed:

• System.Text.Json.SourceGeneration.Tests.UnsupportedTypesTests_Metadata.SerializeUnsupportedType<Int32[,]> (windows-arm64)
• System.Text.Json.SourceGeneration.Tests.UnsupportedTypesTests_Metadata.SerializeUnsupportedType<Boolean[,,,]> (windows-arm64)
• System.Text.Json.SourceGeneration.Tests.JsonCreationHandlingTests_String.CreationHandlingSetWithAttribute_PopulatedPropertyDeserializeInitiallyNull (windows-x64)

All crash with NullReferenceException instead of expected NotSupportedException, followed by a terminal Access Violation.

Affected legs (7 failures in build 1422775):

• windows-arm64 Release NativeAOT_Libs
• windows-x64 Release NativeAOT_Libs
• windows-x64 Release NativeAOT_Checked_Libs
• windows-x64 Release NativeAOT_Checked_Libs_SizeOpt
• windows-x64 Release NativeAOT_Checked_Libs_SpeedOpt
• windows-x86 Release NativeAOT_Libs
• osx-x64 Release NativeAOT_Libs

[FAIL] System.Text.Json.SourceGeneration.Tests.UnsupportedTypesTests_Metadata.SerializeUnsupportedType<Int32[,]>(wrapper: System.Int32[,])
Assert.Throws() Failure: Exception type was not an exact match
Expected: typeof(System.NotSupportedException)
Actual:   typeof(System.NullReferenceException)
---- System.NullReferenceException : Object reference not set to an instance of an object.
[FAIL] System.Text.Json.SourceGeneration.Tests.JsonCreationHandlingTests_String.CreationHandlingSetWithAttribute_PopulatedPropertyDeserializeInitiallyNull(typeWitness: TypeWitness`1 { })
Process terminated. Access Violation: Attempted to read or write protected memory. This is often an indication that other memory is corrupt. The application will be terminated since this platform does not support throwing an AccessViolationException.
   at System.RuntimeExceptionHelpers.FailFast(String, Exception, String, RhFailFastReason, IntPtr, IntPtr)
   at System.RuntimeExceptionHelpers.GetRuntimeException(ExceptionIDs)
   at System.Runtime.EH.GetClasslibException(ExceptionIDs, IntPtr)
   at Xunit.Assert.<RecordExceptionAsync>d__215.MoveNext()

First observed: build 1422775 (2026-05-16). Prior NativeAOT builds (1420289 on May 14, 1419666 on May 14) had different failures. This appears new as of 2026-05-15/16.

Error Message

{
  "ErrorMessage": ["System.Text.Json.SourceGeneration.Roslyn4.4.Tests", "Access Violation: Attempted to read or write protected memory"],
  "BuildRetry": false,
  "ExcludeConsoleLog": false
}

Generated by CI Outer-Loop Failure Scanner · ● 23.8M · ◷

Report

Build	Definition	Test	Pull Request
1425376	dotnet/runtime	System.Text.Json.SourceGeneration.Roslyn4.4.Tests.WorkItemExecution
1424404	dotnet/runtime	System.Text.Json.SourceGeneration.Roslyn4.4.Tests.WorkItemExecution
1424314	dotnet/runtime	System.Text.Json.SourceGeneration.Roslyn4.4.Tests.WorkItemExecution	#128313
1423942	dotnet/runtime	System.Text.Json.SourceGeneration.Roslyn4.4.Tests.WorkItemExecution
1423686	dotnet/runtime	System.Text.Json.SourceGeneration.Roslyn4.4.Tests.WorkItemExecution
1423513	dotnet/runtime	System.Text.Json.SourceGeneration.Roslyn4.4.Tests.WorkItemExecution

Summary

24-Hour Hit Count	7-Day Hit Count	1-Month Count
0	6	6

[MichalStrehovsky]

This and #128291 are likely manifestations of the same underlying issue. Validating a theory in #128313, will probably go to area-codegen-coreclr.

[dotnet-policy-service]

Tagging subscribers to this area: @dotnet/area-system-text-json
See info in area-owners.md if you want to be subscribed.

[MichalStrehovsky]

This has a bunch of failure modes in the CI, this is one of them. It reliably breaks the Windows legs.

00 00000069`d727cb00 00007ff6`acc161e3     KERNELBASE!RaiseFailFastException+0x188
01 00000069`d727d0e0 00007ff6`aee85047     System_Text_Json_SourceGeneration_Roslyn4_4_Tests!S_P_CoreLib_System_Runtime_CachedInterfaceDispatch__RhpCidResolve_Worker+0x33 [/_/src/coreclr/nativeaot/Runtime.Base/src/System/Runtime/CachedInterfaceDispatch.cs @ 37] 
02 00000069`d727d140 00007ff6`ae52f525     System_Text_Json_SourceGeneration_Roslyn4_4_Tests!ReturnFromUniversalTransitionTailCall [D:\a\_work\1\s\src\coreclr\nativeaot\Runtime\amd64\UniversalTransition.asm @ 153] 
03 00000069`d727d1c0 00007ff6`ae537f70     System_Text_Json_SourceGeneration_Roslyn4_4_Tests!S_P_CoreLib_System_Collections_Generic_List_1<System___Canon>___ctor_1+0x65 [/_/src/libraries/System.Private.CoreLib/src/System/Collections/Generic/List.cs @ 83] 
04 00000069`d727d220 00007ff6`acd73f4f     System_Text_Json_SourceGeneration_Roslyn4_4_Tests!System_Text_Json_System_Text_Json_Serialization_ConfigurationList_1<System___Canon>___ctor+0x50 [/_/src/libraries/System.Text.Json/src/System/Text/Json/Serialization/ConfigurationList.cs @ 21] 
05 00000069`d727d270 00007ff6`acdfcd67     System_Text_Json_SourceGeneration_Roslyn4_4_Tests!System_Text_Json_System_Text_Json_JsonSerializerOptions___ctor_0+0xbf [/_/src/libraries/System.Text.Json/src/System/Text/Json/Serialization/JsonSerializerOptions.cs @ 135] 
06 00000069`d727d2c0 00007ff6`aed0a99b     System_Text_Json_SourceGeneration_Roslyn4_4_Tests!System_Text_Json_SourceGeneration_Roslyn4_4_Tests_System_Text_Json_SourceGeneration_Tests_StringSerializerWrapper__GetOptions+0x67 [/_/src/libraries/System.Text.Json/tests/System.Text.Json.SourceGeneration.Tests/Serialization/JsonSerializerWrapper.SourceGen.cs @ 66] 
07 00000069`d727d300 00007ff6`ae550578     System_Text_Json_SourceGeneration_Roslyn4_4_Tests!System_Text_Json_SourceGeneration_Roslyn4_4_Tests_System_Text_Json_SourceGeneration_Tests_StringSerializerWrapper__SerializeWrapper_0<System___Canon>+0x1b [/_/src/libraries/System.Text.Json/tests/System.Text.Json.SourceGeneration.Tests/Serialization/JsonSerializerWrapper.SourceGen.cs @ 29] 
08 00000069`d727d350 00007ff6`ae54fe39     System_Text_Json_SourceGeneration_Roslyn4_4_Tests!System_Text_Json_SourceGeneration_Roslyn4_4_Tests_System_Text_Json_Serialization_Tests_UnsupportedTypesTests___c__DisplayClass6_0_1<System___Canon>__<AsyncCallable>System_Text_Json_SourceGeneration_Roslyn4_4_Tests_System_Text_Json_Serialization_Tests_UnsupportedTypesTests___c__DisplayClass6_0_1___SerializeUnsupportedType_b__0+0xa8 [/_/src/libraries/System.Text.Json/tests/Common/UnsupportedTypesTests.cs @ 73] 
09 00000069`d727d3c0 00007ff6`ad8bcfbe     System_Text_Json_SourceGeneration_Roslyn4_4_Tests!System_Text_Json_SourceGeneration_Roslyn4_4_Tests_System_Text_Json_Serialization_Tests_UnsupportedTypesTests___c__DisplayClass6_0_1<System___Canon>___SerializeUnsupportedType_b__0+0xd9
0a 00000069`d727d460 00007ff6`aeb48f61     System_Text_Json_SourceGeneration_Roslyn4_4_Tests!xunit_assert_Xunit_Assert__RecordExceptionAsync_d__215__MoveNext+0x7e [/_/src/arcade/src/Microsoft.DotNet.XUnitAssert/src/Record.cs @ 138] 
0b 00000069`d727d4b0 00007ff6`ad8b762f     System_Text_Json_SourceGeneration_Roslyn4_4_Tests!S_P_CoreLib_System_Runtime_CompilerServices_AsyncMethodBuilderCore__Start<xunit_assert_Xunit_Assert__RecordExceptionAsync_d__215>+0x61 [/_/src/libraries/System.Private.CoreLib/src/System/Runtime/CompilerServices/AsyncMethodBuilderCore.cs @ 38] 
0c 00000069`d727d510 00007ff6`ae55a233     System_Text_Json_SourceGeneration_Roslyn4_4_Tests!xunit_assert_Xunit_Assert__RecordExceptionAsync+0x2f
0d 00000069`d727d560 00007ff6`aeb4ca55     System_Text_Json_SourceGeneration_Roslyn4_4_Tests!xunit_assert_Xunit_Assert__ThrowsAsync_d__177_1<System___Canon>__MoveNext+0xc3 [/_/src/arcade/src/Microsoft.DotNet.XUnitAssert/src/ExceptionAsserts.cs @ 415] 
0e 00000069`d727d5c0 00007ff6`aed18ecf     System_Text_Json_SourceGeneration_Roslyn4_4_Tests!S_P_CoreLib_System_Runtime_CompilerServices_AsyncMethodBuilderCore__Start<System_Text_Json_System_Text_Json_Serialization_Metadata_JsonTypeInfo_1__DeserializeAsync_d__1_2<JsonSchema_Net_Json_Schema_SchemaValueType__System_Text_Json_System_Text_Json_Serialization_StreamReadBufferState__System___Canon>>+0x75 [/_/src/libraries/System.Private.CoreLib/src/System/Runtime/CompilerServices/AsyncMethodBuilderCore.cs @ 38] 
0f 00000069`d727d630 00007ff6`aed38e0e     System_Text_Json_SourceGeneration_Roslyn4_4_Tests!xunit_assert_Xunit_Assert__ThrowsAsync_1<System___Canon>+0x3f
10 00000069`d727d690 00007ff6`aed12e97     System_Text_Json_SourceGeneration_Roslyn4_4_Tests!System_Text_Json_SourceGeneration_Roslyn4_4_Tests_System_Text_Json_Serialization_Tests_UnsupportedTypesTests__<AsyncCallable>System_Text_Json_SourceGeneration_Roslyn4_4_Tests_System_Text_Json_Serialization_Tests_UnsupportedTypesTests__SerializeUnsupportedType<System___Canon>+0x19e [/_/src/libraries/System.Text.Json/tests/Common/UnsupportedTypesTests.cs @ 73] 
11 00000069`d727d730 00007ff6`aed51822     System_Text_Json_SourceGeneration_Roslyn4_4_Tests!System_Text_Json_SourceGeneration_Roslyn4_4_Tests_System_Text_Json_Serialization_Tests_UnsupportedTypesTests__SerializeUnsupportedType<System___Canon>+0xf7
12 00000069`d727d7f0 00007ff6`acbf432e     System_Text_Json_SourceGeneration_Roslyn4_4_Tests!Internal_CompilerGenerated__Module___<DynamicInvoke>None<Object__Object>+0x22
13 00000069`d727d820 00007ff6`acbf3eef     System_Text_Json_SourceGeneration_Roslyn4_4_Tests!S_P_CoreLib_System_Reflection_DynamicInvokeInfo__InvokeWithFewArguments+0x9e [/_/src/coreclr/nativeaot/System.Private.CoreLib/src/System/Reflection/DynamicInvokeInfo.cs @ 504] 
14 00000069`d727d8e0 00007ff6`acc50d50     System_Text_Json_SourceGeneration_Roslyn4_4_Tests!S_P_CoreLib_System_Reflection_DynamicInvokeInfo__Invoke+0xbf [/_/src/coreclr/nativeaot/System.Private.CoreLib/src/System/Reflection/DynamicInvokeInfo.cs @ 245] 

Looking at the disassembly at the callsite in the test:

        [Theory]
        [MemberData(nameof(GetUnsupportedValues))]
        public async Task SerializeUnsupportedType<T>(ValueWrapper<T> wrapper)
        {
            // ....
            NotSupportedException ex = await Assert.ThrowsAsync<NotSupportedException>(async () => await Serializer.SerializeWrapper(value));
            // ....
        }

00007ff6`ae550556 e8c5516cfe         call    System_Text_Json_SourceGeneration_Roslyn4_4_Tests!S_P_CoreLib_System_Runtime_TypeLoaderExports__GVMLookupForSlot (7ff6acc15720)
00007ff6`ae55055b a802               test    al, 2
00007ff6`ae55055d 7424               je      System_Text_Json_SourceGeneration_Roslyn4_4_Tests!System_Text_Json_SourceGeneration_Roslyn4_4_Tests_System_Text_Json_Serialization_Tests_UnsupportedTypesTests___c__DisplayClass6_0_1<System___Canon>__<AsyncCallable>System_Text_Json_SourceGeneration_Roslyn4_4_Tests_System_Text_Json_Serialization_Tests_UnsupportedTypesTests___c__DisplayClass6_0_1___SerializeUnsupportedType_b__0+0xb3 (7ff6ae550583)
00007ff6`ae55055f 33d2               xor     edx, edx
00007ff6`ae550561 4889542420         mov     qword ptr [rsp+20h], rdx
00007ff6`ae550566 488b5006           mov     rdx, qword ptr [rax+6]
00007ff6`ae55056a 488bcb             mov     rcx, rbx
00007ff6`ae55056d 4c8bce             mov     r9, rsi
00007ff6`ae550570 4533c0             xor     r8d, r8d
00007ff6`ae550573 3909               cmp     dword ptr [rcx], ecx
// ********** We are here **************
00007ff6`ae550575 ff50fe             call    qword ptr [rax-2]
// ********** /We are here **************
00007ff6`ae550578 4885c9             test    rcx, rcx
00007ff6`ae55057b 0f85d4000000       jne     System_Text_Json_SourceGeneration_Roslyn4_4_Tests!System_Text_Json_SourceGeneration_Roslyn4_4_Tests_System_Text_Json_Serialization_Tests_UnsupportedTypesTests___c__DisplayClass6_0_1<System___Canon>__<AsyncCallable>System_Text_Json_SourceGeneration_Roslyn4_4_Tests_System_Text_Json_Serialization_Tests_UnsupportedTypesTests___c__DisplayClass6_0_1___SerializeUnsupportedType_b__0+0x185 (7ff6ae550655)
00007ff6`ae550581 eb18               jmp     System_Text_Json_SourceGeneration_Roslyn4_4_Tests!System_Text_Json_SourceGeneration_Roslyn4_4_Tests_System_Text_Json_Serialization_Tests_UnsupportedTypesTests___c__DisplayClass6_0_1<System___Canon>__<AsyncCallable>System_Text_Json_SourceGeneration_Roslyn4_4_Tests_System_Text_Json_Serialization_Tests_UnsupportedTypesTests___c__DisplayClass6_0_1___SerializeUnsupportedType_b__0+0xcb (7ff6ae55059b)
00007ff6`ae550583 488bcb             mov     rcx, rbx
00007ff6`ae550586 4c8bc6             mov     r8, rsi
00007ff6`ae550589 33d2               xor     edx, edx
00007ff6`ae55058b 4533c9             xor     r9d, r9d
00007ff6`ae55058e 3909               cmp     dword ptr [rcx], ecx
00007ff6`ae550590 ffd0               call    rax

So we set up:

rcx: this
rdx: inst arg
r8: null
r9: value

However the target is:

        public override Task<string> SerializeWrapper<T>(T value, JsonSerializerOptions? options = null)
            => Task.FromResult(JsonSerializer.Serialize(value, GetOptions(options)));

00007ff6`aed0a980 56             push    rsi
00007ff6`aed0a981 53             push    rbx
00007ff6`aed0a982 4883ec38       sub     rsp, 38h
00007ff6`aed0a986 4889542430     mov     qword ptr [rsp+30h], rdx
00007ff6`aed0a98b 488b1a         mov     rbx, qword ptr [rdx]
00007ff6`aed0a98e 4c89442428     mov     qword ptr [rsp+28h], r8
00007ff6`aed0a993 498bd1         mov     rdx, r9
00007ff6`aed0a996 e865230ffe     call    System_Text_Json_SourceGeneration_Roslyn4_4_Tests!System_Text_Json_SourceGeneration_Roslyn4_4_Tests_System_Text_Json_SourceGeneration_Tests_StringSerializerWrapper__GetOptions (7ff6acdfcd00)

We're passing r9 assuming this was the (null) options.

I do not understand why this started failing now, I do not understand why #128313 doesn't repro it.

[dotnet-policy-service]

Tagging subscribers to this area: @JulieLeeMSFT, @jakobbotsch
See info in area-owners.md if you want to be subscribed.

[MichalStrehovsky]

Disabling the test in #128344