Test failure: GC stress assertion (GetComponentSize() <= 2) || IsArray() in methodtable.cpp on x64 (Loader explicit-layout objrefandnonobjrefoverlap/case1)

[JulieLeeMSFT]

Summary:
Under gcstress0x3 on x64, the Loader explicit-layout negative test objrefandnonobjrefoverlap/case1 (which expects a System.TypeLoadException because MyUnion2 overlaps an object field with a non-object field) aborts: while that expected exception propagates, a GC stackwalk validates an object whose MethodTable is inconsistent and trips (GetComponentSize() <= 2) || IsArray() at vm/methodtable.cpp:6304 (exit code -1073740286 instead of the expected 100).

Build Information:

• runtime-coreclr gcstress0x3-gcstress0xc 20260613.1

Error Message:

15:13:05.456 Running test: Loader/classloader/explicitlayout/objrefandnonobjrefoverlap/case1/case1.dll
System.TypeLoadException: Could not load type 'MyUnion2' from assembly 'case1, Version=0.0.0.0, Culture=neutral, PublicKeyToken=null' because it contains an object field at offset 0 that is incorrectly aligned or overlapped by a non-object field.
   at Test.Go()
   at Test.TestEntryPoint()

Assert failure(PID 9684 [0x000025d4], Thread: 4488 [0x1188]): (GetComponentSize() <= 2) || IsArray()

CORECLR! MethodTable::SanityCheck + 0x39 (0x00007ffc`00750a29)
CORECLR! MethodTable::Validate + 0xE (0x00007ffc`00753efe)
CORECLR! Object::ValidateInner + 0x119 (0x00007ffc`00764e99)
CORECLR! Object::Validate + 0xB0 (0x00007ffc`00764d40)
CORECLR! TGcInfoDecoder<AMD64GcInfoEncoding>::ReportStackSlotToGC + 0x165 (0x00007ffc`00c15c35)
CORECLR! TGcInfoDecoder<AMD64GcInfoEncoding>::EnumerateLiveSlots + 0x185D (0x00007ffc`00c111ad)
CORECLR! EECodeManager::EnumGcRefs + 0x2E7 (0x00007ffc`0060f737)
CORECLR! GcStackCrawlCallBack + 0x34B (0x00007ffc`0093f15b)
CORECLR! Thread::StackWalkFramesEx + 0x1FE (0x00007ffc`007cb5ae)
CORECLR! Thread::StackWalkFrames + 0x1D1 (0x00007ffc`007cb381)
    File: D:\a\_work\1\s\src\coreclr\vm\methodtable.cpp:6304
    Image: C:\h\w\B78609C9\p\corerun.exe

Exit Code: -1073740286
Expected: 100
Actual: -1073740286
END EXECUTION - FAILED

Affected test legs:

runtime-coreclr gcstress0x3-gcstress0xc
- coreclr windows x64 Checked gcstress0xc @ Windows.10.Amd64.Open
    - Loader.0.3

Console log: Console Log

Source: runtime-coreclr gcstress0x3-gcstress0xc / coreclr windows x64 Checked gcstress0xc @ Windows.10.Amd64.Open / Loader.0.3

Analysis:
The test case1.dll deliberately loads an explicit-layout struct (MyUnion2) that overlaps an object field with a non-object field, so a System.TypeLoadException is expected. Under GC stress, a stackwalk (Thread::StackWalkFrames -> EECodeManager::EnumGcRefs -> Object::Validate -> MethodTable::Validate -> SanityCheck) runs while this type is in a half-loaded/invalid state and asserts (GetComponentSize() <= 2) || IsArray() at src/coreclr/vm/methodtable.cpp:6304, aborting the process (exit -1073740286). This is a closely-related sibling of the object.cpp:548 !CREATE_CHECK_STRING(pMT && pMT->Validate()) assert seen in the other Loader work items (Loader.1.3/2.3/LoaderClassloaderGenerics) — both are GC object/MethodTable validation tripping on a mid-failed-load type under gcstress — but the verbatim assert site differs.

Related issues:

• Bad GC info during ReadOnlySpan<T> byref copy #128801 — Bad GC info during ReadOnlySpan byref copy (GC stack scanning reports a bad interior pointer; likely the same class of bad-GC-info root cause).
• Test failure: GC stress assertion !CREATE_CHECK_STRING(pMT && pMT->Validate()) in object.cpp on x64 (Loader explicit-layout/generics tests) #129545 — sibling !CREATE_CHECK_STRING(pMT && pMT->Validate()) assert at object.cpp:548 on x64 (Loader.1.3/2.3/Generics).

Milestone: 11.0.0

Note

This issue was generated with assistance from GitHub Copilot (AI).

[dotnet-policy-service]

Tagging subscribers to this area: @JulieLeeMSFT, @jakobbotsch
See info in area-owners.md if you want to be subscribed.

[JulieLeeMSFT]

Please check if it's a duplicate to #129546.

[EgorBo]

Root-cause analysis: this is a VM exception-dispatch / GC-stackwalk bug, not a JIT (codegen) bug

I localized this with a minimal repro + VM instrumentation. The JIT's machine code, GC info and unwind data for the affected method are all correct. The hole is in how the VM's GC stackwalk recovers a callee-saved register holding a GC ref for a managed frame while an exception thrown from a prestub is being dispatched. #129545 and #129546 are the same root cause (different asserts depending on the garbage value).

Minimal repro (x64, reproduces ~100%)

using System;
using System.Collections.Generic;
using System.Reflection;
using System.Runtime.CompilerServices;
using System.Runtime.InteropServices;
using System.Threading;

public struct Wrap<T> { public T o; }

// Invalid explicit layout: overlaps an objref (Wrap<T>) with an int field.
// JIT-compiling Use<T> needs BadGen<T>'s layout, so it throws TypeLoadException from the prestub.
[StructLayout(LayoutKind.Explicit)]
public struct BadGen<T> where T : class
{
    [FieldOffset(0)] public int i;
    [FieldOffset(0)] public Wrap<T> o;
}

public static class Trigger
{
    [MethodImpl(MethodImplOptions.NoInlining)]
    public static int Use<T>() where T : class { BadGen<T> b; b.i = 0; b.o.o = default; return b.i; }
}

public static class Program
{
    static volatile bool s_stop;

    // Distinct reference types so each Use<T> first-call JIT (and its TypeLoad failure) is *fresh*,
    // which widens the window in which a GC can hit while the exception is dispatching.
    static readonly Type[] s_types = MakeTypes();
    static Type[] MakeTypes()
    {
        var list = new List<Type>();
        Type t = typeof(string);
        for (int k = 0; k < 300; k++) { list.Add(t); t = typeof(List<>).MakeGenericType(t); }
        return list.ToArray();
    }

    static void ThrowWorker()
    {
        MethodInfo open = typeof(Trigger).GetMethod("Use");
        int i = 0;
        while (!s_stop)
        {
            // The reflection invoke stub boxes the int return value: that box (a GC ref) is held
            // in a callee-saved register across the call to Use<T>(), which throws from its prestub.
            try { open.MakeGenericMethod(s_types[i++ % s_types.Length]).Invoke(null, null); }
            catch (TargetInvocationException) { }
        }
    }

    static void AllocWorker()
    {
        var keep = new object[64];
        int i = 0;
        while (!s_stop) keep[i++ % keep.Length] = new byte[(i & 0x3FFF) + 32];
    }

    static int Main()
    {
        for (int i = 0; i < Math.Max(2, Environment.ProcessorCount / 2); i++)
            new Thread(ThrowWorker) { IsBackground = true }.Start();
        for (int i = 0; i < 2; i++)
            new Thread(AllocWorker) { IsBackground = true }.Start();
        Thread.Sleep(60_000);
        s_stop = true;
        return 100;
    }
}

Run (checked corerun):

set DOTNET_TieredCompilation=0
set DOTNET_GCStress=0xC      (0x3 reproduces just as reliably)
corerun gchole_minimal.dll

It fires the exact asserts from both issues, e.g. (#129545):

Assert failure: !CREATE_CHECK_STRING(pMT && pMT->Validate())
CORECLR! Object::ValidateInner
CORECLR! Object::Validate
CORECLR! TGcInfoDecoder<AMD64GcInfoEncoding>::ReportRegisterToGC + 0x196
CORECLR! TGcInfoDecoder<AMD64GcInfoEncoding>::ReportSlotToGC
CORECLR! TGcInfoDecoder<AMD64GcInfoEncoding>::EnumerateLiveSlots
CORECLR! EECodeManager::EnumGcRefs
CORECLR! GcStackCrawlCallBack
CORECLR! Thread::StackWalkFramesEx
CORECLR! ScanStackRoots
    File: src/coreclr/vm/object.cpp:548

Mechanism

1. A caller holds a GC reference in a callee-saved register (rbx on x64) live across a call. In the repro the GC ref is the box for the boxed int return value in the reflection invoke stub; in general it's any GC ref kept live across the call (e.g. object Inner<T>(object box){ Use<T>(); return box; }).
2. The callee (Use<T>) can't be JIT-compiled (invalid overlapped layout) and throws TypeLoadException from its prestub. ThePreStub saved the caller's callee-saved registers (incl. the box) into the PrestubMethodFrame's TransitionBlock.
3. While the exception dispatches, the PrestubMethodFrame is popped. A GC then stackwalks and reports the caller frame, but the caller is scanned as the active frame (CrawlFrame::IsActiveFrame()==1, and no PrestubMethodFrame remains in the thread's Frame chain). So rbx is read from the live/redirected context, which no longer holds the box → a bogus OBJECTREF → Object::Validate asserts.

VM-side evidence (I instrumented ReportRegisterToGC → Object::Validate):

• offending method = the caller of the prestub (dynamicClass::InvokeStub_Trigger.Use, or for an equivalent direct repro Trigger.Inner[System.__Canon]), reg=3 (rbx).
• cf->IsActiveFrame()==1; the thread's Frame chain at the failure contains an InlinedCallFrame + ResumableFrame (gcstress redirect) but no PrestubMethodFrame.
• the stack slot it reads rbx from is sometimes the TransitionBlock region and sometimes a far, already-unwound location (slot-sp observed as both -0x38 and -41680) — both garbage by scan time.

The JIT output for the caller is correct — rbx is live at the call site, saved by push rbx, and the unwind info records it:

; Trigger:Inner[System.__Canon](System.Object):System.Object  (partially interruptible)
;  V01 arg0   ref -> rbx   "box"
push     rbx
...
mov      rbx, rdx                       ; rbx = box (GC ref, callee-saved)
call     [Trigger:Use[System.__Canon]():int]   ; throws TypeLoadException from prestub
mov      rax, rbx
...
pop      rbx
; GC info: slot rbx Live at the call site (0x1d); call site recorded
; Unwind:  UWOP_PUSH_NONVOL rbx

Why it's narrow (and why it is not a codegen bug)

• Throwing the same exception from managed code does not repro: the managed callee's physical frame keeps the caller's callee-saved registers recoverable via normal unwind.
• A call with no GC ref live across it does not repro.
• Only prestub-thrown exception + GC ref in a callee-saved register live across the call triggers it.

Notes

• Reproduces under both DOTNET_GCStress=0x3 (alloc/safepoint stress) and 0xC (instruction stress), and with both legacy and new EH (DOTNET_LegacyExceptionHandling=1 still crashes). So it is not new-EH-specific, and in principle a rare real GC during such dispatch could hit it outside gcstress.
• Not a recent regression — it reproduces unchanged at 7da2acb (2026-05-23) and earlier (I built and tested it); the CI "first failure" date is just the intermittent failure crossing the flakiness threshold. So there is no useful bisect target.

Candidate fix area (for the Exceptions team)

The GC stackwalk needs the faulting managed frame's callee-saved-register context to remain valid while a native-origin (prestub) exception dispatches. src/coreclr/vm/stackwalk.cpp (~L1382-1383) notes the ExInfo substitutes a frame's context only "if the ExInfo is due to an exception in managed code"; for a prestub (native) throw, once the PrestubMethodFrame is popped the caller frame is scanned as active with the live context instead of recovering the registers from the PrestubMethodFrame/TransitionBlock (TransitionFrame::UpdateRegDisplay_Impl) or the saved exception context.

Since the JIT output is correct and the defect is in the VM EH/GC-stackwalk path, re-routing from area-CodeGen-coreclr to area-Exceptions-coreclr.