+semver:minor - feat(mocks): runtime auto-stubs for interfaces the source generator cannot see

[thomhurst]

Summary

Fixes #6514 — feature parity with runtime-proxy libraries' auto-substitution. When a loose-mode mock member must produce an interface the source generator produced no mock for — most notably a generic method like IInvocationFeatures.Get<T>() invoked inside a third-party SDK with a T that is internal to that SDK and structurally unnameable by the test assembly — the call fell through to null, surfacing as an NRE deep in SDK code. This was a hard migration blocker from NSubstitute (reported against the Azure Functions Worker SDK).

How it works

The key insight: the generated Get<T>() override is itself generic, so at runtime the engine sees the real System.Type even when the test assembly can't name it. The engine's existing auto-mock fallback (TryGetLooseAutoMockResult → MockRegistry.TryCreateAutoMock) now falls back to RuntimeStubGenerator on a registry miss:

• Emission: a Reflection.Emit type per interface (cached), in a dynamic assembly named DynamicProxyGenAssembly2 carrying Castle DynamicProxy's well-known public key — the exact identity SDKs across the ecosystem already grant InternalsVisibleTo for NSubstitute/Moq compatibility, so every internal interface reachable by those proxies is reachable by these stubs. Dynamic assemblies are never strong-name verified; the public key alone satisfies the grant.
• Behavior: NSubstitute-style defaults, computed lazily and cached per member for stable identity — "" for strings, completed tasks (Task<T>/ValueTask<T> included), empty collections, zeroed value types, recursive stubs for interface members (or a real configurable Mock<T> when a source-generated factory exists for that nested type). Properties round-trip set values. Generic interface methods are supported, including constraint re-emission and out-parameter zeroing.
• Guards: loose mode only (strict still throws); RuntimeFeature.IsDynamicCodeSupported — inert on Native AOT, which keeps the previous default-value behavior; opt-out via settings.Mocks.RuntimeAutoStubs = false; any emission failure (interface with no matching IVT grant, static abstract members, by-ref returns) is a cached miss falling back to the previous behavior. netstandard2.0 builds carry a no-op implementation.
• Stubs are deliberately not configurable or verifiable — their types can't be named in test code (that's the premise). Nameable interfaces keep getting real mocks.

Behavior change

In loose mode, unconfigured interface-returning calls that previously returned null (no generated factory) now return a functional stub on JIT runtimes — hence +semver:minor. Opt-out restores the old behavior globally.

Tests

Issue6514Tests (skipped on Native AOT via the existing SkipIfNotDynamicCodeSupported attribute): the issue's Get<T>-with-internal-type shape (the test project now grants IVT to DynamicProxyGenAssembly2 exactly like real SDKs do), unregistered public interfaces, recursion, task/collection/string defaults, identity caching, registry-preferred real mocks via Mock.Get, value-type defaults unchanged, strict mode unchanged, and the opt-out. Full suites: 1192/1192 (net10.0), 1191/1191 (net9.0), 1169/1169 (net8.0); generator snapshots 90/90; Http 54/54, Logging 31/31.

Docs: new Runtime Auto-Stubs section under Recursive / Auto-Mocking in docs/writing-tests/mocking/advanced.md.

[greptile-apps]

Greptile Summary

Runtime auto-stubs now provide loose-mode defaults for interfaces unavailable to source generation.

• Adds runtime-emitted interface implementations with recursive defaults, property/indexer state, generic methods, and stable per-member identity.
• Integrates runtime stubs into the existing auto-mock fallback with strict-mode, dynamic-code, and global opt-out guards.
• Adds regression coverage and documents runtime behavior and platform limitations.

Confidence Score: 5/5

The PR appears safe to merge.

No blocking failure remains.

Important Files Changed

Filename	Overview
src/TUnit.Mocks/RuntimeStubs/RuntimeStub.cs	Stores property and indexer state and caches defaults by member slot and closed return-type identity, resolving the previously reported generic-instantiation collision.
src/TUnit.Mocks/RuntimeStubs/RuntimeStubGenerator.cs	Emits cached runtime implementations for supported closed interfaces, including generic method signatures, constraints, accessors, events, and out-parameter initialization.
src/TUnit.Mocks/RuntimeStubs/RuntimeStubDefaults.cs	Produces recursive mocks or stubs and conventional defaults for strings, tasks, collections, arrays, nullable types, and value types.
src/TUnit.Mocks/MockEngine.cs	Adds runtime stubs to loose auto-mock fallback and keys the engine cache by member name and exact return-type identity.
tests/TUnit.Mocks.Tests/Issue6514Tests.cs	Covers inaccessible interfaces, recursive defaults, settings and platform guards, generic-instantiation isolation, identity caching, and stateful properties and indexers.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    A[Loose mock member call] --> B{Generated mock factory exists?}
    B -->|Yes| C[Return configurable generated mock]
    B -->|No| D{Runtime auto-stubs enabled and dynamic code supported?}
    D -->|No| E[Return previous default value]
    D -->|Yes| F[Emit or reuse interface stub type]
    F --> G[Create and cache stub per member and return type]
    G --> H[Return recursive default or stored property/indexer value]

Loading

_{Reviews (9): Last reviewed commit: "fix(mocks): preserve custom modifiers on..." | Re-trigger Greptile}

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 722f6f2571

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[chatgpt-codex-connector]

Keep null defaults out of the concurrent cache

When a runtime stub member returns an ordinary class, delegate, nullable value, or another unsupported reference type, RuntimeStubDefaults.GetDefault returns null, but ConcurrentDictionary.GetOrAdd rejects null values and throws ArgumentNullException. Thus loose stubs throw instead of returning the documented default; use a sentinel or bypass caching when the computed value is null.

Useful? React with 👍 / 👎.

[chatgpt-codex-connector]

Include the closed return type in the cache key

For an emitted generic member such as T Get<T>(), every constructed invocation shares the same numeric slot. After Get<int>() caches a boxed integer, a later Get<string>() retrieves that value and the emitted unbox.any/cast fails with InvalidCastException. Key the cache by both slot and the runtime return type so different generic instantiations retain independent defaults.

Useful? React with 👍 / 👎.

[claude]

Review

Clever fix for #6514 — reusing Castle DynamicProxy's well-known public key to piggyback on SDKs' existing InternalsVisibleTo grants is a neat way to reach unnameable internal types without needing per-SDK configuration. AOT/trimming guards, the opt-out flag, and the docs are all solid.

Two correctness issues, one already partially flagged by Greptile:

1. Generic-method return cache collides across type arguments (confirmed, same as Greptile's finding)

RuntimeStub.GetReturnValue(int slot, Type returnType) (RuntimeStub.cs:99-100) keys the cache by slot alone:

protected object? GetReturnValue(int slot, Type returnType)
    => _returnCache.GetOrAdd(slot, static (_, t) => RuntimeStubDefaults.GetDefault(t), returnType);

RuntimeStubGenerator explicitly supports generic methods on the stubbed interface (constraint re-emission, DefineGenericParameters, etc. in RuntimeStubGenerator.cs:490-536), and each generic method gets one fixed slot. If a single stub instance invokes that method twice with different type arguments (e.g. Convert<int>() then Convert<string>()), the second call reuses the first call's cached boxed value and Unbox_Any throws InvalidCastException — the generic-method support is untested for this exact case (Issue6514Tests only exercises Get<T>() on the source-generated mock of IStubFeatures, never a generic method declared on a runtime-stubbed interface).

Suggested fix: key the cache by (int Slot, Type ReturnType) instead of int alone — e.g. ConcurrentDictionary<(int, Type), object?>. That preserves per-instantiation identity stability (the stated goal) while eliminating the collision, and is a minimal change to the existing structure. Please also add a test that invokes a stubbed generic method with two different type arguments on the same stub instance.

2. Unsynchronized ModuleBuilder/TypeBuilder access under TUnit's parallel-by-default execution

EmitStubType (RuntimeStubGenerator.cs:349-425) calls module.DefineType(...) / tb.CreateType() with no synchronization beyond the ConcurrentDictionary.GetOrAdd on _stubTypes. GetOrAdd's factory can run concurrently for different keys — so two tests running in parallel that each first-touch a distinct never-before-stubbed interface will call into EmitStubType on the same shared ModuleBuilder from different threads simultaneously.

ModuleBuilder.DefineType/TypeBuilder.CreateType are documented as not thread-safe on coreclr without external synchronization (see dotnet/runtime#64094, "Generation of dynamic types with multiple threads ... causes runtime corruption"). Notably, Castle DynamicProxy — whose identity this code deliberately impersonates — wraps its own ModuleScope.DefineType in a lock for exactly this reason (castleproject/Core#399); this PR borrows Castle's assembly identity but not its concurrency safeguard. Given TUnit tests run in parallel by default, this looks reachable in normal use, not just a theoretical edge case.

Suggested fix: wrap the type-definition/creation section of EmitStubType (or at minimum module.DefineType(...) through tb.CreateType()) in a static lock, matching Castle's approach.

Everything else — the defaults table, out-parameter zeroing, recursive stub/mock preference order in RuntimeStubDefaults.GetDefault, and the opt-out test — looks correct and well covered.

[thomhurst]

Both P1s reviewed:

Generic return cache conflation — valid, fixed in 9bba13e. T Get<T>() on a stubbed interface shares one member slot across instantiations, so Get<string>() could retrieve Get<int>()'s boxed value and fail the emitted unbox/cast. The cache is now keyed by (slot, closed return type); regression test Stub_Generic_Method_Keeps_Instantiations_Separate covers int/interface/string instantiations plus same-instantiation identity.

"GetOrAdd rejects null values" — not correct, no change. ConcurrentDictionary only rejects null keys; null values are fine for a reference-type TValue. MockEngine.AutoMockCache (ConcurrentDictionary<string, IMock?>) in this same codebase already stores null values on every registry miss. Null defaults (plain classes, delegates, nullable value types) cache and round-trip as null exactly as documented.

[claude]

Review

Overall this is a well-scoped, carefully-guarded feature: gated behind RuntimeFeature.IsDynamicCodeSupported (inert on AOT), a documented opt-out, cached emission with a sane try/catch-to-null-on-miss fallback, and solid generic-constraint re-emission for the Get<T>() case the issue is actually about. The two P1s from the earlier review round (generic-instantiation cache conflation, GetOrAdd-null claim) both look correctly resolved in the current diff — the return cache is now keyed by (slot, closedReturnType) and there's a regression test for it (Stub_Generic_Method_Keeps_Instantiations_Separate).

One correctness issue I think is worth fixing before merge:

Indexer properties ignore their index argument and share a single cached value.

RuntimeStubGenerator.EmitAccessor (src/TUnit.Mocks/RuntimeStubs/RuntimeStubGenerator.cs:314-341) and EmitReturn (:362-394) key a property's cached value purely by slot — the emitted IL for a getter never loads the index parameter(s) onto the stack, and the setter only ever stores the last argument (the value) via SetPropertyValueMethod at that same fixed slot (:324-333). IsSupported (:189-207) only rejects static-abstract members and by-ref returns — it doesn't flag indexers, so an interface with an indexer is still considered "supported" and gets a stub silently emitted.

Concretely, for an interface with TValue this[TKey key] { get; set; } (e.g. IDictionary<,>-shaped interfaces, or something like ASP.NET's IHeaderDictionary — exactly the kind of HTTP "features" interface this PR's own motivating issue is about):

stub["A"] = valueA;
stub["B"] = valueB;
stub["A"]; // returns valueB, not valueA — same slot, index ignored

and for an unset indexer, stub[key1] and stub[key2] return the same cached default instance regardless of key. This is silent — it doesn't throw or fall back to the documented null/previous-default behavior, so it's a plausible source of confusing test failures for exactly the SDK-interface scenarios this feature targets.

Given the existing pattern in the file (reject unsupported shapes in IsSupported and let them fall back to the previous null-returning behavior, as already done for static-abstract members and by-ref returns), I'd suggest adding a check there for indexers (property.GetIndexParameters().Length > 0) rather than trying to key the cache by argument values — that keeps the fix small and consistent with how every other unsupported shape in this emitter is already handled, and it's a safe fallback since loose-mode null is already an accepted (if less convenient) outcome elsewhere in this design.

Everything else — the module/type caching, the DynamicProxyGenAssembly2 identity trick, the generic constraint re-emission, the AOT/netstandard2.0 fallback split — looks solid and well-tested. Nice work on the fast turnaround on the earlier review round.

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 9bba13e66d

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[chatgpt-codex-connector]

Construct ValueTask without ambiguous overload binding

When T is an ordinary reference type for which GetDefault(T) returns null, such as a sealed DTO, this reflective construction must choose between ValueTask<T>(T) and ValueTask<T>(Task<T>). Since null matches both unrelated parameter types, the default binder throws AmbiguousMatchException, so invoking a runtime-stub member returning ValueTask<Dto> fails instead of returning a completed task containing null. Select the constructor whose parameter is exactly T, or use a closed generic helper.

Useful? React with 👍 / 👎.

[thomhurst]

Second review round — both P2s verified and fixed in 5cae648:

Indexer state keyed by index arguments — valid. Emitted accessors now box the index arguments into an object?[] and state lives in a per-(slot, structural index key) map, so stub[1]/stub[2] round-trip independently (multi-parameter indexers included, covered by tests). Defaults remain index-independent for stable identity. Indexers whose index parameters can't be boxed (by-ref/ref-struct/pointer) keep slot-level behavior.

ValueTask<T> ambiguous construction — valid. ValueTask<T> has (T) and (Task<T>) ctors and a null inner default matches both; the ctor is now selected explicitly by parameter type. Test covers ValueTask<Dto> returning a completed task with a null result.

Suites: 1195/1195 net10.0, 1172/1172 net8.0.

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 5cae64836e

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[thomhurst]

Third round — Codex P2 (stub cache keyed by FullName string) verified and fixed in 89c6d6b.

Valid: MockEngine.AutoMockCache keyed memberName + "|" + typeof(TReturn).FullName, so two types with identical namespace-qualified names from different assemblies shared one cache entry — the second invocation retrieved the first type's stub and the (TReturn) cast threw InvalidCastException. The cache (all 9 sites, MockEngine.Typed.cs included) is now keyed by (memberName, Type) — Type identity, never a name string. TryGetAutoMock (no callers anywhere) updated to match.

Regression test emits two dynamic assemblies each declaring Issue6514.Collision.ISameFullName and resolves both through the same stubbed Get<T>() — each gets its own stub implementing its own type.

Suites: 1196/1196 net10.0, 1173/1173 net8.0.

[claude]

Review

Checked this round's changes against the three prior review rounds — the generic-instantiation cache fix ((slot, returnType) in RuntimeStub), the indexer state fix (index-argument-keyed _indexerValues), the ValueTask<T> explicit-ctor fix, and this round's (memberName, Type) cache-key fix (replacing the FullName-string collision across assemblies) are all correctly implemented and match what was described in the commit messages.

One issue from round 1 appears to have been dropped rather than fixed:

ModuleBuilder/TypeBuilder access in EmitStubType is still unsynchronized.

TryCreateStub (src/TUnit.Mocks/RuntimeStubs/RuntimeStubGenerator.cs:83-96) calls _stubTypes.GetOrAdd(interfaceType, ...), whose factory invokes EmitStubType. ConcurrentDictionary.GetOrAdd only avoids running the factory twice for the same key — it does nothing to serialize factory execution across different keys. EmitStubType (:120-124, 189) calls module.DefineType(...) and tb.CreateType() on the single shared _module (EnsureModule, :221-233) with no lock around it. Two tests that each first-touch a distinct, never-before-stubbed interface concurrently — the default mode under TUnit's parallel test execution — will call into DefineType/CreateType on the same ModuleBuilder from different threads at the same time, which is documented as unsafe on coreclr (dotnet/runtime#64094) and is exactly why Castle DynamicProxy — whose assembly identity this PR deliberately reuses — wraps its own ModuleScope.DefineType in a lock.

I flagged this in the first review round; the reply addressing "Both P1s" fixed the generic-cache collision and a separate GetOrAdd-null claim, but this thread-safety item wasn't among them and there's still no lock or test exercising concurrent first-time stub creation in Issue6514Tests.cs.

Suggested fix, consistent with the lock-based pattern already used throughout this same project (MockEngine.cs, MethodSetup.cs, MockRepository.cs, CompositeBehavior.cs all guard shared mutable state this way): wrap the body of EmitStubType (or at least EnsureModule() through tb.CreateType()) in a static lock, e.g.

private static readonly Lock _emitLock = new();
...
lock (_emitLock)
{
    var module = EnsureModule();
    ...
    return tb.CreateType();
}

This keeps the existing ConcurrentDictionary-based per-interface caching (so the common case of a cache hit stays lock-free) while serializing the actual type-emission path, matching Castle's approach for the identity this code borrows.

Everything else in this round — Type-identity cache keys across all call sites in MockEngine.Typed.cs, the TryGetAutoMock signature update, and the new regression test for cross-assembly same-FullName collisions — looks correct and well covered.

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 89c6d6bf54

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[chatgpt-codex-connector]

Route typed cache misses through runtime stubs

When a generated non-generic member has 1–8 arguments and returns an interface without a generated factory—for example, IServiceProvider Resolve(int id), since MemberDiscovery.GetAutoMockFactoryMethod excludes System.* interfaces—the generator selects this typed handler. Its cache factory only queries MockRegistry, so a miss still returns null and never reaches the new RuntimeStubGenerator fallback used by the object-array handler. Apply the runtime-stub fallback to every typed arity as well, preferably through the shared helper.

Useful? React with 👍 / 👎.