Use provider C# reference map with pre-write internalization

[live1206]

Summary

Replaces the C# generator's Roslyn-generated reference-map/internalization path with a provider-based reference map that runs before files are written. The generator now computes reachability and accessibility from the in-memory provider model, filters unreachable providers up front, and removes stale generated files by cleaning src/Generated before writing.

What changed

• Build generated-code reachability from provider metadata, signatures, expressions/bodies, helper dependencies, custom-code roots, API-baseline roots, and provider relationships instead of Roslyn-generated documents.
• Apply internal/public accessibility before output by updating providers directly, including model-factory filtering and XML-doc preservation for internalized types.
• Remove the generated-code Roslyn removal/internalization post-processing path and related stale-file preservation logic.
• Avoid stale generated-source state affecting current output, including old generated accessibility roots and stale ModelFactory usings.
• Keep generated helper, serialization, MRW, custom-code body, and request-body dependencies tied to actual reachable generated code, including parent-namespace custom roots and provider/custom MRW body dependencies.
• Avoid plugin-build stdout/stderr deadlocks by draining both streams asynchronously.

Performance

Latest full-generation benchmark on the updated benchmark branch (mtg-manual-name-reduction-latest-bench, commit 4cf86686e) averaged 544.5 ms / 53.85 MB with the provider reference map enabled vs 847.4 ms / 138.87 MB with it disabled: 35.7% faster and 61.2% less allocation.

Run	Provider reference map off	Provider reference map on	Speedup	Allocation off	Allocation on	Allocation reduction
1	822.2 ms	531.6 ms	35.3%	138.35 MB	51.64 MB	62.7%
2	891.8 ms	567.5 ms	36.4%	138.16 MB	53.88 MB	61.0%
3	828.1 ms	534.4 ms	35.5%	140.10 MB	56.03 MB	60.0%
Average	847.4 ms	544.5 ms	35.7%	138.87 MB	53.85 MB	61.2%

Benchmark artifacts/logs were saved locally under /tmp/typespec-benchmark-results-latest-runs-20260710030912.

Azure SDK management-plane local project-reference regen speedups:

Latest measured PR head: 434e76802 (test(csharp): rename body dependency reference map tests). Each package was regenerated one-by-one with RegenSdkLocal.ps1 -Parallel 1 using the local Azure SDK for .NET project-reference setup.

SDK package	Main baseline	This PR	Improvement
Azure.ResourceManager.Network	00:37:01.9	00:07:00.9	81.1% faster / 5.28x
Azure.ResourceManager.DataFactory	00:13:43.4	00:03:26.2	75.0% faster / 3.99x
Azure.ResourceManager.AppService	00:10:46.9	00:03:17.6	69.5% faster / 3.27x

Validation

• Latest targeted local Azure management-plane regens for Azure.ResourceManager.Network, Azure.ResourceManager.DataFactory, and Azure.ResourceManager.AppService passed one-by-one with no public API listing diffs.
• C# emitter build and focused generator/reference-map tests pass.
• Full Azure data-plane regen completed with no tracked SDK/library diffs.
• Targeted data-plane regen/build checks pass with zero targeted generated diffs for:
  • Azure.Data.AppConfiguration (ResponseError MRW context coverage)
  • Azure.AI.Projects (custom-code parent-namespace body dependency root coverage)
  • Azure.AI.Extensions.OpenAI (polymorphic/unknown model retention coverage)
• Latest main has been merged into the PR branch.

[azure-sdk-automation]

You can try these changes here

🛝 Playground	🌐 Website	🛝 VSCode Extension

[JoshLove-msft]

Correction to my follow-up review (point #1). @live1206 rightly pointed out that RequestHeaderExtensions is a TypeProvider — PipelineRequestHeadersExtensionsDefinition (ScmOutputLibrary.cs:90) — not a raw static file. My "there's no provider to mark" reasoning was wrong; please disregard it.

The real reason the textual .SetDelimited( scan / IsRequestHeadersExtensionsFile special-case is needed is different: the only reference to this provider is the request.Headers.SetDelimited(...) call emitted inside RestClientProvider method bodies, and the provider reference graph builds edges from provider metadata (signatures, properties, fields, attributes, declared helper/body deps) — not from arbitrary method-body expressions. RestClientProvider doesn't declare PipelineRequestHeadersExtensions via BuildHelperDependencyNames / BuildBodyDependencyTypes / IncludeGeneratedBodyReferences, so the graph can't see the body-level usage and the file falls back to the string scan.

So the same marker mechanism you introduced for ClientUriBuilder (IncludeGeneratedBodyReferences) could likely subsume this last special-case: have RestClientProvider declare PipelineRequestHeadersExtensions as a helper/body dependency — ideally conditionally, only when a create-request method actually emits SetDelimited — so the graph roots it precisely and the .SetDelimited( scan + HasCustomRequestHeaderExtensionsReference handling can be dropped from the MTG core. Not blocking, just noting it's the same class of body-reference gap the rest of the PR already handles via markers, so it'd be nice to make it consistent rather than a textual exception.

--generated by Copilot

[JoshLove-msft]

Design suggestion: derive body references from expression trees rather than declared deps + source re-parsing

Following up on the body-reference discussion — the current approach builds graph edges from provider metadata (signatures, properties, fields, attributes, etc.) and then compensates for body-level references three ways: manually-declared HelperDependencyNames / BodyDependencyTypes / IncludeGeneratedBodyReferences, AddGeneratedBodyReferences re-parsing generated source with Roslyn for a curated set of "candidate" providers, and hardcoded fallbacks (the SetDelimited scan, ChangeTracking helper roots, etc.).

The concern is soundness: a pure-metadata graph is an under-approximation of reachability, because a C# type reference can appear anywhere a method body can (new Helper(), SomeStatic.Call(), extension calls like request.Headers.SetDelimited(...), casts, typeof). For a removal/internalization pass, under-approximating is the dangerous direction — you conclude "nothing references X," internalize/remove it, and the break surfaces later as a downstream compile error or a silent public-API regression. Correctness then rests on provider authors remembering to declare every body dependency (a manually-maintained parallel list that will drift) plus the hand-curated candidate list.

Suggestion: walk the body expression trees instead. The bodies already exist as structured MethodBodyStatement / ValueExpression trees before they're rendered to source. Traversing those to collect CSharpType references (constructor targets, static member owners, type literals, generic args, cast targets) would be:

• sound by construction for anything the generator itself emits — it closes the gap generically instead of per-provider declaration, and would subsume the SetDelimited special-case and most BodyDependencyTypes/HelperDependencyNames overrides;
• likely cheaper than AddGeneratedBodyReferences re-parsing generated C# through a Roslyn semantic model.

The fact that the PR re-parses generated source for the client/serialization candidates (rather than walking expression trees) suggests the tree walk wasn't feasible for all bodies — presumably where bodies are raw TypeProvider source or interpolated string literals whose contents aren't structured ValueExpressions. If so, that's the honest residual limitation to document: expression-tree walking would handle the structured majority soundly, with a narrow, explicit fallback (or a CI superset-check against the existing full Roslyn map) for the unstructured remainder — instead of today's broader reliance on manual declarations + name-based special-cases.

Not a merge blocker, but I think it's the change that would turn this from "correct-by-testing" into "correct-by-construction." Would also be worth attributing how much of the ~21% comes from skipping body analysis vs. from pre-write internalization (skipping the Roslyn InternalizeAsync/reduce passes) — if it's mostly the latter, a sound full-body map may retain most of the win.

--generated by Copilot

[JoshLove-msft]

I think splitting this functionality over 10+ files is a bit overkill. Can we consolidate into a few files?

[live1206]

I revisited the split. The analyzer is about 3,000 lines; combining these responsibility-based partials would leave several 700-1,000 line files and make the graph construction, root selection, and candidate rules harder to navigate. I kept the split, but cleaned stale terminology and simplified the candidate flow in b612999.

[JoshLove-msft]

Even though this type is going away, we need to make sure we still have all the same scenarios covered using the replacement type.

[live1206]

Audited the six removed scenarios. Invalid/valid MRW attribute filtering is covered by RemovedProvidersDoNotContributeBuildableAttributes and the surrounding MRW context tests, including mixed kept/removed providers. The two using-removal tests exercised Roslyn source rewriting that no longer exists: generated references are provider-written/global-qualified, and we no longer rewrite source files to remove usings.

[JoshLove-msft]

Why is the suppression check only added for methods? Doesn't the Methods property already handle suppressions?

[live1206]

Methods does not apply CodeGenSuppress; it filters customized members, while IsMethodSuppressed evaluates the enclosing type suppression attributes. The body graph must skip suppressed methods because their generated bodies are not emitted. Other members do not have the equivalent CodeGenSuppress method-signature behavior, so this check is method-specific.

[JoshLove-msft]

Found several correctness gaps in the provider-based reference map that can cause generated compilation failures, API compatibility regressions, or missing MRW registrations.

[jorgerangel-msft]

why is this needed?

[live1206]

This replaces the equivalent Roslyn reference discovery currently used on main for custom code.

The existing map calls SymbolFinder.FindReferencesAsync for each generated type, and it has a separate path that finds references to every extension method. Those references are then attributed to the type declared by the referencing document, including customized documents. That is how a generated provider referenced only from handwritten custom code remains reachable today.

This PR removes that post-generation Roslyn map, so NamedTypeSymbolProvider extracts the same custom-code dependencies once from its existing semantic compilation and exposes them to the provider graph before writing files. The SetDelimited handling corresponds specifically to the old extension-method path: request.Headers.SetDelimited(...) contains no explicit syntax reference to the generated extension class, so the normal type-syntax walk cannot discover its declaring provider. The unresolved method dependency is subsequently matched to the unique generated extension provider by method name and receiver type. Without this replacement edge, a generated type/helper used only by custom code could be removed or internalized.

[live1206]

latest regen: Azure/azure-sdk-for-net#60965 (comment)

[jorgerangel-msft]

As I'm thinking about this some more, does it always make sense that we want to restore a type's attributes from the last GA version? What if a type went from being experimental with the Experimental attribute, then it GAs in the new release. I think we wouldn't want that attribute re-added. I'm leaning towards not having attributes supported as part of the back compat support, at least in this PR, until we have a stronger solution. @JoshLove-msft - any thoughts?