Fix degraded codegen for inner recursive functions under realsig#19882
Open
T-Gro wants to merge 21 commits into
Open
Fix degraded codegen for inner recursive functions under realsig#19882T-Gro wants to merge 21 commits into
T-Gro wants to merge 21 commits into
Conversation
Two codegen bugs fixed: 1. TLR (Top-Level Routing) was disabled under --realsig+ via a blanket short-circuit in InnerLambdasToTopLevelFuncs, causing inner recursive functions to be emitted as closure classes instead of static methods. This produced ~23× perf regression for struct mutual recursion (#17607). Fix: Remove the realsig band-aid. Instead, add a moduleCloc field to IlxGenEnv that always points to the enclosing non-generic module class. TLR-lifted vals (IsCompiledAsTopLevel && !IsMemberOrModuleBinding) are routed to moduleCloc, preventing them from inheriting class typars of a generic enclosing type. 2. Constrained inline generics, when inlined into closures, attached their constraints to the closure class's type params. The Specialize<T> override (from FSharpTypeFunc) must be unconstrained to match its base signature. When constraints leaked, the JIT threw TypeLoadException (#14492). Fix: In EraseClosures CASE 1, strip constraints from both the Specialize override method-typars (CASE 1b) and the later closure class-typars (CASE 1a) at the CASE 1 head. Rewrite stripILGenericParamConstraints via mkILSimpleTypar to be future-proof (clears all constraint fields including CustomAttrsStored which carries IsUnmanagedAttribute). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…sent HOF - verifyILContains now throws on mismatch instead of silently returning CompilationResult.Failure (which callers were ignoring with |> ignore). - Unify checkILPresent/checkILNotPresent via shared checkILFragments HOF. - Expose verifyILPresent in Compiler.fs (symmetric with verifyILNotPresent). - Fix TypeTests.fs assertions exposed by the silent-failure fix. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Contributor
✅ No release notes required |
github-actions
Bot
added
AI-Tooling-Check-Bypassed
Contributor
|
🔍 Tooling Safety Check — Affects-Compiler-Output
|
New test files: - Regression_TLR_MutualInnerRec_StructuralAssertions.fs: 20 tests covering TLR scenarios (generic class, nested module, three-way rec, quotation, value rec) and constraint stripping (IL shape, ILVerify, >5 params CASE 2a, combined TLR+constraint). All run under both realsig on/off. - Regression_Specialize_ConstraintVerification.fs: 14 tests exercising each ILGenericParameterDef field stripped by mkILSimpleTypar (struct, not struct, unmanaged, new(), interface, comparison, combined) via ILVerify + run. - 4 new IL-baseline source files (mutual rec, captured env, generic, Point2D) with Off/On .il.bsl pairs confirming realsig parity. Regenerated baselines: - TestFunction06, TestFunction23: closures replaced by static methods - Match01: clo@4 closure removed (TLR fires under realsig+) - Unmanaged: virtual DirectInvoke → static func@3 Note: Match01 and TestFunction23 .net472.bsl baselines need TEST_UPDATE_BSL=1 regeneration on Windows CI (macOS cannot target net472). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…ix (#14492) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
T-Gro
force-pushed
the
fix/realsign-codegen
branch
from
c47ea8d to
5c08b88
Compare
- Add PR #19882 link to both release note entries - Apply fantomas formatting to il.fs and IlxGen.fs Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Member
|
I don't think it's ready for review, it has some legitimate looking build errors |
…uleCloc When moduleCloc has empty Enclosing (namespace-level / types-only file), the previous routing fell through TypeRefForCompLoc to <PrivateImplementationDetails$AsmName>, a single per-assembly type. Multiple files each with an inner-rec function having the same compiler-generated name (e.g. capture@N in two FSharpEmbedResource-derived modules of FSharp.Build) all dumped into that shared bucket and collided in the IL method table, producing FS2014 `duplicate entry 'capture@83' in method table` at write-time during the bootstrap compilation of FSharp.Build and FSharp.DependencyManager.Nuget under --realsig+. Fix: when moduleCloc.Enclosing is empty, route through CompLocForInitClass instead. TypeNameForInitClass embeds TopImplQualifiedName (per-file) so the lifted val lands in <StartupCode$AsmName>.$FileName, matching the pre-realsig codegen layout and giving the per-file isolation that prevents collisions. The Container<'T>-style fix (moduleCloc with non-empty Enclosing) is preserved unchanged. Verified by rebuilding the compiler with -bootstrap -buildnorealsig:$false; both FSharp.Build.dll and FSharp.DependencyManager.Nuget.dll compile clean. Adds a regression test in Regression_TLR_MutualInnerRec_StructuralAssertions that fails on the previous version and passes after this fix. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
… fix Regenerated from build 1449384 actual IL output: - TestFunction23.fs.RealInternalSignatureOn.OptimizeOn.il.net472.bsl - Match01.fs.RealInternalSignatureOn.il.net472.bsl - Regression_TLR_MutualInnerRec_Point2D.fs.RealInternalSignatureOff.il.bsl - Regression_TLR_MutualInnerRec_Point2D.fs.RealInternalSignatureOn.il.bsl The first two were known stale per the PR description (net472 needs TEST_UPDATE_BSL on Windows CI). The Point2D baselines drift was caused by the duplicate-name fix changing the routing of TLR-lifted vals at namespace-level to the per-file init class. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…rker) Previous extraction grabbed only the first chunk of paginated Actual: output; the correct full IL appears after the "Entire actual:" sentinel. Linux's Point2D baselines are now consistent for both realsig settings. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Windows ildasm 5.x prints whole-number float literals with a trailing dot (`ldc.r8 10.`), Linux ildasm strips the dot entirely (`ldc.r8 10`). Without normalization, any .bsl file with floats inevitably fails on one platform — observed on the new Point2D regression test. The new `unifyFloatLiterals` rule rewrites bare `ldc.r8 -?N` to the dotted form for both expected and actual, keeping comparisons platform- agnostic without forcing baselines to be regenerated per OS. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
abonie
approved these changes
Jun 4, 2026
github-project-automation
Bot
moved this from New
to In Progress
in F# Compiler and Tooling
Comment trimming (all 3 reviewers flagged): - IlxGen.AllocValReprWithinExpr: 9-line block down to 2 lines - IlxGen.moduleCloc field doc + AddEnclosingToEnv: one-liners - EraseClosures CASE 1 unconstrainedGenParams: 3-line narrative down to 1 - Regression_TLR_MutualInnerRec_StructuralAssertions: closureWithConstraint header, namespace-collision test doc, and inline assertion comments - Regression_Specialize_ConstraintVerification module doc: 16 lines down to 3 - ILChecker.unifyFloatLiterals: 3-line comment + redundant (?!\.) regex lookahead Correctness / clarity: - AbstractIL.stripILGenericParamConstraints: keep explicit field-by-field clear with CustomAttrsStored reset, accurate doc — the previous mkILSimpleTypar rewrite silently dropped Variance/MetadataIndex semantics that the comment did not mention - InnerLambdasToTopLevelFuncs: collapse two adjacent Some(f, arity) branches into one predicate (atTopLevel || arity <> 0 || not (isNil tps)) - Combined TLR+constraint test: actually assert constraint stripping (Specialize<class/valuetype patterns) — previously only checked the search@ symbol - TypeTests.fs `M(()) and M() produce same IL method signature`: test name was a silent-failure relic from before verifyILContains threw — assertion clearly shows Unit and int are distinct signatures. Renamed and pointed at #19615. - Drop the now-redundant `Issue 14492: >5 params closure chain produces D-suffix and unconstrained Specialize` Specialize/T duplication — the dedicated test next to it already covers that path; this one keeps the D-suffix-only assertion. No production-code behaviour change beyond field-by-field stripping in stripILGenericParamConstraints; CI baselines unaffected. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…5.5) Tests: - Extract shared `verifyPEAndRun` helper (GPT-5.5: duplicated PEFile + run tail across compileVerifyAndRun and both inline >5-params and Combined test bodies) - Combined test: drop `object Specialize<` positive assertion — the inlined constraint produces no Specialize<> for this closure shape, so the previous assertion broke Linux CI (build 1451155). Negative `Specialize<class`/ `Specialize<valuetype` checks are kept and remain meaningful (vacuously true when no Specialize is emitted, real if one ever appears with leaked constraint). Compiler: - IlxGen.GetEmptyIlxGenEnv + GenerateCode: bind `ccuLoc`/`fragLoc` once instead of computing CompLocForCcu/CompLocForFragment twice per record literal (GPT-5.5: cloc/moduleCloc pairing easy to desynchronize). Deferred (after triage): - verifyILContains vs verifyILPresent rename — all 3 reviewers flagged the name overlap, but the rename touches ~100 callers across the test suite, out of scope for this PR. Both helpers now throw on mismatch (silent failure already fixed); the matching-semantics difference will be documented separately. - withQName near-duplicate with line 1918-1921 cloc reset — sites differ in Range semantics (the latter intentionally omits Range update for FSI fragments), so a shared helper would add overhead more than it removes. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Refute and dismiss B1 ("debug-stepping clobber"): PR #19894 (d0e593f)
landed on origin/main on 2026-06-08 13:11, three days AFTER my last merge
on 2026-06-05 11:35. None of my commits (7757717, e2ed10f, f933f83)
touch the relevant IlxGen.fs lines. Merged origin/main now brings the
fix in — verified `if equals m range0` is present at IlxGen.fs:3178 and
:3743 in the working tree.
Apply M3: Release notes now cite #19075 in the constraint-stripping entry
(test `SRTP member constraint with IDisposable` explicitly targets that
issue's CLR segfault).
Soften M2: Drop the unverified "≈23x" specific number from release notes;
the perf magnitude is documented in #17607 itself with the original repro.
Apply L2: `unmanaged + equality` test now asserts no
`Specialize<valuetype (...modreq...)` or `T<valuetype (...modreq...)`
leakage — exercises the IsUnmanagedAttribute / modreq stripping path that
motivated the CustomAttrsStored clear in stripILGenericParamConstraints.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The 4 new TLR regression tests (Regression_TLR_MutualInnerRec, _CapturedEnv, _Generic, _Point2D) emit byte-identical IL under --realsig+ and --realsig-, which was previously expressed as two duplicated .bsl files per test (_.RealInternalSignatureOff.il.bsl + _.RealInternalSignatureOn.il.bsl). Replace each pair with a single shared file: <test>.fs.il.bsl. The bsl lookup chain in FileInlineDataAttribute.fs:94-106 already falls through to the bare .il.bsl after exhausting realsig-suffixed candidates, so both Realsig=Off and Realsig=On invocations now compare against the same file. Identity becomes a structural property of the test layout instead of a coincidence between two separately-maintained baselines. Reduces baseline byte count by ~50% for these tests and makes any future realsig divergence an immediate failure (the same .bsl can't match two different IL outputs). Note: these tests use no source-level `private` keyword, so they do not exercise realsig's tightened-visibility promise. A genuine `--realsig+` regression test exercising `private` data accessed by a TLR-lifted helper would legitimately produce divergent Off/On baselines and would need to revert to the split form. Left as a follow-up. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Adds runtime smoke tests for the hypothetical access-check failure scenarios identified in the bsl-essence review of this PR. Each tests a distinct shape where this PR's new TLR routing (lifting inner-rec helpers to module/init-class statics under --realsig+) could in principle trip a MethodAccessException / FieldAccessException / TypeAccessException by landing the lifted helper in an IL container that no longer has access to the source-`private` data it touches. Tests, each [<Theory; InlineData(true); InlineData(false)>] for realsig parity, each in its own [<Fact>]-style `let` so failures localize: 1. Module-private value accessed from TLR-lifted inner-rec 2. Type-private static accessed from TLR-lifted inner-rec inside same type 3. Private DU structural compare via TLR-lifted continuation 4. Generic + private nested type captured by TLR-lifted inner-rec Verified locally with the modified compiler: all 8 invocations (4 tests x realsig on/off) compile and run successfully — no access exceptions surface. The tests are therefore positive regressions: they protect future routing / visibility changes from regressing into the hypothetical AV path. Note: the existing 4 `Regression_TLR_MutualInnerRec*.fs` tests use no source-`private` keyword and so do not exercise realsig's tightened visibility promise. This new file fills that gap with runtime evidence. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
ILVerify on this PR (build 1454404) hit the 60-min default timeout and was cancelled mid-Release-build. Investigation against recent main builds: Build 1449768 (pre-#19894) : ILVerify 21.1 min, succeeded Build 1451079 (pre-#19894) : ILVerify 21.2 min, succeeded Build 1453983 (#19894 `d0e593f67`) : ILVerify 36.5 min, succeeded Build 1454404 (this PR HEAD): ILVerify 60.4 min, CANCELLED Two compounding causes: 1. Upstream PR #19894 ("Debug: rework or expressions stepping") added ~14 min to ILVerify on main alone — verified across multiple post-#19894 main builds. ILVerify is the only leg using -bootstrap, so it pays the compiler-build cost x3 (bootstrap, proto, final). 2. This PR makes TLR actually fire under --realsig+ for the first time (the whole point of #17607's fix). The compiler self-build now lifts thousands of inner-rec functions across FCS itself that were previously left as closures. That extra TLR work, multiplied by the x3 bootstrap cycle, pushes ILVerify past 60 min. Neither is "flaky". The work is real and load-bearing. Bumping the per-job timeout to 120 min restores headroom for the bootstrap+proto+final cycle across both Debug and Release configurations. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Wave 2 adversarial exploit (opus 4.8): a TLR-lifted helper emitted at module scope can lose access to source-private members of an enclosing type, throwing MethodAccessException at runtime under --realsig+. F# RecdFields always compile to IL assembly or better, so field access is safe; only val/method references need to be checked. Add SelectTLRVals predicate that walks the binding body and refuses TLR when a non-public val is referenced and realsig+ is on. Adds two regression tests covering the confirmed exploit shape (generic class + type-private static). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Wave 3 (opus 4.7 perf agent) found the previous guard refused TLR for any inner-rec referencing a private val, which silently defeated the PR's perf wins for the very common F# idiom of module-private helpers (state machines, parsers, scanners, predicates). 8 realistic shapes showed 1.33×-3.23× regression vs the pre-fix PR. The MethodAccessException risk only exists when the referenced private val lives in a CLASS/STRUCT — IL 'private' is type-scoped. Module-private vals compile to methods on the same module IL class as the lifted helper, so they remain accessible. Refine SelectTLRVals predicate to refuse only when vref's TryDeclaringEntity is a class/struct (not IsModuleOrNamespace). Wave 2 exploit still fixed (verified locally: W2A20 exit 0; canonical 17607-style pipeline TLR fires, 25ms vs 71ms over-aggressive refusal). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #17607, #14492, and #19075.
Under
--realsig+(since .NET 9), inner recursive functions were not lifted to static methods by TLR, producing closure allocations and losingtail.calls — causing up to 23× perf regression for struct-heavy mutual recursion.Additionally, constrained inline generics inlined into closures could leak type parameter constraints onto the
Specialize<T>override, causingTypeLoadExceptionor CLR segfault at runtime.Note:
.net472.bslbaselines for Match01 and TestFunction23 needTEST_UPDATE_BSL=1regeneration on Windows CI.