Disable graph-based type checking for FSharp.Core

[nojaf]

Compiling the core, a puzzle untamed,
In its depths, mysteries remain unclaimed.
A dance of code, elusive and free,
Not meant to unravel, a digital sea.

In recent days I've been seeing errors when compiling FSharp.Core.fsproj using the new --test:GraphBasedChecking flag. (SDK 8.0.100-preview.7.23376.3)

error FS0192: internal error: Encountered exception when processing item 'PhysicalFile 2'
Unhandled exception. System.Exception: Encountered exception when processing item 'PhysicalFile 2'
 ---> System.Collections.Generic.KeyNotFoundException: The given key was not present in the dictionary.
   at FSharp.Compiler.TypedTree.NonLocalEntityRef.get_Deref() in C:\Users\nojaf\Projects\fsharp\src\Compiler\TypedTree\TypedTree.fs:line 3449
   at FSharp.Compiler.CheckExpressions.UpdateAccModuleOrNamespaceType(TcFileState cenv, TcEnv env, FSharpFunc`2 f) in C:\Users\nojaf\Projects\fsharp\src\Compiler\Checking\CheckExpressions.fs:line 1073
   at FSharp.Compiler.CheckDeclarations.EstablishTypeDefinitionCores.TcMutRecDefns_Phase1@3909-2.Invoke(TcEnv envAbove, Tuple`2 tupledArg) in C:\Users\nojaf\Projects\fsharp\src\Compiler\Checking\CheckDeclarations.fs:line 3913
   at FSharp.Compiler.CheckDeclarations.MutRecShapes.iterWithEnv[a,b,c,d](FSharpFunc`2 f1, FSharpFunc`2 f2, FSharpFunc`2 f3, FSharpFunc`2 f4, a env, FSharpList`1 x) in C:\Users\nojaf\Projects\fsharp\src\Compiler\Checking\CheckDeclarations.fs:line 173
   at FSharp.Compiler.CheckDeclarations.MutRecShapes.iterTyconsWithEnv[a,b,c,d](FSharpFunc`2 f1, a env, FSharpList`1 xs) in C:\Users\nojaf\Projects\fsharp\src\Compiler\Checking\CheckDeclarations.fs:line 179
   at FSharp.Compiler.CheckDeclarations.EstablishTypeDefinitionCores.TcMutRecDefns_Phase1[LetInfo,a,MemberInfo](FSharpFunc`2 mkLetInfo, TcFileState cenv, TcEnv envInitial, ParentRef parent, FSharpSet`1 typeNames, Boolean inSig, UnscopedTyparEnv tpenv, Range m, Range scopem, FSharpOption`1 mutRecNSInfo, FSharpList`1 mutRecDefns) in C:\Users\nojaf\Projects\fsharp\src\Compiler\Checking\CheckDeclarations.fs:line 3908
   at FSharp.Compiler.CheckDeclarations.TcDeclarations.TcMutRecDefinitions(TcFileState cenv, TcEnv envInitial, ParentRef parent, FSharpSet`1 typeNames, UnscopedTyparEnv tpenv, Range m, Range scopem, FSharpOption`1 mutRecNSInfo, FSharpList`1 mutRecDefns, Boolean isMutRec) in C:\Users\nojaf\Projects\fsharp\src\Compiler\Checking\CheckDeclarations.fs:line 4426
   at FSharp.Compiler.CheckDeclarations.TcModuleOrNamespaceElementNonMutRec@4960-1.Invoke(CancellationToken ct) in C:\Users\nojaf\Projects\fsharp\src\Compiler\Checking\CheckDeclarations.fs:line 4979
   at FSharp.Compiler.CheckDeclarations.TcModuleOrNamespaceElementNonMutRec@4960-15.Invoke(CancellationToken ct) in C:\Users\nojaf\Projects\fsharp\src\Compiler\Checking\CheckDeclarations.fs:line 4960
   at FSharp.Compiler.DiagnosticsLogger.DiagnosticsLoggerExtensions.ReraiseIfWatsonable(Exception exn) in C:\Users\nojaf\Projects\fsharp\src\Compiler\Facilities\DiagnosticsLogger.fs:line 425
   at FSharp.Compiler.DiagnosticsLogger.DiagnosticsLoggerExtensions.DiagnosticsLogger.ErrorRecovery(DiagnosticsLogger x, Exception exn, Range m) in C:\Users\nojaf\Projects\fsharp\src\Compiler\Facilities\DiagnosticsLogger.fs:line 476
   at FSharp.Compiler.CheckDeclarations.TcModuleOrNamespaceElementNonMutRec@4960-15.Invoke(CancellationToken ct) in C:\Users\nojaf\Projects\fsharp\src\Compiler\Checking\CheckDeclarations.fs:line 5182
   at FSharp.Compiler.CheckDeclarations.TcModuleOrNamespaceElementsNonMutRec@5196-1.Invoke(CancellationToken ct) in C:\Users\nojaf\Projects\fsharp\src\Compiler\Checking\CheckDeclarations.fs:line 5196
   at FSharp.Compiler.CheckDeclarations.TcModuleOrNamespaceElements@5329-5.Invoke(CancellationToken ct) in C:\Users\nojaf\Projects\fsharp\src\Compiler\Checking\CheckDeclarations.fs:line 5329
   at FSharp.Compiler.CheckDeclarations.TcModuleOrNamespaceElementNonMutRec@5063-13.Invoke(CancellationToken ct) in C:\Users\nojaf\Projects\fsharp\src\Compiler\Checking\CheckDeclarations.fs:line 5063
   at FSharp.Compiler.CheckDeclarations.TcModuleOrNamespaceElementNonMutRec@4960-15.Invoke(CancellationToken ct) in C:\Users\nojaf\Projects\fsharp\src\Compiler\Checking\CheckDeclarations.fs:line 4960
   at FSharp.Compiler.DiagnosticsLogger.DiagnosticsLoggerExtensions.ReraiseIfWatsonable(Exception exn) in C:\Users\nojaf\Projects\fsharp\src\Compiler\Facilities\DiagnosticsLogger.fs:line 425
   at FSharp.Compiler.DiagnosticsLogger.DiagnosticsLoggerExtensions.DiagnosticsLogger.ErrorRecovery(DiagnosticsLogger x, Exception exn, Range m) in C:\Users\nojaf\Projects\fsharp\src\Compiler\Facilities\DiagnosticsLogger.fs:line 476
   at FSharp.Compiler.CheckDeclarations.TcModuleOrNamespaceElementNonMutRec@4960-15.Invoke(CancellationToken ct) in C:\Users\nojaf\Projects\fsharp\src\Compiler\Checking\CheckDeclarations.fs:line 5182
   at FSharp.Compiler.CheckDeclarations.TcModuleOrNamespaceElementsNonMutRec@5196-1.Invoke(CancellationToken ct) in C:\Users\nojaf\Projects\fsharp\src\Compiler\Checking\CheckDeclarations.fs:line 5196
   at FSharp.Compiler.CheckDeclarations.TcModuleOrNamespaceElements@5329-5.Invoke(CancellationToken ct) in C:\Users\nojaf\Projects\fsharp\src\Compiler\Checking\CheckDeclarations.fs:line 5329
   at FSharp.Compiler.CheckDeclarations.TcModuleOrNamespaceElementNonMutRec@5151-14.Invoke(CancellationToken ct) in C:\Users\nojaf\Projects\fsharp\src\Compiler\Checking\CheckDeclarations.fs:line 5151
   at FSharp.Compiler.CheckDeclarations.TcModuleOrNamespaceElementNonMutRec@4960-15.Invoke(CancellationToken ct) in C:\Users\nojaf\Projects\fsharp\src\Compiler\Checking\CheckDeclarations.fs:line 4960
   at FSharp.Compiler.CheckDeclarations.CheckOneImplFile@5544-1.Invoke(CancellationToken ct) in C:\Users\nojaf\Projects\fsharp\src\Compiler\Checking\CheckDeclarations.fs:line 5563
   at FSharp.Compiler.ParseAndCheckInputs.CheckOneInputWithCallback@1568-12.Invoke(CancellationToken ct) in C:\Users\nojaf\Projects\fsharp\src\Compiler\Driver\ParseAndCheckInputs.fs:line 1568
   at FSharp.Compiler.ParseAndCheckInputs.CheckOneInputWithCallback@1490-14.Invoke(CancellationToken ct) in C:\Users\nojaf\Projects\fsharp\src\Compiler\Driver\ParseAndCheckInputs.fs:line 1490
   at FSharp.Compiler.DiagnosticsLogger.DiagnosticsLoggerExtensions.ReraiseIfWatsonable(Exception exn) in C:\Users\nojaf\Projects\fsharp\src\Compiler\Facilities\DiagnosticsLogger.fs:line 425
   at FSharp.Compiler.DiagnosticsLogger.DiagnosticsLoggerExtensions.DiagnosticsLogger.ErrorRecovery(DiagnosticsLogger x, Exception exn, Range m) in C:\Users\nojaf\Projects\fsharp\src\Compiler\Facilities\DiagnosticsLogger.fs:line 476
   at FSharp.Compiler.ParseAndCheckInputs.CheckOneInputWithCallback@1490-14.Invoke(CancellationToken ct) in C:\Users\nojaf\Projects\fsharp\src\Compiler\Driver\ParseAndCheckInputs.fs:line 1609
   at Internal.Utilities.Library.Cancellable.runWithoutCancellation[T](Cancellable`1 comp) in C:\Users\nojaf\Projects\fsharp\src\Compiler\Utilities\illib.fs:line 894
   at FSharp.Compiler.ParseAndCheckInputs.processFile@1840(Boolean priorErrors, TcConfig tcConfig, TcImports tcImports, TcGlobals tcGlobals, FSharpOption`1 prefixPathOpt, NodeToTypeCheck node, ParsedInput input, DiagnosticsLogger logger, TcState currentTcState, Boolean _arg31) in C:\Users\nojaf\Projects\fsharp\src\Compiler\Driver\ParseAndCheckInputs.fs:line 1850
   at FSharp.Compiler.ParseAndCheckInputs.processFile@1876-2.Invoke(NodeToTypeCheck node, Tuple`2 state) in C:\Users\nojaf\Projects\fsharp\src\Compiler\Driver\ParseAndCheckInputs.fs:line 1882
   at FSharp.Compiler.ParseAndCheckInputs.TypeCheckingGraphProcessing.workWrapper@1706.Invoke(FSharpFunc`2 getProcessedNode, NodeInfo`1 node) in C:\Users\nojaf\Projects\fsharp\src\Compiler\Driver\ParseAndCheckInputs.fs:line 1716
   at FSharp.Compiler.GraphChecking.GraphProcessing.processNode@123[Item,Result](FSharpFunc`2 work, CancellationTokenSource localCts, CancellationTokenSource cts, IReadOnlyDictionary`2 nodes, FSharpFunc`2 getItemPublicNode, IncrementableInt processedCount, Object lockObj, FSharpRef`1 exn, GraphNode`2 node) in C:\Users\nojaf\Projects\fsharp\src\Compiler\Driver\GraphChecking\GraphProcessing.fs:line 127
   at FSharp.Compiler.GraphChecking.GraphProcessing.queueNode@114-2.Invoke(Unit unitVar) in C:\Users\nojaf\Projects\fsharp\src\Compiler\Driver\GraphChecking\GraphProcessing.fs:line 114
   at Microsoft.FSharp.Control.AsyncPrimitives.CallThenInvoke[T,TResult](AsyncActivation`1 ctxt, TResult result1, FSharpFunc`2 part2) in C:\Users\nojaf\Projects\fsharp\src\FSharp.Core\async.fs:line 510
   at Microsoft.FSharp.Control.Trampoline.Execute(FSharpFunc`2 firstAction) in C:\Users\nojaf\Projects\fsharp\src\FSharp.Core\async.fs:line 112
   --- End of inner exception stack trace ---
   at FSharp.Compiler.GraphChecking.GraphProcessing.processGraph[Item,Result](IReadOnlyDictionary`2 graph, FSharpFunc`2 work, CancellationToken parentCt) in C:\Users\nojaf\Projects\fsharp\src\Compiler\Driver\GraphChecking\GraphProcessing.fs:line 153
   at FSharp.Compiler.ParseAndCheckInputs.TypeCheckingGraphProcessing.processTypeCheckingGraph(IReadOnlyDictionary`2 graph, FSharpFunc`2 work, TcState emptyState_0, Boolean emptyState_1, CancellationToken ct) in C:\Users\nojaf\Projects\fsharp\src\Compiler\Driver\ParseAndCheckInputs.fs:line 1720
   at FSharp.Compiler.ParseAndCheckInputs.CheckMultipleInputsUsingGraphMode@1866-1.Invoke(FSharpList`1 inputsWithLoggers) in C:\Users\nojaf\Projects\fsharp\src\Compiler\Driver\ParseAndCheckInputs.fs:line 1886
   at FSharp.Compiler.ParseAndCheckInputs.UseMultipleDiagnosticLoggers[a,b](FSharpList`1 inputs, DiagnosticsLogger diagnosticsLogger, FSharpOption`1 eagerFormat, FSharpFunc`2 f) in C:\Users\nojaf\Projects\fsharp\src\Compiler\Driver\ParseAndCheckInputs.fs:line 808
   at FSharp.Compiler.ParseAndCheckInputs.CheckMultipleInputsUsingGraphMode[a](a ctok, FSharpFunc`2 checkForErrors, TcConfig tcConfig, TcImports tcImports, TcGlobals tcGlobals, FSharpOption`1 prefixPathOpt, TcState tcState, FSharpFunc`2 eagerFormat, FSharpList`1 inputs) in C:\Users\nojaf\Projects\fsharp\src\Compiler\Driver\ParseAndCheckInputs.fs:line 1866
   at FSharp.Compiler.ParseAndCheckInputs.CheckClosedInputSet(CompilationThreadToken ctok, FSharpFunc`2 checkForErrors, TcConfig tcConfig, TcImports tcImports, TcGlobals tcGlobals, FSharpOption`1 prefixPathOpt, TcState tcState, FSharpFunc`2 eagerFormat, FSharpList`1 inputs) in C:\Users\nojaf\Projects\fsharp\src\Compiler\Driver\ParseAndCheckInputs.fs:line 1902
   at FSharp.Compiler.Driver.TypeCheck(CompilationThreadToken ctok, TcConfig tcConfig, TcImports tcImports, TcGlobals tcGlobals, DiagnosticsLogger diagnosticsLogger, String assemblyName, TcEnv tcEnv0, FSharpList`1 openDecls0, FSharpList`1 inputs, Exiter exiter) in C:\Users\nojaf\Projects\fsharp\src\Compiler\Driver\fsc.fs:line 169
   at FSharp.Compiler.DiagnosticsLogger.DiagnosticsLoggerExtensions.ReraiseIfWatsonable(Exception exn) in C:\Users\nojaf\Projects\fsharp\src\Compiler\Facilities\DiagnosticsLogger.fs:line 425
   at FSharp.Compiler.DiagnosticsLogger.DiagnosticsLoggerExtensions.DiagnosticsLogger.ErrorRecovery(DiagnosticsLogger x, Exception exn, Range m) in C:\Users\nojaf\Projects\fsharp\src\Compiler\Facilities\DiagnosticsLogger.fs:line 476
   at FSharp.Compiler.Driver.TypeCheck(CompilationThreadToken ctok, TcConfig tcConfig, TcImports tcImports, TcGlobals tcGlobals, DiagnosticsLogger diagnosticsLogger, String assemblyName, TcEnv tcEnv0, FSharpList`1 openDecls0, FSharpList`1 inputs, Exiter exiter) in C:\Users\nojaf\Projects\fsharp\src\Compiler\Driver\fsc.fs:line 181
   at FSharp.Compiler.Driver.main1(CompilationThreadToken ctok, String[] argv, LegacyReferenceResolver legacyReferenceResolver, Boolean bannerAlreadyPrinted, ReduceMemoryFlag reduceMemoryUsage, CopyFSharpCoreFlag defaultCopyFSharpCore, Exiter exiter, IDiagnosticsLoggerProvider diagnosticsLoggerProvider, DisposablesTracker disposables) in C:\Users\nojaf\Projects\fsharp\src\Compiler\Driver\fsc.fs:line 687
   at FSharp.Compiler.Driver.CompileFromCommandLineArguments(CompilationThreadToken ctok, String[] argv, LegacyReferenceResolver legacyReferenceResolver, Boolean bannerAlreadyPrinted, ReduceMemoryFlag reduceMemoryUsage, CopyFSharpCoreFlag defaultCopyFSharpCore, Exiter exiter, IDiagnosticsLoggerProvider loggerProvider, FSharpOption`1 tcImportsCapture, FSharpOption`1 dynamicAssemblyCreator) in C:\Users\nojaf\Projects\fsharp\src\Compiler\Driver\fsc.fs:line 1256
   at FSharp.Compiler.CommandLineMain.main(String[] argv) in C:\Users\nojaf\Projects\fsharp\src\fsc\fscmain.fs:line 79
   at FSharp.Compiler.DiagnosticsLogger.DiagnosticsLoggerExtensions.ReraiseIfWatsonable(Exception exn) in C:\Users\nojaf\Projects\fsharp\src\Compiler\Facilities\DiagnosticsLogger.fs:line 425
   at FSharp.Compiler.DiagnosticsLogger.DiagnosticsLoggerExtensions.DiagnosticsLogger.ErrorRecovery(DiagnosticsLogger x, Exception exn, Range m) in C:\Users\nojaf\Projects\fsharp\src\Compiler\Facilities\DiagnosticsLogger.fs:line 476
   at FSharp.Compiler.DiagnosticsLogger.errorRecovery(Exception exn, Range m) in C:\Users\nojaf\Projects\fsharp\src\Compiler\Facilities\DiagnosticsLogger.fs:line 577
   at FSharp.Compiler.CommandLineMain.main(String[] argv) in C:\Users\nojaf\Projects\fsharp\src\fsc\fscmain.fs:line 96
One or more errors occurred. (Command execution failed because the underlying process (fsc.exe#45060) returned a non-zero exit code (-532462766).

I was able to reproduce this behaviour for multiple files in FSharp.Core which leads me to suspect that this is a problem with mutable state, threads and a race condition.

I'd like to stress that so far, I have not seen this behaviour outside of FSharp.Core! Keeping that in mind, I've done some digging and found that the problem occurs when compilingCanonicalFslibModuleType is true:

fsharp/src/Compiler/Checking/CheckBasics.fs

Line 364 in 8a6a476

compilingCanonicalFslibModuleType = (isSig || not haveSig) && g.compilingFSharpCore

The error happens in UpdateAccModuleOrNamespaceType, KeyNotFoundException can be raised for various modules:

fsharp/src/Compiler/Checking/CheckExpressions.fs

Lines 1067 to 1075 in 3183d55

	let UpdateAccModuleOrNamespaceType (cenv: cenv) env f =
	// When compiling FSharp.Core, modify the fslib CCU to ensure forward stable references used by
	// the compiler can be resolved ASAP. Not at all pretty but it's hard to
	// find good ways to do references from the compiler into a term graph.
	if cenv.compilingCanonicalFslibModuleType then
	let nleref = mkNonLocalEntityRef cenv.thisCcu (arrPathOfLid env.ePath)
	let modul = nleref.Deref
	modul.entity_modul_type <- MaybeLazy.Strict (f true modul.ModuleOrNamespaceType)
	SetCurrAccumulatedModuleOrNamespaceType env (f false (GetCurrAccumulatedModuleOrNamespaceType env))

let modul = nleref.Deref can raise an exception.

It is my understanding that

fsharp/src/Compiler/Checking/CheckDeclarations.fs

Lines 4819 to 4825 in 681069f

	and TcSignatureElements cenv parent endm env xml mutRecNSInfo defs =
	cancellable {
	// Ensure the .Deref call in UpdateAccModuleOrNamespaceType succeeds
	if cenv.compilingCanonicalFslibModuleType then
	let checkXmlDocs = cenv.diagnosticOptions.CheckXmlDocs
	let xmlDoc = xml.ToXmlDoc(checkXmlDocs, Some [])
	ensureCcuHasModuleOrNamespaceAtPath cenv.thisCcu env.ePath env.eCompPath xmlDoc

fsharp/src/Compiler/TypedTree/TypedTreeOps.fs

Lines 1191 to 1212 in 6cb9792

	/// When compiling FSharp.Core.dll we have to deal with the non-local references into
	/// the library arising from env.fs. Part of this means that we have to be able to resolve these
	/// references. This function artificially forces the existence of a module or namespace at a
	/// particular point in order to do this.
	let ensureCcuHasModuleOrNamespaceAtPath (ccu: CcuThunk) path (CompPath(_, cpath)) xml =
	let scoref = ccu.ILScopeRef
	let rec loop prior_cpath (path: Ident list) cpath (modul: ModuleOrNamespace) =
	let mtype = modul.ModuleOrNamespaceType
	match path, cpath with
	| hpath :: tpath, (_, mkind) :: tcpath ->
	let modName = hpath.idText
	if not (Map.containsKey modName mtype.AllEntitiesByCompiledAndLogicalMangledNames) then
	let mty = Construct.NewEmptyModuleOrNamespaceType mkind
	let cpath = CompPath(scoref, prior_cpath)
	let smodul = Construct.NewModuleOrNamespace (Some cpath) taccessPublic hpath xml [] (MaybeLazy.Strict mty)
	mtype.AddModuleOrNamespaceByMutation smodul
	let modul = Map.find modName mtype.AllEntitiesByCompiledAndLogicalMangledNames
	loop (prior_cpath @ [(modName, Namespace true)]) tpath tcpath modul
	| _ -> ()
	loop [] path cpath ccu.Contents

should add some modules or namespaces that will later be found by nleref.Deref. Because of the parallel nature, this appears to be no longer happening as expected. Hence the KeyNotFoundException.

mtype.AddModuleOrNamespaceByMutation smodul clearly mutates the data in

fsharp/src/Compiler/TypedTree/TypedTree.fs

Lines 1996 to 2000 in 5483858

	/// Mutation used during compilation of FSharp.Core.dll
	member _.AddModuleOrNamespaceByMutation(modul: ModuleOrNamespace) =
	entities <- QueueList.appendOne entities modul
	modulesByDemangledNameCache <- None
	allEntitiesByMangledNameCache <- None

which I believe is unpredictable right now during graph mode.

I'm not quite sure about the exact details of what is transpiring here. If someone knows more about compiling FSharp.Core, I'd be happy to chat/pair. For now, I propose disabling graph-based type-checking for FSharp.Core. Understandably, this is rather anti-climactic, but I'm not yet convinced the problem can occur outside FSharp.Core, so this seems a sensible approach. As we already had a bit of a hack for FSharp.Core ("prim-types-prelude.fsi" as a dependency for all files), maybe it is not meant to be.

Anyway, I'm happy to hear your thoughts and collaborate.

//cc @safesparrow