Allow FileWritesShared to be tracked and cleaned with an opt-in flag

[baronfel]

This allows features like the .NET SDK's Artifacts Layout to correctly track and clean outputs that aren't under the project directory, but are in project-isolated bubbles.

Part of dotnet/sdk#49582.

Context

The core problem is that during the Publish, one of the things that happens is that we call the _CleanGetCurrentAndPriorFileWrites target from Microsoft.Common.CurrentVersion.targets. The purpose of this Target is to identify all of the files that were written and that are deleteable from the current build. These files are then tracked in a file that future builds use as an input to their own cleanup operations. The dotnet build -c Release command does read this file to delete files as necessary, but in this circumstance the file simply lacks all of the content that the self-contained build dumps into the artifacts/bin/myproj/release directory.

I'll inline the Target so we can discuss it:

msbuild/src/Tasks/Microsoft.Common.CurrentVersion.targets

Lines 5733 to 5806 in 25d1e4c

	<!--
	============================================================
	_CleanGetCurrentAndPriorFileWrites
	Get the list of files built in the current build and in prior builds.
	============================================================
	-->
	<Target
	Name="_CleanGetCurrentAndPriorFileWrites"
	DependsOnTargets="_CheckForCompileOutputs;_SGenCheckForOutputs">
	<!-- Read the list of files produced by a prior builds from disk. -->
	<ReadLinesFromFile File="$(IntermediateOutputPath)$(CleanFile)">
	<Output TaskParameter="Lines" ItemName="_CleanUnfilteredPriorFileWrites"/>
	</ReadLinesFromFile>
	<!--
	Convert the list of references to the absolute paths so we can make valid comparisons
	across two lists
	-->
	<ConvertToAbsolutePath Paths="@(_ResolveAssemblyReferenceResolvedFiles)">
	<Output TaskParameter="AbsolutePaths" ItemName="_ResolveAssemblyReferenceResolvedFilesAbsolute"/>
	</ConvertToAbsolutePath>
	<!--
	Subtract any resolved assembly files from *prior* file writes because deleting
	these would break subsequent builds because the assemblies would be unresolvable.
	-->
	<ItemGroup>
	<_CleanPriorFileWrites Include="@(_CleanUnfilteredPriorFileWrites)" Exclude="@(_ResolveAssemblyReferenceResolvedFilesAbsolute)"/>
	</ItemGroup>
	<!--
	Of shareable files, keep only those that are in the project's directory.
	We never clean shareable files outside of the project directory because
	the build may be to a common output directory and other projects may need
	them.
	Only subtract the outputs from ResolveAssemblyReferences target because that's the
	only "Resolve" target that tries to resolve assemblies directly from the output
	directory.
	-->
	<FindUnderPath Path="$(MSBuildProjectDirectory)" Files="@(FileWritesShareable)" UpdateToAbsolutePaths="true">
	<Output TaskParameter="InPath" ItemName="FileWrites"/>
	</FindUnderPath>
	<!-- Find all files in the final output directory. -->
	<FindUnderPath Path="$(OutDir)" Files="@(FileWrites)" UpdateToAbsolutePaths="true">
	<Output TaskParameter="InPath" ItemName="_CleanCurrentFileWritesInOutput"/>
	</FindUnderPath>
	<!-- Find all files in the intermediate output directory. -->
	<FindUnderPath Path="$(IntermediateOutputPath)" Files="@(FileWrites)" UpdateToAbsolutePaths="true">
	<Output TaskParameter="InPath" ItemName="_CleanCurrentFileWritesInIntermediate"/>
	</FindUnderPath>
	<!--
	Subtract any resolved assembly files from *current* file writes because deleting
	these would break subsequent builds because the assemblies would be unresolvable.
	Only subtract the outputs from ResolveAssemblyReferences target because that's the
	only "Resolve" target that tries to resolve assemblies directly from the output
	directory.
	-->
	<ItemGroup>
	<_CleanCurrentFileWritesWithNoReferences Include="@(_CleanCurrentFileWritesInOutput);@(_CleanCurrentFileWritesInIntermediate)" Exclude="@(_ResolveAssemblyReferenceResolvedFilesAbsolute)"/>
	</ItemGroup>
	<!-- Remove duplicates from files produced in this build. -->
	<RemoveDuplicates Inputs="@(_CleanCurrentFileWritesWithNoReferences)" >
	<Output TaskParameter="Filtered" ItemName="_CleanCurrentFileWrites"/>
	</RemoveDuplicates>
	</Target>

The specific part of this Target that is failing is the part that tries to locate any FileWritesShareable MSBuild Items that live under the MSBuildProjectDirectory (meaning, the directory of the project file currently being built. FileWritesShareable are automatically tracked by the build, and for the publish do contain all of the runtime files that need to be cleaned. However, when the BaseOutputPath is directed into someplace that isn't under MSBuildProjectDirectory (as is the case with the artifacts layout), none of these FileWritesShareable will be found (because they are all in some location not under MSBuildProjectDirectory).

Changes Made

Adds an opt-in flag that consumers like the .NET SDK can set so that potentially-shared files outside of the directory bubble are tracked for cleanup.

Testing

Manual testing by passing the property in and verifying FileListAbsolute.txt content.

Notes

[Copilot]

Pull Request Overview

This PR adds an opt-in flag to allow the tracking and cleanup of FileWritesShareable items located outside the MSBuildProjectDirectory, enabling scenarios like the .NET SDK's Artifacts Layout.

• Introduces the TrackFileWritesShareableOutsideOfProjectDirectory property with a default value of false.
• Updates the FilesToTrackFromOutputDirectories item group to conditionally include FileWritesShareable based on the opt-in flag.
• Modifies the FindUnderPath tasks for the output and intermediate directories to use the updated FilesToTrackFromOutputDirectories.

[baronfel]

@rainersigwald I introduced a bug here. If we're not allowing cleaning FileWritesShared outside of the bubble, we should still clean them from inside the bubble. Instead of this Task's output going to FileWrites, it should go to FilesToTrackFromOutputDirectories.

    <!-- Even if we don't allow cleaning filewrites from outside of the project bubble, we should still include FileWritesShareable that are inside the bubble -->
    <FindUnderPath Condition="!$(TrackFileWritesShareableOutsideOfProjectDirectory)" Path="$(MSBuildProjectDirectory)" Files="@(FileWritesShareable)" UpdateToAbsolutePaths="true">
      <Output TaskParameter="InPath" ItemName="FilesToTrackFromOutputDirectories"/>
    </FindUnderPath>

This is the cause of several of the test failures in dotnet/sdk#49740.

[baronfel]

Reverted due to a performance regression:
#12192