Skip to content

Generate prebuilt report and include it in the intermediate nupkg (arcade-powered source-build) - #6121

Merged
dagood merged 7 commits into
dotnet:masterfrom
dagood:check-prebuilts
Sep 4, 2020
Merged

Generate prebuilt report and include it in the intermediate nupkg (arcade-powered source-build)#6121
dagood merged 7 commits into
dotnet:masterfrom
dagood:check-prebuilts

Conversation

@dagood

@dagood dagood commented Sep 3, 2020

Copy link
Copy Markdown
Member

Add Microsoft.DotNet.SourceBuild.Tasks. This is separate from Microsoft.DotNet.Arcade.Sdk because the dependencies it takes on NuGet libraries are in danger of conflicting with what's loaded by Arcade already. See #6014. The tasks in this new package let us track prebuilts, for dotnet/source-build#1715.

To use the tasks in Microsoft.DotNet.SourceBuild.Tasks, we restore that package in Tools.proj if we're running source-build, and then use it from a new build step AfterSourceBuild.proj. (Since we're relying on Execute restoring Tools.proj, we now need to add a build step rather than doing everything in BeforeTargets=Execute.) AfterSourceBuild.proj only runs if '$(ArcadeBuildFromSource)' == 'true', so I believe these changes will have no effect on existing infra.

I also moved the intermediate nupkg generation from a BeforeTargets="Execute" target to AfterSourceBuild.proj, so I could include the prebuilt report generated in AfterSourceBuild.proj in the intermediate nupkg. This is for dotnet/source-build#1725.

I included a test project for the new tasks. Currently there's only a tiny test to justify the project's existence. The tasks are simply copied from dotnet/source-build, so there isn't any significant test coverage yet. I want to make sure we have a test project ready so we don't have a barrier to entry to adding regression tests and such in the future.

@dagood dagood self-assigned this Sep 3, 2020
Comment on lines -26 to +30
Version="[%(ExactVersion)]"/>
Version="[%(SourceBuildIntermediateNupkgReference.ExactVersion)]"/>

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I had to make this explicit to avoid MSB4096, which seems to me to be an MSBuild bug: dotnet/msbuild#5707. I don't think it's worth adding a comment (when you're just reading through the file, it doesn't seem distracting to me), but it looks odd in this diff.

@dagood
dagood marked this pull request as ready for review September 3, 2020 20:56
@am11

am11 commented Sep 3, 2020

Copy link
Copy Markdown
Member

@dagood, a heads up. :) we have removed See the LICENSE file in the project root for more information. lines in #6111. It can easily be removed by, for example, running the following in macOS bash:

$ git remote add dotnet https://github.com/dotnet/arcade

# delete entire line containing the unwanted license third line in C# code, which are part of your PR branch (HEAD to dotnet/master)
$ git show HEAD...dotnet/master --name-only --pretty="" :/*.cs |\
    xargs grep -l "See the LICENSE file in the project root for more information." |\
    xargs sed -i '' '/See the LICENSE file in the project root for more information/d'

# delete part of line containing license third line in rest of the files, which are part of your PR (HEAD to dotnet/master)
$ git show HEAD...dotnet/master --name-only --pretty="" |\
    xargs grep -l "See the LICENSE file in the project root for more information." |\
    xargs sed -i '' 's/ See the LICENSE file in the project root for more information.//g'

# middle grep is there to avoid autofixing of abnormal file-endings (and errors line `sed: RE error: illegal byte sequence`)

after that it can be verified that there are no files containing the unwanted third line:

$ git grep -l 'See the LICENSE file in the project root for more information'
# or without -l to see the file contents

@dagood

dagood commented Sep 3, 2020

Copy link
Copy Markdown
Member Author

Interesting, thanks. Should probably put that info in #6111, since I'd expect this to happen a lot with that PR going in so quietly (as far as I can tell), and it would be good to have a common resource.

@am11

am11 commented Sep 3, 2020

Copy link
Copy Markdown
Member

Sure, added a comment here: #6111 (comment). It was a bit different script than for the on-going PRs (git ls-files instead of git show --name-only --pretty="").

@dagood

dagood commented Sep 4, 2020

Copy link
Copy Markdown
Member Author

I included a test project for the new tasks. Currently there's only a tiny test to justify the project's existence. The tasks are simply copied from dotnet/source-build, so there isn't any significant test coverage yet. I want to make sure we have a test project ready so we don't have a barrier to entry to adding regression tests and such in the future.

I'm going to remove the new test project, because tests are currently failing for me locally, even without my changes, so I can't efficiently look into the problems yet: #6126. No need to sort through this now.

@dagood

dagood commented Sep 4, 2020

Copy link
Copy Markdown
Member Author

@markwilkie @mmitche PTAL, in particular around the new step I'm adding to Execute.

@mmitche
mmitche requested a review from chcosta September 4, 2020 17:18
Comment thread src/Microsoft.DotNet.SourceBuild/tasks/src/EnumerableExtensions.cs Outdated
Comment thread src/Microsoft.DotNet.SourceBuild/tasks/src/UsageReport/WriteUsageBurndownData.cs Outdated
}

public override int GetHashCode() => (
PackageIdentity,

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does PackageIdentity have an appropriate GetHashCode for this?

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(or is a value type)

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is the ref type over at https://github.com/NuGet/NuGet.Client/blob/dev/src/NuGet.Core/NuGet.Packaging/Core/PackageIdentity.cs, and it has some nondefault implementation, but I can't say for sure. Seems to have worked fine for us so far though.

Comment thread src/Microsoft.DotNet.SourceBuild/tasks/src/UsageReport/WriteUsageBurndownData.cs Outdated
public const string CreationTimePropertyName = "BuildOutputPropsCreationTime";

[Required]
public ITaskItem[] NuGetPackages { get; set; }

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you include the expected metadata for this task item here?

Directory.CreateDirectory(Path.GetDirectoryName(OutputPath));

using (var outStream = File.Open(OutputPath, FileMode.Create))
using (var sw = new StreamWriter(outStream, new UTF8Encoding(false)))

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there a reason we're writing the xml text explicitly this rather using an XmlDocument or other such method?

@dagood

dagood commented Sep 4, 2020

Copy link
Copy Markdown
Member Author

@mmitche I intentionally didn't review the content of the CS files, they are just copied from source-build. Feel free to leave comments but in the interest of moving forward I think it's best if I don't address them now.

(Edit: I filed dotnet/source-build#1730 to track this.)

Add Microsoft.DotNet.SourceBuild.Tasks. Restore it in Tools.proj and use it from a build step proj. Move the intermediate nupkg generation to the build step so that the prebuilt report is available to insert into the int nupkg.
@dagood

dagood commented Sep 4, 2020

Copy link
Copy Markdown
Member Author

Addressed comments. I also rebased to split out the changes I made to the files post-copy into a separate commit: b9a8d2c. There were also some ternary statements doing some redundant Element calls that I fixed up with NullAsEmpty, removed some usings, and of course removed // See the LICENSE file in the project root for more information lines.

@dagood
dagood merged commit c7fbbab into dotnet:master Sep 4, 2020
@dagood
dagood deleted the check-prebuilts branch September 4, 2020 20:16
dagood added a commit to dagood/arcade that referenced this pull request Feb 2, 2021
…cade-powered source-build) (dotnet#6121)

# Conflicts:
#	Arcade.sln
#	src/Microsoft.DotNet.Arcade.Sdk/tools/Build.proj
#	src/Microsoft.DotNet.Arcade.Sdk/tools/DefaultVersions.props
dagood added a commit to dagood/arcade that referenced this pull request Feb 3, 2021
…cade-powered source-build) (dotnet#6121)

# Conflicts:
#	Arcade.sln
#	src/Microsoft.DotNet.Arcade.Sdk/tools/Build.proj
#	src/Microsoft.DotNet.Arcade.Sdk/tools/DefaultVersions.props
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants