Skip to content

Refactor common variables used for the "send runtime tests to helix" steps into a shared template that all of our pipelines that run runtime tests can use.#81926

Merged
jkoritzinsky merged 12 commits into
dotnet:mainfrom
jkoritzinsky:runtime-test-variables-template
Feb 23, 2023

Conversation

@jkoritzinsky
Copy link
Copy Markdown
Member

@jkoritzinsky jkoritzinsky commented Feb 9, 2023

This will help us be able to be more consistent with our variables by giving us a place to centralize variables we use in the split-job run-test-job.yml model and the single-job build-runtime-tests-and-send-to-helix.yml model.

Extracted from #81924

…steps into a shared template that all of our pipelines that run runtime tests can use.
@jkoritzinsky jkoritzinsky requested a review from radical as a code owner February 9, 2023 23:39
@ghost
Copy link
Copy Markdown

ghost commented Feb 9, 2023

I couldn't figure out the best area label to add to this PR. If you have write-permissions please help me learn by adding exactly one area label.

@ghost ghost assigned jkoritzinsky Feb 9, 2023
@ghost
Copy link
Copy Markdown

ghost commented Feb 10, 2023

Tagging subscribers to this area: @dotnet/runtime-infrastructure
See info in area-owners.md if you want to be subscribed.

Issue Details

This will help us be able to be more consistent with our variables by giving us a place to centralize variables we use in the split-job run-test-job.yml model and the single-job build-runtime-tests-and-send-to-helix.yml model.

Extracted from #81924

Author: jkoritzinsky
Assignees: jkoritzinsky
Labels:

area-Infrastructure

Milestone: -

# example:
# - [key]: [value]
- ${{ if and(eq(variable.name, ''), eq(variable.group, '')) }}:
- ${{ if and(eq(variable.name, ''), eq(variable.group, ''), eq(variable.template, '')) }}:
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Arcade PR is here: dotnet/arcade#12524

Comment thread eng/pipelines/runtime.yml Outdated
Comment on lines +1246 to +1252
extraVariablesTemplates:
- template: /eng/pipelines/common/templates/runtimes/test-variables.yml
forwardedParameters:
- osGroup
- osSubgroup
- runtimeFlavor
- runtimeVariant
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there a way to avoid this added noise? I see that this is now added to every runtime tests build leg. Can't the build-runtime-tests-and-send-to-helix.yml have this attached?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll try to figure out a way to make this less noisy.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tried to attach it to build-runtime-tests-and-send-to-helix.yml, but I couldn't get it to work.

The problem is that variables in YAML are a job level concept, but we're trying to add extra variables for a specific step. We can make the extra variables templates have the same parameters passed as the extra steps templates, but it doesn't look like we can attach variables to a step template unless we make our "extra steps" templates into job templates that wrap the global-build-job template, which I don't like.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've done the simplification I can (inline the most common "forwarded parameters" so we don't need to manually forward the common ones), but I can't do any more as you can't templatize what type of template a file is.

@jkoritzinsky
Copy link
Copy Markdown
Member Author

Windows failure is #82140.

@ViktorHofer ViktorHofer requested a review from a team February 17, 2023 12:50
@ViktorHofer
Copy link
Copy Markdown
Member

@dotnet/runtime-infrastructure can you please take a look? Asking as I don't have any cycles right now.

Comment thread eng/pipelines/runtime.yml
extraStepsParameters:
creator: dotnet-bot
testBuildArgs: nativeaot tree nativeaot
liveLibrariesBuildConfig: Release
Copy link
Copy Markdown
Member

@agocke agocke Feb 22, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It looks like this code is passing -lc Release above. How does the liveLibrariesBuildConfig setting affect this?

Copy link
Copy Markdown
Member Author

@jkoritzinsky jkoritzinsky Feb 22, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The liveLibrariesBuildConfig for the extra steps template needs to match whatever is passed for the -lc parameter above if the libraries are built different than the runtime. The nativeaot-post-build-steps.yml template assumes that the passed in $(_BuildConfig) was used to build the runtime.

Copy link
Copy Markdown
Member

@agocke agocke left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@jkoritzinsky
Copy link
Copy Markdown
Member Author

Test failure is known and this change doesn't affect the build leg that timed out.

@jkoritzinsky jkoritzinsky merged commit 1afd89f into dotnet:main Feb 23, 2023
@jkoritzinsky jkoritzinsky deleted the runtime-test-variables-template branch February 23, 2023 22:49
@ghost ghost locked as resolved and limited conversation to collaborators Mar 26, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants