This repository was archived by the owner on Jan 23, 2023. It is now read-only.
Increase timeout on Linux, change macOS pool and move arm64 legs around - #38343
Merged
Conversation
Member
|
Was just chatting with @safern about this yesterday. I had a PR that hit the timeout but then succeeded even though the leg failed in AzDO. I acutally spot checked a few tests to see if it was generally slower, but at least the ones I looked at seemed comparable to x64. Makes me think we might actually be hitting some sort of transient hang, or perhaps a couple slow machines in the pool. I wish we had a way of looking at the test execution data in some sort of reports to see if we can spot patterns. Anyhow, I think this will help as a bandaid, but I think there's something more fishy going on here. |
ericstj
approved these changes
Jun 7, 2019
Member
|
Yeah, there's something very wrong with how things are running. This PR's Linux arm64 leg has now been in the Send to Helix step for 1h49m, and 0 test suites have completed.
cc: @MattGal |
Member
Author
|
cc @MattGal |
ViktorHofer
added
the
* NO MERGE *
Member
|
Was just chatting with @safern about this yesterday. I had a PR that hit the timeout but then succeeded even though the leg failed in AzDO. I actually spot checked a few tests to see if it was generally slower, but at least the ones I looked at seemed comparable to x64. Makes me think we might actually be hitting some sort of reporting lag. Is there any way we can look at Helix's responsiveness here? |
Member
|
FYI: here's the one I was looking at https://dev.azure.com/dnceng/public/_build/results?buildId=213414. |
Member
|
Perhaps it's just slow to start the work? Backed up queue? |
Member
Author
|
If this build would have taken 4 minutes longer it would have failed 🗡 |
Member
Author
|
I'll wait for Matt's response, he should know more. |
Member
|
Seems reasonable as a workaround; in the end though from what I've seen recently we really need to figure out how to send less work to this queue til we have > 29 machines (currently hardware is ordered but not provisioned) |
ViktorHofer
force-pushed
the
ViktorHofer-patch-1
branch
from
June 11, 2019 19:57
38aee58 to
b6add86
Compare
ViktorHofer
removed
the
* NO MERGE *
ViktorHofer
force-pushed
the
ViktorHofer-patch-1
branch
2 times, most recently
from
June 11, 2019 21:26
71f6958 to
a12f944
Compare
ViktorHofer
changed the title
ViktorHofer
changed the title
ViktorHofer
force-pushed
the
ViktorHofer-patch-1
branch
from
June 13, 2019 15:55
a12f944 to
2fc357a
Compare
Member
Author
|
If CI turns green I'll merge this today to unblock the hanging PRs. This contains multiple fixes, I updated the title and description. |
Member
Author
|
@ericstj Internal CLR error again. |
dagood
reviewed
Jun 16, 2019
| _BuildConfig: Debug | ||
| ${{ if eq(parameters.fullMatrix, 'false') }}: | ||
| x64_Release: | ||
| _BuildConfig: Release |
Member
There was a problem hiding this comment.
It looks like this plus:
${{ if eq(parameters.fullMatrix, 'true') }}:
x64_Debug:
_BuildConfig: Debugis causing the official Linux x64 artifacts to be built Debug mode rather than Release mode:
https://dev.azure.com/dnceng/internal/_build/results?buildId=223624 (before)
https://dev.azure.com/dnceng/internal/_build/results?buildId=223845 (after)
I was taking a look at this because the uptake of CoreFX into Core-Setup started failing from the first to the second. Debug mode definitely seems wrong to me, although I don't understand how it would be related to the break. (dotnet/core-setup#6817)
/cc @wtgodbe
Member
There was a problem hiding this comment.
Debug mode definitely seems wrong to me, although I don't understand how it would be related to the break
We don't embed the IBC optimization data in Debug builds so this very likely is the reason for the break. Thanks for root causing this! It wouldn't have occurred to me that this could be an official build configuration issue. (I started looking at product build on my local machine...)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
8 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This increases the timeout for all Linux jobs as the condition in corefx-base.yml is currently based on the job. As this change is temporary until we figured out what is causing the long test executions I'm ok with enabling it for all Linux legs.
Other changes:
Hosted macOSwhich runs with macOS 10.14 latest.cc @stephentoub