Cherry-pick changes from master to implement support for building on internal Jenkins. - #4224
Merged
Merged
Conversation
rolfbjarne
commented
Jun 12, 2018
rolfbjarne
commented
Member
- 9cdaf2b [jenkins] Cherry-pick a series of fixes for internal jenkins support from the d15-8 branch ([jenkins] Cherry-pick a series of fixes for internal jenkins support from the d15-8 branch #4195)
- ae7c72c Add support for building on Jenkins. (Add support for building on Jenkins. #4159)
- fa7f52d [jenkins] Improve api/generator diff reporting to say if there were changes, and if they were breaking or not. ([jenkins] Improve api/generator diff reporting to say if there were changes, and if they were breaking or not. #4138)
- f7527aa [jenkins] Revert unintended change that prevents the keychain from unlocking properly. ([jenkins] Revert unintended change that prevents the keychain from unlocking properly. #4158)
- e0e2f93 [jenkins] Fix scripts to be shellcheck-happy. ([jenkins] Fix scripts to be shellcheck-happy. #4148)
- 96d3445 [tests] Make sure accessing the keychain doesn't make macOS show any dialogs. ([tests] Make sure accessing the keychain doesn't make macOS show any dialogs. #4141)
- bece50d [jenkins] Don't treat API/Generator diff failures as errors. ([jenkins] Don't treat API/Generator diff failures as errors. #4127)
- c35adef [jenkins] Update the urls for html reports. ([jenkins] Update the urls for html reports. #4119)
- ccb2f59 [jenkins] Make it possible to skip API comparison by applying a label to a pull request. ([jenkins] Make it possible to skip API comparison by applying a label to a pull request. #3994)
- d7efd34 [tools] Improve error reporting when trying to create API/generator diff for a pull request with conflicts. ([tools] Improve error reporting when trying to create API/generator diff for a pull request with conflicts. #3966)
- fb322c4 [jenkins] Clean keystore on bots before running tests ([jenkins] Clean keystore on bots before running tests #3754)
- 61ec520 Bump maccore.
Because if invalid data gets into the store then some unit tests will always fail on that particular bot. Fix xamarin/maccore#640
…iff for a pull request with conflicts. (dotnet#3966) We can't create an API/generator diff for a pull request with conflicts, so detect this scenario and show a better error than this (from dotnet#3961): Comparing the changes between origin/pr/3961/merge^1 and HEAD: fatal: ambiguous argument 'origin/pr/3961/merge^1..HEAD': unknown revision or path not in the working tree. Use '--' to separate paths from revisions, like this: 'git <command> [<revision>...] -- [<file>...]' fatal: ambiguous argument 'origin/pr/3961/merge^1': unknown revision or path not in the working tree. Use '--' to separate paths from revisions, like this: 'git <command> [<revision>...] -- [<file>...]'
… to a pull request. (dotnet#3994) Implement support for skipping API comparison by applying a label to a pull request. This also required some refactoring to move existing code to fetch the labels for a pull request to a separate script.
* [jenkins] Update the urls for html reports. The urls for html reports changed as a consequence of a Jenkins update [1], so update how we construct the links accordingly. See also: * https://xamarinhq.slack.com/archives/C03CCJNR7/p1526646040000120 * https://xamarinhq.slack.com/archives/C03CCJNR7/p1526653512000249 [1] https://wiki.jenkins.io/display/JENKINS/HTML+Publisher+Plugin (update to v1.16) * Remove extra closing parenthesis.
…4127) This way such failures won't make the build show up as failed, which may cause other tooling to behave differently (and non-optimal).
A few general categories of fixes: * Sprinkle lots of quotes everywhere. * Don't use environment variables in the format string to printf, instead pass them as arguments. * Don't use backticks to execute commands (it's deprecated), use the new "$(...)" syntax instead.
…locking properly. (dotnet#4158)
…hanges, and if they were breaking or not. (dotnet#4138) For the api diff for this PR we now show: * `🔥 breaking changes 🔥`: If there are any breaking changes in the api diff (for this PR). * `please review changes`: If there are any non-breaking changes in the api diff. * `no change`: If the api diff is empty. For the generator diff we show: * `only version changes`: If there were only changes related to version numbers (since the XI/XM version number is added to the generator, that version number will always show up as a diff when comparing the generated source code) * `please review changes`: If anything other that version numbers changed in the generated source code.
Add support for building on internal Jenkins. Jenkins has been configured to build every branch on xamarin/xamarin-macios that contains a `jenkins/Jenkinsfile`, which means it will start working as soon as this PR is merged. Results will be posted as statuses on each commit, which can be viewed using the url `https://github.com/xamarin/xamarin-macios/commits/<branch>`:  * The `continuous-integration/jenkins/branch` status links to the jenkins job. * The other two are XI and XM packages (the `Jenkins-` prefix will be removed once we officially switch from Wrench to Jenkins). More detailed information will be added as a comment to each commit, which can be seen by clicking on the commit and scrolling to the bottom (url of the format `https://github.com/xamarin/xamarin-macios/commit/<sha1>`)  Unfortunately GitHub does not display the commit statuses when viewing a single commit, so to view those statuses you'll have to view the list of commits (the `/commits/` url). Tip: it's possible to use `<sha1>` instead of `<branch>` (and vice versa for that matter) if you're interested in the statuses of a particular commit. Pull requests will also be built (only from contributors with write access), but by default nothing will be done (the job will exit immediately, although a green check mark will still show up). Jenkins will **not** add a comment in the pull request in this case. However, if the label `build-package` [1] is set for a pull request, the internal jenkins job will run (it will do everything except the local xharness test run: this includes creating and publishing packages, creating various diffs, run tests on older macOS versions, test docs, etc). A detailed comment will also be added to the pull request (see below for multiple examples), which means that there will be two Jenkins comments: one for the public Jenkins which builds every PR, and one for the internal Jenkins [2]. [1] I don't quite like the name of the label, because it doesn't get even close to explain all that will actually happen, but `run-on-internal-jenkins-and-create-package` is a bit too long IMHO... Also it's non-obvious that this is the label to apply if the reason for executing on the internal jenkins is some other reason (for instance to test a maccore bump). Other ideas: * `run-internal-jenkins`: doesn't make it obvious that a package will be created (which is probably the most common reason to want to run on internal jenkins) * We could have multiple labels that mean the same thing: `build-package`, `internal-build`, `run-internal-jenkins`, etc, but it's redundant and I don't quite like it either. * Any other ideas? [2] I'm noticing now that these two look quite similar and this might end up confusing (the main difference is that the comment from the public jenkins will say **Build success/failure** and **Build comment file:** at the top. If something goes wrong the failure will also show up differently). Should this be made clearer?
…from the d15-8 branch (dotnet#4195) * [Jenkins] Create artifacts.json and set a GH status as 'Jenkins: Artifacts'. * [jenkins] Include the url in artifacts.json * [jenkins] Add sha256 checksum to artifacts.json as well. * [Jenkins] Enable xamarin before provisioning so that we auto-provision Xcode. * [jenkins] Fix passing flags to configure. Quoting empty CONFIGURE_FLAGS ends up doing this: ./configure "" --disable-ios-device and since configure parses arguments until it finds an empty argument, it would stop parsing at the first argument, effectively not disabling the device build. So don't quote CONFIGURE_FLAGS when invoking configure. shellcheck doesn't quite like this, but the better code is much more complex, and not really needed, so just add an exception.
rolfbjarne
added
the
build-package
spouliot
approved these changes
Jun 12, 2018
Member
Author
|
🔥 Jenkins job (on internal Jenkins) failed in stage 'Provisioning' 🔥 : hudson.AbortException: script returned exit code 2 |
Contributor
Build failure
|
dalexsoto
approved these changes
Jun 12, 2018
VincentDondain
approved these changes
Jun 12, 2018
Contributor
Build failure
|
Member
Author
|
🔥 Jenkins job (on internal Jenkins) failed in stage 'Package XM tests' 🔥 : org.jenkinsci.plugins.workflow.steps.FlowInterruptedException ✅ Build succeeded |
Member
Author
|
🔥 Jenkins job (on internal Jenkins) failed in stage 'Build' 🔥 : hudson.AbortException: script returned exit code 2 |
…tween different remotes for dependencies. (dotnet#4060) Put the shell code for resetting README dependencies in a shell script instead of embedded in the Makefile so that it's easier to write, read and debug. Also add support for switching between different remotes for README dependencies (this means that `make reset-X` will now work fine if `X`'s remote changed). A side effect is that all README dependencies will now end up with a 'xamarin' remote in addition to the 'origin' remote, but this should have no other effect.
Contributor
Build failure
|
It turns out that some of the vstool versions (from VSfM) we allow require a mono 5.10.0.171+, so just bump to latest stable.
Member
Author
|
🔥 Jenkins job (on internal Jenkins) failed in stage 'Package XM tests' 🔥 : hudson.AbortException: script returned exit code 2 ✅ Build succeeded |
Member
Author
|
🔥 Jenkins job (on internal Jenkins) failed in stage 'Package XM tests' 🔥 : hudson.AbortException: script returned exit code 2 ✅ Build succeeded |
Contributor
Build failure
|
Fixes an issue with the packaged XM tests, where they don't build because we try to build 32-bit tests.
spouliot
approved these changes
Jun 13, 2018
Member
Author
|
🔥 Jenkins job (on internal Jenkins) failed in stage 'Test docs' 🔥 : hudson.AbortException: script returned exit code 2 ✅ Build succeeded |
Contributor
Build failure
|
Member
Author
|
✅ Jenkins job (on internal Jenkins) succeeded ✅ Build succeeded |
jstedfast
approved these changes
Jun 14, 2018
Contributor
Build failure
|
Member
Author
|
Test failure is known (#4241). |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
6 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.