Add support for building on Jenkins. - #4159
Conversation
Build failure
|
|
🔥 Jenkins job failed in stage 'Running XM tests on '10.10'' 🔥 : hudson.AbortException: script returned exit code 2 ✅ Build succeeded |
Build success
|
|
🔥 Jenkins job failed in stage 'Running XM tests on '10.10'' 🔥 : hudson.AbortException: script returned exit code 2 ✅ Build succeeded |
Build success
|
|
🔥 Jenkins job failed in stage 'Running XM tests on '10.10'' 🔥 : hudson.AbortException: script returned exit code 2 ✅ Build succeeded |
Build success
|
|
🔥 Jenkins job failed in stage 'Running XM tests on '10.9'' 🔥 : hudson.AbortException: script returned exit code 2 ✅ Build succeeded |
|
The test failures private Jenkins are known (tests fail on older macOS bots due to #4129). |
|
@xamarin/macios This is ready for review now. Please also take a look at the job at private jenkins to see if there are any questions/improvements that can be done (The |
| createFinalStatus = true | ||
| skipLocalTestRunReason = "" | ||
|
|
||
| def abortExecutingBuilds () |
There was a problem hiding this comment.
Since this is all commented, do we need it in the diff? Or should it be deleted?
There was a problem hiding this comment.
It will be working pretty soon (needs some changes to Jenkins itself first), but I can remove it.
| try { | ||
| sh ("curl -vf -H 'Authorization: token ${GITHUB_PAT_TOKEN}' --output '${outputFile}' '${url}'") | ||
| github_pull_request_info = readJSON (file: outputFile) | ||
| echo ("Got pull request info: ${github_pull_request_info}") |
There was a problem hiding this comment.
Do we want to log the exception for easier debugging?
There was a problem hiding this comment.
The exception is not caught, it will bubble up and eventually fail the job.
| def labels = pinfo ["labels"] | ||
| if (labels != null) { | ||
| for (int i = 0; i < labels.size (); i++) { | ||
| def label = labels [i] |
There was a problem hiding this comment.
Why not labels[i]["name"]? Also, I suppose that we are 100% sure that label does have the "name" key, right?
There was a problem hiding this comment.
This is how it ended up after I removed all the debug spew that was between these two statements 😄
And yes, every label has a name, this comes from GitHub's API, which is considered stable on their end.
| withCredentials ([string (credentialsId: 'macios_github_comment_token', variable: 'GITHUB_COMMENT_TOKEN')]) { | ||
| sh ("curl -i -H 'Authorization: token ${GITHUB_COMMENT_TOKEN}' ${url} --data '@${jsonFile}'") | ||
| } | ||
| } finally { |
There was a problem hiding this comment.
Same as previous, do we want to log exceptions?
| def pinfo = githubGetPullRequestInfo () | ||
| def labels = pinfo ["labels"] | ||
| if (labels != null) { | ||
| for (int i = 0; i < labels.size (); i++) { |
There was a problem hiding this comment.
AFAIK groovy has foreach loops, like label.each { // your code }, should we use them?
There was a problem hiding this comment.
Yeah, we could. OTOH doesn't matter in any way, and the current code works (and is more familiar to us C# developers), so I prefer the current version.
| try { | ||
| sh (script: "grep '^@MonkeyWrench: ...Summary: ' '${outputFile}' > ${tmpfile}", returnStatus: true /* don't throw exceptions if something goes wrong */) | ||
| def lines = readFile ("${tmpfile}").split ("\n") | ||
| for (int i = 0; i < lines.length; i++) { |
There was a problem hiding this comment.
Same, we could use each here.
| def uploadingFiles = findFiles (glob: "package/*") | ||
| def manifest = "" | ||
| def metadata = "[\n" | ||
| for (int i = 0; i < uploadingFiles.length; i++) { |
There was a problem hiding this comment.
Same, should we use each?
dalexsoto
left a comment
There was a problem hiding this comment.
[1] I don't quite like the name of the label, because it doesn't get even close to explain all that will actually happen, but
run-on-internal-jenkins-and-create-packageis a bit too long IMHO... Also it's non-obvious that this is the label to apply if the reason for executing on the internal jenkins is some other reason (for instance to test a maccore bump). Other ideas:
run-internal-jenkins: doesn't make it obvious that a package will be created (which is probably the most common reason to want to run on internal jenkins)- We could have multiple labels that mean the same thing:
build-package,internal-build,run-internal-jenkins, etc, but it's redundant and I don't quite like it either.- Any other ideas?
What about internal-build-and-package? Most of the time we are the ones applying the labels and we could document them in the maccore wiki including the need of applying it when bumping maccore. I am not really keen to the idea of having multiple labels meaning the same thing.
Tooling wise it looks good to me 👍
spouliot
left a comment
There was a problem hiding this comment.
Q: How can we identify PR-based packages ?
Should we overload a variable (maybe PACKAGE_REV) to a known value so we don't get duplicates ? (or at least no duplicate with normal builds)
I guess the branch name will also point the the PR name (but we don't really control the names)
Build success
|
|
🔥 Jenkins job failed in stage 'Running XM tests on '10.8'' 🔥 : hudson.AbortException: script returned exit code 2 ✅ Build succeeded |
VincentDondain
left a comment
There was a problem hiding this comment.
I like the label build-package it's indeed likely what we care about, the fact it's internal jenkins doesn't matter much IMHO and I see the fact that we're running a larger subset of the tests as a requirement to build a package (:
I do think we should update the status messages, right now they are too similar. The titles should clearly explain what is what. In addition, it might be a good idea to have a 1 liner description of the differences between what internal and external Jenkins do.
Also why do we have 🔥 Jenkins job failed in stage 'Running XM tests on '10.8'' 🔥 : hudson.AbortException: script returned exit code 2 at the top and not a 🔥 Test run failed step.
|
I think it's because it's running on older macOS bots |
Right now nothing will show that it was built from a PR (except the download link itself, and indirectly from the hash in I don't quite like messing with the version numbers themselves, since that ends up being "hardcoded knowledge", and it also makes version comparisons non-intuitive. However, I can make the actual branch name (and something like Does that sound like a plan? [1] I can even make it say something very explicit, like the full url of the PR: |
Build success
|
|
🔥 Jenkins job (on internal Jenkins) failed in stage 'Running XM tests on '10.8'' 🔥 : java.io.IOException: java.nio.file.NoSuchFileException: /Users/builder/jenkins/workspace/xamarin-macios/xamarin-macios/ℹ️ Test run skipped: Not running tests here because they're run on public Jenkins. ✅ Build succeeded |
Build success
|
|
🔥 Jenkins job (on internal Jenkins) failed in stage 'Running XM tests on '10.8'' 🔥 : hudson.AbortException: script returned exit code 2 ✅ Build succeeded |
|
@spouliot: this is how the version shows up now: |
Build success
|
|
🔥 Jenkins job (on internal Jenkins) failed in stage 'Running XM tests on '10.7'' 🔥 : hudson.AbortException: script returned exit code 2 ✅ Build succeeded |
|
build |
Build success
|
Add support for building on internal Jenkins. Jenkins has been configured to build every branch on xamarin/xamarin-macios that contains a `jenkins/Jenkinsfile`, which means it will start working as soon as this PR is merged. Results will be posted as statuses on each commit, which can be viewed using the url `https://github.com/xamarin/xamarin-macios/commits/<branch>`:  * The `continuous-integration/jenkins/branch` status links to the jenkins job. * The other two are XI and XM packages (the `Jenkins-` prefix will be removed once we officially switch from Wrench to Jenkins). More detailed information will be added as a comment to each commit, which can be seen by clicking on the commit and scrolling to the bottom (url of the format `https://github.com/xamarin/xamarin-macios/commit/<sha1>`)  Unfortunately GitHub does not display the commit statuses when viewing a single commit, so to view those statuses you'll have to view the list of commits (the `/commits/` url). Tip: it's possible to use `<sha1>` instead of `<branch>` (and vice versa for that matter) if you're interested in the statuses of a particular commit. Pull requests will also be built (only from contributors with write access), but by default nothing will be done (the job will exit immediately, although a green check mark will still show up). Jenkins will **not** add a comment in the pull request in this case. However, if the label `build-package` [1] is set for a pull request, the internal jenkins job will run (it will do everything except the local xharness test run: this includes creating and publishing packages, creating various diffs, run tests on older macOS versions, test docs, etc). A detailed comment will also be added to the pull request (see below for multiple examples), which means that there will be two Jenkins comments: one for the public Jenkins which builds every PR, and one for the internal Jenkins [2]. [1] I don't quite like the name of the label, because it doesn't get even close to explain all that will actually happen, but `run-on-internal-jenkins-and-create-package` is a bit too long IMHO... Also it's non-obvious that this is the label to apply if the reason for executing on the internal jenkins is some other reason (for instance to test a maccore bump). Other ideas: * `run-internal-jenkins`: doesn't make it obvious that a package will be created (which is probably the most common reason to want to run on internal jenkins) * We could have multiple labels that mean the same thing: `build-package`, `internal-build`, `run-internal-jenkins`, etc, but it's redundant and I don't quite like it either. * Any other ideas? [2] I'm noticing now that these two look quite similar and this might end up confusing (the main difference is that the comment from the public jenkins will say **Build success/failure** and **Build comment file:** at the top. If something goes wrong the failure will also show up differently). Should this be made clearer?
* Add support for building on Jenkins. (#4159) Add support for building on internal Jenkins. Jenkins has been configured to build every branch on xamarin/xamarin-macios that contains a `jenkins/Jenkinsfile`, which means it will start working as soon as this PR is merged. Results will be posted as statuses on each commit, which can be viewed using the url `https://github.com/xamarin/xamarin-macios/commits/<branch>`:  * The `continuous-integration/jenkins/branch` status links to the jenkins job. * The other two are XI and XM packages (the `Jenkins-` prefix will be removed once we officially switch from Wrench to Jenkins). More detailed information will be added as a comment to each commit, which can be seen by clicking on the commit and scrolling to the bottom (url of the format `https://github.com/xamarin/xamarin-macios/commit/<sha1>`)  Unfortunately GitHub does not display the commit statuses when viewing a single commit, so to view those statuses you'll have to view the list of commits (the `/commits/` url). Tip: it's possible to use `<sha1>` instead of `<branch>` (and vice versa for that matter) if you're interested in the statuses of a particular commit. Pull requests will also be built (only from contributors with write access), but by default nothing will be done (the job will exit immediately, although a green check mark will still show up). Jenkins will **not** add a comment in the pull request in this case. However, if the label `build-package` [1] is set for a pull request, the internal jenkins job will run (it will do everything except the local xharness test run: this includes creating and publishing packages, creating various diffs, run tests on older macOS versions, test docs, etc). A detailed comment will also be added to the pull request (see below for multiple examples), which means that there will be two Jenkins comments: one for the public Jenkins which builds every PR, and one for the internal Jenkins [2]. [1] I don't quite like the name of the label, because it doesn't get even close to explain all that will actually happen, but `run-on-internal-jenkins-and-create-package` is a bit too long IMHO... Also it's non-obvious that this is the label to apply if the reason for executing on the internal jenkins is some other reason (for instance to test a maccore bump). Other ideas: * `run-internal-jenkins`: doesn't make it obvious that a package will be created (which is probably the most common reason to want to run on internal jenkins) * We could have multiple labels that mean the same thing: `build-package`, `internal-build`, `run-internal-jenkins`, etc, but it's redundant and I don't quite like it either. * Any other ideas? [2] I'm noticing now that these two look quite similar and this might end up confusing (the main difference is that the comment from the public jenkins will say **Build success/failure** and **Build comment file:** at the top. If something goes wrong the failure will also show up differently). Should this be made clearer? * Make the Jenkins packages official. * [Jenkins] Create artifacts.json and set a GH status as 'Jenkins: Artifacts'. * [jenkins] Include the url in artifacts.json * [jenkins] Add sha256 checksum to artifacts.json as well. * [Jenkins] Enable xamarin before provisioning so that we auto-provision Xcode. * [jenkins] Fix passing flags to configure. Quoting empty CONFIGURE_FLAGS ends up doing this: ./configure "" --disable-ios-device and since configure parses arguments until it finds an empty argument, it would stop parsing at the first argument, effectively not disabling the device build. So don't quote CONFIGURE_FLAGS when invoking configure. shellcheck doesn't quite like this, but the better code is much more complex, and not really needed, so just add an exception.
Add support for building on internal Jenkins. Jenkins has been configured to build every branch on xamarin/xamarin-macios that contains a `jenkins/Jenkinsfile`, which means it will start working as soon as this PR is merged. Results will be posted as statuses on each commit, which can be viewed using the url `https://github.com/xamarin/xamarin-macios/commits/<branch>`:  * The `continuous-integration/jenkins/branch` status links to the jenkins job. * The other two are XI and XM packages (the `Jenkins-` prefix will be removed once we officially switch from Wrench to Jenkins). More detailed information will be added as a comment to each commit, which can be seen by clicking on the commit and scrolling to the bottom (url of the format `https://github.com/xamarin/xamarin-macios/commit/<sha1>`)  Unfortunately GitHub does not display the commit statuses when viewing a single commit, so to view those statuses you'll have to view the list of commits (the `/commits/` url). Tip: it's possible to use `<sha1>` instead of `<branch>` (and vice versa for that matter) if you're interested in the statuses of a particular commit. Pull requests will also be built (only from contributors with write access), but by default nothing will be done (the job will exit immediately, although a green check mark will still show up). Jenkins will **not** add a comment in the pull request in this case. However, if the label `build-package` [1] is set for a pull request, the internal jenkins job will run (it will do everything except the local xharness test run: this includes creating and publishing packages, creating various diffs, run tests on older macOS versions, test docs, etc). A detailed comment will also be added to the pull request (see below for multiple examples), which means that there will be two Jenkins comments: one for the public Jenkins which builds every PR, and one for the internal Jenkins [2]. [1] I don't quite like the name of the label, because it doesn't get even close to explain all that will actually happen, but `run-on-internal-jenkins-and-create-package` is a bit too long IMHO... Also it's non-obvious that this is the label to apply if the reason for executing on the internal jenkins is some other reason (for instance to test a maccore bump). Other ideas: * `run-internal-jenkins`: doesn't make it obvious that a package will be created (which is probably the most common reason to want to run on internal jenkins) * We could have multiple labels that mean the same thing: `build-package`, `internal-build`, `run-internal-jenkins`, etc, but it's redundant and I don't quite like it either. * Any other ideas? [2] I'm noticing now that these two look quite similar and this might end up confusing (the main difference is that the comment from the public jenkins will say **Build success/failure** and **Build comment file:** at the top. If something goes wrong the failure will also show up differently). Should this be made clearer?
Add support for building on internal Jenkins.
Jenkins has been configured to build every branch on xamarin/xamarin-macios that contains a
jenkins/Jenkinsfile, which means it will start working as soon as this PR is merged.Results will be posted as statuses on each commit, which can be viewed using the url
https://github.com/xamarin/xamarin-macios/commits/<branch>:continuous-integration/jenkins/branchstatus links to the jenkins job.Jenkins-prefix will be removed once we officially switch from Wrench to Jenkins).More detailed information will be added as a comment to each commit, which can be seen by clicking on the commit and scrolling to the bottom (url of the format
https://github.com/xamarin/xamarin-macios/commit/<sha1>)Unfortunately GitHub does not display the commit statuses when viewing a single commit, so to view those statuses you'll have to view the list of commits (the
/commits/url). Tip: it's possible to use<sha1>instead of<branch>(and vice versa for that matter) if you're interested in the statuses of a particular commit.Pull requests will also be built (only from contributors with write access), but by default nothing will be done (the job will exit immediately, although a green check mark will still show up). Jenkins will not add a comment in the pull request in this case.
However, if the label
build-package[1] is set for a pull request, the internal jenkins job will run (it will do everything except the local xharness test run: this includes creating and publishing packages, creating various diffs, run tests on older macOS versions, test docs, etc). A detailed comment will also be added to the pull request (see below for multiple examples), which means that there will be two Jenkins comments: one for the public Jenkins which builds every PR, and one for the internal Jenkins [2].[1] I don't quite like the name of the label, because it doesn't get even close to explain all that will actually happen, but
run-on-internal-jenkins-and-create-packageis a bit too long IMHO... Also it's non-obvious that this is the label to apply if the reason for executing on the internal jenkins is some other reason (for instance to test a maccore bump). Other ideas:run-internal-jenkins: doesn't make it obvious that a package will be created (which is probably the most common reason to want to run on internal jenkins)build-package,internal-build,run-internal-jenkins, etc, but it's redundant and I don't quite like it either.[2] I'm noticing now that these two look quite similar and this might end up confusing (the main difference is that the comment from the public jenkins will say Build success/failure and Build comment file: at the top. If something goes wrong the failure will also show up differently). Should this be made clearer?