let bot run EESSI test suite during test phase (part of bot build procedure)#467

Merged

boegel merged 74 commits into

EESSI:2023.06-software.eessi.iofrom

casparvl:run_test_suite_based_on_pr366

Mar 26, 2024

Commits on Oct 14, 2023

first version of bot/test.sh script
truib
committed

Commits on Oct 15, 2023

Commits on Feb 5, 2024

Second attempt at having the bot run the test suite, now based on Thomas' PR 366
Caspar van Leeuwen
committed
Merge upstream into old feature branch based on pr366
Caspar van Leeuwen
committed
Make a very simple change to an easystack file so we are able to trigger the bot build and test procedures and see if this PR actually works
Caspar van Leeuwen
committed

Commits on Feb 6, 2024

Make scripts executable
Caspar van Leeuwen
committed
Check for existence of SLURM output first
Caspar van Leeuwen
committed
Clarify return messages from the bot
Caspar van Leeuwen
committed
Use EESSI_SOFTWARE_PATH instead of EASYBUILD_INSTALLPATH to set the modulepath. It should be the same, and we no longer configure EasyBuild, since we don't use it when running the test suite
Caspar van Leeuwen
committed
Do git clone in a seperate script, so that that can be run in the prefix layer
Caspar van Leeuwen
committed
Should use the one from current dir
Caspar van Leeuwen
committed
Remove quotes, see if that helps
Caspar van Leeuwen
committed
Apparently, we need to remove curly braces too
Caspar van Leeuwen
committed
Make sure this actually gets reported as failure, as the ReFrame runtime fails to run the test suite (it is not an indication of tests failing themselves).
Caspar van Leeuwen
committed
Fix the actual issue: make sure that two arguments are interpreted seperately by not quoting them
Caspar van Leeuwen
committed
See if we can make check-test formatting more fancy
Caspar van Leeuwen
committed
Replace the env var for the result file
Caspar van Leeuwen
committed
Add missing functions
Caspar van Leeuwen
committed
Remove artefacts from reporting
Caspar van Leeuwen
committed
Add a reason for the failure
Caspar van Leeuwen
committed
Fixed missing bracket
Caspar van Leeuwen
committed
Add reporting of the ReFrame result
Caspar van Leeuwen
committed
Add missing bracket. Again
Caspar van Leeuwen
committed
Add some echo for debugging
Caspar van Leeuwen
committed
more debugging output
Caspar van Leeuwen
committed
Lets try some nicer formatting with headers etc. See if this actually prints something.
Caspar van Leeuwen
committed
Inverse check, it was incorrect
Caspar van Leeuwen
committed
Remove debugging output
Caspar van Leeuwen
committed
Correct the regex pattern to account for the fact that the slurm output file contains special characters in that pattern that are NOT whitespace
Caspar van Leeuwen
committed
Corrected typo
Caspar van Leeuwen
committed
Trigger failure in the workflow on purpose, to show what that looks like
Caspar van Leeuwen
committed
Change order of checking error codes. If the ReFrame tests ran and we find that pattern, that should take priority. Other errors might have come from errors in the build phase...
Caspar van Leeuwen
committed

Commits on Feb 7, 2024

Make sure we don't overwrite the grepped result from failures
Caspar van Leeuwen
committed
Fix plural/singular typo
Caspar van Leeuwen
committed
Be a bit more expressive about what we are doing
Caspar van Leeuwen
committed
Change to dedicated config file for testing by the bot in the build job
Caspar van Leeuwen
committed
Make it verbose, see if we can see why autodetect fails
Caspar van Leeuwen
committed
Add ReFrame config template. Not that four template variables need to be replaced for this to be a valid ReFrame config file
Caspar van Leeuwen
committed
Use template config file and make replacements based on lscpu
Caspar van Leeuwen
committed
Fix incorrect variable name
Caspar van Leeuwen
committed
Missing comma
Caspar van Leeuwen
committed
Corrected typo
Caspar van Leeuwen
committed
Remove verbose flags
Caspar van Leeuwen
committed
Pick name without spaces, to avoid issues with directories
Caspar van Leeuwen
committed

Commits on Feb 8, 2024

Pick more meaningful name for SLURM variable, namely SLURM_OUTPUT_FOUND
Caspar van Leeuwen
committed
Correct erroneous status
Caspar van Leeuwen
committed
Rename CoDeList variable to comment_details_list
Caspar van Leeuwen
committed
Remove commented section, it is not relevant
Caspar van Leeuwen
committed
Update run_tests.sh

casparvl
and
boegel
authored
Fix mistake in replacing SLURM with SLURM_OUTPUT_FOUND
Caspar van Leeuwen
committed
Merge branch 'run_test_suite_based_on_pr366' of github.com:casparvl/software-layer into run_test_suite_based_on_pr366
Caspar van Leeuwen
committed
Change name of the template for the ReFrame config file to make clear it is a template. Then, alter the sed commands so that they output to a new file: the actual config file
Caspar van Leeuwen
committed
Fix typo in variable
Caspar van Leeuwen
committed
No longer needed
Caspar van Leeuwen
committed
Added header, rephrased some comments
Caspar van Leeuwen
committed
Fix headers
Caspar van Leeuwen
committed
Load default ReFrame version
Caspar van Leeuwen
committed
Point to docs for ReFrame config
Caspar van Leeuwen
committed
Fix silly mistake: I was overwriting the ReFrame config file with every sed call
Caspar van Leeuwen
committed
See if @boegel is correct and if we don't actually need ANY software change to have the bot run...
Caspar van Leeuwen
committed

Commits on Feb 14, 2024

Add escape character to star, so that the regex pattern prints correctly on GitHub in the report
Caspar van Leeuwen
committed

Commits on Feb 15, 2024

Commits on Feb 19, 2024

Remove manual setup of MODULEPATH etc, and source the regular EESSI init script to do this
Caspar van Leeuwen
committed

Commits on Mar 1, 2024

'which' is not working in the container, command -v is
Caspar van Leeuwen
committed
Workaround for 'PSM3 can't open nic unit' error
Caspar van Leeuwen
committed

Commits on Mar 22, 2024

Merge branch '2023.06-software.eessi.io' into run_test_suite_based_on_pr366
casparvl
authored