USHIFT-6905: consolidate CI test scenarios to reduce VM count#6591
Merged
openshift-merge-bot[bot] merged 8 commits intoApr 28, 2026
Merged
Conversation
TLS v1.3 enforcement is a transport-layer concern validated by multi-config-standard1 which already applies TLS v1.3 and runs standard1 tests. Dedicated TLS v1.3 scenarios for standard1/standard2 are redundant. Also remove --exclude version from multi-config-standard1 so that version tests now run there, and drop the stale comment that referenced the old exclusion. Removes 6 VMs (2 per platform: el9/el10 bootc, ostree). Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> pre-commit.check-secrets: ENABLED
Piggyback the configuration test suite onto the dual-stack VM which is already provisioned with a compatible environment. The dual-stack scenario is renamed to dual-stack-configuration to reflect the combined coverage. Removes 3 VMs (1 per platform: el9/el10 bootc, ostree). Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> pre-commit.check-secrets: ENABLED
Consolidate the FIPS test suite into the ai-model-serving-online VM by switching to the FIPS image, adding --fips to launch_vm for kernel-level FIPS enforcement, and appending suites/fips/ to the test run. Both scenarios already skip on aarch64. Removes 2 VMs (1 per platform: el9/el10 bootc; no ostree equivalent). Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> pre-commit.check-secrets: ENABLED
Merge the standalone upgrade+standard1/standard2 scenarios into the LVMS upgrade scenarios. Each LVMS VM now runs the full LVMS lifecycle (create/check/upgrade/validate/cleanup) followed by a standard suite, eliminating the need for separate standard-only VMs. The split follows a consistent pattern across el9/el10 bootc and ostree: - lvms-standard1: LVMS lifecycle + ginkgo (y2 only) + standard1 with EXPECTED_OS_VERSION + --vm_disksize 30 - lvms-standard2: LVMS lifecycle + standard2 + --vm_disksize 30 For el10, the el98→el102 cross-major upgrade scenarios also absorb LVMS lifecycle testing with --vm_disksize 30. Removes 12 VMs (4 per platform: el9/el10 bootc, ostree) and absorbs the el10 el98→el102 LVMS scenario into the standard1/standard2 cross-major scenarios. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> pre-commit.check-secrets: ENABLED
Remove the .disabled suffix from el98 RPM standard1, standard2, and upgrade scenarios to restore their execution in CI. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> pre-commit.check-secrets: ENABLED
Change set -ux to set -eux so that cleanup failures (API server unreachable, RBAC errors, etc.) abort the scenario instead of silently exiting 0 and letting subsequent standard suite tests run on a degraded cluster. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> pre-commit.check-secrets: ENABLED
The tuned image overwrites /etc/tuned/microshift-baseline-variables.conf with test-specific values, causing rpm -V --configfiles to fail in the version.robot RPM integrity check. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> pre-commit.check-secrets: ENABLED
openshift-ci-robot
added
the
jira/valid-reference
|
@agullon: This pull request references USHIFT-6905 which is a valid jira issue. Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the task to target the "5.0.0" version, but no target version was set. DetailsIn response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository. |
openshift-ci
Bot
added
the
approved
Contributor
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Repository YAML (base), Central YAML (inherited) Review profile: CHILL Plan: Enterprise Run ID: 📒 Files selected for processing (4)
✅ Files skipped from review due to trivial changes (1)
🚧 Files skipped from review as they are similar to previous changes (3)
WalkthroughMultiple test scenario scripts were reorganized: numerous old scenario files were removed, several new LVMS-aware upgrade scenarios were added, VM provisioning now often sets a 30 GB disk, FIPS/configuration test selections expanded, and a helper script's Bash flags were tightened. Changes
Sequence Diagram(s)sequenceDiagram
participant CI as CI/orchestrator
participant Provisioner as VM Provisioner
participant VM as host1 (VM)
participant MicroShift as MicroShift
participant LVMS as LVMS
participant TestRunner as run_tests
CI->>Provisioner: request VM (start_image, --vm_disksize 30, --fips?)
Provisioner->>VM: create and boot
VM->>MicroShift: start services
MicroShift->>CI: ready (greenboot)
CI->>TestRunner: run pre-upgrade LVMS workload creation on VM
TestRunner->>LVMS: create workloads & validate resources
CI->>TestRunner: invoke upgrade tests (TARGET_REF, BOOTC_REGISTRY)
TestRunner->>VM: perform upgrade
VM->>MicroShift: upgraded, restart
TestRunner->>LVMS: validate workload persistence & resources post-upgrade
TestRunner->>LVMS: cleanup workloads
CI->>TestRunner: run post-upgrade standard suites (suites/standard1|standard2)
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~25 minutes 🚥 Pre-merge checks | ✅ 11 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (11 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@test/scenarios/releases/el96-yminus2`@el98-lrel@lvms-standard1.sh:
- Line 9: The start_image value currently hardcodes the major version
("4.${YMINUS2_MINOR_VERSION}") which can break when branch/version mappings
change; update the start_image assignment to use the variables for both major
and minor versions (e.g., "${YMINUS2_MAJOR_VERSION}.${YMINUS2_MINOR_VERSION}")
so start_image is constructed from ${YMINUS2_MAJOR_VERSION} and
${YMINUS2_MINOR_VERSION} instead of the literal "4".
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Repository YAML (base), Central YAML (inherited)
Review profile: CHILL
Plan: Enterprise
Run ID: bcbf56d7-ea8f-40b6-afdc-5ad95ba6835e
📒 Files selected for processing (48)
scripts/lvms-helpers/cleanupWorkload.shtest/scenarios-bootc/el10/releases/el102-lrel@ai-model-serving-online-fips.shtest/scenarios-bootc/el10/releases/el102-lrel@configuration.shtest/scenarios-bootc/el10/releases/el102-lrel@dual-stack-configuration.shtest/scenarios-bootc/el10/releases/el102-lrel@fips.shtest/scenarios-bootc/el10/releases/el102-lrel@multi-config-standard1.shtest/scenarios-bootc/el10/releases/el102-lrel@tlsv13-standard1.shtest/scenarios-bootc/el10/releases/el102-lrel@tlsv13-standard2.shtest/scenarios-bootc/el10/releases/el96-y1@el102-lrel@lvms-standard1.shtest/scenarios-bootc/el10/releases/el96-y1@el102-lrel@lvms-standard2.shtest/scenarios-bootc/el10/releases/el96-y1@el102-lrel@standard1.shtest/scenarios-bootc/el10/releases/el96-y1@el102-lrel@standard2.shtest/scenarios-bootc/el10/releases/el96-y2@el102-lrel@lvms-standard1.shtest/scenarios-bootc/el10/releases/el96-y2@el102-lrel@lvms-standard2.shtest/scenarios-bootc/el10/releases/el96-y2@el102-lrel@standard1.shtest/scenarios-bootc/el10/releases/el96-y2@el102-lrel@standard2.shtest/scenarios-bootc/el10/releases/el98-lrel@el102-lrel@standard1.shtest/scenarios-bootc/el10/releases/el98-lrel@el102-lrel@standard2.shtest/scenarios-bootc/el9/releases/el96-y1@el98-lrel@lvms-standard1.shtest/scenarios-bootc/el9/releases/el96-y1@el98-lrel@lvms-standard2.shtest/scenarios-bootc/el9/releases/el96-y1@el98-lrel@standard1.shtest/scenarios-bootc/el9/releases/el96-y1@el98-lrel@standard2.shtest/scenarios-bootc/el9/releases/el96-y2@el98-lrel@lvms-standard1.shtest/scenarios-bootc/el9/releases/el96-y2@el98-lrel@lvms-standard2.shtest/scenarios-bootc/el9/releases/el96-y2@el98-lrel@standard1.shtest/scenarios-bootc/el9/releases/el96-y2@el98-lrel@standard2.shtest/scenarios-bootc/el9/releases/el98-lrel@ai-model-serving-online-fips.shtest/scenarios-bootc/el9/releases/el98-lrel@configuration.shtest/scenarios-bootc/el9/releases/el98-lrel@dual-stack-configuration.shtest/scenarios-bootc/el9/releases/el98-lrel@fips.shtest/scenarios-bootc/el9/releases/el98-lrel@multi-config-standard1.shtest/scenarios-bootc/el9/releases/el98-lrel@tlsv13-standard1.shtest/scenarios-bootc/el9/releases/el98-lrel@tlsv13-standard2.shtest/scenarios/releases/el96-yminus1@el98-lrel@lvms-standard1.shtest/scenarios/releases/el96-yminus1@el98-lrel@lvms-standard2.shtest/scenarios/releases/el96-yminus1@el98-lrel@standard1.shtest/scenarios/releases/el96-yminus1@el98-lrel@standard2.shtest/scenarios/releases/el96-yminus2@el98-lrel@lvms-standard1.shtest/scenarios/releases/el96-yminus2@el98-lrel@lvms-standard2.shtest/scenarios/releases/el96-yminus2@el98-lrel@standard1.shtest/scenarios/releases/el96-yminus2@el98-lrel@standard2.shtest/scenarios/releases/el98-lrel@configuration.shtest/scenarios/releases/el98-lrel@dual-stack-configuration.shtest/scenarios/releases/el98-lrel@tlsv13-standard1.shtest/scenarios/releases/el98-lrel@tlsv13-standard2.shtest/scenarios/releases/el98@rpm-standard1.shtest/scenarios/releases/el98@rpm-standard2.shtest/scenarios/releases/el98@rpm-upgrade.sh
💤 Files with no reviewable changes (25)
- test/scenarios-bootc/el10/releases/el102-lrel@multi-config-standard1.sh
- test/scenarios-bootc/el9/releases/el98-lrel@multi-config-standard1.sh
- test/scenarios/releases/el96-yminus1@el98-lrel@standard2.sh
- test/scenarios-bootc/el9/releases/el98-lrel@configuration.sh
- test/scenarios-bootc/el9/releases/el96-y2@el98-lrel@standard2.sh
- test/scenarios-bootc/el9/releases/el98-lrel@tlsv13-standard1.sh
- test/scenarios-bootc/el10/releases/el102-lrel@fips.sh
- test/scenarios-bootc/el10/releases/el96-y2@el102-lrel@standard1.sh
- test/scenarios/releases/el96-yminus2@el98-lrel@standard2.sh
- test/scenarios-bootc/el9/releases/el98-lrel@tlsv13-standard2.sh
- test/scenarios-bootc/el10/releases/el96-y1@el102-lrel@standard2.sh
- test/scenarios-bootc/el10/releases/el96-y1@el102-lrel@standard1.sh
- test/scenarios/releases/el98-lrel@tlsv13-standard1.sh
- test/scenarios-bootc/el9/releases/el96-y1@el98-lrel@standard2.sh
- test/scenarios/releases/el98-lrel@tlsv13-standard2.sh
- test/scenarios-bootc/el9/releases/el98-lrel@fips.sh
- test/scenarios-bootc/el10/releases/el96-y2@el102-lrel@standard2.sh
- test/scenarios-bootc/el10/releases/el102-lrel@configuration.sh
- test/scenarios/releases/el96-yminus2@el98-lrel@standard1.sh
- test/scenarios-bootc/el9/releases/el96-y1@el98-lrel@standard1.sh
- test/scenarios/releases/el98-lrel@configuration.sh
- test/scenarios-bootc/el10/releases/el102-lrel@tlsv13-standard2.sh
- test/scenarios-bootc/el9/releases/el96-y2@el98-lrel@standard1.sh
- test/scenarios-bootc/el10/releases/el102-lrel@tlsv13-standard1.sh
- test/scenarios/releases/el96-yminus1@el98-lrel@standard1.sh
pacevedom
reviewed
Apr 28, 2026
Refactor the start_image variable in multiple scenario scripts to use the PREVIOUS_MAJOR_VERSION environment variable for improved versioning consistency. This change ensures that the correct image tags are utilized across different scenarios, enhancing maintainability and clarity. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> pre-commit.check-secrets: ENABLED
Contributor
Author
|
/test e2e-aws-tests-release |
Contributor
Author
|
/test e2e-aws-tests-arm |
Contributor
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: agullon, pacevedom The full list of commands accepted by this bot can be found here. The pull request process is described here DetailsNeeds approval from an approver in each of these files:
Approvers can indicate their approval by writing |
Contributor
Author
|
/override ci/prow/e2e-aws-tests-release |
Contributor
|
@agullon: Overrode contexts on behalf of agullon: ci/prow/e2e-aws-tests-release, ci/prow/e2e-aws-tests-release-arm DetailsIn response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. |
Contributor
Author
|
/verified by @agullon |
openshift-ci-robot
added
the
verified
|
@agullon: This PR has been marked as verified by DetailsIn response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository. |
Contributor
Author
|
/cherrypick release-4.22 |
|
@agullon: once the present PR merges, I will cherry-pick it on top of DetailsIn response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. |
Contributor
Author
|
/retest |
Contributor
Contributor
Author
|
/retest |
Contributor
Author
|
/override ci/prow/e2e-aws-tests-bootc-release-el10 |
Contributor
|
@agullon: Overrode contexts on behalf of agullon: ci/prow/e2e-aws-tests-bootc-release-el10 DetailsIn response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. |
Contributor
|
@agullon: all tests passed! Full PR test history. Your PR dashboard. DetailsInstructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here. |
|
@agullon: #6591 failed to apply on top of branch "release-4.22": DetailsIn response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. |
4 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Consolidate CI test scenarios to reduce VM count without losing test coverage. Tests that ran in dedicated VMs are now piggy-backed onto VMs already provisioned for compatible environments.
The changes are symmetric across el9 and el10 for the y1/y2 upgrade paths and standalone scenarios. el10 has additional operations for the el98→el102 cross-major-version upgrade path which has no el9 equivalent.
Additionally,
cleanupWorkload.shis fixed to useset -eux(wasset -ux) so that cleanup failures abort the scenario instead of silently continuing.test/scenarios-bootc/el9/releases/(8 deleted, 4 renamed, 2 created, 1 modified → net -6 VMs)Deleted (8):
el98-lrel@tlsv13-standard1.sh— TLS v1.3 + standard1 already covered bymulti-config-standard1el98-lrel@tlsv13-standard2.sh— TLS v1.3 enforcement is transport-layer; validated bymulti-config-standard1running standard1, so dedicated standard2+TLS v1.3 testing is redundantel98-lrel@configuration.sh— configuration suite moved intodual-stack-configurationel98-lrel@fips.sh— FIPS suite moved intoai-model-serving-online-fipsel96-y1@el98-lrel@standard1.sh— standard1 suite moved intoel96-y1@lvms-standard1el96-y1@el98-lrel@standard2.sh— standard2 suite moved intoel96-y1@lvms-standard2el96-y2@el98-lrel@standard1.sh— standard1 suite moved intoel96-y2@lvms-standard1el96-y2@el98-lrel@standard2.sh— standard2 suite moved intoel96-y2@lvms-standard2Renamed (4):
el98-lrel@dual-stack.sh→el98-lrel@dual-stack-configuration.sh— addedsuites/configuration/el98-lrel@ai-model-serving-online.sh→el98-lrel@ai-model-serving-online-fips.sh— switched tofipsimage, added--fipstolaunch_vm, addedsuites/fips/el96-y1@el98-lrel@lvms.sh→el96-y1@el98-lrel@lvms-standard1.sh— added--vm_disksize 30el96-y2@el98-lrel@lvms.sh→el96-y2@el98-lrel@lvms-standard2.sh— removed ginkgo, addedsuites/standard2/after LVMS cleanupCreated (2):
el96-y1@el98-lrel@lvms-standard2.sh— LVMS lifecycle + standard2 after upgrade (replaces deleted standard2)el96-y2@el98-lrel@lvms-standard1.sh— LVMS lifecycle + ginkgo + standard1 after upgrade (replaces deleted standard1)Modified (1):
el98-lrel@multi-config-standard1.sh— removed--exclude versionand stale comment (now the only place running standard1 with TLS v1.3)test/scenarios-bootc/el10/releases/(8 deleted, 5 renamed, 1 created, 3 modified → net -7 VMs)Same as el9 for standalone and y1/y2 upgrade scenarios, plus additional changes for the el98→el102 cross-major-version upgrade path (no el9 equivalent).
Deleted (8):
el102-lrel@tlsv13-standard1.sh— TLS v1.3 + standard1 already covered bymulti-config-standard1el102-lrel@tlsv13-standard2.sh— TLS v1.3 enforcement is transport-layer; validated bymulti-config-standard1running standard1, so dedicated standard2+TLS v1.3 testing is redundantel102-lrel@configuration.sh— configuration suite moved intodual-stack-configurationel102-lrel@fips.sh— FIPS suite moved intoai-model-serving-online-fipsel96-y1@el102-lrel@standard1.sh— standard1 suite moved into upgrade LVMS scenariosel96-y1@el102-lrel@standard2.sh— standard2 suite moved into upgrade LVMS scenariosel96-y2@el102-lrel@standard1.sh— standard1 suite moved intoel96-y2@lvms-standard1el96-y2@el102-lrel@standard2.sh— standard2 suite moved intoel96-y2@lvms-standard2Renamed (4 + 1 el98→el102):
el102-lrel@dual-stack.sh→el102-lrel@dual-stack-configuration.sh— addedsuites/configuration/el102-lrel@ai-model-serving-online.sh→el102-lrel@ai-model-serving-online-fips.sh— switched tofipsimage, added--fipstolaunch_vm, addedsuites/fips/el96-y1@el102-lrel@lvms.sh→el96-y1@el102-lrel@lvms-standard1.sh— added--vm_disksize 30el96-y2@el102-lrel@lvms.sh→el96-y2@el102-lrel@lvms-standard1.sh— added--vm_disksize 30el98-lrel@el102-lrel@lvms.sh→el96-y1@el102-lrel@lvms-standard2.sh— changed upgrade path from el98→el102 to el96-y1→el102, removed ginkgo, added standard2, added--vm_disksize 30(el98 LVMS tests absorbed into modifiedel98-lrel@el102-lrel@standard1/2)Created (1):
el96-y2@el102-lrel@lvms-standard2.sh— LVMS lifecycle + standard2 after upgradeModified (1 + 2 el98→el102):
el102-lrel@multi-config-standard1.sh— removed--exclude versionand stale commentel98-lrel@el102-lrel@standard1.sh— expanded with LVMS lifecycle (create/check/upgrade/validate/cleanup) +--vm_disksize 30, absorbing oldel98-lrel@el102-lrel@lvms.shel98-lrel@el102-lrel@standard2.sh— expanded with LVMS lifecycle (create/check/upgrade/validate/cleanup) +--vm_disksize 30, absorbing oldel98-lrel@el102-lrel@lvms.shtest/scenarios/releases/(ostree) (7 deleted, 3 renamed, 2 created, 3 re-enabled → net -2 VMs)Equivalent to el9 bootc changes (no FIPS or AI model serving scenarios in ostree).
Deleted (7):
el98-lrel@tlsv13-standard1.sh— TLS v1.3 + standard1 already covered byginkgo-multi-configel98-lrel@tlsv13-standard2.sh— TLS v1.3 enforcement is transport-layer; validated byginkgo-multi-configrunning standard1, so dedicated standard2+TLS v1.3 testing is redundantel98-lrel@configuration.sh— configuration suite moved intodual-stack-configurationel96-yminus1@el98-lrel@standard1.sh— standard1 suite moved intoyminus1@lvms-standard1el96-yminus1@el98-lrel@standard2.sh— standard2 suite moved intoyminus1@lvms-standard2el96-yminus2@el98-lrel@standard1.sh— standard1 suite moved intoyminus2@lvms-standard1el96-yminus2@el98-lrel@standard2.sh— standard2 suite moved intoyminus2@lvms-standard2Renamed (3):
el98-lrel@dual-stack.sh→el98-lrel@dual-stack-configuration.sh— addedsuites/configuration/el96-yminus1@el98-lrel@lvms.sh→el96-yminus1@el98-lrel@lvms-standard1.sh— added--vm_disksize 30el96-yminus2@el98-lrel@lvms.sh→el96-yminus2@el98-lrel@lvms-standard2.sh— removed ginkgo, addedsuites/standard2/after LVMS cleanup, added--vm_disksize 30Created (2):
el96-yminus1@el98-lrel@lvms-standard2.sh— LVMS lifecycle + standard2 after upgradeel96-yminus2@el98-lrel@lvms-standard1.sh— LVMS lifecycle + ginkgo + standard1 after upgradeRe-enabled (3):
el98@rpm-standard1.sh(was.disabled)el98@rpm-standard2.sh(was.disabled)el98@rpm-upgrade.sh(was.disabled)scripts/lvms-helpers/(1 modified)cleanupWorkload.sh— changedset -uxtoset -euxso cleanup failures abort the scenarioTotal: 23 scenarios deleted, 12 renamed, 5 created, 3 re-enabled, 1 script fixed → net -15 VMs
Test plan
🤖 Generated with Claude Code
Cherry-picked from #6588 (release-4.22)
Summary by CodeRabbit
Tests
Chores