-
Notifications
You must be signed in to change notification settings - Fork 221
docs(ci): document release smoke testing process #4449
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
openshift-merge-bot
merged 2 commits into
redhat-developer:main
from
gustavolira:RHIDP-8640
Mar 24, 2026
Merged
Changes from all commits
Commits
Show all changes
2 commits
Select commit
Hold shift + click to select a range
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,102 @@ | ||
| # Release Smoke Testing Process | ||
|
|
||
| This document describes how to run the existing CI test suite against a productized (Red Hat) RHDH | ||
| image to validate a new release candidate before shipping. | ||
|
|
||
| The approach is simple: create a PR that overrides the community image references with the | ||
| productized image, and let CI do the rest. | ||
|
|
||
| **Reference PR:** <https://github.com/redhat-developer/rhdh/pull/4431> | ||
|
|
||
| ## Step 1: Override Variables in `openshift-ci-tests.sh` | ||
|
|
||
| Edit `.ci/pipelines/openshift-ci-tests.sh` and add the following exports **before** the `main()` | ||
| function: | ||
|
|
||
| ```bash | ||
| export CHART_VERSION="1.9-211-CI" | ||
| export HELM_CHART_URL="oci://quay.io/rhdh/chart" | ||
| export QUAY_REPO="rhdh/rhdh-hub-rhel9" | ||
| export TAG_NAME="1.9-211" | ||
| ``` | ||
|
|
||
| Replace the version and build numbers to match the release candidate you are testing. | ||
|
|
||
| - `CHART_VERSION` — Helm chart version to install (append `-CI` suffix). | ||
| - `HELM_CHART_URL` — OCI registry URL for the Helm chart. | ||
| - `QUAY_REPO` — Productized image repository (instead of the community `rhdh-community/rhdh`). | ||
| - `TAG_NAME` — Image tag for the release candidate. | ||
|
|
||
| ## Step 2: Override Image in `values_showcase.yaml` | ||
|
|
||
| Edit `.ci/pipelines/value_files/values_showcase.yaml` and update the `upstream.backstage.image` | ||
| section: | ||
|
|
||
| ```yaml | ||
| upstream: | ||
| backstage: | ||
| image: | ||
| pullPolicy: Always | ||
| registry: quay.io | ||
| repository: rhdh/rhdh-hub-rhel9 # was: rhdh-community/rhdh | ||
| tag: "1.9-211" # was: next-1.9 | ||
| ``` | ||
|
|
||
| ## Step 3: Override Image in `values_showcase-rbac.yaml` | ||
|
|
||
| Apply the same image changes in `.ci/pipelines/value_files/values_showcase-rbac.yaml`: | ||
|
|
||
| ```yaml | ||
| upstream: | ||
| backstage: | ||
| image: | ||
| pullPolicy: Always | ||
| registry: quay.io | ||
| repository: rhdh/rhdh-hub-rhel9 # was: rhdh-community/rhdh | ||
| tag: "1.9-211" # was: next-1.9 | ||
| ``` | ||
|
|
||
| ## Step 4: Create the PR | ||
|
|
||
| Push the branch and open a PR. The default test suite runs automatically on PR creation and on every | ||
| new push. | ||
|
|
||
| ### Skip the community image build | ||
|
|
||
| Since you are testing a pre-built productized image, there is no need to build the community image. | ||
| Add `[skip-build]` to your commit messages to save time: | ||
|
|
||
| ``` | ||
| chore: test rhdh 1.9-211 [skip-build] | ||
| ``` | ||
|
|
||
| ## Running Additional Test Suites | ||
|
|
||
| To see the full list of available test suites, post the following comment on the PR: | ||
|
|
||
| ``` | ||
| /test ? | ||
| ``` | ||
|
|
||
| CI will reply with a list of test suite commands. Copy the desired command and post it as a new | ||
| comment to trigger that suite. | ||
|
|
||
| ## Testing on a Custom OpenShift Cluster | ||
|
|
||
| By default, CI tests run on ephemeral clusters from the cluster pool. To test on a specific | ||
| OpenShift cluster instead, add the following overrides to `openshift-ci-tests.sh` (alongside the | ||
| exports from Step 1): | ||
|
|
||
| ```bash | ||
| export K8S_CLUSTER_URL='https://api.<your-cluster>:443' | ||
| export OCM_CLUSTER_TOKEN=$K8S_CLUSTER_TOKEN_TEMPORARY | ||
| export K8S_CLUSTER_TOKEN=$K8S_CLUSTER_TOKEN_TEMPORARY | ||
|
|
||
| # Recalculate derived variables (originally computed in env_variables.sh before the override) | ||
| export K8S_CLUSTER_TOKEN_ENCODED=$(printf "%s" "$K8S_CLUSTER_TOKEN" | base64 | tr -d '\n') | ||
| export K8S_CLUSTER_API_SERVER_URL=$(printf "%s" "$K8S_CLUSTER_URL" | base64 | tr -d '\n') | ||
| export K8S_SERVICE_ACCOUNT_TOKEN=$K8S_CLUSTER_TOKEN_ENCODED | ||
| ``` | ||
|
|
||
| Before pushing, update the `K8S_CLUSTER_TOKEN_TEMPORARY` secret in Vault with the login token from | ||
|
josephca marked this conversation as resolved.
|
||
| your target cluster. | ||
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
These steps are completely redundant. They are basically placeholders which are getting overriden by
helm upgrade -iwhich uses this function for params:rhdh/.ci/pipelines/lib/helm.sh
Lines 214 to 225 in 30f6369
It's actually pretty misleading, and we should replace them or delete them from the value files.
Uh oh!
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
PR to remove them as a proof: #4457