Skip to content

ci(backstage): skip plugin downloads in PR checks and use chart defaults - #480

Merged
rm3l merged 10 commits into
redhat-developer:mainfrom
rm3l:chore/backstage-ci-skip-plugins-next-catalog-index
Jul 21, 2026
Merged

ci(backstage): skip plugin downloads in PR checks and use chart defaults#480
rm3l merged 10 commits into
redhat-developer:mainfrom
rm3l:chore/backstage-ci-skip-plugins-next-catalog-index

Conversation

@rm3l

@rm3l rm3l commented Jul 20, 2026

Copy link
Copy Markdown
Member

Description of the change

Speed up backstage chart PR tests and align CI with chart defaults:

  • Use next catalog index tag: switch global.catalogIndex.image.tag from 1.10 to next
  • Skip dynamic plugin downloads in CI: set global.dynamic.includes, lightspeed.plugins,
    and orchestrator.plugins to [] in all CI values files — the chart features under test
    (deployment, config, probes) don't depend on actual plugins
  • Restore plugins in nightly: add helm_extra_args input to the test-charts composite action
    and generate a nightly override from values.yaml (via yq) so nightly runs still exercise
    the full plugin installation stack
  • Use chart-default backstage image in PR tests: remove the image repository/tag overrides
    from the PR test workflow so the chart's own values.yaml defaults are used

Which issue(s) does this PR fix or relate to

How to test changes / Special notes to the reviewer

  • PR tests should be faster since dynamic plugins are no longer downloaded and the catalog index is not pulled.
  • Nightly tests still exercise plugin installation by generating and applying an override values file extracted from values.yaml.
  • PR tests now use the backstage image defined in values.yaml instead of overriding it with rhdh/rhdh-hub-rhel9:latest.

Checklist

  • For each Chart updated, version bumped in the corresponding Chart.yaml according to Semantic Versioning.
  • For each Chart updated, variables are documented in the values.yaml and added to the corresponding README.md.
  • JSON Schema template updated and re-generated the raw schema via the pre-commit hook.
  • Tests pass using the Chart Testing tool and the ct lint command.
  • If you updated the orchestrator-infra chart, make sure the versions of the Knative CRDs are aligned with the versions of the CRDs installed by the OpenShift Serverless operators declared in the values.yaml file. See Installing Knative Eventing and Knative Serving CRDs for more details.

@rm3l
rm3l requested a review from a team as a code owner July 20, 2026 21:47
@rm3l

rm3l commented Jul 20, 2026

Copy link
Copy Markdown
Member Author

/cherry-pick release-1.9
/cherry-pick release-1.10

@openshift-cherrypick-robot

Copy link
Copy Markdown

@rm3l: once the present PR merges, I will cherry-pick it on top of release-1.10, release-1.9 in new PRs and assign them to you.

Details

In response to this:

/cherry-pick release-1.9
/cherry-pick release-1.10

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@rhdh-qodo-merge

Copy link
Copy Markdown

PR Summary by Qodo

CI: skip Backstage plugin downloads in PR checks, keep nightly coverage

✨ Enhancement ⚙️ Configuration changes 📝 Documentation 🕐 20-40 Minutes

Grey Divider

AI Description

• Switch Backstage catalog index default image tag to "next".
• Disable dynamic plugin installation in CI values to speed PR chart tests.
• Restore plugin coverage in nightly via generated override and extra Helm args.
Diagram

graph TD
  A["PR CI values (plugins off)"] --> B["test-charts action"] --> C["ct install (helm)"] --> D["Backstage chart"]
  E["nightly workflow"] --> F["yq: build override"] --> G["nightly override values"] --> B
  H["values.yaml (defaults)"] --> F
Loading
High-Level Assessment

The following are alternative approaches to this PR:

1. Cache plugin downloads in CI
  • ➕ Keeps PR checks exercising the full plugin install path
  • ➕ Potentially fastest steady-state if cache hit rates are high
  • ➖ Cache invalidation/versioning complexity for plugin artifacts
  • ➖ More moving parts (cache keys, storage limits, partial hits) and harder debugging when caches go stale
2. Maintain a dedicated “CI-minimal” plugin bundle image
  • ➕ Deterministic installs without network/plugin resolution during CI
  • ➕ Still covers plugin loading paths (from pre-baked artifacts)
  • ➖ Requires building/publishing an additional image and keeping it in sync
  • ➖ Still not equivalent to exercising real catalog-index driven discovery/install

Recommendation: Current approach is a good tradeoff for PR signal-to-time ratio: PR checks validate chart wiring (resources, config, probes) without paying the dynamic plugin download cost, while nightly restores meaningful coverage of the plugin installation stack via an override derived from values.yaml. If nightly coverage is considered critical, add a small guard (e.g., a comment/test) to ensure the yq-generated override continues to include all plugin-related keys as values.yaml evolves.

Files changed (14) +69 / -4

Documentation (1) +1 / -1
README.mdDocument default catalog index tag as next +1/-1

Document default catalog index tag as next

• Updates the documented default global.catalogIndex.image.tag from 1.10 to next to match chart defaults.

charts/backstage/README.md

Other (13) +68 / -3
action.ymlAdd helm_extra_args passthrough to ct install +8/-0

Add helm_extra_args passthrough to ct install

• Introduces a new composite-action input (helm_extra_args) and wires it to chart-testing via ct install --helm-extra-args. This enables workflows to pass additional Helm arguments like an extra values file without overloading --set-based extra_helm_args.

.github/actions/test-charts/action.yml

nightly.yamlGenerate Backstage nightly override and pass via helm_extra_args +7/-0

Generate Backstage nightly override and pass via helm_extra_args

• Adds a step to generate /tmp/backstage-nightly-values.yaml from charts/backstage/values.yaml using yq, extracting plugin-related defaults. Passes the generated file to the test-charts action so nightly runs still exercise the full plugin installation configuration while PR CI stays fast.

.github/workflows/nightly.yaml

default-values.yamlDisable dynamic plugin installs in CI default values +8/-0

Disable dynamic plugin installs in CI default values

• Sets global.dynamic.includes to [] and clears global.lightspeed.plugins for CI to avoid downloading/installing dynamic plugins during PR chart tests. Adds explanatory comments about CI speed and test scope.

charts/backstage/ci/default-values.yaml

with-custom-dynamic-pvc-claim-spec-values.yamlDisable dynamic plugin installs in CI PVC-claim-spec scenario +7/-0

Disable dynamic plugin installs in CI PVC-claim-spec scenario

• Overrides global.dynamic.includes to [] and clears global.lightspeed.plugins to prevent plugin downloads in this CI test variant.

charts/backstage/ci/with-custom-dynamic-pvc-claim-spec-values.yaml

with-custom-image-for-test-pod-values.yamlDisable dynamic plugin installs in CI custom test-pod image scenario +8/-0

Disable dynamic plugin installs in CI custom test-pod image scenario

• Adds global.dynamic.includes: [] and global.lightspeed.plugins: [] to speed this CI scenario by skipping dynamic plugin downloads.

charts/backstage/ci/with-custom-image-for-test-pod-values.yaml

with-lightspeed-disabled-values.yamlSkip dynamic plugins even when Lightspeed is disabled +4/-0

Skip dynamic plugins even when Lightspeed is disabled

• Adds global.dynamic.includes: [] and sets lightspeed.plugins: [] in the Lightspeed-disabled CI values, ensuring plugin downloads are skipped consistently across CI variants.

charts/backstage/ci/with-lightspeed-disabled-values.yaml

with-lightspeed-service-host.yamlSkip dynamic plugins and override Lightspeed plugins for CI +5/-0

Skip dynamic plugins and override Lightspeed plugins for CI

• Sets global.dynamic.includes: [] and clears lightspeed.plugins, with a FIXME note indicating this override can be removed once the next catalog index stabilizes for Lightspeed references.

charts/backstage/ci/with-lightspeed-service-host.yaml

with-orchestrator-and-dynamic-plugins-npmrc-values.yamlDisable plugin downloads for orchestrator + npmrc CI scenario +5/-0

Disable plugin downloads for orchestrator + npmrc CI scenario

• Sets global.dynamic.includes: [] to prevent downloading included dynamic plugins, and clears global.lightspeed.plugins and orchestrator.plugins to avoid plugin installation work in CI while keeping orchestrator enabled/configured.

charts/backstage/ci/with-orchestrator-and-dynamic-plugins-npmrc-values.yaml

with-orchestrator-values.yamlDisable plugin downloads for orchestrator CI scenario +5/-0

Disable plugin downloads for orchestrator CI scenario

• Sets global.dynamic.includes: [] and clears global.lightspeed.plugins and orchestrator.plugins so the orchestrator-enabled CI scenario avoids dynamic plugin installation overhead.

charts/backstage/ci/with-orchestrator-values.yaml

with-test-pod-disabled-values.yamlDisable dynamic plugin installs in CI test-pod-disabled scenario +8/-0

Disable dynamic plugin installs in CI test-pod-disabled scenario

• Adds global.dynamic.includes: [] and global.lightspeed.plugins: [] so this CI variant skips dynamic plugin downloads like the other CI values files.

charts/backstage/ci/with-test-pod-disabled-values.yaml

values.schema.jsonUpdate schema default catalog index tag to next +1/-1

Update schema default catalog index tag to next

• Changes the generated JSON schema default for global.catalogIndex.image.tag to next, keeping schema and values aligned.

charts/backstage/values.schema.json

values.schema.tmpl.jsonUpdate schema template default catalog index tag to next +1/-1

Update schema template default catalog index tag to next

• Updates the schema template default for global.catalogIndex.image.tag to next so future schema regenerations preserve the new default.

charts/backstage/values.schema.tmpl.json

values.yamlSet default catalog index tag to next +1/-1

Set default catalog index tag to next

• Updates global.catalogIndex.image.tag from 1.10 to next, aligning Backstage chart defaults with nightly images and upcoming catalog index content.

charts/backstage/values.yaml

Switch the catalog index image tag from 1.10 to next, and disable
dynamic plugin downloads in all CI values files to speed up PR test
cycles. The chart features under test (deployment, config, probes)
don't depend on actual plugins.

- Set global.dynamic.includes to [] in every CI values file
- Set lightspeed.plugins and orchestrator.plugins to [] in CI
- Add helm_extra_args input to the test-charts action
- Generate a nightly override from values.yaml to restore default
  plugin configuration in nightly runs

Assisted-by: Claude
@rm3l
rm3l force-pushed the chore/backstage-ci-skip-plugins-next-catalog-index branch from 3b72a23 to d963323 Compare July 20, 2026 21:50
@rhdh-qodo-merge

rhdh-qodo-merge Bot commented Jul 20, 2026

Copy link
Copy Markdown

Code Review by Qodo

🐞 Bugs (1) 📘 Rule violations (0) 📜 Skill insights (0)

Context used
⚠️ Tickets: not configured — ticket URL found in PR but could not be fetched — check ticket provider credentials

Grey Divider


Action required

1. Nightly override ignored 🐞 Bug ☼ Reliability
Description
The nightly workflow checks out each matrix.branch and then invokes the local composite action
(./.github/actions/test-charts) while passing helm_extra_args, so only branches that contain the
updated action implementation will apply the generated override values file. For other branches in
the nightly matrix, the override can be silently skipped, meaning nightlies won’t exercise the
plugin-installation path as intended.
Code

.github/workflows/nightly.yaml[R111-119]

+      - name: Generate nightly values override for backstage chart
+        if: steps.check.outputs.exists == 'true' && matrix.chart == 'backstage'
+        run: |
+          yq e '{"global": {"dynamic": {"includes": .global.dynamic.includes}, "lightspeed": {"plugins": .global.lightspeed.plugins}}, "orchestrator": {"plugins": .orchestrator.plugins}}' \
+            charts/backstage/values.yaml > /tmp/backstage-nightly-values.yaml
+
      - name: Test charts
        if: steps.check.outputs.exists == 'true'
        uses: ./.github/actions/test-charts
Relevance

⭐⭐⭐ High

They’ve handled nightly matrix branch/action input drift before via explicit compatibility steps (PR
#437).

PR-#437
PR-#346

ⓘ Recommendations generated based on similar findings in past PRs

Evidence
The nightly workflow checks out each target branch before calling a local-path action, so the action
definition/inputs used at runtime come from that checked-out branch. The new override is passed via
a newly-added action input (helm_extra_args) which only takes effect if the branch’s action.yml
implements it; the workflow itself also notes that action inputs can differ across release branches.

.github/workflows/nightly.yaml[54-59]
.github/workflows/nightly.yaml[94-110]
.github/workflows/nightly.yaml[111-126]
.github/actions/test-charts/action.yml[4-23]
.github/actions/test-charts/action.yml[175-229]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
Nightly checks out `matrix.branch` and then uses `./.github/actions/test-charts`. Because local actions are resolved from the checked-out workspace, any branch that doesn’t yet contain the new `helm_extra_args` input/logic will not apply the generated `--values` override, so the nightly job won’t reliably re-enable plugin installation.

## Issue Context
The workflow is intentionally testing multiple release branches, and it already acknowledges that the composite action’s inputs can differ across branches. Adding a new input (`helm_extra_args`) exacerbates this: the workflow may pass it, but the checked-out branch’s action may not consume it.

## Fix Focus Areas
- .github/workflows/nightly.yaml[54-59]
- .github/workflows/nightly.yaml[111-126]
- .github/actions/test-charts/action.yml[4-23]
- .github/actions/test-charts/action.yml[175-223]

### Concrete fix options (pick one)
1) **Use a versioned action implementation independent of the checked-out branch**
  - Checkout the workflow repo (main) into a dedicated path for the action, and checkout the target branch into another path for the charts under test.
  - Invoke the composite action from the “main action” checkout, while pointing `ct` at the charts directory from the target-branch checkout.

2) **Backport/synchronize the composite action change to all branches in the nightly matrix**
  - Ensure every `release-*` branch includes the `helm_extra_args` input and the corresponding `ct install --helm-extra-args` wiring.

3) **Avoid relying on the composite action for this branch-specific behavior**
  - For nightly only, run `ct install` directly in the workflow (or wrap it in a separate action that is always taken from main) so the override is applied consistently regardless of the tested branch.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools


Grey Divider

Qodo Logo

@rhdh-qodo-merge rhdh-qodo-merge Bot added documentation Improvements or additions to documentation enhancement New feature or request Tests labels Jul 20, 2026
rm3l added 2 commits July 21, 2026 08:48
Add --proto =https to the curl command so redirects cannot downgrade
to plain HTTP.

Assisted-by: Claude
Comment thread .github/actions/test-charts/action.yml Fixed
@rm3l

rm3l commented Jul 21, 2026

Copy link
Copy Markdown
Member Author

/cherry-pick release-1.9
/cherry-pick release-1.10

@openshift-cherrypick-robot

Copy link
Copy Markdown

@rm3l: once the present PR merges, I will cherry-pick it on top of release-1.10, release-1.9 in new PRs and assign them to you.

Details

In response to this:

/cherry-pick release-1.9
/cherry-pick release-1.10

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

Stop overriding the backstage image repository and tag in the PR test
workflow. The chart defaults from values.yaml are used instead.

Assisted-by: Claude
@rm3l rm3l changed the title ci: skip plugin downloads in PR checks but reenable in nightly to speed things up ci(backstage): skip plugin downloads in PR checks and use chart defaults Jul 21, 2026
rm3l added 6 commits July 21, 2026 09:23
Run helm template for each CI values file with the default plugin
configuration restored on top, before creating the KinD cluster.
This catches template rendering regressions in PR checks without
the cost of a full install.

Assisted-by: Claude
The bulk-import plugins are incompatible with the current Backstage
version. Replace them with scaffolder-relation-processor and tech-radar
from the rhdh-plugin-export-overlays OCI registry, which are both
compatible (bs_1.52.0) and require no extra configuration.

Assisted-by: Claude
Rename with-lightspeed-service-host.yaml to
with-lightspeed-service-host-values.yaml so it matches the
*-values.yaml pattern used by ct and the helm template sanity check.
Also fix the glob to use *-values.yaml only.

Assisted-by: Claude
ct's --upgrade flag runs a same-version self-upgrade test (helm install
+ helm upgrade) for every CI values file, even when the cross-version
upgrade is already skipped due to a major version bump. This doubles
test time without meaningful coverage. Conditionally omit --upgrade
when a major version bump is detected vs. the target branch.
@sonarqubecloud

Copy link
Copy Markdown

@rm3l

rm3l commented Jul 21, 2026

Copy link
Copy Markdown
Member Author

Merging to test the nightly workflows.

@rm3l
rm3l merged commit 4e4dc71 into redhat-developer:main Jul 21, 2026
9 of 10 checks passed
@rm3l
rm3l deleted the chore/backstage-ci-skip-plugins-next-catalog-index branch July 21, 2026 23:48
@openshift-cherrypick-robot

Copy link
Copy Markdown

@rm3l: #480 failed to apply on top of branch "release-1.9":

Applying: chore(backstage): use next catalog index and skip plugin downloads in CI
Using index info to reconstruct a base tree...
M	.github/actions/test-charts/action.yml
M	.github/workflows/nightly.yaml
M	charts/backstage/Chart.yaml
M	charts/backstage/README.md
M	charts/backstage/ci/default-values.yaml
A	charts/backstage/ci/with-lightspeed-disabled-values.yaml
A	charts/backstage/ci/with-lightspeed-service-host.yaml
M	charts/backstage/ci/with-orchestrator-and-dynamic-plugins-npmrc-values.yaml
M	charts/backstage/ci/with-orchestrator-values.yaml
M	charts/backstage/values.schema.json
M	charts/backstage/values.schema.tmpl.json
M	charts/backstage/values.yaml
Falling back to patching base and 3-way merge...
Auto-merging .github/actions/test-charts/action.yml
CONFLICT (content): Merge conflict in .github/actions/test-charts/action.yml
Auto-merging .github/workflows/nightly.yaml
CONFLICT (content): Merge conflict in .github/workflows/nightly.yaml
Auto-merging charts/backstage/Chart.yaml
CONFLICT (content): Merge conflict in charts/backstage/Chart.yaml
Auto-merging charts/backstage/README.md
CONFLICT (content): Merge conflict in charts/backstage/README.md
Auto-merging charts/backstage/ci/default-values.yaml
CONFLICT (content): Merge conflict in charts/backstage/ci/default-values.yaml
CONFLICT (modify/delete): charts/backstage/ci/with-lightspeed-disabled-values.yaml deleted in HEAD and modified in chore(backstage): use next catalog index and skip plugin downloads in CI.  Version chore(backstage): use next catalog index and skip plugin downloads in CI of charts/backstage/ci/with-lightspeed-disabled-values.yaml left in tree.
CONFLICT (modify/delete): charts/backstage/ci/with-lightspeed-service-host.yaml deleted in HEAD and modified in chore(backstage): use next catalog index and skip plugin downloads in CI.  Version chore(backstage): use next catalog index and skip plugin downloads in CI of charts/backstage/ci/with-lightspeed-service-host.yaml left in tree.
Auto-merging charts/backstage/ci/with-orchestrator-and-dynamic-plugins-npmrc-values.yaml
CONFLICT (content): Merge conflict in charts/backstage/ci/with-orchestrator-and-dynamic-plugins-npmrc-values.yaml
Auto-merging charts/backstage/ci/with-orchestrator-values.yaml
CONFLICT (content): Merge conflict in charts/backstage/ci/with-orchestrator-values.yaml
Auto-merging charts/backstage/values.schema.json
CONFLICT (content): Merge conflict in charts/backstage/values.schema.json
Auto-merging charts/backstage/values.schema.tmpl.json
Auto-merging charts/backstage/values.yaml
CONFLICT (content): Merge conflict in charts/backstage/values.yaml
error: Failed to merge in the changes.
hint: Use 'git am --show-current-patch=diff' to see the failed patch
hint: When you have resolved this problem, run "git am --continue".
hint: If you prefer to skip this patch, run "git am --skip" instead.
hint: To restore the original branch and stop patching, run "git am --abort".
hint: Disable this message with "git config set advice.mergeConflict false"
Patch failed at 0001 chore(backstage): use next catalog index and skip plugin downloads in CI

Details

In response to this:

/cherry-pick release-1.9
/cherry-pick release-1.10

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@openshift-cherrypick-robot

Copy link
Copy Markdown

@rm3l: #480 failed to apply on top of branch "release-1.10":

Applying: chore(backstage): use next catalog index and skip plugin downloads in CI
Using index info to reconstruct a base tree...
M	.github/actions/test-charts/action.yml
M	.github/workflows/nightly.yaml
M	charts/backstage/Chart.yaml
M	charts/backstage/README.md
M	charts/backstage/ci/with-orchestrator-and-dynamic-plugins-npmrc-values.yaml
M	charts/backstage/ci/with-orchestrator-values.yaml
M	charts/backstage/values.schema.json
M	charts/backstage/values.schema.tmpl.json
M	charts/backstage/values.yaml
Falling back to patching base and 3-way merge...
Auto-merging .github/actions/test-charts/action.yml
CONFLICT (content): Merge conflict in .github/actions/test-charts/action.yml
Auto-merging .github/workflows/nightly.yaml
CONFLICT (content): Merge conflict in .github/workflows/nightly.yaml
Auto-merging charts/backstage/Chart.yaml
CONFLICT (content): Merge conflict in charts/backstage/Chart.yaml
Auto-merging charts/backstage/README.md
CONFLICT (content): Merge conflict in charts/backstage/README.md
Auto-merging charts/backstage/ci/with-orchestrator-and-dynamic-plugins-npmrc-values.yaml
CONFLICT (content): Merge conflict in charts/backstage/ci/with-orchestrator-and-dynamic-plugins-npmrc-values.yaml
Auto-merging charts/backstage/ci/with-orchestrator-values.yaml
CONFLICT (content): Merge conflict in charts/backstage/ci/with-orchestrator-values.yaml
Auto-merging charts/backstage/values.schema.json
Auto-merging charts/backstage/values.schema.tmpl.json
Auto-merging charts/backstage/values.yaml
error: Failed to merge in the changes.
hint: Use 'git am --show-current-patch=diff' to see the failed patch
hint: When you have resolved this problem, run "git am --continue".
hint: If you prefer to skip this patch, run "git am --skip" instead.
hint: To restore the original branch and stop patching, run "git am --abort".
hint: Disable this message with "git config set advice.mergeConflict false"
Patch failed at 0001 chore(backstage): use next catalog index and skip plugin downloads in CI

Details

In response to this:

/cherry-pick release-1.9
/cherry-pick release-1.10

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation enhancement New feature or request Tests

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants