Skip to content

test(e2e): enable instance-health-check and smoke-test on the cluster-free harness - #5057

Merged
openshift-merge-bot[bot] merged 6 commits into
redhat-developer:mainfrom
gustavolira:rhidp-13501-cluster-free-more-specs
Jul 7, 2026
Merged

test(e2e): enable instance-health-check and smoke-test on the cluster-free harness#5057
openshift-merge-bot[bot] merged 6 commits into
redhat-developer:mainfrom
gustavolira:rhidp-13501-cluster-free-more-specs

Conversation

@gustavolira

@gustavolira gustavolira commented Jul 6, 2026

Copy link
Copy Markdown
Member

Description

Widens the cluster-free E2E harness (RHIDP-13501) with the next seven specs from the enablement queue in docs/e2e-tests/layer-migration-matrix.md — the PR check goes from 4 to 14 test cases:

  • instance-health-check.spec.ts — GET /healthcheck, zero plugin/config dependencies.
  • smoke-test.spec.ts — guest sign-in + home-page welcome heading.
  • home-page-customization.spec.ts — all three tests; the CI home-page card customization (Placeholder/Markdown/Featured Docs/Random Joke/Top + Recently Visited) is mirrored into app-config.local-e2e.yaml.
  • plugins/frontend/sidebar.spec.ts — CI menu customization (Test enabled/nested, techdocs Favorites → Docs, Test_i) mirrored into the overlay, plus the techdocs frontend plugin added to the harness OCI set (the /docs TechDocsIndexPage route is a dynamic plugin and 404s without it).
  • settings.spec.ts — CI i18n.locales + userSettings.persistence mirrored, and the guest ownership entities ("Guest User", member of team-a — ingested from Keycloak in CI) provided as a minimal User/Group pair via a catalog.locations file entry; the guest sign-in resolver then issues the same ownership refs as in-cluster.
  • plugins/user-settings-info-card.spec.ts — CI buildInfo card customization mirrored.
  • plugins/application-provider.spec.ts + plugins/application-listener.spec.ts — the application provider/listener test plugins are OCI-only builds that CI installs through its Helm values (values_showcase.yaml); the harness now installs the same packages with the same pluginConfig.

All specs run unmodified apart from the @cluster-free tag and the testMatch allowlist entries. One commit per spec (or small pair), each validated green locally before push.

Skipped (documented in docs/e2e-tests/local-e2e-harness.md): plugins/licensed-users-info-backend — the plugin is not published to the overlays OCI registry.

Enabling change: /healthcheck on the frontend origin

instance-health-check and smoke-test hit /healthcheck relative to the frontend baseURL. In-cluster that works because the backend serves the app and the health endpoint on a single origin; the legacy app dev server previously had no route for it. This PR adds a dev-only proxy entry to packages/app/package.json (consumed by the janus-cli webpack dev server) forwarding /healthcheck to the backend, mirroring the production single-origin behavior. The e2e-cluster-free.yaml path filter now also watches packages/app/package.json since the harness depends on it.

Validation

  • 14/14 tests green locally on playwright.legacy-local.config.ts; each commit also validated by the "E2E Cluster-free / e2e" PR check (~4-5 min).
  • fmt / lint / prettier clean. The pre-existing tsc error in playwright/support/coverage/test.ts is unrelated (touched by refactor(e2e): consolidate auth and runtime test seams #5022).

Part of epic RHIDP-13501 (E2E Test Optimization). Follow-up to #5005.

🤖 Generated with Claude Code

…-free harness

Both specs run unmodified apart from the @cluster-free tag (RHIDP-13501,
enablement order per docs/e2e-tests/layer-migration-matrix.md).

They reach /healthcheck through the frontend origin, which in production is
served by the backend on the same origin. The legacy app dev server now
mirrors that: a proxy entry in packages/app/package.json forwards
/healthcheck to the backend (consumed by the janus-cli webpack dev server,
dev-only). The workflow path filter gains packages/app/package.json since
the harness depends on it.

Verified locally: 6/6 tests green on playwright.legacy-local.config.ts.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@codecov

codecov Bot commented Jul 6, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 54.77%. Comparing base (db354ff) to head (82e4a42).
⚠️ Report is 3 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #5057      +/-   ##
==========================================
- Coverage   55.39%   54.77%   -0.62%     
==========================================
  Files         122      110      -12     
  Lines        2365     2147     -218     
  Branches      563      542      -21     
==========================================
- Hits         1310     1176     -134     
+ Misses       1049      969      -80     
+ Partials        6        2       -4     
Flag Coverage Δ
rhdh 54.77% <ø> (-0.62%) ⬇️

Continue to review full report in Codecov by Harness.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update db354ff...82e4a42. Read the comment docs.

🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

All three tests, unmodified apart from the @cluster-free tag. The CI
home-page card customization (Placeholder/Markdown/Featured Docs/Random
Joke/Top + Recently Visited, from the CI dynamic-plugins-config.yaml
configmap) is mirrored into app-config.local-e2e.yaml. The mountPoints
array replaces the static app-config.dynamic-plugins.yaml one on config
merge, so the mirror is the CI list verbatim (a superset of the static
list, keeping guest-signin's SearchBar/Quick Access/Starred cards).

Verified locally: 9/9 tests green on playwright.legacy-local.config.ts.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@github-actions

github-actions Bot commented Jul 6, 2026

Copy link
Copy Markdown
Contributor

The container image build workflow finished with status: cancelled.

…harness

Unmodified apart from the @cluster-free tag. Two enabling changes:

- Mirror the CI menu customization into app-config.local-e2e.yaml
  (Test enabled/nested main-menu items and the techdocs Favorites -> Docs
  and Test_i items, from the CI dynamic-plugins-config.yaml configmap).
- Add the techdocs frontend plugin (OCI) to the harness set: the spec
  navigates to /docs, whose TechDocsIndexPage route is a dynamic plugin
  (404 off-cluster without it). Its route/menu config already lives in the
  static app-config.dynamic-plugins.yaml, so no pluginConfig is needed.

Verified locally: 10/10 tests green on playwright.legacy-local.config.ts.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@github-actions

github-actions Bot commented Jul 6, 2026

Copy link
Copy Markdown
Contributor

The container image build workflow finished with status: cancelled.

Unmodified apart from the @cluster-free tag. Three enabling mirrors of the
CI configmap (app-config-rhdh.yaml) in app-config.local-e2e.yaml:

- i18n.locales — the settings language toggle only renders with multiple
  locales configured (the spec switches to French).
- userSettings.persistence: browser.
- Guest ownership entities: CI ingests the guest user ("Guest User",
  member of team-a) from Keycloak via keycloakOrg; the harness ingests an
  equivalent minimal User/Group pair from
  e2e-tests/local-harness/guest-ownership-entities.yaml through a
  catalog.locations file entry (file targets resolve relative to the
  backend cwd, packages/backend). The guest sign-in resolver then issues
  ownership refs including team-a exactly as in-cluster.

Verified locally: 11/11 tests green on playwright.legacy-local.config.ts.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@github-actions

github-actions Bot commented Jul 6, 2026

Copy link
Copy Markdown
Contributor

The container image build workflow finished with status: cancelled.

…e harness

Unmodified apart from the @cluster-free tag. The CI buildInfo card
customization ("RHDH Build info": TechDocs builder/Authentication
provider/RBAC entries, overrideBuildInfo) is mirrored from the CI
app-config-rhdh.yaml configmap into app-config.local-e2e.yaml.

Verified locally: 12/12 tests green on playwright.legacy-local.config.ts.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@github-actions

github-actions Bot commented Jul 6, 2026

Copy link
Copy Markdown
Contributor

The container image build workflow finished with status: cancelled.

…ter-free harness

Unmodified apart from the @cluster-free tag. The application-provider-test
and application-listener-test plugins are OCI-only builds (not part of the
repo's dynamic-plugins source tree): CI installs them from ghcr through its
Helm values (values_showcase.yaml), and the harness now installs the same
packages with the same pluginConfig via local-harness/dynamic-plugins.yaml.

Also documents the one spec from the enablement queue that cannot run yet:
plugins/licensed-users-info-backend, whose plugin is not published to the
overlays OCI registry.

Verified locally: 14/14 tests green on playwright.legacy-local.config.ts.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@github-actions

github-actions Bot commented Jul 6, 2026

Copy link
Copy Markdown
Contributor

The container image build workflow finished with status: cancelled.

@sonarqubecloud

sonarqubecloud Bot commented Jul 6, 2026

Copy link
Copy Markdown

@github-actions

github-actions Bot commented Jul 6, 2026

Copy link
Copy Markdown
Contributor

Image was built and published successfully. It is available at:

@gustavolira

Copy link
Copy Markdown
Member Author

/agentic_review

@rhdh-qodo-merge

Copy link
Copy Markdown

Code Review by Qodo

🐞 Bugs (0) 📘 Rule violations (1) 📜 Skill insights (0)

Context used
⚠️ Tickets: not configured — ticket URL found in PR but could not be fetched — check ticket provider credentials
✅ Compliance rules (platform): 46 rules

Grey Divider


Informational

1. settings.spec.ts lacks beforeAll 📘 Rule violation ⚙ Maintainability
Description
e2e-tests/playwright/e2e/settings.spec.ts and
e2e-tests/playwright/e2e/plugins/frontend/sidebar.spec.ts lack a test.beforeAll hook that pushes
the required component annotation, and instead currently add annotations in test.beforeEach. This
violates the requirement that each spec file must declare its component annotation in
test.beforeAll for consistent reporting.
Code

e2e-tests/playwright/e2e/settings.spec.ts[26]

+  test(`Verify settings page`, { tag: "@cluster-free" }, async () => {
Relevance

⭐ Low

Repo historically adds component annotations in beforeEach (not beforeAll), e.g., reporting metadata
in PRs #3618/#3333.

PR-#3618
PR-#3333

ⓘ Recommendations generated based on similar findings in past PRs

Evidence
PR Compliance ID 1834 requires every spec file to include a test.beforeAll hook that pushes a
component annotation. In both settings.spec.ts and sidebar.spec.ts, there is no beforeAll hook
present at all—only a beforeEach that pushes annotations—so neither file follows the mandated
pattern and therefore fails the compliance requirement.

Rule 1834: Every e2e spec file must have a component annotation in test.beforeAll
e2e-tests/playwright/e2e/settings.spec.ts[12-22]
e2e-tests/playwright/e2e/plugins/frontend/sidebar.spec.ts[10-24]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
Both `e2e-tests/playwright/e2e/settings.spec.ts` and `e2e-tests/playwright/e2e/plugins/frontend/sidebar.spec.ts` are missing a `test.beforeAll` hook that pushes the required component annotation; the annotation is currently added in `test.beforeEach`.

## Issue Context
Compliance (PR Compliance ID 1834) requires component annotations to be present in `test.beforeAll` for every spec file to ensure consistent reporting.

## Fix Focus Areas
- e2e-tests/playwright/e2e/settings.spec.ts[12-22]
- e2e-tests/playwright/e2e/plugins/frontend/sidebar.spec.ts[10-24]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools


Grey Divider

Qodo Logo

@openshift-ci openshift-ci Bot added the lgtm label Jul 7, 2026
@openshift-merge-bot
openshift-merge-bot Bot merged commit 034e653 into redhat-developer:main Jul 7, 2026
25 checks passed
@openshift-ci

openshift-ci Bot commented Jul 7, 2026

Copy link
Copy Markdown

@gustavolira: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name Commit Details Required Rerun command
ci/prow/e2e-ocp-helm 82e4a42 link unknown /test e2e-ocp-helm

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

gustavolira added a commit to gustavolira/rhdh that referenced this pull request Jul 7, 2026
…at-developer#5057)

Marks the 8 newly-validated specs green (harness now at 9 specs / 14 test
cases on the PR check), records the enabling mechanics, and flags the one
remaining candidate (licensed-users-info) as blocked on OCI publication.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
gustavolira added a commit to gustavolira/rhdh that referenced this pull request Jul 7, 2026
… wording

- 2026-07-07 update: 10 specs (not 9) — the list itself names ten files.
- Drop the positional "queue above" reference (the queue list no longer
  precedes it) and past-tense the 2026-07-02 "now runs" so the two dated
  updates stop contradicting each other.
- Self-date the PR redhat-developer#4967 "open" claim and add PR redhat-developer#5057 to References.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
gustavolira added a commit to gustavolira/rhdh that referenced this pull request Jul 7, 2026
… wording

- 2026-07-07 update: 10 specs (not 9) — the list itself names ten files.
- Drop the positional "queue above" reference (the queue list no longer
  precedes it) and past-tense the 2026-07-02 "now runs" so the two dated
  updates stop contradicting each other.
- Self-date the PR redhat-developer#4967 "open" claim and add PR redhat-developer#5057 to References.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
gustavolira added a commit to gustavolira/rhdh that referenced this pull request Jul 20, 2026
… wording

- 2026-07-07 update: 10 specs (not 9) — the list itself names ten files.
- Drop the positional "queue above" reference (the queue list no longer
  precedes it) and past-tense the 2026-07-02 "now runs" so the two dated
  updates stop contradicting each other.
- Self-date the PR redhat-developer#4967 "open" claim and add PR redhat-developer#5057 to References.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
openshift-merge-bot Bot pushed a commit that referenced this pull request Jul 20, 2026
* docs(e2e): add E2E layer migration matrix (RHIDP-15076)

Phase-1 deliverable of RHIDP-15076 (E2E Test Optimization epic
RHIDP-13501): classifies all 30 e2e specs by target layer (L1-L4b),
maps which are supplementable by Layer 3 component tests or the
cluster-free L4a harness, and includes the companion analysis of the
rhdh-plugin-export-overlays test system. Referenced from the Jira
story's closing comment; updated 2026-07-02 with the L4a harness
validation results from PR #5005 (4 tests green cluster-free).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* docs(e2e): address review — reconcile counts, durable references, README link

- Fix the Tally L3 count (10, not 9) so the column totals sum to the
  30-spec heading; reconcile the heading itself (29 on main + #19 pending
  in PR #4967).
- "runs 4 tests" -> "runs 2 specs (4 test cases)" — the doc's accounting
  unit is the spec; note that spec numbers refer to the matrix below.
- Soften "fully covers the 12 pure-backend workspaces" to load + API
  surface: scaffolder-backend-module-kubernetes also has a UI e2e that
  needs the render harness, so "fully" overstated the native-smoke scope.
- Replace short commit hashes and the fork-only
  RHIDP-13235-layer3-component-tests branch name with the durable PR
  reference (rhdh#4864, closed) — hashes on a mutable branch dangle after
  a rebase or branch deletion.
- Give DRAFT a promotion condition (groomed into RHIDP-13528/13529).
- Link the matrix from docs/e2e-tests/README.md ("Adding a Test") so the
  doc is discoverable outside the Jira comment.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* docs(e2e): correct licensed-users-info blocker — internal backend plugin, not OCI

* docs(e2e): address review — fix spec count, dangling reference, dated wording

- 2026-07-07 update: 10 specs (not 9) — the list itself names ten files.
- Drop the positional "queue above" reference (the queue list no longer
  precedes it) and past-tense the 2026-07-02 "now runs" so the two dated
  updates stop contradicting each other.
- Self-date the PR #4967 "open" claim and add PR #5057 to References.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* docs(e2e): record the merged overlays native smoke harness (#2714/#2731)

The companion analysis recommended rebuilding PR #2231's in-process idea on
the published CLI — that landed in the overlay repo on 2026-07-06/07 as
smoke-tests-native (overlays#2714, ~20x vs the Docker smoke) plus workspace
mode (overlays#2731), running as a dedicated native-smoke.yaml workflow
alongside the Docker one. Mark recommendation 1 as landed and add both PRs
to References.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* docs(e2e): address review — reconcile landed-vs-target claims and stale count

- "Every remaining candidate landed" contradicted the #2 bullet in the same
  section — now "every candidate except #2".
- The native-harness recommendation read as if the full Docker-smoke
  replacement had shipped; scope it to what overlays#2714 landed (backend
  boots), with catalog-extending modules and the frontend bundle probe
  still pending (same qualifier in the bucket table).
- ROI paragraph still said 9 L3 candidates after the tally fix made it 10.
- Date-mark the pre-#2714 two-tier takeaway so it points at the update.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* docs(e2e): correct #4864 status and refresh matrix against main

Re-verified the document against origin/main (2026-07-20):

- rhdh#4864 was **merged** on 2026-06-01, not "closed, not merged". The six
  Layer 3 tests are on main under packages/app/src — replace the prose
  descriptions with the actual file paths so the reference stays durable.
- Batch 1 no longer asks to "close out" #5/#6/#8/#9; that work landed.
- #18 github-happy-path is no longer a showcase `.fixme`; #5022 parked it at
  e2e-tests/playwright/blocked/github-happy-path.blocked.ts.
- Spell out how the "29 specs on main" count is composed (28 *.spec.ts + the
  blocked one) so it can be re-checked mechanically.
- Refresh dates; #4967 is still open as of 2026-07-20.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants