refactor(e2e): refactor runtime tests into TypeScript - #4809
Conversation
|
Skipping CI for Draft Pull Request. |
|
/review |
|
/test e2e-ocp-operator-nightly |
|
/test e2e-ocp-v4-18-helm-nightly |
PR Reviewer Guide 🔍Warning
Here are some key observations to aid the review process:
|
|
/agentic_review |
Code Review by Qodo
Context used✅ Tickets:
RHIDP-9140 1. KubeClient ignores env token
|
|
/test e2e-ocp-helm-nightly |
4ec83ad to
d383484
Compare
|
The container image build workflow finished with status: |
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #4809 +/- ##
==========================================
- Coverage 55.39% 54.77% -0.62%
==========================================
Files 122 110 -12
Lines 2365 2147 -218
Branches 563 518 -45
==========================================
- Hits 1310 1176 -134
+ Misses 1048 970 -78
+ Partials 7 1 -6
Continue to review full report in Codecov by Harness.
🚀 New features to boost your workflow:
|
d383484 to
af1c911
Compare
|
/test e2e-ocp-operator-nightly |
|
/test e2e-ocp-helm-nightly |
|
The container image build workflow finished with status: |
|
/retest |
|
/test e2e-ocp-operator-nightly |
af1c911 to
91c4806
Compare
|
/test e2e-ocp-helm-nightly |
91c4806 to
ce2f25e
Compare
|
/test e2e-ocp-helm-nightly |
- Replace 8 inline Buffer.from() base64 calls with base64Encode/Decode from helper.ts (keycloak.ts, api-helper.ts, annotator.spec.ts, scaffolder-relation-processor.spec.ts) - Fix method name typo: createCongifmap -> createConfigMap in KubeClient - Use kubeClient.createConfigMap() in runtime-deploy.ts instead of inline coreV1Api.createNamespacedConfigMap() calls Assisted-by: OpenCode
Move reusable utilities out of runtime-deploy.ts and runtime-config.ts into shared modules: helper.ts: - run() — shell command execution with stdout/stderr capture - discoverRouterBase() — OpenShift cluster router base discovery - ImageRef interface, imageRefToString(), parseCatalogIndexImage() — image reference parsing utilities kube-client.ts (KubeClient class): - createNamespace() — 409-safe namespace creation - deleteNamespaceIfExists() — 404-safe deletion wrapping deleteNamespaceAndWait() runtime-config.ts re-exports ImageRef, imageRefToString, and parseCatalogIndexImage from helper.ts so existing callers are unaffected. runtime-deploy.ts now imports all shared utilities instead of defining them locally, reducing file size by ~100 lines. Assisted-by: OpenCode
- Guard against undefined host in RDS/Azure DB test loops by skipping
individual DB versions when their host env var is not set, instead of
crashing with a TypeError in clearDatabase/configurePostgresCredentials
- Fix RELEASE_NAME default mismatch in verify-schema-mode.spec.ts
('developer-hub' -> 'rhdh') to match runtime-config.ts and kube-client.ts
- Use resolveInstallMethod() in getRhdhDeploymentName() instead of
duplicating install method detection logic
- Rewrite updateConfigMapTitle as a thin wrapper around patchAppConfig,
eliminating ~65 lines of duplicated read-modify-write ConfigMap logic
- Extract deployment env var manipulation from postgres-config.ts into
KubeClient methods (removeContainerEnvVars, addContainerEnvVarsFromSecret)
so postgres-config.ts stays at the 'what' level while KubeClient handles
'how' to patch deployments
Assisted-by: OpenCode
- Extract BACKSTAGE_CR_API_VERSION constant in runtime-config.ts and import it in runtime-deploy.ts, so there is a single place to update when the CRD version bumps - Document the workers:1 assumption on the module-level deployed flag in runtime-deploy.ts to help future readers - Add comment explaining why the operator path uses a computed route URL rather than cluster discovery (deterministic naming convention) Assisted-by: OpenCode
Each runtime spec now calls ensureRuntimeDeployed() in its own beforeAll instead of relying on alphabetical file discovery order (configuration-test/ running before external-database/ and plugin-division-mode-schema/). The call is idempotent — when tests run in order it detects the existing ready deployment and returns immediately. When a spec is run standalone via --grep, it deploys RHDH first. This removes the hidden ordering dependency that the old explicit `dependencies: [SHOWCASE_RUNTIME_DB]` used to make visible. Assisted-by: OpenCode
KubeClient now uses loadFromDefault() so these env vars are no longer consumed by runtime-deploy.ts. Remove them from the header to keep the documentation accurate. Assisted-by: OpenCode
Change config.image from a plain {registry, repository, tag} object
to ImageRef which includes a separator field (':' for tags, '@' for
digests). This fixes digest-pinned images (e.g. repo@sha256:...)
being incorrectly joined with ':' in generateBackstageCR().
The same ImageRef/imageRefToString() is already used for
catalogIndex — this unifies both image references under the same
type and construction logic.
Assisted-by: OpenCode
Centralize the backstage-backend container name as a named constant in kube-client.ts alongside getRhdhDeploymentName(). Replace hardcoded strings in runtime-config.ts, postgres-config.ts, and schema-mode-setup.ts. rhdh-deployment.ts (auth-providers) also uses the same string but is left for a future PR to avoid expanding scope. Assisted-by: OpenCode
Extract the duplicated `tag.startsWith("sha256:") ? "@" : ":"
separator logic into a buildImageRef() helper in helper.ts. Used by
both runtime-config.ts and rhdh-deployment.ts to construct ImageRef
from individual registry/repository/tag env vars.
Assisted-by: OpenCode
Deduplicate the overlapping POSTGRES_* env var key lists across postgres-config.ts and schema-mode-setup.ts into a shared POSTGRES_ENV_KEYS constant. The external-DB function extends it with PGSSLMODE and NODE_EXTRA_CA_CERTS via POSTGRES_CRED_ENV_KEYS. Assisted-by: OpenCode
Move configureSchemaMode() and the default DB user/password constants from runtime-deploy.ts to schema-mode-db.ts where the rest of the schema-mode database utilities live. runtime-deploy.ts imports and calls the function — it no longer owns any schema-mode-specific logic. Assisted-by: OpenCode
updateConfigMapTitle was a thin wrapper around patchAppConfig with an unused _configMapName parameter and a single caller. Inline the title-update logic into config-map.spec.ts and remove the method from KubeClient. Assisted-by: OpenCode
Rename POSTGRES_CRED_ENV_KEYS to postgresCredEnvKeys — private (non-exported) const must be camelCase per the eslint @typescript-eslint/naming-convention rule. Assisted-by: OpenCode
The placeholder postgres-cred secret doesn't contain a POSTGRES_DB key, causing CreateContainerConfigError when prepareForExternalDatabase() adds a secretKeyRef for it. External DB tests don't need POSTGRES_DB — Backstage auto-creates per-plugin databases when it's unset. Schema-mode tests are unaffected; they manage env vars via their own configureSchemaMode() / schema-mode-setup.ts mechanism. Assisted-by: OpenCode
After rebasing onto main, the kube-client.ts monolith was split into kube-client/ directory modules. Add methods and constants that our PR introduced to the new modular structure: - BACKSTAGE_BACKEND_CONTAINER constant - patchAppConfig, jsonPatchDeployment, restartDeploymentWithRetry - removeContainerEnvVars, addContainerEnvVarsFromSecret - waitForBackstageCrd, createConfigMap, deleteNamespaceIfExists, createNamespace - getRhdhDeploymentName: use resolveInstallMethod() for INSTALL_METHOD env var Fix all oxlint violations to comply with strict + pedantic linting: - Replace || with ?? for nullish coalescing - Add explicit nullish/empty checks for strict-boolean-expressions - Fix setTimeout in Promise executors for strict-void-return - Move inline comments to separate lines - Add type annotations and safe type assertions - Swap negated conditions to positive form Assisted-by: OpenCode
…states The checkWaitingContainerState function had inverted logic — it treated transient states like PodInitializing and ContainerCreating as failures while silently ignoring actual failure states like CrashLoopBackOff and ImagePullBackOff. This caused the runtime tests to fail immediately during operator deployment because the init container (install-dynamic-plugins) triggers PodInitializing state, which was incorrectly flagged as a pod failure. Assisted-by: OpenCode
Runtime tests that navigate to the RHDH UI establish WebSocket connections (event-stream). When the test body completes, Playwright's page fixture teardown hangs for up to 10 minutes waiting for the browser context to close because the WebSocket connections are never dropped. Fix: navigate to about:blank at the end of each test that uses the page fixture to close all active connections before Playwright tears down the context. Also add a 10-minute project-level timeout for the SHOWCASE_RUNTIME project — runtime tests restart the RHDH deployment (ConfigMap changes, external DB reconfiguration, schema-mode setup) which takes 60-90s per restart. Assisted-by: OpenCode
daeefd1 to
ac701b4
Compare
|
/test e2e-ocp-operator-nightly |
|
@zdrapela: The following tests failed, say
Full PR test history. Your PR dashboard. DetailsInstructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here. |
036635a
into
redhat-developer:main
|
Reconcile e2e hardening with main's runtime/kube-client refactor and fix worker-scoped rhdhPage fixture crash when testInfo.titlePath is unset. Co-authored-by: Cursor <cursoragent@cursor.com>
…er#4809) * fix(e2e): consolidate SHOWCASE_RUNTIME_DB into SHOWCASE_RUNTIME project Merge the SHOWCASE_RUNTIME_DB Playwright project into SHOWCASE_RUNTIME to simplify runtime test execution. All runtime tests now run sequentially in a single project (workers: 1) with no inter-project dependencies. Key changes: - Deploy runtime with internal PostgreSQL (Helm sub-chart / operator- managed) instead of external Crunchy DB - External DB tests (RDS, Azure) switch to external DB at runtime via prepareForExternalDatabase() which patches ConfigMap + adds env vars - Operator uses separate rhdh-runtime-config secret for RHDH_RUNTIME_URL to avoid POSTGRES_* env var conflicts with internal DB - schema-mode-env.sh and schema-mode-setup.ts support operator-specific service/secret naming (backstage-psql-*) and POSTGRESQL_ADMIN_PASSWORD preservation - SSL connection conditionally applied (skip for internal DB, enable for external DB) - Added waitForRuntimeDeploymentReady() for config-map tests - Added restartWithRetry() for operator reconciliation resilience - Removed old resource files (values-showcase-postgres.yaml, rds-app-config.yaml, rhdh-start-runtime.yaml) Assisted-by: OpenCode * refactor(e2e): extract shared utilities and eliminate duplication - Extract resolveInstallMethod(), base64Encode/Decode to helper.ts - Deduplicate AppConfigYaml interface into runtime-config.ts - Add patchAppConfig(), restartDeploymentWithRetry(), jsonPatchDeployment() to KubeClient — eliminates repeated ConfigMap patch + deployment restart patterns across postgres-config.ts and schema-mode-setup.ts - Export getKubeApiErrorMessage, run(), discoverRouterBase() for reuse - Deduplicate getDeploymentName() via getRhdhDeploymentName() - Handle @sha256: digest refs in parseCatalogIndexImage() - Type BackstageCR return value Assisted-by: OpenCode * fix(e2e): address review findings - Fix parseCatalogIndexImage digest reconstruction: add ImageRef.separator field (':' for tags, '@' for digests) and imageRefToString() helper so downstream consumers reconstruct refs correctly - Fix patchAppConfig no-op detection: compare YAML before/after mutator to skip unnecessary ConfigMap writes and misleading log messages - Fix removeSchemaModePatchedEnvVars: add explicit backstageIdx === -1 guard with warning log (consistent with ensurePostgresCredEnvVars) Assisted-by: OpenCode * refactor(e2e): unify KubeClient with rhdh-deployment.ts patterns - KubeClient constructor: switch from loadFromOptions() with explicit K8S_CLUSTER_URL/K8S_CLUSTER_TOKEN env vars to loadFromDefault() which reads the kubeconfig file (set by oc login / kubectl config). This aligns with how rhdh-deployment.ts (auth-providers) connects. - Extract waitForBackstageCrd() as standalone function in kube-client.ts, used by both runtime-deploy.ts and rhdh-deployment.ts — eliminates duplicated CRD polling loops. - rhdh-deployment.ts: adopt base64Encode() from helper.ts, getKubeApiErrorMessage() from kube-client.ts for safe error logging. Assisted-by: OpenCode * refactor(e2e): standardize on js-yaml, drop yaml package Migrate rhdh-deployment.ts from the `yaml` npm package to `js-yaml` which is already used by kube-client.ts and runtime-config.ts. The two packages have incompatible APIs (`yaml.parse/stringify` vs `yaml.load/dump`) despite being imported under the same alias, creating a copy-paste trap for developers. Changes: - rhdh-deployment.ts: yaml.parse() -> yaml.load(), yaml.stringify() -> yaml.dump() - package.json: remove direct `yaml` dependency from e2e-tests Assisted-by: OpenCode * refactor(e2e): adopt shared helpers and fix createConfigMap typo - Replace 8 inline Buffer.from() base64 calls with base64Encode/Decode from helper.ts (keycloak.ts, api-helper.ts, annotator.spec.ts, scaffolder-relation-processor.spec.ts) - Fix method name typo: createCongifmap -> createConfigMap in KubeClient - Use kubeClient.createConfigMap() in runtime-deploy.ts instead of inline coreV1Api.createNamespacedConfigMap() calls Assisted-by: OpenCode * refactor(e2e): extract shared utilities from runtime-*.ts Move reusable utilities out of runtime-deploy.ts and runtime-config.ts into shared modules: helper.ts: - run() — shell command execution with stdout/stderr capture - discoverRouterBase() — OpenShift cluster router base discovery - ImageRef interface, imageRefToString(), parseCatalogIndexImage() — image reference parsing utilities kube-client.ts (KubeClient class): - createNamespace() — 409-safe namespace creation - deleteNamespaceIfExists() — 404-safe deletion wrapping deleteNamespaceAndWait() runtime-config.ts re-exports ImageRef, imageRefToString, and parseCatalogIndexImage from helper.ts so existing callers are unaffected. runtime-deploy.ts now imports all shared utilities instead of defining them locally, reducing file size by ~100 lines. Assisted-by: OpenCode * fix(e2e): address review findings in runtime test refactor - Guard against undefined host in RDS/Azure DB test loops by skipping individual DB versions when their host env var is not set, instead of crashing with a TypeError in clearDatabase/configurePostgresCredentials - Fix RELEASE_NAME default mismatch in verify-schema-mode.spec.ts ('developer-hub' -> 'rhdh') to match runtime-config.ts and kube-client.ts - Use resolveInstallMethod() in getRhdhDeploymentName() instead of duplicating install method detection logic - Rewrite updateConfigMapTitle as a thin wrapper around patchAppConfig, eliminating ~65 lines of duplicated read-modify-write ConfigMap logic - Extract deployment env var manipulation from postgres-config.ts into KubeClient methods (removeContainerEnvVars, addContainerEnvVarsFromSecret) so postgres-config.ts stays at the 'what' level while KubeClient handles 'how' to patch deployments Assisted-by: OpenCode * fix(e2e): extract API version constant and clarify assumptions - Extract BACKSTAGE_CR_API_VERSION constant in runtime-config.ts and import it in runtime-deploy.ts, so there is a single place to update when the CRD version bumps - Document the workers:1 assumption on the module-level deployed flag in runtime-deploy.ts to help future readers - Add comment explaining why the operator path uses a computed route URL rather than cluster discovery (deterministic naming convention) Assisted-by: OpenCode * fix(e2e): call ensureRuntimeDeployed in every runtime spec Each runtime spec now calls ensureRuntimeDeployed() in its own beforeAll instead of relying on alphabetical file discovery order (configuration-test/ running before external-database/ and plugin-division-mode-schema/). The call is idempotent — when tests run in order it detects the existing ready deployment and returns immediately. When a spec is run standalone via --grep, it deploys RHDH first. This removes the hidden ordering dependency that the old explicit `dependencies: [SHOWCASE_RUNTIME_DB]` used to make visible. Assisted-by: OpenCode * fix(e2e): drop stale K8S_CLUSTER_URL/TOKEN from header comment KubeClient now uses loadFromDefault() so these env vars are no longer consumed by runtime-deploy.ts. Remove them from the header to keep the documentation accurate. Assisted-by: OpenCode * refactor(e2e): use ImageRef for main container image Change config.image from a plain {registry, repository, tag} object to ImageRef which includes a separator field (':' for tags, '@' for digests). This fixes digest-pinned images (e.g. repo@sha256:...) being incorrectly joined with ':' in generateBackstageCR(). The same ImageRef/imageRefToString() is already used for catalogIndex — this unifies both image references under the same type and construction logic. Assisted-by: OpenCode * refactor(e2e): extract BACKSTAGE_BACKEND_CONTAINER constant Centralize the backstage-backend container name as a named constant in kube-client.ts alongside getRhdhDeploymentName(). Replace hardcoded strings in runtime-config.ts, postgres-config.ts, and schema-mode-setup.ts. rhdh-deployment.ts (auth-providers) also uses the same string but is left for a future PR to avoid expanding scope. Assisted-by: OpenCode * refactor(e2e): extract buildImageRef helper for separator detection Extract the duplicated `tag.startsWith("sha256:") ? "@" : ":" separator logic into a buildImageRef() helper in helper.ts. Used by both runtime-config.ts and rhdh-deployment.ts to construct ImageRef from individual registry/repository/tag env vars. Assisted-by: OpenCode * refactor(e2e): extract shared POSTGRES_ENV_KEYS constant Deduplicate the overlapping POSTGRES_* env var key lists across postgres-config.ts and schema-mode-setup.ts into a shared POSTGRES_ENV_KEYS constant. The external-DB function extends it with PGSSLMODE and NODE_EXTRA_CA_CERTS via POSTGRES_CRED_ENV_KEYS. Assisted-by: OpenCode * refactor(e2e): move schema-mode config to schema-mode-db.ts Move configureSchemaMode() and the default DB user/password constants from runtime-deploy.ts to schema-mode-db.ts where the rest of the schema-mode database utilities live. runtime-deploy.ts imports and calls the function — it no longer owns any schema-mode-specific logic. Assisted-by: OpenCode * refactor(e2e): remove updateConfigMapTitle, use patchAppConfig directly updateConfigMapTitle was a thin wrapper around patchAppConfig with an unused _configMapName parameter and a single caller. Inline the title-update logic into config-map.spec.ts and remove the method from KubeClient. Assisted-by: OpenCode * fix(e2e): fix naming convention lint error for private constant Rename POSTGRES_CRED_ENV_KEYS to postgresCredEnvKeys — private (non-exported) const must be camelCase per the eslint @typescript-eslint/naming-convention rule. Assisted-by: OpenCode * fix(e2e): don't inject POSTGRES_DB env var in external DB tests The placeholder postgres-cred secret doesn't contain a POSTGRES_DB key, causing CreateContainerConfigError when prepareForExternalDatabase() adds a secretKeyRef for it. External DB tests don't need POSTGRES_DB — Backstage auto-creates per-plugin databases when it's unset. Schema-mode tests are unaffected; they manage env vars via their own configureSchemaMode() / schema-mode-setup.ts mechanism. Assisted-by: OpenCode * fix(e2e): adapt new KubeClient methods to modular kube-client structure After rebasing onto main, the kube-client.ts monolith was split into kube-client/ directory modules. Add methods and constants that our PR introduced to the new modular structure: - BACKSTAGE_BACKEND_CONTAINER constant - patchAppConfig, jsonPatchDeployment, restartDeploymentWithRetry - removeContainerEnvVars, addContainerEnvVarsFromSecret - waitForBackstageCrd, createConfigMap, deleteNamespaceIfExists, createNamespace - getRhdhDeploymentName: use resolveInstallMethod() for INSTALL_METHOD env var Fix all oxlint violations to comply with strict + pedantic linting: - Replace || with ?? for nullish coalescing - Add explicit nullish/empty checks for strict-boolean-expressions - Fix setTimeout in Promise executors for strict-void-return - Move inline comments to separate lines - Add type annotations and safe type assertions - Swap negated conditions to positive form Assisted-by: OpenCode * fix(e2e): fix inverted pod failure detection for transient container states The checkWaitingContainerState function had inverted logic — it treated transient states like PodInitializing and ContainerCreating as failures while silently ignoring actual failure states like CrashLoopBackOff and ImagePullBackOff. This caused the runtime tests to fail immediately during operator deployment because the init container (install-dynamic-plugins) triggers PodInitializing state, which was incorrectly flagged as a pod failure. Assisted-by: OpenCode * fix(e2e): fix runtime test timeouts caused by WebSocket teardown hang Runtime tests that navigate to the RHDH UI establish WebSocket connections (event-stream). When the test body completes, Playwright's page fixture teardown hangs for up to 10 minutes waiting for the browser context to close because the WebSocket connections are never dropped. Fix: navigate to about:blank at the end of each test that uses the page fixture to close all active connections before Playwright tears down the context. Also add a 10-minute project-level timeout for the SHOWCASE_RUNTIME project — runtime tests restart the RHDH deployment (ConfigMap changes, external DB reconfiguration, schema-mode setup) which takes 60-90s per restart. Assisted-by: OpenCode



Goal
Consolidate
SHOWCASE_RUNTIME_DBintoSHOWCASE_RUNTIME— a single Playwright project that deploys RHDH from TypeScript (runtime-deploy.ts) and runs all runtime tests sequentially (workers: 1).What the runtime tests actually test
All 4 test files are infrastructure-level tests that only need a basic RHDH instance up and serving the UI:
config-map.spec.tsapp.titlechange propagates after deployment restartverify-schema-mode.spec.tspluginDivisionMode: schemaon a restricted (NOCREATEDB) DB userverify-tls-config-with-external-rds.spec.tsverify-tls-config-with-external-azure-db.spec.tsNone exercise catalog, scaffolder, TechDocs, search, Kubernetes, ArgoCD, or any other domain-specific plugin. They only need guest auth and the core backend serving the UI.
Key changes
Single source of truth (
runtime-config.ts)All deployment configuration is generated from a single TypeScript module — no static YAML files. The same config produces both Helm values and Operator Backstage CR, ensuring the two install methods stay in sync.
--setflags for image/cluster. Lightspeed disabled viaglobal.lightspeed.enabled: false. PVC for dynamic-plugins-root persists plugins across restarts.includes: [], plugins: []), and Backstage CR programmatically. Lightspeed disabled viaflavours: [].global.catalogIndex.image.*(matching CI'shelm::get_image_params()); Operator pushes env var withcontainers: ["install-dynamic-plugins"]. When not set, chart/operator defaults apply.Deployment moved from shell to TypeScript
CI scripts (
ocp-nightly.sh,ocp-operator.sh) are reduced to thin wrappers that set env vars and invoke Playwright. All deployment logic lives inruntime-deploy.ts:ensureRuntimeDeployed()— idempotent, called from the first test'sbeforeAllhelm upgrade -i) and Operator (Backstage CR) install methodspostgres-config.ts) reconfigure the running instance at runtimeShared utilities extracted
resolveInstallMethod(),base64Encode/Decode()→helper.tspatchAppConfig(),restartDeploymentWithRetry(),jsonPatchDeployment()→KubeClientAppConfigYamlinterface,ImageReftype,imageRefToString()→runtime-config.tsgetDeploymentName()in schema-mode-setup now delegates togetRhdhDeploymentName()run(),discoverRouterBase()exported fromruntime-deploy.tsFiles deleted (replaced by TypeScript generation)
.ci/pipelines/resources/postgres-db/values-showcase-postgres.yaml→ generated bygenerateHelmValuesYaml().ci/pipelines/resources/postgres-db/rds-app-config.yaml→ generated bygenerateAppConfigYaml().ci/pipelines/resources/rhdh-operator/rhdh-start-runtime.yaml→ generated bygenerateBackstageCR()Jira: https://redhat.atlassian.net/browse/RHIDP-9140
Jira: https://redhat.atlassian.net/browse/RHIDP-9141