feat(install-dynamic-plugins): port from Python to TypeScript/Node.js - #4574
Merged
openshift-merge-bot[bot] merged 9 commits intoMay 27, 2026
Conversation
Code Review by Qodo
1.
|
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
Review Summary by QodoPort install-dynamic-plugins from Python to TypeScript/Node.js with performance improvements
WalkthroughsDescription• **Complete rewrite**: Replaces Python-based install-dynamic-plugins.py with a TypeScript/Node.js
implementation (18 modules, 105 Jest tests)
• **Performance improvements**: Incorporates parallel OCI downloads, shared image cache, and
streaming SHA verification from the install-dynamic-plugins-fast.py POC
• **New package structure**: scripts/install-dynamic-plugins/ with TypeScript sources, bundled to
single dist/install-dynamic-plugins.cjs (~412 KB)
• **Runtime contract unchanged**: Same dynamic-plugins.yaml input schema, same output format, same
lock-file behavior, same {{inherit}} semantics
• **Resource-conscious concurrency**: Respects cgroup CPU limits with default workers `max(1,
min(floor(cpus/2), 6)), configurable via DYNAMIC_PLUGINS_WORKERS`
• **Memory-efficient**: Streaming tar extraction and SHA verification; typical 10-plugin run uses
20–80 MB peak RSS
• **Security parity**: Path traversal prevention, zip-bomb detection, symlink validation, device
file rejection, SRI integrity verification, registry fallback
• **CI updated**: Replaced pytest with npm run tsc && npm test plus bundle freshness
verification
• **Container image**: Updated Containerfile to copy .cjs bundle instead of .py; wrapper shell
script execs node instead of python
• **Deleted**: install-dynamic-plugins.py (1288 LOC), test_install-dynamic-plugins.py (3065
LOC), pytest.ini
• **Documentation**: Comprehensive README, updated user docs and CI references
Diagramflowchart LR
A["Python Script<br/>install-dynamic-plugins.py"] -->|"Replaced by"| B["TypeScript/Node.js<br/>18 modules + 105 tests"]
B -->|"Bundled to"| C["dist/install-dynamic-plugins.cjs<br/>~412 KB"]
C -->|"Copied in"| D["Container Image<br/>Containerfile"]
E["Parallel OCI<br/>Downloads"] -->|"Incorporated"| B
F["Shared Image<br/>Cache"] -->|"Incorporated"| B
G["Streaming SHA<br/>Verification"] -->|"Incorporated"| B
H["pytest"] -->|"Replaced by"| I["npm test<br/>Jest 105 tests"]
File Changes1. scripts/install-dynamic-plugins/src/index.ts
|
Contributor
|
The container image build workflow finished with status: |
Contributor
|
my browser cannot even load the 11,5k-line file 😅 |
Member
Author
@rostalan No need to review that file, it's the auto-generated esbuild bundle of src/. Just pushed a commit marking it as linguist-generated so GitHub collapses it in the diff. Only src/ and tests/ need review; CI verifies the bundle is up-to-date on every push. |
Contributor
|
The container image build workflow finished with status: |
gustavolira
added a commit
to gustavolira/rhdh
that referenced
this pull request
Apr 13, 2026
… feedback Addresses review feedback on PR redhat-developer#4574: - Extract shared helpers into `src/util.ts`: `fileExists`, `isInside`, `isPlainObject`, `isAllowedEntryType`, `markAsFresh`. Removes 4x duplication across `index.ts`, `catalog-index.ts`, `installer-oci.ts`, `merger.ts`, and `tar-extract.ts`. - Unify `catalog-index.ts` tar filter with `tar-extract.ts`: oversized entries now throw `InstallException` instead of being silently dropped; `OldFile` and `ContiguousFile` are accepted (were previously excluded by mistake). Uses the `isAllowedEntryType` helper. - Extract `markAsFresh(installed, pluginPath)` helper used by both installers to drop stale hash entries after a successful install. - `installer-npm.ts`: use `npm pack --json` instead of parsing the last line of text stdout (warnings on stdout would shift the filename). Also simplify the integrity-check flow — one gate that throws on missing hash, then one verify call (was two overlapping conditionals). - Split `Plugin` → `PluginSpec` (YAML schema) + `Plugin` (internal, with `_level`/`plugin_hash`/`version`). Makes it explicit which fields originate from user YAML vs runtime state. - `lock-file.ts`: add `DYNAMIC_PLUGINS_LOCK_TIMEOUT_MS` (default 10 min) so a stale lock from a `kill -9`'d process no longer wedges the init container forever. New test covers the timeout path. - Drop the broken `lint:check` script — it had `|| true` silencing every lint error and there is no ESLint config in the package. - `README.md`: remove stale reference to non-existent `cli.ts`, document the new lock-timeout env var, mention `util.ts`. Tests: 115 (was 114). Bundle rebuilt (413.4 KB).
Contributor
|
The container image build workflow finished with status: |
Contributor
|
The container image build workflow finished with status: |
gustavolira
added a commit
to gustavolira/rhdh
that referenced
this pull request
Apr 13, 2026
Resolves the 21 issues flagged by SonarQube on PR redhat-developer#4574: Prototype pollution (CodeQL, merger.ts) - `deepMerge` now assigns via `Object.defineProperty` (bypasses the `__proto__` setter on `Object.prototype`) in addition to the existing `FORBIDDEN_KEYS` guard. CodeQL recognizes this pattern. Redundant type assertions - `index.ts:180`: drop `pc as Record<string, unknown>` — use the `isPlainObject` type guard already imported from `util.ts`. - `installer-npm.ts:37`, `installer-oci.ts:35`: replace `(plugin.pluginConfig ?? {}) as Record<string, unknown>` with a typed local variable. - `installer-oci.ts:41,51,71,78`: drop `as string` casts by restructuring the `isAlreadyInstalled` helper with proper `undefined` checks. - `merger.ts:136-140`: replace `.slice(-1)[0] as string` with `.at(-1) ?? ''`. - `merger.ts:215`: `ReadonlyArray<keyof Plugin | string>` collapses to `ReadonlyArray<string>`. Cognitive complexity reductions - `installOciPlugin` (17 → ~10): extract `resolvePullPolicy` and `isAlreadyInstalled` helpers. - `mergeOciPlugin` (20 → ~12): extract `resolveInherit`. - `npmPluginKey` (16 → ~7): extract `tryParseAlias`, `isGitLikeSpec`, `stripRefSuffix`. - `ociPluginKey`: extract `autoDetectPluginPath`. Modern JS / readability (es2015-es2022) - `integrity.ts`: `charCodeAt` → `codePointAt` (es2015). - `oci-key.ts`: use `String.raw` for the regex pieces containing `\s`, `\d`, `\]`, `\\` instead of escaped string literals (es2015). - `oci-key.ts:escape`: `.replace(/.../g, ...)` → `.replaceAll(...)` (es2021). - `plugin-hash.ts`: pass an explicit code-point comparator to `sort` so deterministic-hash behavior is spelled out. `localeCompare` is NOT used — it varies per-locale and would break hash stability. All 115 tests still pass. Bundle rebuilt (415.1 KB).
Contributor
|
The container image build workflow finished with status: |
Member
Author
|
/review |
PR Reviewer Guide 🔍(Review updated until commit ca0c4c6)Warning
Here are some key observations to aid the review process:
|
Contributor
|
The container image build workflow finished with status: |
gustavolira
added a commit
to gustavolira/rhdh
that referenced
this pull request
Apr 13, 2026
… feedback Addresses review feedback on PR redhat-developer#4574: - Extract shared helpers into `src/util.ts`: `fileExists`, `isInside`, `isPlainObject`, `isAllowedEntryType`, `markAsFresh`. Removes 4x duplication across `index.ts`, `catalog-index.ts`, `installer-oci.ts`, `merger.ts`, and `tar-extract.ts`. - Unify `catalog-index.ts` tar filter with `tar-extract.ts`: oversized entries now throw `InstallException` instead of being silently dropped; `OldFile` and `ContiguousFile` are accepted (were previously excluded by mistake). Uses the `isAllowedEntryType` helper. - Extract `markAsFresh(installed, pluginPath)` helper used by both installers to drop stale hash entries after a successful install. - `installer-npm.ts`: use `npm pack --json` instead of parsing the last line of text stdout (warnings on stdout would shift the filename). Also simplify the integrity-check flow — one gate that throws on missing hash, then one verify call (was two overlapping conditionals). - Split `Plugin` → `PluginSpec` (YAML schema) + `Plugin` (internal, with `_level`/`plugin_hash`/`version`). Makes it explicit which fields originate from user YAML vs runtime state. - `lock-file.ts`: add `DYNAMIC_PLUGINS_LOCK_TIMEOUT_MS` (default 10 min) so a stale lock from a `kill -9`'d process no longer wedges the init container forever. New test covers the timeout path. - Drop the broken `lint:check` script — it had `|| true` silencing every lint error and there is no ESLint config in the package. - `README.md`: remove stale reference to non-existent `cli.ts`, document the new lock-timeout env var, mention `util.ts`. Tests: 115 (was 114). Bundle rebuilt (413.4 KB).
gustavolira
force-pushed
the
feat/install-dynamic-plugins-ts
branch
from
April 13, 2026 19:48
b76ee6a to
dd84f75
Compare
gustavolira
added a commit
to gustavolira/rhdh
that referenced
this pull request
Apr 13, 2026
Resolves the 21 issues flagged by SonarQube on PR redhat-developer#4574: Prototype pollution (CodeQL, merger.ts) - `deepMerge` now assigns via `Object.defineProperty` (bypasses the `__proto__` setter on `Object.prototype`) in addition to the existing `FORBIDDEN_KEYS` guard. CodeQL recognizes this pattern. Redundant type assertions - `index.ts:180`: drop `pc as Record<string, unknown>` — use the `isPlainObject` type guard already imported from `util.ts`. - `installer-npm.ts:37`, `installer-oci.ts:35`: replace `(plugin.pluginConfig ?? {}) as Record<string, unknown>` with a typed local variable. - `installer-oci.ts:41,51,71,78`: drop `as string` casts by restructuring the `isAlreadyInstalled` helper with proper `undefined` checks. - `merger.ts:136-140`: replace `.slice(-1)[0] as string` with `.at(-1) ?? ''`. - `merger.ts:215`: `ReadonlyArray<keyof Plugin | string>` collapses to `ReadonlyArray<string>`. Cognitive complexity reductions - `installOciPlugin` (17 → ~10): extract `resolvePullPolicy` and `isAlreadyInstalled` helpers. - `mergeOciPlugin` (20 → ~12): extract `resolveInherit`. - `npmPluginKey` (16 → ~7): extract `tryParseAlias`, `isGitLikeSpec`, `stripRefSuffix`. - `ociPluginKey`: extract `autoDetectPluginPath`. Modern JS / readability (es2015-es2022) - `integrity.ts`: `charCodeAt` → `codePointAt` (es2015). - `oci-key.ts`: use `String.raw` for the regex pieces containing `\s`, `\d`, `\]`, `\\` instead of escaped string literals (es2015). - `oci-key.ts:escape`: `.replace(/.../g, ...)` → `.replaceAll(...)` (es2021). - `plugin-hash.ts`: pass an explicit code-point comparator to `sort` so deterministic-hash behavior is spelled out. `localeCompare` is NOT used — it varies per-locale and would break hash stability. All 115 tests still pass. Bundle rebuilt (415.1 KB).
Contributor
|
The container image build workflow finished with status: |
Member
Author
|
/review |
|
Persistent review updated to latest commit ee984d5 |
…deQL The shared Set.has() lookup wasn't recognised by CodeQL's js/prototype-polluting-function rule as an exhaustive sanitizer, even though Object.defineProperty already bypasses the __proto__ setter. Replace it with the inlined string-literal pattern CodeQL accepts. Closes CodeQL alert redhat-developer#116 on PR redhat-developer#4574 (safeSet at merger.ts:36).
Contributor
|
The container image build workflow finished with status: |
Member
Author
|
/review |
|
Persistent review updated to latest commit ca0c4c6 |
…-developer#4574 - catalog-index.ts: invert the eq===-1 branch so the positive case comes first (Sonar S3923 'Unexpected negated condition'). - index.ts: drop five redundant type assertions on PluginSpec / Plugin / IncludePluginList — TS already narrows correctly thanks to PluginSpec being a structural subset of Plugin and Array of being assignable to ReadonlyArray. - merger.ts: * Split preMergeOciDisabledState into top-level helpers (processOciEntry, recordEntryState, recordRegistryPath, validateAmbiguousPathless, effectiveRegistryDisabled, computeDisabledRegistries) so the orchestrator stays well under the cognitive-complexity ceiling. * Replace the nested ternary in the explicit-paths sort with String#localeCompare. * Replace 'bucket && bucket.size === 1' with optional-chain 'bucket?.size === 1'. * Drop a stray NUL byte in the entry-key template that crept in via an early Edit and was making the file impossible to diff in a few tools. 155 vitest cases still pass; bundle rebuilt.
Contributor
|
The container image build workflow finished with status: |
…dir names Qodo's persistent review flagged that the explicit 'name=<ref>' form of EXTRA_CATALOG_INDEX_IMAGES does not validate 'name', so an operator could accidentally (or maliciously) extract an extra catalog index outside of '<entitiesDir>/extra/' by passing names containing '..' or path separators. The auto-derived form is already safe because imageRefToSubdirectory strips '/', ':', and '@'. Add isSafeSubdirectoryName() that rejects empty names, '.', '..', and any name containing '/' or '\\'. Parsing logs a warning and skips the entry instead of crashing the installer, matching the existing 'empty image reference' handling. 6 new vitest cases cover the rejected forms and document that the check is character-based (so URL-encoded separators like '%2F' are accepted verbatim rather than decoded). Bundle rebuilt (225.6 kB).
Contributor
|
The container image build workflow finished with status: |
Targeted cleanups identified in the in-conversation review. No behaviour
changes outside of the two defense-in-depth guards listed below.
Correctness/security:
- installer-npm.ts: validate that the filename emitted by 'npm pack
--json' is flat (no '/', no '\\', no leading '..') before joining it
to 'destination'. A registry returning '../evil.tgz' would otherwise
let extraction escape.
- installer-oci.ts: replace 'pkg.split("!")' with an indexOf-based
splitOciPackage helper so plugin paths containing a literal '!' are no
longer silently truncated. Applied to both the install path and the
Always-policy digest comparison.
- catalog-index.ts: re-validate 'subdirectory' inside
extractExtraCatalogIndex itself instead of trusting all callers to go
through parseExtraCatalogIndexImages. Five new vitest cases cover the
rejected forms.
- index.ts: drop the existsSyncSafe wrapper and use Node's existsSync
directly (the wrapper was a literal reimplementation that also
conflated 'doesn't exist' with 'no read permission').
- image-cache.ts: surface a typed InstallException with the image ref
when 'io.backstage.dynamic-packages' is unparseable, instead of
letting JSON.parse crash the install.
Maintainability:
- types.ts: new effectivePullPolicy() helper centralises the ':latest!'
fallback that was duplicated between installer-oci.ts and
definitelyNoOp in index.ts.
- merger.ts: copyPluginFields now uses a for-of loop with safeSet
(same prototype-pollution guard as deepMerge) instead of building an
intermediate object via Object.fromEntries. effectiveRegistryDisabled
drops the 'as string' cast via destructuring + undefined check.
IncludePluginList downgraded from exported to internal.
- tar-extract.ts: comment the POSIX assumption around the
pluginPathBoundary check; the trailing slash is unambiguous now.
- index.ts: destructure process.argv to remove the 'as string' cast.
- installer-npm.ts: extract isNpmPackJsonEntry() type guard so the
install path stays linear.
- concurrency.ts: replace the magic-number worker caps (6/3) with
MAX_OCI_WORKERS / MAX_NPM_WORKERS named constants.
- catalog-index.ts: simplify isSafeSubdirectoryName (the second clause
was unreachable once '/' and '\\' were rejected upfront).
166 vitest cases (was 161; +5 new defense-in-depth tests). Bundle
rebuilt (226.0 kB).
Contributor
|
The container image build workflow finished with status: |
- Generic 'safeSet<T extends object>' removes the awkward 'dst as unknown as Record<string, unknown>' double-cast in copyPluginFields. The Object.defineProperty signature already accepts any object, so the constraint widening is enough. - Drop the redundant 'as unknown' on parseYaml's return in mergePluginsFromFile — yaml v2's parse() is already typed unknown. No behaviour changes. 166 vitest cases still pass.
Contributor
|
The container image build workflow finished with status: |
The 'Upload test results to Codecov' step at the monorepo level looks at $RUNNER_TEMP/test-results/ for JUnit reports. Because this PR doesn't touch any yarn workspace, 'yarn run test --affected' skips all workspace tests and the directory ends up empty — codecov[bot] then posts a 'JUnit XML file not found' warning on the PR. Run vitest with the junit reporter so it writes its own report into $RUNNER_TEMP/test-results/install-dynamic-plugins.junit.xml; the existing test-results-action then finds it (alongside any workspace reports) and the warning stops firing. No new upload step needed — test-results-action already searches that directory. Verified locally: 'JUNIT report written to /tmp/.../junit.xml' (166 testcases serialized).
Contributor
|
The container image build workflow finished with status: |
…pload The previous attempt at producing a JUnit report didn't take effect because the step ordering put it AFTER the 'Upload test results to Codecov' step. The Codecov log confirmed this: 14:33:08 error - No JUnit XML reports found. 14:33:16 JUNIT report written to .../install-dynamic-plugins.junit.xml Move the install-dynamic-plugins test step before the monorepo 'Run tests' so its JUnit report lands in $RUNNER_TEMP/test-results/ in time for the existing upload step to pick it up. The bundle freshness check stays where it was (after the uploads) so it doesn't delay the upload-bearing path further.
|
jonkoops
approved these changes
May 27, 2026
openshift-merge-bot
Bot
merged commit May 27, 2026
105732d
into
redhat-developer:main
22 checks passed
6 tasks
gustavolira
added a commit
to redhat-developer/rhdh-plugins
that referenced
this pull request
Jun 5, 2026
…install-dynamic-plugins (#3246) * feat(install-dynamic-plugins): import package from redhat-developer/rhdh Migrates scripts/install-dynamic-plugins/ from redhat-developer/rhdh#4574 into this repo as @red-hat-developer-hub/install-dynamic-plugins so it can be published to npm and consumed by the RHDH init-container without curl-by-SHA. Runtime contract (CLI args, env vars, plugin-hash format, on-disk layout, tar/OCI security guards) preserved verbatim. Build remains a single self-contained .cjs via esbuild. Tests migrated from vitest to jest to align with the repo's backstage-cli pipeline (14 suites / 166 tests pass). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(install-dynamic-plugins): address self-review feedback - Add bin/install-dynamic-plugins shim and have package.json bin point at it (matches the convention used by extensions-cli, translations-cli, and rhdh-repo-tools). Split src/cli.ts as the esbuild entry so the bundle no longer needs the require.main guard or a shebang banner. - Stop committing dist/install-dynamic-plugins.cjs; the release pipeline rebuilds via the customBuild path, and a new prepack script makes yarn npm publish self-healing for local runs. - Drop the .js suffix from relative imports across src/ so the package matches the rest of the repo and the jest moduleNameMapper workaround is no longer needed. - Consolidate the tsconfigs: the inner package extends the workspace tsconfig and only declares what differs. - Add why-it's-intentional comments to the two eslint-disable lines (PullPolicy const+type pair, tar.x filter inside a sequential loop). - README now leads with the npm/npx usage path; the RHDH init-container section is below. tar/yaml stay in dependencies (not devDependencies as the review suggested) — @backstage/no-undeclared-imports flagged the source imports, and the repo convention treats bundling as opaque. 166/166 tests pass, tsc/lint/prettier clean, bin shim and bundle both exit 0 on the empty-config smoke run. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(install-dynamic-plugins): unblock CI api-reports step The CI step "check api reports and generate API reference" runs `backstage-repo-tools api-reports --ci` before the build, and that tool requires the bin file to introspect the CLI. The previous shim did a plain `require('../dist/install-dynamic-plugins.cjs')`, which failed under CI because dist/ is no longer committed and the build hasn't run yet. - Switch the bin shim to the local-vs-installed pattern used by every other CLI in the repo (extensions-cli, translations-cli, rhdh-repo-tools): when `src/` exists (monorepo), load TS directly via `@backstage/cli/config/nodeTransform`; otherwise require the built bundle (npm-installed scenario). - Add `--help` / `-h` handling to main() so the api-reports tool can introspect the CLI usage without creating a stray `--help/` directory. - Commit the generated `cli-report.md`. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * chore(install-dynamic-plugins): address SonarCloud findings - Use String.raw for strings containing backslash literals so the source reads with one '\' instead of '\\\\' (catalog-index.ts log message, extra-catalog-index.test.ts subdirectory fixtures, skopeo.test.ts shell escape). - Switch the OCI regex builder to a joined string array — eliminates the nested template literals SonarCloud was flagging on oci-key.ts (and reads much better). - Object.prototype.hasOwnProperty.call -> Object.hasOwn in merger.test.ts (ES2022, available since Node 16.9). - String#replace(/'/g, ...) -> String#replaceAll("'", ...) in skopeo.test.ts (ES2021). - Hoist test helpers (stageLayer, fakeImageCache) out of their describe blocks so they aren't re-defined on every test. - Drop the redundant parseMaxEntrySize(undefined) call in types.test.ts — the parameter already defaults to process.env.MAX_ENTRY_SIZE. 166/166 tests still pass. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * refactor(install-dynamic-plugins): parse argv with cleye Switches the hand-rolled `process.argv` + USAGE-string handling in main() to `cleye` — the same parser every `@backstage/cli-module-*` package uses (already in our transitive deps). Aligns with the Backstage CLI convention requested during PR review. Existing surface preserved: - positional `<dynamic-plugins-root>` (required, exit 1 if absent) - `--help` / `-h` prints usage and exits 0 - normal run still exits with the installer's status code Bundle grew from 226 kB -> 267 kB (cleye + type-flag minified). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * refactor(install-dynamic-plugins): adopt cli-module convention while keeping the bundled bin Aligns with the @backstage/cli-module-* convention without losing the self-contained bundled artifact: - Rename to @red-hat-developer-hub/cli-module-install-dynamic-plugins, backstage.role: cli-module. - src/installer.ts holds the install pipeline and main() (formerly src/index.ts). - New src/index.ts default-exports `createCliModule(...)` registering an `install` command whose loader is src/command.ts. Exposes the package through backstage-cli discovery — `backstage-cli install <dir>` works when the package is a dependency. - src/cli.ts (the esbuild entry) keeps invoking installer.main() directly, so the bundled .cjs stays self-contained: no @backstage/cli-node and no keytar gymnastics in the bin path. - Build is dual now — `backstage-cli package build && node esbuild.config.mjs`. backstage-cli emits dist/index.cjs.js (the cli-module export) and the unbundled supporting modules; esbuild emits dist/install-dynamic-plugins.cjs (the standalone bin). Both are published. - bin shim's installed branch now requires the bundled .cjs explicitly rather than going through `main` — that keeps direct/npx/init-container invocations at ~60 ms cold start instead of paying the cli-module dispatch cost. - main() now takes optional `args` and `programName` so the cli-module loader can pass the command's argv slice and have `--help` print the real invocation (`install-dynamic-plugins install …`). - @backstage/cli-node added as a runtime dependency. It is only loaded by the cli-module discovery path; the bundled bin never imports it. 166/166 tests pass; tsc/lint/prettier/api-reports clean. Bundle size unchanged at ~267 KB. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * refactor(install-dynamic-plugins): drop custom esbuild bundle, ship unbundled cli-module Replaces the dual build (esbuild bundle + backstage-cli) with plain `backstage-cli package build` — the standard Backstage cli-module pattern. The bundle and the keytar gymnastics only existed to satisfy RHDH's init-container `COPY` of a single self-contained `.cjs`; that consumption model is moving to `npm install` (redhat-developer/rhdh#4908), so the single-file requirement is going away. What's left in the package: - `src/installer.ts`: install pipeline + `main(args, programName)`. - `src/index.ts`: `createCliModule(...)` default export, registers the `install` command. Discovered by `backstage-cli` when this package is a dependency of a host project. - `src/command.ts`: thin loader that calls `installer.main`. - `bin/install-dynamic-plugins`: fast-path shim that loads `dist/installer.cjs.js` directly and runs `main(process.argv.slice(2))`, bypassing `@backstage/cli-node`'s `runCliModule` dispatch — saves ~80 ms of cold start for direct/`npx`/init-container invocations. The cli-module discovery path still goes through `runCliModule` and pays the dispatch cost where it belongs. Removed: - `esbuild.config.mjs` (the custom bundle config) - `src/cli.ts` (the esbuild entry) - `dist/install-dynamic-plugins.cjs` from `files` (no longer produced) - `esbuild` devDependency 166/166 tests pass; tsc/lint/prettier/api-reports clean. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * docs(install-dynamic-plugins): drop stale references to the esbuild bundle The README and the createCliModule docstring still referenced the self-contained bundle that the package no longer ships: - `npm run build` description and committed-bundle CI-check section. - `node install-dynamic-plugins.cjs "$1"` wrapper line in "How RHDH consumes it" — replaced with a pointer to redhat-developer/rhdh#4908. - `src/index.ts` describing the bin path as the bundle. - Source layout was missing `index.ts` / `command.ts` / `installer.ts`. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(install-dynamic-plugins): address PR review — align with upstream cli-module contract Applies the four findings from @schultzp2020's review: - package.json reshaped to match the upstream `@backstage/cli-module-*` contract: `main`/`types` point at `src/index.ts` for local dev, `publishConfig` overrides with `dist/index.cjs.js`/`dist/index.d.ts`, `prepack` and `postpack` wire up `backstage-cli package prepack/postpack` so the published artifact has the correct entry points, and `files` is `["bin", "dist"]` (the old `dist/**/*.js` glob excluded `.d.ts`). - bin wrapper now uses `@backstage/cli-node/config/nodeTransform.cjs` (the cli-node variant) instead of `@backstage/cli/config/nodeTransform.cjs`. `@backstage/cli` is only a devDependency — the previous import would have broken `npx`/installed-package invocations. - bin wrapper now goes through `runCliModule(...)` like every other `@backstage/cli-module-*` package. The earlier fast path bypassed it to save ~80 ms of cold start, but per @schultzp2020 that cost is one-time per init-container run (not per-plugin), and going through the standard dispatch gives us `--version`/`--help` and future runCliModule improvements for free. - installer.ts: added an `isPlainObject` guard for the parsed main config, mirroring the existing guard on include files. A YAML scalar or array in `dynamic-plugins.yaml` now throws a clear `InstallException` instead of a confusing downstream `TypeError`. Also drops the stale `install-dynamic-plugins.sh` wrapper from the package — it pointed at the long-removed esbuild bundle and was no longer listed in `files`. 166/166 tests pass; tsc/lint/prettier/api-reports clean. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
5 tasks
gustavolira
added a commit
to gustavolira/rhdh
that referenced
this pull request
Jun 11, 2026
… era The dynamic-plugins installer was migrated to TypeScript in redhat-developer#4574 and the vendored TypeScript copy was removed earlier in this PR, so the pytest test infrastructure no longer has anything live to test. - Deletes python/requirements-dev.in and python/requirements-dev.txt (pytest, pytest-cov, pytest-mock — and their transitive deps). - Drops the .github/workflows/pr.yaml "Install Python dependencies" step that installed all three requirements files; no subsequent step in the PR workflow uses Python. - Cleans the stale comment in codecov.yml referencing the removed install-dynamic-plugins pytest flag. python/requirements.txt and python/requirements-build.txt stay — they are consumed by the TechDocs venv build in the Containerfile (line 237) for mkdocs / mkdocs-techdocs-core / plantuml-markdown, which is unrelated. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
openshift-merge-bot Bot
pushed a commit
that referenced
this pull request
Jun 16, 2026
…arn workspace (#4908) * [DRAFT] chore(install-dynamic-plugins): consume installer from npm Replaces the COPY of scripts/install-dynamic-plugins/{install-dynamic-plugins.cjs, install-dynamic-plugins.sh} with an `npm install` of @red-hat-developer-hub/cli-module-install-dynamic-plugins (built and published out of redhat-developer/rhdh-plugins). This unblocks the cli-module structure on the rhdh-plugins side — it lets that package use the standard `backstage-cli package build` (unbundled, multi-file dist) instead of a custom esbuild bundle with a keytar stub. See the conversation context: redhat-developer/rhdh-plugins#3254 Backward compatibility is preserved by writing a tiny `/opt/app-root/src/install-dynamic-plugins.sh` shim that delegates to the npm-installed bin, so the Helm chart and Operator init-container spec continue to invoke `./install-dynamic-plugins.sh /dynamic-plugins-root` unchanged. DRAFT — DO NOT MERGE: blocked on redhat-developer/rhdh-plugins#3254 (or the unbundled successor) being merged and published to npm. Opened for review of the consumption pattern and to back the cold-start benchmark posted in Slack. Trade-off summary (cold-start benchmark on empty config): - Current (bundled .cjs, 231 KB single file): ~89 ms warm cache (median) - Proposed (npm install, 25 MB node_modules): ~180 ms warm cache (median) The ~90 ms gap is the module-resolution overhead of unbundled Node — paid once per pod start. Image build time also gets +`npm install` of ~25 MB (one extra layer), offset by deleting ~7000 lines of vendored installer script from this repo in a follow-up. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fixup: address PR review on the install-dynamic-plugins Containerfile step - Install into /opt/dynamic-plugins-installer (its own dir, no package.json) instead of /opt/app-root/src so npm cannot honor the yarn workspace and perturb the production tree that `yarn workspaces focus` built at line 208. - Delegate the shim to `node_modules/.bin/install-dynamic-plugins` (the symlink npm creates from the package's bin field) instead of reaching into the package's internal layout. - Add `--no-save --omit=dev` so npm doesn't write a package-lock.json into the installer dir and doesn't fetch devDependencies. - Pin the installer to an exact version (0.1.0) so image builds are reproducible. - Add a build-time smoke check (`install-dynamic-plugins --help`) so a missing or renamed CLI entrypoint fails the image build instead of the init container at pod start. The hermetic-build concern (npm reaching the public registry when this Containerfile runs under Konflux with networking disabled) is acknowledged separately in the PR description — it's the real gating work and is not addressed by this commit. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fixup: drop synthetic `install` subcommand from the shim The unbundled cli-module variant ships a fast-path bin that calls the installer directly (bypassing @backstage/cli-node's runCliModule dispatch), so the published binary takes the dynamic-plugins-root as a positional without a subcommand prefix — matching the original CLI surface. Verified locally: $ /opt/dynamic-plugins-installer/node_modules/.bin/install-dynamic-plugins /dynamic-plugins-root exits 0 on an empty config. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * chore(install-dynamic-plugins): consume installer through dynamic-plugins yarn workspace Pivots #4908 from `RUN npm install …` to a plain yarn dependency in dynamic-plugins/package.json. The existing `yarn install --immutable` at line 151 of this Containerfile pulls the installer as part of the same yarn run that already brings in the rest of the dynamic-plugins deps, so the bin lands at /opt/app-root/src/dynamic-plugins/node_modules/.bin/install-dynamic-plugins and the final stage just writes a shim pointing there. Why yarn and not npm: scripts/local-hermeto-build.sh:213 already prefetches yarn deps for ./dynamic-plugins via cachi2/hermeto. No infra change is needed in this repo or in the midstream Konflux pipeline — the hermetic build "just works". The earlier npm-install approach required wiring npm into hermeto's fetch-deps, which is real work spanning two repos. Validated locally: yarn install of the unbundled cli-module variant (via a tarball of the fast-path build) produces .bin/install-dynamic-plugins and the smoke invocation on an empty dynamic-plugins.yaml exits 0. dynamic-plugins/yarn.lock is intentionally NOT regenerated in this commit — that has to happen against the published @red-hat-developer-hub/cli-module-install-dynamic-plugins@0.1.0 once redhat-developer/rhdh-plugins ships it. Marking the PR as Draft until then. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fixup: shim uses install subcommand + forwards all positional args The rhdh-plugins side reverted the bin shim to the standard `runCliModule` dispatch (redhat-developer/rhdh-plugins#3246 review), so the installed bin now expects an `install` subcommand. Updating the wrapper to match and to forward `"$@"` instead of `"$1"` so any extra positional argument the Helm chart or Operator passes is preserved. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * chore: pin cli-module-install-dynamic-plugins to 0.2.0 and regen yarn.lock redhat-developer/rhdh-plugins#3246 merged and the changesets release published @red-hat-developer-hub/cli-module-install-dynamic-plugins as 0.2.0 (the changesets minor bump rolled past 0.1.0 because of the existing `0.0.0` workspace version). Updates the dynamic-plugins/package.json pin to 0.2.0 and regenerates dynamic-plugins/yarn.lock so `yarn install --immutable` (line 151 of the Containerfile) and the hermeto yarn prefetch at scripts/local-hermeto-build.sh:213 both resolve the package. The Build Image GitHub Actions job should now go green. Verified locally: yarn install resolves the package, the bin symlink lands at dynamic-plugins/node_modules/.bin/install-dynamic-plugins, and invoking it with `install <dir>` exits 0 on an empty config (going through the standard @backstage/cli-node runCliModule dispatch like the final Containerfile shim does). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * chore: drop vendored scripts/install-dynamic-plugins/ now that it's on npm The dynamic-plugins installer is consumed from @red-hat-developer-hub/cli-module-install-dynamic-plugins via the yarn dependency declared in dynamic-plugins/package.json, so the vendored copy under scripts/install-dynamic-plugins/ is no longer used by the Containerfile and can be removed. Deletes: - scripts/install-dynamic-plugins/ (46 files, ~7000 lines) Cleans up the references that pointed at it: - .gitattributes: drop linguist-generated marker for the bundled .cjs - .dockerignore: drop the dist/ exceptions that kept the bundle in the build context - codecov.yml: drop the install-dynamic-plugins flag (used to track Python coverage from the pre-TS era) - .github/workflows/pr.yaml: drop the vitest + bundle-up-to-date checks - .github/workflows/coverage-baseline.yml: drop the pytest baseline - docs/dynamic-plugins/installing-plugins.md: link to the rhdh-plugins source instead of the vendored path - docs/coverage/e2e-rhdh.md: drop the vitest row from the coverage table - .rulesync/rules/ci-e2e-testing.md + the derived files under .claude/, .cursor/, .opencode/: update the installer link Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix: move installer dep to root package.json so the bin survives stage 3 The Containerfile's build stage at line 175 wipes everything under dynamic-plugins/ except dist/, which deleted dynamic-plugins/node_modules/.bin/ along with everything else. The final stage then failed at runtime with exit code 127 because the bin wasn't there: /bin/sh: line 1: /opt/app-root/src/dynamic-plugins/node_modules/.bin/install-dynamic-plugins: No such file or directory Moves @red-hat-developer-hub/cli-module-install-dynamic-plugins from dynamic-plugins/package.json to the repo root package.json. The bin then survives the `yarn workspaces focus --all --production` at line 208 and lands at /opt/app-root/src/node_modules/.bin/ where the final stage's shim can reach it. Hermeto already prefetches yarn deps for the repo root (scripts/local-hermeto-build.sh:213, `{"type": "yarn", "path": "."}`), so no infra change. Also corrects the shim path: the published package uses the string form of `bin` ("bin/install-dynamic-plugins"), and yarn names the symlink after the package's base name (after the scope), so the actual bin lands at node_modules/.bin/cli-module-install-dynamic-plugins, not node_modules/.bin/install-dynamic-plugins. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * chore: drop pytest residue left by the Python install-dynamic-plugins era The dynamic-plugins installer was migrated to TypeScript in #4574 and the vendored TypeScript copy was removed earlier in this PR, so the pytest test infrastructure no longer has anything live to test. - Deletes python/requirements-dev.in and python/requirements-dev.txt (pytest, pytest-cov, pytest-mock — and their transitive deps). - Drops the .github/workflows/pr.yaml "Install Python dependencies" step that installed all three requirements files; no subsequent step in the PR workflow uses Python. - Cleans the stale comment in codecov.yml referencing the removed install-dynamic-plugins pytest flag. python/requirements.txt and python/requirements-build.txt stay — they are consumed by the TechDocs venv build in the Containerfile (line 237) for mkdocs / mkdocs-techdocs-core / plantuml-markdown, which is unrelated. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * chore: stop tracking and ignore the Claude Code session lock file `.claude/scheduled_tasks.lock` is a Claude Code runtime artifact (per-session pid lock); it got accidentally included by `git add -A` in the previous commit. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix: address PR review — smoke check `install`, stop pointing at line numbers, drop orphaned comments - Build-time smoke check now invokes `install --help`, not just `--help`. A top-level `--help` only proves the bin resolves; the shim hardcodes the `install` subcommand, so if a future release of the package renames/removes it, the failure should land at image build time, not at pod start. cleye prints the subcommand help and exits 0 before validating the required `<dynamic-plugins-root>` positional, so this is a strict upgrade with no behavioural change for the current published package. - Replace hardcoded line-number references in the Containerfile comment (already drifted: line 175 → actual 173) with the step banners `=== YARN WORKSPACES FOCUS ===` and `=== DELETE DYNAMIC PLUGINS/* ===`, which survive edits to the surrounding RUN steps. Same treatment for scripts/local-hermeto-build.sh: point at the `{"type": "yarn", "path": "."}` fetch-deps entry instead of a line number. - Drop the orphaned `.dockerignore` comment that explained the now-removed `!scripts/install-dynamic-plugins/dist` re-include lines and was reading as if it described `**/node_modules`. - Drop the orphaned `.gitattributes` section header `# Generated bundles — collapsed in GitHub diffs, …` left behind when the `dist/install-dynamic-plugins.cjs linguist-generated=true` entry was removed. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix: move installer dep to packages/backend to satisfy monorepo:check The sherif rule in `yarn run monorepo:check` rejects `dependencies` on the private root package.json — the rationale being that dependencies vs devDependencies is a no-op for a private package and creates confusion. CI fails on that check today. Moves `@red-hat-developer-hub/cli-module-install-dynamic-plugins` from the root package.json to `packages/backend/package.json`, keeping the load-bearing behaviour: backend is `private: true` and gets included by `yarn workspaces focus --all --production`, so the bin still hoists to `/opt/app-root/src/node_modules/.bin/cli-module-install-dynamic-plugins` where the Containerfile shim picks it up. Semantically this is the right home anyway — the backend is the runtime that the init-container runs alongside. Containerfile comment updated to point at the new declaration site. Verified locally: - yarn run monorepo:check → No issues found - node_modules/.bin/cli-module-install-dynamic-plugins install --help exits 0 Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
6 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Replaces the Python-based
install-dynamic-plugins.pyinit-container script with a TypeScript/Node.js implementation, incorporating all improvements from theinstall-dynamic-plugins-fast.pyPOC (#4523): parallel OCI downloads, shared image cache, streaming SHA verification.Motivation: The Python script was a longstanding duplication target for export overlays (see rhdh-plugin-export-overlays#2231). A Node.js implementation removes the Python dependency from the init-container runtime, enables reuse across RHDH core and overlays, and adopts the faster parallel architecture natively.
What changed
scripts/install-dynamic-plugins/— 18 TypeScript modules + 9 Jest test files (105 tests)dist/install-dynamic-plugins.cjs(~412 KB), committed and verified fresh in CI (same pattern as.yarn/releases/yarn-*.cjs).cjsbundle instead of.py; wrapper shell script execsnodeinstead ofpythonnpm run tsc && npm test+ bundle freshness check (replacespytest)install-dynamic-plugins.py(1288 LOC),test_install-dynamic-plugins.py(3065 LOC),pytest.iniKey design decisions
Runtime contract is unchanged
Same
dynamic-plugins.yamlinput schema, sameapp-config.dynamic-plugins.yamloutput, samedynamic-plugin-config.hash/dynamic-plugin-image.hashfiles, same lock-file behaviour, same{{inherit}}semantics and OCI path auto-detection.Resource-conscious concurrency
availableParallelism()respects cgroup CPU limits (init containers often get 0.5 CPU)max(1, min(floor(cpus/2), 6))— cap avoids exhausting registry/networkDYNAMIC_PLUGINS_WORKERS=<n>Memory: streaming everywhere
node-tarstreams extraction — no full-archive read into RAMnode:cryptopipeline for SHA integrity — chunks through the hashSecurity parity with Python
.., absolute)tar-extract.tsMAX_ENTRY_SIZE)tar-extract.ts,catalog-index.tstar-extract.tstar-extract.tspackage/prefix enforced for NPM tarballstar-extract.tssha256/sha384/sha512)integrity.tsregistry.access.redhat.com/rhdh→quay.io/rhdhimage-resolver.tsTest plan
npm run tscpasses (strict mode,noUncheckedIndexedAccess)npm test— 105 Jest tests pass (npm-key, oci-key, integrity, tar-extract, merger, concurrency, lock-file, image-resolver, plugin-hash)npm run buildproduces freshdist/install-dynamic-plugins.cjs.cjs(CI will verify)fast.pybaseline (~2:42 for full catalog)Compatibility
skopeois still installed for OCI inspectioninstall-dynamic-plugins.shwrapper contract unchanged (./install-dynamic-plugins.sh /dynamic-plugins-root)Related