[main] fix(lightspeed): fix rag-content folder ownership in init container

[rm3l]

Manual cherrypick of #3042

[sonarqubecloud]

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

[rhdh-qodo-merge]

PR Summary by Qodo

Fix Lightspeed RAG init container permissions for shared rag-content volume

🐞 Bug fix ⚙️ Configuration changes 🕐 20-40 Minutes

AI Description

• Prevent PermissionError on EKS/AKS by ensuring the notebooks directory exists before runtime.
• Copy RAG seed data without preserving restrictive source permissions.
• Tighten chmod behavior to only grant needed rwX on model/vector DB paths.

Diagram

graph TD
R["/rag (image data)"] --> C{{"cp --no-preserve"}} --> V[("Shared /data volume")] --> M["mkdir notebooks"] --> P["chmod a+rwX"] --> S["Llama-stack sidecar"]
S --> W["Write faiss_store.db"]
subgraph Legend
direction LR
_rect["Workload/Container"] ~~~ _dec{{"Operation"}} ~~~ _vol[("Volume")]
end

Loading

High-Level Assessment

The following are alternative approaches to this PR:

1. Set pod fsGroup / runAsGroup for the Lightspeed pod

• ➕ Kubernetes enforces group ownership on mounted volumes; less reliance on chmod
• ➕ More consistent across clusters where SCC/fsGroup defaults differ
• ➖ May not be available/allowed under some restricted PSP/PSS policies
• ➖ Still needs directory creation if the writer expects a subdir to exist

2. Replace inline shell with a small init script (ConfigMap or image entrypoint)

• ➕ More readable and testable than a long sh -c one-liner
• ➕ Easier to add error handling (set -euo pipefail), logging, and idempotency
• ➖ Adds an additional artifact to maintain (script + mounting)
• ➖ Slightly more moving parts in the deployment manifests

Recommendation: The PR’s approach (copy without preserving mode/ownership, explicitly mkdir the notebooks directory, and apply rwX permissions) is appropriate for a backport: it is minimal, targeted at the failure mode on EKS/AKS, and low risk. If the issue recurs across other volumes or clusters, consider adding an explicit fsGroup/runAsGroup to make volume permissions policy-driven instead of relying on chmod.

Files changed (5) +5 / -5

Bug fix (3) +3 / -3

rhdh-flavour-lightspeed-config_v1_configmap.yamlFix RAG init container copy + permissions for notebooks path +1/-1

Fix RAG init container copy + permissions for notebooks path

• Changes the init container command to copy RAG data without preserving mode/ownership, explicitly creates /data/vector_db/notebooks, and applies rwX permissions to embeddings_model and vector_db. This prevents runtime PermissionError when the sidecar writes FAISS state on EKS/AKS-style volume mounts.

bundle/rhdh/manifests/rhdh-flavour-lightspeed-config_v1_configmap.yaml

deployment.yamlUpdate Lightspeed init container RAG seed/permissions command +1/-1

Update Lightspeed init container RAG seed/permissions command

• Aligns the Lightspeed flavour deployment with the fixed init container command: cp --no-preserve, mkdir notebooks, and chmod a+rwX on required paths. Ensures the sidecar can create/write under the shared rag-content volume.

config/profile/rhdh/default-config/flavours/lightspeed/deployment.yaml

install.yamlPropagate Lightspeed RAG init container permission fix into dist install manifest +1/-1

Propagate Lightspeed RAG init container permission fix into dist install manifest

• Carries the same init container command fix into the rendered distribution install YAML so downstream installs get the corrected behavior. Prevents PermissionError on clusters without OpenShift-style default fsGroup behavior.

dist/rhdh/install.yaml

Other (2) +2 / -2

backstage-operator.clusterserviceversion.yamlRefresh CSV createdAt timestamp +1/-1

Refresh CSV createdAt timestamp

• Updates the generated ClusterServiceVersion metadata timestamp. No functional/operator behavior changes are introduced.

bundle/backstage.io/manifests/backstage-operator.clusterserviceversion.yaml

backstage-operator.clusterserviceversion.yamlRefresh RHDH CSV createdAt timestamp +1/-1

Refresh RHDH CSV createdAt timestamp

• Updates the generated ClusterServiceVersion metadata timestamp for the RHDH bundle. No functional/operator behavior changes are introduced.

bundle/rhdh/manifests/backstage-operator.clusterserviceversion.yaml

[rhdh-qodo-merge]

Code Review by Qodo

🐞 Bugs (1) 📘 Rule violations (0) 📜 Skill insights (0)

Context used

✅ Compliance rules (platform): 18 rules

1. Overbroad RAG write perms 🐞 Bug ⛨ Security

Description

The Lightspeed init container recursively applies chmod -R a+rwX to both /data/embeddings_model
and the entire /data/vector_db tree, making the embedding model and prebuilt RAG indices
world-writable by any process in the pod. This enables tampering/corruption of the model and RAG
stores (including product docs), which can change retrieval results and undermine answer integrity.

Code

config/profile/rhdh/default-config/flavours/lightspeed/deployment.yaml[12]

+            - "echo 'Copying RAG data...'; cp -r --no-preserve=mode,ownership /rag/. /data/ && mkdir -p /data/vector_db/notebooks && chmod -R a+rwX /data/embeddings_model /data/vector_db && echo 'Copy complete.'"

Relevance

⭐⭐ Medium

No precedent on chmod/world-writable; least-privilege permission hardening was rejected in PR #2141;
security concerns sometimes partial (#2293).

PR-#2141
PR-#2293

ⓘ Recommendations generated based on similar findings in past PRs

Evidence

The PR changes the initContainer command to chmod -R a+rwX /data/embeddings_model /data/vector_db,
which grants write access broadly. The Lightspeed config shows distinct DB paths for product docs vs
notebooks and uses /rag-content/embeddings_model as the embedding model location, so broad write
permissions cover assets that are expected to be static inputs.

config/profile/rhdh/default-config/flavours/lightspeed/deployment.yaml[6-23]
config/profile/rhdh/default-config/flavours/lightspeed/configmap-files.yaml[163-218]
dist/rhdh/install.yaml[3283-3305]

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
The initContainer command makes `/data/embeddings_model` and all of `/data/vector_db` world-writable (`a+rwX`). This is broader than necessary and allows in-pod tampering of integrity-sensitive RAG assets.

## Issue Context
The configuration distinguishes between a product-docs vector store and a notebooks vector store, implying only the notebooks store needs write access.

## Fix Focus Areas
- config/profile/rhdh/default-config/flavours/lightspeed/deployment.yaml[12-12]
- dist/rhdh/install.yaml[3289-3296]
- bundle/rhdh/manifests/rhdh-flavour-lightspeed-config_v1_configmap.yaml[267-274]

## Suggested change
1. Keep creating the notebooks directory.
2. Restrict permissions by intent, e.g.:
  - Make embeddings model read-only: `chmod -R a+rX /data/embeddings_model`
  - Make product-docs vector DB read-only (or at least not world-writable): `chmod -R a+rX /data/vector_db/rhdh_product_docs`
  - Make only notebooks writable: `chmod -R a+rwX /data/vector_db/notebooks`

(Alternatively, prefer group-based permissions using `fsGroup` + `chmod -R g+rwX` instead of world-writable perms.)

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

---