[RHDHBUGS-2804] Fix Developer Lightspeed Safety Guard

[Jdubrick]

Description

• In the safety path it was assuming you had the model running
• Moves to always deploying this safety guard locally to avoid issues in both Ollama and BYOM scenarios

Which issue(s) does this PR fix or relate to

https://issues.redhat.com/browse/RHDHBUGS-2804

PR acceptance criteria

• Tests updated and passing
• Documentation updated
• Built-in TechDocs updated if needed. Note that TechDocs changes may need to be reviewed by a Product Manager and/or Architect to ensure content accuracy, clarity, and alignment with user needs.

How to test changes / Special notes to the reviewer

[rhdh-qodo-merge]

PR Reviewer Guide 🔍

Here are some key observations to aid the review process:

⏱️ Estimated effort to review: 3 🔵🔵🔵⚪⚪

🔒 No security concerns identified

⚡ Recommended focus areas for review Healthcheck Bug The new healthcheck checks for the inference model with grep -q $$OLLAMA_MODEL, which may fail if the environment variable is unset/empty or not expanded as expected by Compose, and it doesn’t apply the same defaulting logic used in the ollama pull command. Consider defaulting in the healthcheck as well (and ensuring the variable escaping/expansion is correct) so the container doesn’t get stuck as unhealthy. healthcheck: test: ["CMD-SHELL", "ollama list | grep -q $$OLLAMA_MODEL && ollama list | grep -q $${SAFETY_MODEL:-llama-guard3:8b} && [ -f /tmp/ollama-model-ready ]"] interval: 10s timeout: 10s retries: 40 Behavior Change detect_compose_config now returns external-guard when safety-ollama is running without ollama. Validate that all downstream logic (callers/case statements that consume this function output) explicitly handles this new value; otherwise stop behavior could regress for BYO + Safety Guard mode. detect_compose_config() { local runtime=$1 local has_ollama=false local has_safety_ollama=false local has_lightspeed=false # Check for Lightspeed-specific containers if is_container_running "$runtime" "ollama"; then has_ollama=true fi if is_container_running "$runtime" "safety-ollama"; then has_safety_ollama=true fi if is_container_running "$runtime" "lightspeed-core-service"; then has_lightspeed=true fi # If no Lightspeed containers are running, return empty if [[ "$has_lightspeed" == false ]]; then echo "" return fi # Determine provider type: # - ollama container (inference) running → Ollama provider # - safety-ollama container (no ollama) → BYO provider with local safety guard # - neither → BYO provider without safety guard local provider="" if [[ "$has_ollama" == true ]]; then provider="ollama" else provider="external" fi # safety-ollama is the dedicated safety container used in BYO + Safety Guard mode if [[ "$has_safety_ollama" == true && "$has_ollama" == false ]]; then echo "external-guard" return fi # Check safety guard mode by inspecting the mounted run.yaml file local safety_guard_mode safety_guard_mode=$(check_safety_guard_mode "$runtime")

📚 Focus areas based on broader codebase context Hardening The newly added safety-ollama container runs without any explicit container hardening settings (e.g., non-root, dropped Linux capabilities, seccomp profile, privilege escalation disabled). Since this service processes untrusted user content for safety filtering, align its runtime restrictions with the repo’s existing hardened container defaults to reduce blast radius. (Ref 2) # Dedicated Ollama instance for the llama-guard safety model only # Uses a separate container name to avoid collision with the inference Ollama service safety-ollama: image: docker.io/ollama/ollama:0.10.0 container_name: safety-ollama volumes: - safety_ollama_data:/root/.ollama env_file: - path: "./default.env" required: true - path: "./.env" required: false command: > "ollama serve & sleep 5 && ollama pull ${SAFETY_MODEL:-llama-guard3:8b} && touch /tmp/ollama-model-ready && wait" entrypoint: [ "sh", "-c" ] restart: unless-stopped healthcheck: test: ["CMD-SHELL", "ollama list | grep -q $${SAFETY_MODEL:-llama-guard3:8b} && [ -f /tmp/ollama-model-ready ]"] interval: 10s timeout: 10s retries: 40 start_period: 90s Reference reasoning: The existing deployment values define a strict containerSecurityContext baseline (non-root, allowPrivilegeEscalation: false, drop all capabilities, seccompProfile: RuntimeDefault). Applying analogous restrictions in the compose service (where possible) would bring the new container in line with established security posture.

📄 References redhat-developer/rhdh/scripts/rhdh-openshift-setup/values.yaml [238-252] redhat-developer/rhdh/scripts/rhdh-openshift-setup/values.yaml [271-310] redhat-developer/rhdh/scripts/rhdh-openshift-setup/values.yaml [143-157] redhat-developer/rhdh/scripts/rhdh-openshift-setup/values.yaml [79-102] redhat-developer/rhdh-operator/pkg/model/testdata/rhdh-deployment.yaml [61-90] redhat-developer/rhdh-operator/pkg/model/testdata/rhdh-deployment.yaml [91-93] redhat-developer/rhdh-operator/dist/rhdh/install.yaml [2913-2945] redhat-developer/rhdh-operator/dist/rhdh/install.yaml [331-335]

[rhdh-qodo-merge]

PR Type

Enhancement, Bug fix

---

Description

• Deploys safety guard locally via dedicated Ollama container for all provider combinations

• Eliminates manual safety configuration requirements for both Ollama and BYO scenarios

• Adds safety-ollama container to avoid port collisions and simplify setup

• Clarifies environment variable descriptions and improves documentation

• Fixes typos and healthcheck configurations in compose files

---

File Walkthrough

	Relevant files
Documentation	start-lightspeed.sh Clarify automatic safety guard provisioning in menu            developer-lightspeed/scripts/start-lightspeed.sh Updated safety guard menu descriptions to clarify automatic local provisioning Changed messaging from "defaults to llama-guard3:8b with Ollama" to "automatically provisions llama-guard3:8b locally" Simplified user guidance by emphasizing "no additional configuration needed" +3/-3      default.env Improve safety environment variable documentation                default.env Clarified that SAFETY_MODEL must be a llama-guard variant (e.g. llama-guard3:8b) Documented that SAFETY_URL is auto-configured by compose files and should not be set manually Explained that SAFETY_API_KEY is only needed for remote endpoints, not for local safety-ollama +7/-3      README.md Simplify safety guard documentation and remove manual config requirements developer-lightspeed/README.md Updated safety guard configuration section to emphasize automatic provisioning for all provider combinations Removed requirement for manual SAFETY_URL and SAFETY_API_KEY configuration in BYO scenarios Added warning about performance impact of running llama-guard locally on constrained systems Simplified setup combinations table to show safety guard is auto-configured regardless of provider Clarified that safety-ollama is a dedicated container for BYO + safety guard mode +17/-25
Enhancement	stop-lightspeed.sh Add safety-ollama container detection and routing                developer-lightspeed/scripts/stop-lightspeed.sh Added detection for safety-ollama container to identify dedicated safety guard deployments Introduced new provider type "external-guard" for BYO model with local safety guard Enhanced logic to distinguish between Ollama inference, BYO, and BYO with safety guard scenarios +15/-1    compose-with-safety-guard.yaml Introduce dedicated safety-ollama container for BYO provider developer-lightspeed/compose-with-safety-guard.yaml Added new safety-ollama service as dedicated Ollama container for safety model only Configured safety-ollama with separate volume and healthcheck for llama-guard model Added service dependencies for llama-stack and lightspeed-core-service on safety-ollama Set SAFETY_URL environment variable to point to dedicated safety-ollama container Added safety_ollama_data volume for persistent safety model storage +41/-0
Bug fix	compose-with-ollama.yaml Add Ollama URL configuration and fix healthcheck                  developer-lightspeed/compose-with-ollama.yaml Added OLLAMA_URL environment variable set to http://ollama:11434 Fixed healthcheck to use correct service endpoint +1/-0      compose-with-safety-guard-ollama.yaml Fix safety guard healthcheck pattern matching                        developer-lightspeed/compose-with-safety-guard-ollama.yaml Fixed healthcheck regex pattern to match exact safety model version llama-guard3:8b Changed from partial match $${SAFETY_MODEL:-llama-guard3} to full version match +1/-1      run-no-guard.yaml Fix typo in configuration file comment                                      developer-lightspeed/configs/extra-files/run-no-guard.yaml Fixed typo: changed "diasbeld" to "disabled" in comment +1/-1

---

[rhdh-qodo-merge]

PR Code Suggestions ✨

Explore these optional code suggestions:

Category	Suggestion	Impact
High-level	✅ Consider defaulting to a smaller safety model Suggestion Impact: Updated both compose-with-safety-guard.yaml and compose-with-safety-guard-ollama.yaml to default SAFETY_MODEL to llama-guard3:1b instead of llama-guard3:8b (including corresponding healthcheck grep). The second file also adjusted healthcheck syntax while making the same default-model change. code diff: # File: developer-lightspeed/compose-with-safety-guard.yaml @@ -16,12 +16,12 @@ required: false command: > "ollama serve & sleep 5 && - ollama pull ${SAFETY_MODEL:-llama-guard3:8b} && + ollama pull ${SAFETY_MODEL:-llama-guard3:1b} && touch /tmp/ollama-model-ready && wait" entrypoint: [ "sh", "-c" ] restart: unless-stopped healthcheck: - test: ["CMD-SHELL", "ollama list | grep -q $${SAFETY_MODEL:-llama-guard3:8b} && [ -f /tmp/ollama-model-ready ]"] + test: ["CMD-SHELL", "ollama list | grep -q $${SAFETY_MODEL:-llama-guard3:1b} && [ -f /tmp/ollama-model-ready ]"] interval: 10s timeout: 10s retries: 40 # File: developer-lightspeed/compose-with-safety-guard-ollama.yaml @@ -7,10 +7,10 @@ command: > "ollama serve & sleep 5 && ollama pull ${OLLAMA_MODEL:-llama3.2:1b} && - ollama pull ${SAFETY_MODEL:-llama-guard3:8b} && + ollama pull ${SAFETY_MODEL:-llama-guard3:1b} && touch /tmp/ollama-model-ready && wait" healthcheck: - test: ["CMD-SHELL", "ollama list | grep -q $$OLLAMA_MODEL && ollama list | grep -q $${SAFETY_MODEL:-llama-guard3:8b} && [ -f /tmp/ollama-model-ready ]"] + test: ollama list | grep -q $$OLLAMA_MODEL && ollama list | grep -q $${SAFETY_MODEL:-llama-guard3:1b} && [ -f /tmp/ollama-model-ready ] To improve performance and reduce resource usage for users on less powerful machines, consider changing the default safety model from the resource-intensive llama-guard3:8b to the smaller llama-guard3:1b. This provides a better out-of-the-box experience. Examples: developer-lightspeed/compose-with-safety-guard.yaml [19-24] ollama pull ${SAFETY_MODEL:-llama-guard3:8b} && touch /tmp/ollama-model-ready && wait" entrypoint: [ "sh", "-c" ] restart: unless-stopped healthcheck: test: ["CMD-SHELL", "ollama list | grep -q $${SAFETY_MODEL:-llama-guard3:8b} && [ -f /tmp/ollama-model-ready ]"] developer-lightspeed/compose-with-safety-guard-ollama.yaml [10-13] ollama pull ${SAFETY_MODEL:-llama-guard3:8b} && touch /tmp/ollama-model-ready && wait" healthcheck: test: ["CMD-SHELL", "ollama list | grep -q $$OLLAMA_MODEL && ollama list | grep -q $${SAFETY_MODEL:-llama-guard3:8b} && [ -f /tmp/ollama-model-ready ]"] Solution Walkthrough: Before: # developer-lightspeed/compose-with-safety-guard.yaml services: safety-ollama: command: > "ollama serve & sleep 5 && ollama pull ${SAFETY_MODEL:-llama-guard3:8b} && touch /tmp/ollama-model-ready && wait" healthcheck: test: ["CMD-SHELL", "ollama list | grep -q $${SAFETY_MODEL:-llama-guard3:8b} && ..."] # developer-lightspeed/compose-with-safety-guard-ollama.yaml services: ollama: command: > "... && ollama pull ${SAFETY_MODEL:-llama-guard3:8b} && ..." After: # developer-lightspeed/compose-with-safety-guard.yaml services: safety-ollama: command: > "ollama serve & sleep 5 && ollama pull ${SAFETY_MODEL:-llama-guard3:1b} && touch /tmp/ollama-model-ready && wait" healthcheck: test: ["CMD-SHELL", "ollama list | grep -q $${SAFETY_MODEL:-llama-guard3:1b} && ..."] # developer-lightspeed/compose-with-safety-guard-ollama.yaml services: ollama: command: > "... && ollama pull ${SAFETY_MODEL:-llama-guard3:1b} && ..." Suggestion importance[1-10]: 8 __ Why: The suggestion correctly identifies that defaulting to the llama-guard3:8b model can cause significant performance issues, and proposing a smaller default (llama-guard3:1b) is a valid and impactful improvement for user experience.	Medium
Possible issue	Improve detection logic for configuration Refactor the "external-guard" detection logic to only check for the presence of the safety-ollama container, avoiding potential conflicts with other containers named ollama. developer-lightspeed/scripts/stop-lightspeed.sh [98-102] # safety-ollama is the dedicated safety container used in BYO + Safety Guard mode -if [[ "$has_safety_ollama" == true && "$has_ollama" == false ]]; then +if [[ "$has_safety_ollama" == true ]]; then echo "external-guard" return fi Apply / Chat Suggestion importance[1-10]: 7 __ Why: The suggestion correctly identifies a potential bug where a name collision with an unrelated ollama container could break the shutdown script's logic, and the proposed fix makes the detection more robust.	Medium
	Anchor grep pattern for exact match To prevent false positives in the ollama service healthcheck, anchor the grep pattern with ^ to ensure it matches the model name at the beginning of the line. developer-lightspeed/compose-with-safety-guard-ollama.yaml [12-13] healthcheck: - test: ["CMD-SHELL", "ollama list | grep -q $$OLLAMA_MODEL && ollama list | grep -q $${SAFETY_MODEL:-llama-guard3:8b} && [ -f /tmp/ollama-model-ready ]"] + test: ["CMD-SHELL", "ollama list | grep -q \"^$$OLLAMA_MODEL\" && ollama list | grep -q \"^$${SAFETY_MODEL:-llama-guard3:8b}\" && [ -f /tmp/ollama-model-ready ]"] Apply / Chat Suggestion importance[1-10]: 5 __ Why: The suggestion correctly identifies a potential for false positives in the healthcheck grep and proposes a more robust solution by anchoring the pattern, which is a valid improvement.	Low
Update

[Jdubrick]

FYI @karthikjeeyar @maysunfaisal

[karthikjeeyar]

@Jdubrick I tried running ollama + with safety guard path, and I am see the following error.

  File "/opt/homebrew/Cellar/podman-compose/1.4.0/libexec/lib/python3.13/site-packages/podman_compose.py", line 1239, in container_to_args
    raise ValueError("'CMD_SHELL' takes a single string after it")
ValueError: 'CMD_SHELL' takes a single string after it
Error: executing /opt/homebrew/bin/podman-compose -f /Users/kjeeyar/Documents/latest/rhdh-local/compose.yaml -f /Users/kjeeyar/Documents/latest/rhdh-local/developer-lightspeed/compose-with-ollama.yaml -f /Users/kjeeyar/Documents/latest/rhdh-local/developer-lightspeed/compose-with-safety-guard-ollama.yaml up -d: exit status 1

When using Ollama with the safety guard, the script loads both compose-with-ollama.yaml and compose-with-safety-guard-ollama.yaml. Both define healthcheck.test for the ollama service, so podman-compose concatenates list fields, so the merged config becomes:

["CMD-SHELL", "cmd1", "CMD-SHELL", "cmd2"] instead of a single ["CMD-SHELL", "command"]

I dont know how it was working previously, maybe I have updated the podman-compose/podman version recently.

My podman version is

podman-compose version 1.4.0
podman version 5.5.2

[karthikjeeyar]

@Jdubrick You can also add this ollama + safety guard to our test matrix here - https://github.com/redhat-developer/rhdh-local/blob/main/.github/workflows/test.yml#L53-L54 , this way if it really breaks we can catch it early.

[Jdubrick]

@Jdubrick I tried running ollama + with safety guard path, and I am see the following error.

  File "/opt/homebrew/Cellar/podman-compose/1.4.0/libexec/lib/python3.13/site-packages/podman_compose.py", line 1239, in container_to_args
    raise ValueError("'CMD_SHELL' takes a single string after it")
ValueError: 'CMD_SHELL' takes a single string after it
Error: executing /opt/homebrew/bin/podman-compose -f /Users/kjeeyar/Documents/latest/rhdh-local/compose.yaml -f /Users/kjeeyar/Documents/latest/rhdh-local/developer-lightspeed/compose-with-ollama.yaml -f /Users/kjeeyar/Documents/latest/rhdh-local/developer-lightspeed/compose-with-safety-guard-ollama.yaml up -d: exit status 1

When using Ollama with the safety guard, the script loads both compose-with-ollama.yaml and compose-with-safety-guard-ollama.yaml. Both define healthcheck.test for the ollama service, so podman-compose concatenates list fields, so the merged config becomes:

["CMD-SHELL", "cmd1", "CMD-SHELL", "cmd2"] instead of a single ["CMD-SHELL", "command"]

I dont know how it was working previously, maybe I have updated the podman-compose/podman version recently.

My podman version is

podman-compose version 1.4.0
podman version 5.5.2

@karthikjeeyar it may be the podman-compose versus using docker compose under the hood for Podman. I changed from the list syntax to just a string, that might help combat the concatenation you're seeing

[Jdubrick]

Seems like the quay pull for the failing test timed out or something but I don't have permissions to re-run

[karthikjeeyar]

/approve
/lgtm

[karthikjeeyar]

/cc @rm3l