Developer Lightspeed in RHDH local

[karthikjeeyar]

Description

This PR adds support for integrating Developer Lightspeed into the rhdh-local environment, providing a generative AI-powered virtual assistant within the RHDH console.
Key Additions and Changes:

• Conditional Lightspeed App Config Support:
  The startup script wait-for-plugins-and-start.sh now conditionally includes app-config.lightspeed.local.yaml as an extra configuration file, allowing users to easily enable Developer Lightspeed features for local development.
• Documentation:
  Added a comprehensive guide developer-lightspeed-guide.md detailing how to set up and use Developer Lightspeed in rhdh-local. The guide covers prerequisites, configuration steps, environment variable setup, and verification instructions
• Example Config Files:
  Provided example override files for both dynamic plugins and app config to simplify the onboarding process for Lightspeed users.
• Environment Variable Integration:
  The setup leverages environment variables for secure and flexible configuration of Lightspeed server details and API keys.
• Manual and Automated Steps:
  The guide and scripts support both quick-start (copy-paste) and manual merge approaches for users with existing customizations.
• How it Works:
  • Users can enable Lightspeed by copying the example config files and setting the required environment variables.
  • The system will automatically detect and use the Lightspeed configuration if present, without disrupting existing user or legacy configs.
  • The documentation provides step-by-step instructions for setup, verification, and cleanup.

Which issue(s) does this PR fix or relate to

• Fixes
  https://issues.redhat.com/browse/RHDHPAI-964

PR acceptance criteria

• Tests
• Documentation

How to test changes / Special notes to the reviewer

Follow this guide to get started with Developer lightspeed - developer-lightspeed/README.md

Summary by Sourcery

Enable Developer Lightspeed integration in the RHDH local environment by extending startup scripts, configuration files, compose overlays, and documentation.

New Features:

• Add support for the Developer Lightspeed generative AI assistant in rhdh-local via dynamic plugins and backend service.
• Introduce a compose-with-lightspeed overlay to launch the Lightspeed core service alongside RHDH.

Enhancements:

• Update startup script to detect and include a lightspeed-specific app-config file if present.
• Extend the base app-config.yaml with placeholder Lightspeed server entries sourced from environment variables.

Documentation:

• Add a detailed Developer Lightspeed setup guide and update the README with a link to the new guide.

Chores:

• Provide example override files for dynamic plugins and lightspeed-specific app-config.
• Add a default rcsconfig.yaml for Lightspeed environment and example app-config templates.

Summary by Sourcery

Enable Developer Lightspeed integration in the RHDH local environment by extending startup scripts, adding compose overlays and example configs, updating CI workflows, and providing detailed setup documentation

New Features:

• Integrate Developer Lightspeed AI assistant into rhdh-local via dynamic plugins and backend service
• Add compose overlays for both minimal and full (with Ollama) Lightspeed setups

Enhancements:

• Update startup script to detect and merge a lightspeed-specific app-config file alongside existing configs
• Leverage environment variables for flexible Lightspeed server and API key configuration

CI:

• Add GitHub Actions jobs to test minimal and full Developer Lightspeed compose scenarios

Documentation:

• Add a comprehensive Developer Lightspeed setup guide and update the README to reference it

Chores:

• Provide example override configurations for dynamic plugins, app-config, and road-core-service
• Include default.env entries and example templates to simplify Lightspeed onboarding

[sourcery-ai]

Reviewer's Guide

This PR integrates Developer Lightspeed into the RHDH local environment by extending the startup script to conditionally load a Lightspeed-specific app config, updating CI workflows to test Lightspeed compose overlays, and adding comprehensive documentation, compose files, and configuration templates for the new Lightspeed service.

Class diagram for Lightspeed app config structure

classDiagram
    class LightspeedConfig {
      +bool questionValidation
      +Server[] servers
    }
    class Server {
      +string id
      +string url
      +string token
    }
    LightspeedConfig "1" o-- "many" Server

Loading

Flow diagram for conditional loading of Lightspeed app config in startup script

flowchart TD
    Start([Start])
    CheckUserConfig{User app-config.local.yaml exists?}
    CheckLegacyConfig{Legacy app-config.local.yaml exists?}
    CheckLightspeedConfig{Lightspeed app-config.lightspeed.local.yaml exists?}
    BuildArgs["Build EXTRA_CONFIGS with found configs"]
    BuildCLIArgs["Build EXTRA_CLI_ARGS from EXTRA_CONFIGS"]
    StartBackend["Start Backstage backend with CLI args"]

    Start --> CheckUserConfig
    CheckUserConfig -- Yes --> BuildArgs
    CheckUserConfig -- No --> CheckLegacyConfig
    CheckLegacyConfig -- Yes --> BuildArgs
    CheckLegacyConfig -- No --> CheckLightspeedConfig
    BuildArgs --> CheckLightspeedConfig
    CheckLightspeedConfig -- Yes --> BuildArgs
    CheckLightspeedConfig -- No --> BuildCLIArgs
    CheckLightspeedConfig -- Yes --> BuildCLIArgs
    BuildCLIArgs --> StartBackend

Loading

File-Level Changes

Change	Details	Files
Extend startup script to load Lightspeed app-config if present	Introduce LIGHTSPEED_APP_CONFIG variable pointing to lightspeed config file Accumulate extra configs (user, legacy, lightspeed) into a single loop Generate EXTRA_CLI_ARGS dynamically from accumulated config list	wait-for-plugins-and-start.sh
Update CI workflow to include Lightspeed compose tests	Add developer-lightspeed-minimal job using compose.yaml and compose overlay Add developer-lightspeed job using compose-with-ollama overlay	.github/workflows/test.yml
Add Developer Lightspeed integration artifacts and documentation	Add detailed setup guide and verification steps in developer-lightspeed/docs/README.md Provide compose overlays (with and without Ollama) for Lightspeed service Include dynamic plugin override and app-config example templates Supply extra road-core-service config for Lightspeed backend	developer-lightspeed/docs/README.md developer-lightspeed/compose.yaml developer-lightspeed/compose-with-ollama.yaml developer-lightspeed/configs/app-config/app-config.lightspeed.local.example.yaml developer-lightspeed/configs/dynamic-plugins/dynamic-plugins.lightspeed.yaml developer-lightspeed/configs/extra-files/road-core-service-config.yaml
Update main README to reference the new Lightspeed guide	Add link to Developer Lightspeed Guide in feature list	README.md

---

Tips and commands

Interacting with Sourcery

• Trigger a new review: Comment @sourcery-ai review on the pull request.
• Continue discussions: Reply directly to Sourcery's review comments.
• Generate a GitHub issue from a review comment: Ask Sourcery to create an
  issue from a review comment by replying to it. You can also reply to a
  review comment with @sourcery-ai issue to create an issue from it.
• Generate a pull request title: Write @sourcery-ai anywhere in the pull
  request title to generate a title at any time. You can also comment
  @sourcery-ai title on the pull request to (re-)generate the title at any time.
• Generate a pull request summary: Write @sourcery-ai summary anywhere in
  the pull request body to generate a PR summary at any time exactly where you
  want it. You can also comment @sourcery-ai summary on the pull request to
  (re-)generate the summary at any time.
• Generate reviewer's guide: Comment @sourcery-ai guide on the pull
  request to (re-)generate the reviewer's guide at any time.
• Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
  pull request to resolve all Sourcery comments. Useful if you've already
  addressed all the comments and don't want to see them anymore.
• Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
  request to dismiss all existing Sourcery reviews. Especially useful if you
  want to start fresh with a new review - don't forget to comment
  @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

• Enable or disable review features such as the Sourcery-generated pull request
  summary, the reviewer's guide, and others.
• Change the review language.
• Add, remove or edit custom review instructions.
• Adjust other review settings.

Getting Help

• Contact our support team for questions or feedback.
• Visit our documentation for detailed guides and information.
• Keep in touch with the Sourcery team by following us on X/Twitter, LinkedIn or GitHub.

[sourcery-ai]

Hey @karthikjeeyar - I've reviewed your changes and they look great!

Prompt for AI Agents

Please address the comments from this code review:
## Individual Comments

### Comment 1
<location> `additional-config-guides/lightspeed-guide.md:12` </location>
<code_context>
+**Prerequisites**
+
+- Ensure you have **Podman** and **podman-compose** installed and available in your PATH.
+- You should have a working RHDH local environment.
+- Access to a Lightspeed server and API key.
+
</code_context>

<issue_to_address>
Use 'an' instead of 'a' before 'RHDH local environment'.

'RHDH' is pronounced with a vowel sound, so 'an' is correct here.
</issue_to_address>

---

Sourcery is free for open source - if you like our reviews please consider sharing them ✨

• X
• Mastodon
• LinkedIn
• Facebook

_{Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.}

[karthikjeeyar]

cc: @benwilcock @kadel

[benwilcock]

I think the guide or the changes might have missed the example .env changes required because when I try and boot the container I get:

WARN[0000] The "LIGHTSPEED_SERVER_TOKEN" variable is not set. Defaulting to a blank string. 
WARN[0000] The "LIGHTSPEED_SERVER_ID" variable is not set. Defaulting to a blank string. 
WARN[0000] The "LIGHTSPEED_SERVER_URL" variable is not set. Defaulting to a blank string. 
WARN[0000] The "LIGHTSPEED_SERVER_TOKEN" variable is not set. Defaulting to a blank string. 

The default settings would need adding to the default.env in order to be picked up with minimal changes.

[benwilcock]

I now have the ENV vars in the default.env, but I still get the warnings!?

However, the VARS are in place - I can see them when I ssh into the container and run env

[benwilcock]

I think the RCS config may need some further work. When I go into the Lightspeed GUI and and ask a question, I get back an error message:

failed to create message: Error from road-core server: Provider 'ollama' is not a valid provider. Valid providers are: ['dummy', '']

I think it maybe an issue in the LLM providers section? Here's the config I've been using on my server for comparison:

llm_providers:
  - name: dummy 
    type: openai 
    url: http://dummy.com
    models:
      - name: dummymodel # Dummy model for testing purposes
  - name: ollama
    type: openai
    url: "http://192.168.1.70:11434/v1/" # URL of the LLM server to that RCS will use for completions
    # credentials_path: /opt/app-root/src/cluster-key.txt
    disable_model_check: true # Disable model check to allow any model to be used
      
ols_config:
  user_data_collection:
    log_level: "DEBUG"
    feedback_disabled: false # Feedback collection is enabled
    feedback_storage: "/opt/app-root/src/user-data/feedback" # Feedback stored to disk
  reference_content:
    product_docs_index_path: "./vector_db/rhdh_product_docs/1.6" # RHDH RAG embeddings
    product_docs_index_id: rhdh-product-docs-1_6
    embeddings_model_path: "./embeddings_model"
  conversation_cache:
    type: memory # Chat storage mechanism
    memory:
      max_entries: 1000
  logging_config:
    app_log_level: info
    lib_log_level: warning
    uvicorn_log_level: info
    suppress_metrics_in_log: false
    suppress_auth_checks_warning_in_log: false
  authentication_config:
    module: "noop"
  default_provider: dummy
  default_model: dummymodel # This is only used if the client fails to send a model choice in the request
  query_validation_method: disabled
dev_config:
  enable_dev_ui: false
  disable_auth: true # Allow use without authentication
  disable_tls: true # Allow use via HTTP
  enable_system_prompt_override: true # Allow the system prompt to be overwritten
user_data_collector_config:
  user_agent: "lightspeed-operator/user-data-collection cluster/{cluster_id}"
  ingress_url: "https://example.ingress.com/upload"

[karthikjeeyar]

The problem was default.env variables are not mounted into road-core, podman only picks up .env file. I am now mounting default.env and .env file to the road-core-service explicitly so that it will now pick up from both default.env and .env (if available) files.

I have fixed it now, RCS should pick up the LIGHTSPEED server details from env correctly.

[karthikjeeyar]

FYI @benwilcock you do not need ollama in llm_providers in rcsconfig.yaml anymore, now that the default.env values are properly injected into rcs, the Lightpseed server details from the app-config.lightspeed.yaml will automatically picked by road-core.

[benwilcock]

I've tried again by pulling the latest. Now I get a new error: failed to create message: Error fetching completions from ollama: FetchError: request to http://0.0.0.0:8080/v1/streaming_query?user_id=user%3Adevelopment%2Fguest failed, reason: connect ECONNREFUSED 0.0.0.0:8080

[rm3l]

/lgtm

[Jdubrick]

lgtm

[karthikjeeyar]

@rm3l I have updated the documetation to add table of contents and reordered the cleanup section.

[benwilcock]

It would be good if we could get this merged in to coincide with the release of 1.7, specifically the docs - they have a link to the RHDH Local repo and mention that Lightspeed can be tested using RHDH Local.

[rm3l]

/lgtm

[rm3l]

Merging this PR since it has been approved by several people.