RHIDP-12949 Llama stack library mode changes

workspaces/lightspeed/.changeset/afraid-pugs-impress.md

@@ -0,0 +1,6 @@
+---
+'@red-hat-developer-hub/backstage-plugin-lightspeed-backend': minor
+'@red-hat-developer-hub/backstage-plugin-lightspeed': minor
+---
+
+Migrated AI Notebooks from direct Llama Stack server to Lightspeed-Core integration

workspaces/lightspeed/app-config.yaml

@@ -18,8 +18,15 @@ organization:
 
 # Disable AI Notebooks feature by default
 lightspeed:
-  aiNotebooks:
+  notebooks:
     enabled: false
+    queryDefaults:
+      model: redhataillama-31-8b-instruct
+      provider_id: vllm
+    sessionDefaults:
+      provider_id: notebooks
+      embedding_model: ${LLAMA_STACK_EMBEDDING_MODEL}
+      embedding_dimension: 768
 
 backend:
   # Used for enabling authentication, secret is shared by all backend plugins

workspaces/lightspeed/plugins/lightspeed-backend/README.md

@@ -89,7 +89,10 @@ For user-facing feature documentation, see the [Lightspeed Frontend README](../l
 
 #### Prerequisites
 
-AI Notebooks requires a **Llama Stack service** to be running. Llama Stack provides the vector database, embeddings, and RAG capabilities.
+AI Notebooks requires:
+
+- **Lightspeed Core service** to be running (provides the backend API proxy)
+- **Llama Stack service** to be accessible from Lightspeed Core (provides vector database, embeddings, and RAG capabilities)
 
 For Llama Stack setup and configuration, refer to the [Llama Stack documentation](https://github.com/llamastack/llama-stack).
 
@@ -99,21 +102,23 @@ To enable AI Notebooks, add the following configuration to your `app-config.yaml
 
 ```yaml
 lightspeed:
-  aiNotebooks:
-    enabled: true # Enable AI Notebooks feature (default: false)
+  servicePort: 8080 # Optional: Lightspeed Core service port (default: 8080)
 
-    # Required when enabled: Llama Stack service configuration
-    llamaStack:
-      port: 8321 # Llama Stack API endpoint (required, commonly 8321)
+  notebooks:
+    enabled: false # Enable AI Notebooks feature (default: false)
 
-      # Optional embedding configuration
-      embeddingModel: sentence-transformers/nomic-ai/nomic-embed-text-v1.5 # (default shown)
-      embeddingDimension: 768 # Embedding vector dimension (default: 768)
-      vectorIo:
-        providerId: rhdh-docs # Vector store provider ID (default: rhdh-docs)
+    # Required: Query defaults for RAG queries
+    # Both model and provider_id must be configured together
+    queryDefaults:
+      model: llama3.1-8b-instruct # Model to use for answering queries
+      provider_id: ollama # AI provider for the query model
 
-    # Optional: File processing timeout (default: 30000ms = 30 seconds)
-    fileProcessingTimeoutMs: 30000
+    # Required: Session defaults for creating vector stores
+    # All three fields are required when Notebooks is enabled
+    sessionDefaults:
+      provider_id: notebooks # Vector store provider ID (must match Llama Stack config)
+      embedding_model: sentence-transformers/all-mpnet-base-v2 # Model for generating embeddings
+      embedding_dimension: 768 # Embedding vector dimension (must match model output)
 
     # Optional: Chunking strategy for document processing
     chunkingStrategy:
@@ -125,15 +130,37 @@ lightspeed:
 
 **Configuration Options**:
 
-- **`enabled`**: Enable or disable the AI Notebooks feature (default: `false`)
-- **`llamaStack.port`**: Port of the Llama Stack service (default: `8321`)
-- **`llamaStack.embeddingModel`**: Model used for generating embeddings (default: `sentence-transformers/nomic-ai/nomic-embed-text-v1.5`)
-- **`llamaStack.embeddingDimension`**: Dimension of embedding vectors (default: `768`)
-- **`llamaStack.vectorIo.providerId`**: Vector store provider in Llama Stack config (default: `rhdh-docs`)
-- **`fileProcessingTimeoutMs`**: Timeout for file processing in milliseconds (default: `30000`)
-- **`chunkingStrategy.type`**: Document chunking strategy - `auto` (automatic) or `static` (fixed size) (default: `auto`)
-- **`chunkingStrategy.maxChunkSizeTokens`**: Maximum chunk size in tokens for static chunking (default: `512`)
-- **`chunkingStrategy.chunkOverlapTokens`**: Token overlap between chunks for static chunking (default: `50`)
+**Core Settings**:
+
+- **`lightspeed.servicePort`** _(optional)_: Port where Lightspeed Core service is running (default: `8080`). The backend connects to Lightspeed Core at `http://0.0.0.0:{servicePort}` to proxy vector store operations.
+
+**Notebooks Settings**:
+
+- **`Notebooks.enabled`** _(optional)_: Enable or disable the AI Notebooks feature (default: `false`)
+
+**Query Defaults** _(required when enabled)_:
+
+- **`queryDefaults.model`** _(required)_: The LLM model to use for answering RAG queries. Must be available in the configured provider.
+- **`queryDefaults.provider_id`** _(required)_: The AI provider identifier for the query model (e.g., `ollama`, `vllm`). Both `model` and `provider_id` must be configured together.
+
+**Session Defaults** _(required when enabled)_:
+
+- **`sessionDefaults.provider_id`** _(required)_: Vector store provider identifier. Must match a provider configured in your Llama Stack instance (e.g., `notebooks`, `chromadb`). This determines where document embeddings are stored.
+- **`sessionDefaults.embedding_model`** _(required)_: The embedding model to use for converting documents to vectors (e.g., `sentence-transformers/all-mpnet-base-v2`). Must be available in Llama Stack.
+- **`sessionDefaults.embedding_dimension`** _(required)_: Dimension of the embedding vectors produced by the embedding model. Must match the model's output dimension (commonly `768`, `384`, or `1536`).
+
+**Chunking Strategy** _(optional)_:
+
+- **`chunkingStrategy.type`** _(optional)_: Document chunking strategy - `auto` (automatic, default) or `static` (fixed size)
+- **`chunkingStrategy.maxChunkSizeTokens`** _(optional)_: Maximum chunk size in tokens for static chunking (default: `512`)
+- **`chunkingStrategy.chunkOverlapTokens`** _(optional)_: Token overlap between chunks for static chunking (default: `50`)
+
+**Where to Find These Values**:
+
+- **Provider IDs**: Check your Llama Stack configuration file for configured providers (both for models and vector stores)
+- **Model names**: Available models are listed in your Llama Stack provider configuration
+- **Embedding dimensions**: Refer to the embedding model's documentation (e.g., `all-mpnet-base-v2` outputs 768 dimensions)
+- **Lightspeed Core port**: Check your Lightspeed Core service deployment configuration
 
 #### API Endpoints
 
@@ -144,20 +171,26 @@ When enabled, AI Notebooks exposes the following REST API endpoints:
 
 - **Sessions**:
   - `POST /lightspeed/ai-notebooks/v1/sessions` - Create a new session
-  - `GET /lightspeed/ai-notebooks/v1/sessions` - List all sessions
-  - `GET /lightspeed/ai-notebooks/v1/sessions/:sessionId` - Get session details
-  - `PUT /lightspeed/ai-notebooks/v1/sessions/:sessionId` - Update session
+  - `GET /lightspeed/ai-notebooks/v1/sessions` - List all sessions for the current user
+  - `PUT /lightspeed/ai-notebooks/v1/sessions/:sessionId` - Update session details
   - `DELETE /lightspeed/ai-notebooks/v1/sessions/:sessionId` - Delete session
 
 - **Documents**:
-  - `POST /lightspeed/ai-notebooks/v1/sessions/:sessionId/documents/upload` - Upload document
-  - `GET /lightspeed/ai-notebooks/v1/sessions/:sessionId/documents` - List documents
-  - `PUT /lightspeed/ai-notebooks/v1/sessions/:sessionId/documents/:documentId` - Update document
-  - `DELETE /lightspeed/ai-notebooks/v1/sessions/:sessionId/documents/:documentId` - Delete document
+  - `PUT /lightspeed/ai-notebooks/v1/sessions/:sessionId/documents` - Upload or update a document (multipart/form-data)
+  - `GET /lightspeed/ai-notebooks/v1/sessions/:sessionId/documents` - List all documents in a session
+  - `GET /lightspeed/ai-notebooks/v1/sessions/:sessionId/documents/:documentId/status` - Get document processing status
+  - `DELETE /lightspeed/ai-notebooks/v1/sessions/:sessionId/documents/:documentId` - Delete a document
 
 - **Queries**:
   - `POST /lightspeed/ai-notebooks/v1/sessions/:sessionId/query` - Query documents with RAG
 
+**Notes**:
+
+- All endpoints require authentication (user context is automatically provided by Backstage)
+- All `/v1/*` endpoints require the `lightspeed.notebooks.use` permission
+- Document endpoints verify session ownership before allowing operations
+- `documentId` in paths is the document title (URL-encoded for special characters)
+
 #### Permission Framework Support for AI Notebooks
 
 When RBAC is enabled, users need the following permission to use AI Notebooks:

workspaces/lightspeed/plugins/lightspeed-backend/__fixtures__/lightspeedCoreHandlers.ts

@@ -0,0 +1,237 @@
+/*
+ * Copyright Red Hat, Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+import { http, HttpResponse, type HttpHandler } from 'msw';
+
+// Use port 7007 for tests (matches notebooksRouter.test.ts config)
+const TEST_LIGHTSPEED_SERVICE_PORT = 7007;
+export const LIGHTSPEED_CORE_ADDR = `http://0.0.0.0:${TEST_LIGHTSPEED_SERVICE_PORT}`;
+
+// Mock session data
+export const mockSession1 = {
+  session_id: 'session-1',
+  name: 'Test Session 1',
+  user_id: 'user:default/guest',
+  description: 'Test description',
+  created_at: '2024-01-01T00:00:00.000Z',
+  updated_at: '2024-01-01T00:00:00.000Z',
+  metadata: {
+    embedding_model: 'sentence-transformers/nomic-ai/nomic-embed-text-v1.5',
+    embedding_dimension: 768,
+    provider_id: 'notebooks',
+    conversation_id: null,
+  },
+};
+
+export const mockSession2 = {
+  session_id: 'session-2',
+  name: 'Test Session 2',
+  user_id: 'user:default/guest',
+  description: 'Another test',
+  created_at: '2024-01-02T00:00:00.000Z',
+  updated_at: '2024-01-02T00:00:00.000Z',
+  metadata: {
+    embedding_model: 'sentence-transformers/nomic-ai/nomic-embed-text-v1.5',
+    embedding_dimension: 768,
+    provider_id: 'notebooks',
+    conversation_id: 'conv-1',
+  },
+};
+
+export const mockFile1 = {
+  id: 'file-1',
+  created_at: 1704067200,
+  status: 'completed' as const,
+  attributes: {
+    document_id: 'test-document',
+    user_id: 'user:default/guest',
+    title: 'Test Document',
+    session_id: 'session-1',
+    source_type: 'text',
+    created_at: '2024-01-01T00:00:00.000Z',
+  },
+};
+
+// In-memory storage for tests
+const vectorStores = new Map<string, any>();
+const files = new Map<string, any>();
+const vectorStoreFiles = new Map<string, any[]>();
+
+export function resetMockStorage() {
+  vectorStores.clear();
+  files.clear();
+  vectorStoreFiles.clear();
+}
+
+/**
+ * MSW handlers for lightspeed-core vector store endpoints
+ * These mock the endpoints created in lightspeed-core that proxy to llama stack
+ */
+export const lightspeedCoreHandlers: HttpHandler[] = [
+  // Create vector store
+  http.post(`${LIGHTSPEED_CORE_ADDR}/v1/vector-stores`, async ({ request }) => {
+    const body = (await request.json()) as any;
+    const id = `vs-${Date.now()}`;
+    const vectorStore = {
+      id,
+      name: body.name,
+      embedding_model: body.embedding_model,
+      embedding_dimension: body.embedding_dimension,
+      provider_id: body.provider_id,
+      metadata: body.metadata || {},
+    };
+    vectorStores.set(id, vectorStore);
+    vectorStoreFiles.set(id, []);
+    return HttpResponse.json(vectorStore);
+  }),
+
+  // Get vector store
+  http.get(`${LIGHTSPEED_CORE_ADDR}/v1/vector-stores/:id`, ({ params }) => {
+    const { id } = params;
+    const vectorStore = vectorStores.get(id as string);
+    if (!vectorStore) {
+      return HttpResponse.json(
+        { detail: 'Vector store not found' },
+        { status: 404 },
+      );
+    }
+    return HttpResponse.json(vectorStore);
+  }),
+
+  // Update vector store
+  http.put(
+    `${LIGHTSPEED_CORE_ADDR}/v1/vector-stores/:id`,
+    async ({ params, request }) => {
+      const { id } = params;
+      const vectorStore = vectorStores.get(id as string);
+      if (!vectorStore) {
+        return HttpResponse.json(
+          { detail: 'Vector store not found' },
+          { status: 404 },
+        );
+      }
+      const body = (await request.json()) as any;
+      const updated = {
+        ...vectorStore,
+        metadata: body.metadata || vectorStore.metadata,
+      };
+      vectorStores.set(id as string, updated);
+      return HttpResponse.json(updated);
+    },
+  ),
+
+  // Delete vector store
+  http.delete(`${LIGHTSPEED_CORE_ADDR}/v1/vector-stores/:id`, ({ params }) => {
+    const { id } = params;
+    if (!vectorStores.has(id as string)) {
+      return HttpResponse.json(
+        { detail: 'Vector store not found' },
+        { status: 404 },
+      );
+    }
+    vectorStores.delete(id as string);
+    vectorStoreFiles.delete(id as string);
+    return HttpResponse.json({ deleted: true });
+  }),
+
+  // List vector stores
+  http.get(`${LIGHTSPEED_CORE_ADDR}/v1/vector-stores`, () => {
+    const data = Array.from(vectorStores.values());
+    return HttpResponse.json({ data });
+  }),
+
+  // Upload file
+  http.post(`${LIGHTSPEED_CORE_ADDR}/v1/files`, async () => {
+    const fileId = `file-${Date.now()}`;
+    const file = {
+      id: fileId,
+      created_at: Date.now(),
+      purpose: 'assistants',
+    };
+    files.set(fileId, file);
+    return HttpResponse.json(file);
+  }),
+
+  // Add file to vector store
+  http.post(
+    `${LIGHTSPEED_CORE_ADDR}/v1/vector-stores/:id/files`,
+    async ({ params, request }) => {
+      const { id } = params;
+      const vectorStore = vectorStores.get(id as string);
+      if (!vectorStore) {
+        return HttpResponse.json(
+          { detail: 'Vector store not found' },
+          { status: 404 },
+        );
+      }
+
+      const body = (await request.json()) as any;
+      const vectorStoreFile = {
+        id: body.file_id,
+        status: 'completed' as const,
+        created_at: Date.now(),
+        chunks_count: 1,
+        attributes: body.attributes || {},
+      };
+
+      const storeFiles = vectorStoreFiles.get(id as string) || [];
+      storeFiles.push(vectorStoreFile);
+      vectorStoreFiles.set(id as string, storeFiles);
+
+      return HttpResponse.json(vectorStoreFile);
+    },
+  ),
+
+  // List files in vector store
+  http.get(
+    `${LIGHTSPEED_CORE_ADDR}/v1/vector-stores/:id/files`,
+    ({ params }) => {
+      const { id } = params;
+      const storeFiles = vectorStoreFiles.get(id as string) || [];
+      return HttpResponse.json({ data: storeFiles });
+    },
+  ),
+
+  // Get file from vector store
+  http.get(
+    `${LIGHTSPEED_CORE_ADDR}/v1/vector-stores/:id/files/:fileId`,
+    ({ params }) => {
+      const { id, fileId } = params;
+      const storeFiles = vectorStoreFiles.get(id as string) || [];
+      const file = storeFiles.find(f => f.id === fileId);
+      if (!file) {
+        return HttpResponse.json({ detail: 'File not found' }, { status: 404 });
+      }
+      return HttpResponse.json(file);
+    },
+  ),
+
+  // Delete file from vector store
+  http.delete(
+    `${LIGHTSPEED_CORE_ADDR}/v1/vector-stores/:id/files/:fileId`,
+    ({ params }) => {
+      const { id, fileId } = params;
+      const storeFiles = vectorStoreFiles.get(id as string) || [];
+      const fileIndex = storeFiles.findIndex(f => f.id === fileId);
+      if (fileIndex === -1) {
+        return HttpResponse.json({ detail: 'File not found' }, { status: 404 });
+      }
+      storeFiles.splice(fileIndex, 1);
+      vectorStoreFiles.set(id as string, storeFiles);
+      return HttpResponse.json({ deleted: true });
+    },
+  ),
+];