Conversation
The prior workflow-issues learning assumed inline ACL payloads and legacy acl_apply staging, which now contradicts the current datasource workflow and frontend contract. This refresh replaces the stale learning with a successor doc reflecting workspace/datasource binding orchestration plus table-permissions governance, and adds AGENTS discoverability guidance for docs/solutions. Constraint: docs/solutions is ignored by default in this repo and required force-add for replacement doc Rejected: In-place update of the old learning | stale filename and ACL-framed narrative would keep drift risk high Confidence: high Scope-risk: narrow Reversibility: clean Directive: Treat /api/v1/datasources/workflow as context-binding orchestration; keep governance writes in table-permissions APIs Tested: Manual evidence verification against current DTO/service/tests and README contracts Not-tested: No lint/test suite run (docs and AGENTS changes only)
Keep workflow learning aligned with table-permissions reality
Strengthen the v1.3 index plan with an explicit cross-phase dependency graph, full R1-R36 ownership mapping, unified gate evidence schema, and rollback matrix. This keeps phase promotion decisions executable instead of narrative. Constraint: Plan artifacts under docs/ are ignored by default and require force-add for versioned review handoff Rejected: Keep grouped requirement ranges only | Could not provide per-requirement accountability across gates Directive: Do not rename gate evidence fields without updating this index contract first Confidence: high Scope-risk: narrow Tested: lsp_diagnostics on updated plan file (0 diagnostics) Tested: requirement coverage script confirms R1-R36 complete with no missing IDs Not-tested: repository lint/test/build (docs-only change)
Deepened the R6 plan with quantitative multi-datasource gates, cache/budget validation strategy, graph acceleration fallback controls, release/rollback evidence requirements, and a complete test coverage matrix. Constraint: R6 must preserve R2-R5 Postgres fallback and existing security/audit gates Constraint: docs/plans is gitignored and requires targeted force-add for tracked updates Rejected: Keep qualitative gate wording only | Cannot automate release blocking without hard thresholds and sample windows Rejected: Enable graph acceleration by default | Violates rollback-first optionality for R6 Confidence: high Scope-risk: narrow Directive: Do not tune R6 thresholds or fallback triggers without updating r6-scale-gate-spec and evidence checklist together Tested: mcp__omx_code_intel__.lsp_diagnostics on docs/plans/2026-04-17-006-feat-rag-r6-scale-optimization-plan.md (0 errors) Tested: rg absolute-path scan on plan file (no /Users/ or Windows absolute paths) Not-tested: Project lint/test/build (doc-only planning change)
Deepened both phase plans to close ce:plan quality gaps around requirements trace continuity, cross-phase dependency gating, and system-wide impact articulation.\n\nAdded explicit open-question classifications, rationale-backed technical decisions, fuller implementation-unit test coverage expectations, and operational evidence/rollout notes so execution can proceed without re-litigating stage contracts. Constraint: Changes must stay within two user-specified plan files Constraint: docs/plans files are ignored by default and require force-add for commit Rejected: Minimal section patching only | left unresolved continuity and test-coverage ambiguity Directive: Do not start R4/R5 execution until R3 exit artifacts (semantic snapshot, skill digest, replay evidence) are present Confidence: high Scope-risk: narrow Tested: Section presence/trace coverage/open-question classification checks via rg Not-tested: Repository-wide lint/typecheck/test gates (docs-only change)
…out ambiguity Deepened the two R2 plan artifacts to make requirement ownership, cross-phase handoff, unit-level dependencies, repo-relative file scope, scenario-complete testing, gate metrics, and rollback evidence explicit. This reduces planning-time interpretation drift before ce:work. Constraint: Requested scope is limited to two existing R2 plan documents Constraint: Keep content at planning level and avoid implementation code Rejected: Leave requirements trace at range-only granularity | cannot prove R6-R24 ownership per phase Rejected: Add code-level pseudo implementation details | violates plan-only boundary and increases churn risk Confidence: high Scope-risk: narrow Directive: Keep R2 gate thresholds centralized in 003 and do not split metric definitions across multiple docs without updating the gate spec Tested: pnpm run format:check Tested: npx tsc --noEmit --pretty false --project apps/backend/tsconfig.json Tested: npx tsc --noEmit --pretty false --project apps/frontend/tsconfig.json Not-tested: Runtime feature tests (docs-only change)
- Introduced the RAG module, including services for index building, retrieval, and reranking. - Added new models for RAG documents, chunks, index versions, and run replays in the Prisma schema. - Updated the application module to integrate the RAG module across various components. - Enhanced knowledge retrieval and SQL generation processes to utilize RAG context, improving query accuracy and context awareness. - Established new database migrations to support the RAG data structures and relationships. This commit strengthens the system's capability to handle retrieval-augmented generation tasks, ensuring better context management and data integrity.
- Introduced the Semantic Registry module, including models for semantic versions and terms in the Prisma schema. - Updated the application module to integrate the Semantic Registry module across various components. - Enhanced the agent's planning nodes to incorporate semantic versioning and risk management, improving query accuracy and context awareness. - Implemented caching mechanisms for planner services to optimize performance and reduce redundant computations. - Established new database migrations to support the semantic registry data structures and relationships. This commit strengthens the system's capability to manage semantic data, ensuring better version control and risk assessment in planning processes.
- Updated the RAG module to include new services for managing delivery contracts, improving the handling of run data. - Enhanced the ChatService to attach delivery contracts to SQL runs, ensuring better tracking and management of delivery-related information. - Introduced new methods for loading replay records and triggering memory promotions based on delivery contracts. - Updated the backend workflows to include R6 scenario package tests and evidence collection, enhancing the testing framework for RAG functionalities. - Improved health check endpoints to provide detailed RAG quality metrics, ensuring better monitoring and diagnostics. This commit strengthens the RAG module's capabilities in managing delivery contracts and enhances the overall quality assurance processes within the system.
- Updated AGENTS.md to ensure consistency between `run.trace.promptTemplate` and `delivery.evidence.promptTemplate` fields during prompt template integration. - Expanded README.md with new health check endpoints for glossary context and additional API routes for glossary and prompt management. - Introduced new JSON files for review processes, including adversarial, correctness, and data migrations, to enhance governance and compliance checks. - Improved backend workflows to include R6 scenario tests and evidence collection, ensuring robust testing for new features. This commit strengthens the governance framework for agent operations and enhances API contract compliance, improving overall system integrity and reliability.
- Introduced a new `.graphify_python` file to specify the Python environment for graphify operations. - Updated `AGENTS.md` with guidelines for utilizing the graphify knowledge graph, including rules for navigating and updating the graph. - Added new JSON cache files for various modules to support graphify functionality, enhancing the system's ability to manage and visualize dependencies. - Created a `.codex/hooks.json` file to implement pre-tool use hooks for checking the existence of the knowledge graph. This commit enhances the agent's capabilities by integrating graphify support, improving documentation, and establishing a foundation for better dependency management and visualization.
- Added execution tracking sections to R2, R3, R4, and R5 plans, specifying dedicated execution logs for each phase. - Introduced boundary and edge case catalogs to outline critical scenarios and handling strategies across phases. - Updated requirements trace to ensure clarity on cross-phase dependencies and governance. - Enhanced documentation to support better tracking of execution status and evidence collection for each phase. This commit strengthens the planning framework for RAG phases, ensuring robust execution tracking and comprehensive handling of edge cases, thereby improving overall system reliability and governance.
…tation - Introduced a new governance module to encapsulate governance-related functionalities, including datasource and settings management. - Updated AGENTS.md and README.md to include governance terminology hard cut specifications and relevant checks. - Added a governance terminology check script and CI workflow to enforce compliance with new governance standards. - Refactored existing code to replace legacy routing and fields with updated governance terminology, ensuring consistency across the application. - Created a new Governance Hard Cut Progress document to track the status and details of the governance transition. This commit strengthens the governance framework by formalizing terminology standards and enhancing documentation, ensuring better compliance and clarity across the system.
…ule restructuring - Introduced a new backend business capability topology specification in AGENTS.md, detailing the structure and dependencies for backend modules. - Updated README.md to include the new backend capability topology guidelines, ensuring clarity on module dependencies and restrictions. - Refactored the backend application module to integrate the new ConversationModule, consolidating related functionalities and improving modularity. - Adjusted various modules (Chat, Agent, etc.) to align with the new structure, ensuring proper imports and dependencies. - Enhanced CI workflows to include checks for backend capability boundaries, reinforcing compliance with the new specifications. This commit strengthens the backend architecture by formalizing capability boundaries and improving module organization, leading to better maintainability and clarity in the system.
- Updated .gitignore to include new output directories for graphify operations. - Enhanced AGENTS.md with additional restrictions on module imports to enforce governance boundaries. - Refactored agent and conversation modules to replace legacy data module imports with new persistence and query modules, improving modularity and compliance with updated architecture. - Introduced new use cases for chat session management, including execute-message and run-view functionalities, to streamline chat operations and enhance user experience. - Improved session lifecycle management with robust error handling and policy checks to ensure compliance with governance standards. This commit strengthens the backend architecture by refining module dependencies, enhancing governance compliance, and improving chat session management capabilities.
- Updated KnowledgeChatSupportFacade to implement KnowledgeFacadeContract, improving type safety and modularity. - Introduced new contracts for glossary, memory, and semantic registry services, enhancing the overall architecture. - Refactored KnowledgeModule to provide dependencies for the new contracts, ensuring better integration and maintainability. - Enhanced graph report with updated node and edge counts, reflecting recent changes in the system. This commit strengthens the knowledge module's architecture by formalizing contracts and improving service integration, leading to better maintainability and clarity in the system.
- Updated package.json to include a default value for BACKEND_CAPABILITY_BOUNDARY_CONVERSATION_KNOWLEDGE_BASELINE in the capability boundary check command. - Refactored conversation agent modules to replace legacy imports with new knowledge facade contracts, improving modularity and compliance with governance boundaries. - Introduced new compatibility bridges for glossary, memory, and semantic registry modules, formalizing their integration and enhancing maintainability. - Enhanced unit tests to validate the new compatibility checks and ensure proper handling of knowledge imports. This commit strengthens the backend architecture by formalizing knowledge integration, improving compliance with governance standards, and enhancing the overall modularity of the system.
- Updated import paths for the Prisma client in the glossary and semantic registry services to correct relative paths, ensuring proper module resolution. - Refactored the Dialog component in the frontend to utilize forward refs for better integration with React's ref system, enhancing component flexibility and usability. - Improved the Graph Report by updating word counts to reflect recent changes in the corpus, ensuring accurate reporting. This commit enhances module import accuracy and improves component structure in the frontend, contributing to better maintainability and functionality across the application.
- Updated the README.md to include support for an optional `contextEnvelope` in request bodies, improving the clarity of context management. - Refactored agent types to incorporate `ContextEnvelope`, `GraphEffectiveContextSummary`, and `GraphContextConflictHint`, enhancing type safety and modularity. - Improved the GraphBuilderService to build and include context evidence in SQL run traces, allowing for better context differentiation in responses. - Enhanced the ClarifyNode to utilize context information for more accurate clarification prompts, improving user interaction. - Introduced a new Slot Filling Context module to streamline the decision-making process for missing critical slots in user queries. - Updated various services and controllers to support the new context envelope structure, ensuring consistent handling across the application. This commit strengthens the agent's ability to manage context effectively, leading to improved user experience and more accurate query handling.
…-filling-exec Enhance context handling and clarify agent interactions
…d-cut-exec Codex/governance terminology hard cut exec
Codex/rag v1 3 team exec tracking
- Added a comprehensive section in AGENTS.md and README.md for the Text2SQL + RAG full process understanding, including links to canonical documents and maintenance scripts. - Updated graph reports to reflect the latest corpus statistics, including an increase in files and word count, as well as adjustments in node and edge counts. - Improved community cohesion metrics in the graph report, highlighting changes in community structures and node connections. This commit strengthens the documentation and reporting framework, providing clearer guidance for users and enhancing the accuracy of system metrics.
- Modified .gitignore to include specific graphify output directory patterns for better file management. - Enhanced settings.local.json with additional Bash commands for improved local script execution, facilitating easier access to documentation and file analysis. These changes streamline the development process and improve the organization of generated files.
…handling - Introduced the `SemanticSpineSnapshot` model in the Prisma schema to manage semantic snapshots, including relationships to `SqlRun`. - Created a migration script to establish the new `semantic_spine_snapshots` table with appropriate constraints and indexes. - Added new scripts for collecting relationship and semantic spine shadow gate metrics, improving data reporting capabilities. - Enhanced the `LangGraph` state and handlers to incorporate context pack information and relationship correction logic, improving the agent's ability to manage semantic relationships. These changes enhance the backend's data handling and improve the overall architecture for managing semantic snapshots and relationships.
- Added new endpoints for saving views from runs, improving user interaction with saved SQL queries. - Introduced the `modelingGraphRevisions` model in the Prisma schema to track revisions of modeling graphs, enhancing data integrity and version control. - Updated various services and scripts to incorporate modeling workspace metrics, including thresholds for deployment and schema changes. - Enhanced the `collect-modeling-parity-shadow-gate` script to evaluate modeling workspace metrics, improving reporting capabilities. - Updated documentation in AGENTS.md and README.md to reflect new modeling workspace functionalities and metrics. These changes improve the backend's ability to manage and report on modeling workspaces, enhancing user experience and data accuracy.
- Added support for modeling revision in various components, including `BuildSemanticQueryNode`, `DeliveryContractMapper`, and `RagRetrievalService`, improving the accuracy of semantic processing. - Updated context pack handling to accommodate both camelCase and snake_case formats, ensuring compatibility with different data sources. - Enhanced semantic hints to reflect the presence or absence of modeling revisions, providing clearer guidance for relationship management. - Improved unit tests to validate the new modeling revision logic and context pack parsing, ensuring robust functionality. These changes enhance the system's ability to manage modeling revisions and context packs, leading to improved semantic query processing and user experience.
- Updated the `generatedAt` timestamps in gate summary JSON files for relationship platform, semantic spine, and modeling parity shadows to reflect the latest generation time. - Changed `strictMode` from `true` to `false` in the modeling parity shadow gate summary to allow for more flexible processing. - Enhanced the semantic plan service to handle degraded contexts without grounding evidence, ensuring the SQL path is maintained in such scenarios. - Introduced a tolerance for gate comparison in the text2sql evaluation service to prevent false negatives due to rounding errors.
- Added new endpoints for managing RAG task configurations, including listing, upserting, and health checking. - Implemented a new `RagTaskConfig` model and service to handle RAG task settings. - Enhanced the `SettingsService` to integrate RAG task configuration management. - Updated the `EmbeddingRouterService` and related components to utilize RAG task configurations for embedding runtime resolution. - Introduced coverage gap handling in the semantic plan service to improve error reporting and decision-making in SQL generation.
- Introduced new evaluation gates for Text2SQL v2, including strict modes for evaluation and focused coverage. - Updated CI references in documentation to reflect new gate commands and their expected outputs. - Enhanced the text2sql evaluation service to include detailed traceability metrics, ensuring comprehensive coverage reporting. - Added new traceability fields in evaluation cases to track fixture families, behavior tests, and flow nodes. - Improved error handling in SQL generation to account for metadata and general queries, preventing unnecessary SQL execution. - Updated test fixtures to include traceability information for better integration testing.
- Introduced a new method `runOperationalFailure` in `FormatAnswerNode` to handle operational failures with user guidance. - Updated `RetrieveKnowledgeNode` to support datasource and allowed tables in its input, improving context handling. - Enhanced `SqlGenerationService` to utilize semantic shortcuts for generating SQL queries, improving efficiency and error recovery. - Implemented checks in `SqlReadonlyTool` to prevent introspection of schema system tables, ensuring security compliance. - Added new tests for RAG provider model previews and SQL generation shortcuts, ensuring robust functionality and error handling.
Introduce trace.v2 runtimePlan, artifactRefs, and Smart Defaults evidence across the shared contract, LangGraph runtime, delivery projection, and stream state summaries.
Extend focused and eval gates so runtime intelligence requires producer-backed artifact refs, stream lifecycle coverage, Smart Defaults evidence, and no-legacy owner scans.
Define additive plan ledger obligations, generation claims, validation fulfillment, and delivery summary fields so the v2 trace can carry SQL intent contracts without changing runtime plan semantics.
Build semantic plan obligations, block failed hard blockers before SQL generation, record fulfillment claims on generated SQL, and validate SQL against the ledger before execution or correction.
Add ledger fixture families, rollout metrics, and unit coverage for obligation creation, generation claims, validation fulfillment, correction grounding, and safe delivery surfaces.
…-spine feat(conversation): hard-cut text2sql v2 workflow spine and RAG runtime gates
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.