Skip to content

feat: ship track-centric research context stack#356

Merged
jerry609 merged 6 commits into
devfrom
feat/track-context-epic-325
Mar 12, 2026
Merged

feat: ship track-centric research context stack#356
jerry609 merged 6 commits into
devfrom
feat/track-context-epic-325

Conversation

@jerry609

Copy link
Copy Markdown
Owner

Summary

  • add a backend track context read model and consolidated /api/research/tracks/{track_id}/context endpoint
  • migrate ResearchPageNew to the consolidated track context contract and make context builds pass explicit track_id
  • wrap track-scoped memory access behind an application service instead of route-local stitching
  • document the track-centric research model and stabilize singleton-sensitive integration tests

Linked Issues

Validation

  • pytest tests/unit/test_research_track_context_service.py -q
  • pytest tests/integration/test_research_track_context_routes.py tests/unit/test_research_track_context_service.py -q
  • pytest tests/unit/test_research_context_route_explicit_track.py -q
  • pytest tests/unit/test_track_memory_service.py tests/integration/test_research_track_memory_routes.py -q
  • pytest tests/unit/test_research_track_context_service.py tests/integration/test_research_track_context_routes.py tests/unit/test_research_context_route_explicit_track.py tests/unit/test_track_memory_service.py tests/integration/test_research_track_memory_routes.py tests/integration/test_scope_and_acceptance_criteria_hooks.py -q
  • node ./web/node_modules/vitest/vitest.mjs run web/src/components/research/ResearchTrackContextPanel.test.tsx
  • node ./web/node_modules/eslint/bin/eslint.js web/src/components/research/ResearchPageNew.tsx web/src/components/research/ResearchTrackContextPanel.tsx web/src/components/research/ResearchTrackContextPanel.test.tsx web/src/lib/types.ts
  • cd web && ./node_modules/.bin/next build

Copilot AI review requested due to automatic review settings March 12, 2026 04:57
@vercel

vercel Bot commented Mar 12, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
paper-bot Ready Ready Preview, Comment Mar 12, 2026 5:22am

@coderabbitai

coderabbitai Bot commented Mar 12, 2026

Copy link
Copy Markdown

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: 5df9e6fc-f828-4383-b44e-94f1b84b93c1

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Post copyable unit tests in a comment
  • Commit unit tests in branch feat/track-context-epic-325

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@gemini-code-assist

Copy link
Copy Markdown

Summary of Changes

Hello, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request refactors the research workspace to be track-centric, establishing ResearchTrack as the primary aggregate for managing research state. It introduces new application services and a consolidated API endpoint to provide a consistent and stable view of a research track's context, including its tasks, milestones, memory, and feedback. This change streamlines data access for the frontend, making track-related operations more explicit and maintainable, and lays the groundwork for more robust track management features.

Highlights

  • New Track-Centric Read Model: Introduced a backend ResearchTrackContextService to consolidate research workspace state around the ResearchTrack aggregate root, providing a stable snapshot of track-related data.
  • Consolidated Track Context API Endpoint: Exposed a new GET /api/research/tracks/{track_id}/context endpoint that delivers a comprehensive snapshot including track details, tasks, milestones, memory summaries, feedback, and saved papers.
  • Dedicated Track Memory Service: Implemented a TrackMemoryService to centralize and abstract track-scoped memory access, mutations, and resolution logic, reducing boilerplate in route handlers.
  • Frontend Migration to New Endpoint: Migrated ResearchPageNew to consume the new consolidated track context endpoint and explicitly pass track_id for context builds, improving data consistency and reducing server-side activation state.
  • New Design Documentation: Added a new design document (docs/design/research-track-context-model.md) detailing the purpose, API surface, backend entry points, and contributor guidance for the track-centric research model.

🧠 New Feature in Public Preview: You can now enable Memory to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console.

Changelog
  • docs/design/research-track-context-model.md
    • Added a new design document outlining the purpose, stable track snapshot, track memory ownership, and contributor/test guidance for the research track context model.
  • src/paperbot/api/routes/research.py
    • Added imports for new application services and Pydantic models for track context.
    • Introduced helper functions _build_track_context_service and _build_track_memory_service for service instantiation.
    • Defined new Pydantic response models for TrackContextMemorySummaryResponse, TrackContextFeedbackSummaryResponse, TrackContextSavedPapersResponse, and TrackContextResponse.
    • Added a new _serialize_track_context_response function to convert service snapshots to API response models.
    • Implemented the GET /api/research/tracks/{track_id}/context endpoint to retrieve the consolidated track context.
    • Refactored _resolve_track_scope_id to delegate logic to TrackMemoryService.
    • Updated create_memory_item and suggest_memories to use the new track memory service for scope resolution and refined error messages.
    • Migrated list_memory_inbox, bulk_moderate, bulk_move, and clear_track_memory endpoints to utilize the TrackMemoryService for their core logic, simplifying route handlers and improving error handling.
  • src/paperbot/application/ports/init.py
    • Added new port interfaces FeedbackPort, MemoryPort, ResearchTrackReadPort, and TrackMemoryStorePort to the application layer's __all__ export.
  • src/paperbot/application/ports/research_track_read_port.py
    • Created a new ResearchTrackReadPort protocol defining read-only operations for track context aggregation, such as retrieving track details, tasks, milestones, feedback, saved papers, and evaluation summaries.
  • src/paperbot/application/ports/track_memory_store_port.py
    • Created a new TrackMemoryStorePort protocol for track-scoped memory store operations, including listing, retrieving by ID, searching, soft deleting by scope, and bulk updating memory items.
  • src/paperbot/application/services/init.py
    • Exported ResearchTrackContextService and TrackMemoryService from the application services module.
  • src/paperbot/application/services/research_track_context_service.py
    • Added a new ResearchTrackContextService to compose a track-scoped read model.
    • Defined dataclasses TrackContextQuery, TrackMemoryStats, TrackFeedbackSummary, TrackSavedPaperSummary, and TrackContextSnapshot for structuring track context data.
    • Implemented get_track_context method to aggregate data from ResearchTrackReadPort and MemoryPort into a TrackContextSnapshot.
  • src/paperbot/application/services/track_memory_service.py
    • Added a new TrackMemoryService to encapsulate track-scoped memory resolution and mutations.
    • Defined custom exceptions TrackMemoryScopeError and TrackMemoryValidationError for specific memory service failures.
    • Introduced dataclasses TrackMemoryScope, TrackMemoryClearResult, and TrackMemoryBulkResult for structured service responses.
    • Implemented methods for resolve_scope_id, list_inbox, clear_track_memory, bulk_moderate, and bulk_move to centralize memory management logic.
  • tests/integration/test_research_track_context_routes.py
    • Added new integration tests to verify the functionality of the GET /api/research/tracks/{track_id}/context endpoint, including correct data aggregation and 404 error handling for missing tracks.
  • tests/integration/test_research_track_memory_routes.py
    • Added new integration tests for track-scoped memory routes, specifically testing list_memory_inbox's use of active track scope and clear_track_memory's error handling for inaccessible tracks.
  • tests/integration/test_scope_and_acceptance_criteria_hooks.py
    • Updated the test client setup to ensure proper resetting of research_module singletons for improved test isolation.
  • tests/unit/test_research_context_route_explicit_track.py
    • Added new unit tests to confirm that the /api/research/context route correctly processes an explicit track_id without requiring track activation.
  • tests/unit/test_research_track_context_service.py
    • Added new unit tests for ResearchTrackContextService, verifying its ability to aggregate track-scoped reads and handle scenarios where a track is missing.
  • tests/unit/test_track_memory_service.py
    • Added new unit tests for TrackMemoryService, covering list_inbox functionality, clear_track_memory reporting, bulk_move validation, and require_track_scope error handling.
  • web/src/components/research/ResearchPageNew.tsx
    • Imported ResearchTrackContextPanel and ResearchTrackContextResponse type.
    • Added new state variables trackContext and trackContextLoading to manage the track context snapshot.
    • Modified the activeTrack memoized selector to prioritize data from trackContext.
    • Implemented refreshTrackContext function to fetch the latest track context from the new API endpoint.
    • Added a useEffect hook to automatically load and update trackContext when the activeTrackId changes.
    • Updated the build_context_pack API call to explicitly include track_id.
    • Integrated calls to refreshTrackContext after track update and clear memory operations to ensure UI consistency.
    • Rendered the new ResearchTrackContextPanel component to display the track context and added badges for loading status and pending memory items.
  • web/src/components/research/ResearchTrackContextPanel.tsx
    • Added a new React component ResearchTrackContextPanel to visually present the consolidated track context.
    • Implemented buildStatItems helper function to create compact statistical cards from the track context data.
    • Designed the panel to display track details, recent tasks, milestones, memory statistics (approved, pending, top tags), feedback summary, and evaluation coverage.
  • web/src/lib/types.ts
    • Added new TypeScript interfaces for ResearchTrackContextTask, ResearchTrackContextMilestone, ResearchTrackContextMemorySummary, ResearchTrackContextFeedbackItem, ResearchTrackContextFeedbackSummary, ResearchTrackSavedPaperPreview, ResearchTrackContextSavedPapersSummary, and ResearchTrackContextResponse to align with the new backend API models.
Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature Command Description
Code Review /gemini review Performs a code review for the current pull request in its current state.
Pull Request Summary /gemini summary Provides a summary of the current pull request in its current state.
Comment @gemini-code-assist Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help /gemini help Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for GitHub and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

  1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Refactors the research domain to treat ResearchTrack as the aggregate root by introducing a consolidated, track-centric context snapshot on the backend and migrating the web UI to consume it, while centralizing track-scoped memory operations behind an application service.

Changes:

  • Add ResearchTrackContextService read model plus GET /api/research/tracks/{track_id}/context endpoint and corresponding response types.
  • Introduce TrackMemoryService facade and update research memory routes to use it for scope resolution, inbox, bulk ops, and clear operations.
  • Update ResearchPageNew to fetch/render the consolidated track snapshot and pass explicit track_id into context builds; add unit/integration tests and design docs.

Reviewed changes

Copilot reviewed 18 out of 18 changed files in this pull request and generated no comments.

Show a summary per file
File Description
web/src/lib/types.ts Adds TypeScript types for the consolidated track context response contract.
web/src/components/research/ResearchTrackContextPanel.tsx New UI panel to render a “Track Snapshot” from the consolidated context payload.
web/src/components/research/ResearchTrackContextPanel.test.tsx Adds Vitest coverage for stat building and snapshot rendering.
web/src/components/research/ResearchPageNew.tsx Migrates to the consolidated track context endpoint and sends explicit track_id in context builds.
src/paperbot/application/services/research_track_context_service.py Implements the track-centric context read model and snapshot aggregation.
src/paperbot/application/services/track_memory_service.py Adds a track-scoped memory facade for resolution and mutations.
src/paperbot/application/services/init.py Exposes the new services via the package export list.
src/paperbot/application/ports/research_track_read_port.py Defines the read port used to assemble track context snapshots.
src/paperbot/application/ports/track_memory_store_port.py Defines the store port needed by track memory orchestration.
src/paperbot/application/ports/init.py Re-exports the newly introduced ports.
src/paperbot/api/routes/research.py Adds the consolidated track context endpoint and routes memory ops through TrackMemoryService.
tests/unit/test_research_track_context_service.py Unit tests for track context aggregation and missing-track behavior.
tests/integration/test_research_track_context_routes.py Integration coverage for the new context endpoint (200/404 cases).
tests/unit/test_research_context_route_explicit_track.py Ensures context builds pass explicit track_id without forcing activation.
tests/unit/test_track_memory_service.py Unit tests for track memory scope resolution and operations.
tests/integration/test_research_track_memory_routes.py Integration tests for track-scoped memory inbox/clear behaviors.
tests/integration/test_scope_and_acceptance_criteria_hooks.py Stabilizes tests by resetting route-level singletons/collectors.
docs/design/research-track-context-model.md Documents the track-centric research model, ownership boundaries, and test guidance.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

You can also share your feedback on Copilot code review. Take the survey.

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces a significant and well-executed refactoring by creating a track-centric research context stack. The introduction of ResearchTrackContextService and TrackMemoryService successfully encapsulates business logic, moving it out of the API route handlers and making the codebase more modular and maintainable. The new /api/research/tracks/{track_id}/context endpoint provides a clean, consolidated read model for the frontend, which has been effectively migrated in ResearchPageNew.tsx. The addition of application-layer ports (ResearchTrackReadPort, TrackMemoryStorePort) is a great use of dependency inversion. The accompanying documentation and comprehensive tests (both unit and integration) are excellent and make the changes easy to understand and verify.

I have one suggestion regarding performance optimization in the ResearchTrackContextService to ensure the new services scale well with growing data.

Comment on lines +148 to +192
def _build_memory_stats(
self,
*,
user_id: str,
track_id: int,
query: TrackContextQuery,
) -> TrackMemoryStats:
scope_id = str(track_id)
approved_items = self._memory_store.list_memories(
user_id=user_id,
limit=query.memory_scan_limit,
scope_type="track",
scope_id=scope_id,
status="approved",
include_pending=True,
include_deleted=False,
)
pending_items = self._memory_store.list_memories(
user_id=user_id,
limit=query.memory_scan_limit,
scope_type="track",
scope_id=scope_id,
status="pending",
include_pending=True,
include_deleted=False,
)
all_items = approved_items + pending_items
tag_counts: Counter[str] = Counter()
latest_memory_at: Optional[str] = None
for item in all_items:
for raw_tag in item.get("tags") or []:
tag = str(raw_tag).strip()
if tag:
tag_counts[tag] += 1
latest_memory_at = self._pick_latest_timestamp(
latest_memory_at,
str(item.get("updated_at") or item.get("created_at") or "") or None,
)
return TrackMemoryStats(
total_items=len(all_items),
approved_items=len(approved_items),
pending_items=len(pending_items),
top_tags=[tag for tag, _ in tag_counts.most_common(query.top_tag_limit)],
latest_memory_at=latest_memory_at,
)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

This method is inefficient as it fetches up to 1000 memory items (500 approved, 500 pending) from the database and then performs aggregations in Python. This can lead to significant performance degradation and memory usage as the number of memories per track increases.

A better approach would be to delegate the aggregation to the database. I suggest adding a new method to the MemoryPort and its implementation, such as get_memory_stats_for_scope, which would compute the required statistics (total_items, approved_items, pending_items, top_tags, latest_memory_at) using efficient SQL queries (e.g., COUNT, GROUP BY, MAX).

This same principle of database-side aggregation should also be applied to _build_feedback_summary and _build_saved_paper_summary for similar performance benefits.

…c-325

# Conflicts:
#	web/src/components/research/ResearchPageNew.tsx
@jerry609
jerry609 merged commit 4095876 into dev Mar 12, 2026
3 of 4 checks passed
@jerry609
jerry609 deleted the feat/track-context-epic-325 branch March 12, 2026 05:50
jerry609 added a commit that referenced this pull request Mar 13, 2026
* fix: validate studio output dirs and add p2c module design docs

:wq

* feat(PaperToContext): M3
#140
Closes #140

* feat(p2c): implement ContextEngineBridge to inject user context into extraction Related to #157

* activate paper-scope memory read/write path. Raletd to #158

* fix(p2c): address Gemini code review issues from PR. Related to #157

* fix(p2c): address Gemini code review issues. Related to #158

* fix(p2c): sanitize XML tag content to prevent tag-escape prompt injection

* feat(memory): add FTS5 full-text search and sqlite-vec hybrid search. Related to #161

* feat(p2c): persist CodeMemory experiences to SQLite. Related to #162

* address Gemini code review issues from PR #224 and #225

* fix(memory): address Gemini code review issues from #153 epic audit

P0 fixes:
- Replace unbounded daemon threads with ThreadPoolExecutor(max_workers=2)
  for embedding writes; atexit + close() ensure graceful shutdown so
  in-flight embeddings are not lost on process exit
- Restore apprise>=1.9.0 and feedgen>=1.0.0 removed in epic branch,
  which would have broken Epic #179 push/RSS features on merge

P1 fixes:
- _escape_fts: replace double-quote-only escaping with a whitelist regex
  [A-Za-z0-9_+-] so FTS5 operators (*, NEAR, NOT, ^) cannot alter query
  semantics; empty queries now short-circuit to []
- _hybrid_merge: skip items with None/invalid id instead of defaulting
  to id=0, preventing silent score collisions across unrelated records
- ReproExperienceStore: add application-level dedup check + UNIQUE
  constraint (paper_id, pattern_type, content) + IntegrityError fallback
  to prevent duplicate experiences from accumulating across retries

Migration:
- 0022_repro_experience_dedup: adds uq_repro_exp_paper_type_content
  unique constraint to existing repro_code_experience table

Tests: 47 unit tests pass (+3 new tests covering the fixes above)

* fix: harden repro experience isolation and wire persistence into repro pipeline

Add user-scoped isolation for repro_code_experience and enforce dedup semantics with migration 0022_repro_experience_dedup.

Inject ReproExperienceStore through ReproAgent/Orchestrator/CodingAgent/GenerationNode and propagate user_id/pack_id in generation, verification, and debugging persistence paths.

Update /api/gen-code to accept user_id and extend unit tests for user isolation and persistence behavior.

* feat(memory): introduce memory decay mechanism (#163)

Add decay-aware scoring that combines relevance (confidence), recency
(exponential decay with 90-day half-life), and usage frequency to
re-rank search results. New memories default to expires_at = created_at
+ 365 days. search_memories() now auto-touches usage on hits.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat(memory): add cross-track batch search (#164)

Add search_memories_batch() that queries multiple scope_ids in a single
SQL call, eliminating the N+1 loop in build_context_pack(). Engine now
uses this batch method for cross-track memory retrieval.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat(context): implement layered context loading (#165)

Refactor build_context_pack() into 4 layer methods:
- Layer 0: user profile (cached with 5-min TTL, ~200 tokens)
- Layer 1: track context — tasks/milestones (~500 tokens)
- Layer 2: query-relevant + cross-track memories (~1000 tokens)
- Layer 3: paper-scoped memories (on-demand)

Return value adds context_layers metadata while remaining fully
backward-compatible.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* refactor(memory): align decay with OpenClaw patterns

- Use standard ln(2)/halfLifeDays lambda formula (matches OpenClaw
  temporal-decay.ts:toDecayLambda)
- Default half-life lowered to 30 days (was 90, now matches OpenClaw)
- Evergreen memories (global scope, preference kind) are immune to
  recency decay (inspired by OpenClaw isEvergreenMemoryPath)
- Add _to_decay_lambda() and _is_evergreen_memory() helpers
- Expand tests for lambda math, half-life precision, and evergreen logic

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix(context): isolate layer0 cache and touch batch hits (#236)

* feat(memory): upgrade batch retrieval with hybrid and MMR options (#237)

* feat(context): add embedding fallback chain and token guard config (#238)

* fix(memory): bound batch hybrid candidates by scope_ids

* docs: update README with macOS python3 tips

* docs: update README with more badges

* docs: update README with deleting extra badges

* feat(search): add offline retrieval benchmark harness

Add a deterministic retrieval benchmark fixture, scorer, CLI, docs, and CI smoke gate for PaperSearchService.\n\nCloses #284\nRefs #283

* feat(context): add offline context-engine benchmark

Add deterministic fixtures, scoring, CLI, and smoke coverage for layered assembly, token guard, and advisory routing in ContextEngine.\n\nCloses #286\nRefs #283

* feat(memory): add scope isolation acceptance bench

Add an offline scope-isolation benchmark for memory retrieval paths, extend the metric collector with cross-user and cross-scope leak rates, and wire the new check into CI.\n\nCloses #285\nRefs #283

* feat(memory): add offline injection robustness detector

Add a deterministic prompt-injection pattern detector, labeled offline fixtures, an acceptance benchmark, and CI coverage for Injection Robustness L1.\n\nCloses #287\nRefs #283

* feat(memory): add offline performance benchmark harness

Add a deterministic synthetic benchmark for memory search latency baselines across 10k/100k/1M scales, plus docs and smokeable unit coverage.\n\nCloses #288\nRefs #283

* feat: add ROI benchmark for repro memory

Closes #289

Refs #283

* docs: add MemoryBench epic completion report

Refs #283

* docs: add runtime memory benchmark report

Refs #283

* Docs: update README with our new name (#290)

* docs: update README with macOS python3 tips

* docs: update README with more badges

* docs: update README with deleting extra badges

* docs: update README with new god name

---------

Co-authored-by: 林杰 <linjie@linjiedeMacBook-Air.local>
Co-authored-by: 林杰 <linjie@v8d1ef64c.dip.tu-dresden.de>
Co-authored-by: 林杰 <linjie@v8d1ef43e.dip.tu-dresden.de>
Co-authored-by: 林杰 <linjie@v8d1ef6c5.dip.tu-dresden.de>

* Fix: show friendly error when backend is unreachable (#291)

* docs: update README with macOS python3 tips

* docs: update README with more badges

* docs: update README with deleting extra badges

* docs: update README with new god name

---------

Co-authored-by: 林杰 <linjie@linjiedeMacBook-Air.local>
Co-authored-by: 林杰 <linjie@v8d1ef64c.dip.tu-dresden.de>
Co-authored-by: 林杰 <linjie@v8d1ef43e.dip.tu-dresden.de>
Co-authored-by: 林杰 <linjie@v8d1ef6c5.dip.tu-dresden.de>

* feat: improve memory ROI and effectiveness benchmarks

* fix: avoid importing missing data template in main

* fix: stabilize live memory roi benchmark

* feat: expand multi-session memory effectiveness benchmark

* feat: implement MemoryBench evaluation suite with 4 bench suites

Add comprehensive memory module evaluation aligned with LongMemEval
(ICLR 2025), LoCoMo (ACL 2024), Mem0, and Letta benchmarks.

- Retrieval Bench v2: IR metrics (Recall@5=0.873, MRR@10=0.731, nDCG@10=0.747)
  with 40 annotated queries across 5 question types and 5 memory dimensions
- Scope Isolation + CRUD: zero-leak verification across user x scope matrix,
  Mem0-aligned CRUD lifecycle (add/update/delete/dedup)
- Context Extraction: L0-L3 layer completeness, precision, token budget guard,
  TrackRouter accuracy (100%), graceful degradation
- Injection Robustness L1: offline pattern detection (0% pollution, 0% FP)
- Fixture dataset: 45 memories (2 users), 12 injection patterns
- Testing documentation with methodology, validity analysis, and results

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* docs: add MemoryBench quantitative results to README

Add evaluation results section with metrics from 4 bench suites:
- Retrieval quality (Recall@5=0.873, MRR@10=0.731, nDCG@10=0.747)
- Scope isolation + CRUD lifecycle (zero leaks, all CRUD pass)
- Context extraction (100% precision, 100% router accuracy)
- Injection robustness (0% pollution, 0% false positive)

Includes LoCoMo question-type breakdown and run instructions.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat(AgentSwarm): add Claude/Codex workflow, workspace persistence, and human review UX (#296)

* feat(AgentSwarm): add Claude/Codex workflow, workspace persistence, and human review UX

- 添加 /api/agent-board 路由和 Swarm Commander/Dispatcher 基础架构

- 将生成的任务文件持久化到已配置的工作区

- 在工作区中为每个任务生成用户审核文档(reviews/<任务>-user-review.md)

- 添加代理看板 UI,包含看板、任务详细日志和人工审核操作

- 要求“全部运行”任务设置工作区,并添加 VS Code 打开操作

- 实现 Studio 存储和 SSE 任务的更新/插入/同步行为

- 为路由流程、持久化和超时处理添加单元/功能测试

- 更新 Studio 布局、PostCSS 配置、后端 URL 助手和锁定文件

Closes #197

* (fix)修改AI review问题

* refactor: share research fetch helpers (#295)

Co-authored-by: 林杰 <linjie@v8d1ef6c5.dip.tu-dresden.de>

* feat: add research/yearCombobox. Realted to #297

* refactor(research): dedupe fetch helpers across Research pages; fix Radix popover import; add @radix-ui/react-popover dep

* fix(web): move @radix-ui/react-popover to dependencies

* fix(research): keep track list order stable across activation

* refactor(research): share stable track merge helper

* fix(research): make paper feedback togglable

* feat(research): hide 'Open Discovery Workspace' behind feature flag

* feat(research): gate track memory button behind feature flag (#333)

* feat(research): gate track memory button behind feature flag

* fix(research): avoid reactivating already active track

* ci(vercel): add auto deploy workflow for dev branch

* feat: calm dashboard workspace (#334)

* feat: harden search and connector infrastructure (#339)

* refactor(infra): unify async request layer for connectors

Implements issue #262 by moving Arxiv/OpenAlex/PapersCool connectors onto the shared async transport with retry support, updating async call sites, and adding focused tests.

* perf(infra): batch OpenAlex ID lookups

Implements issue #267 by replacing per-ID OpenAlex work fetches with batched filter queries and adds focused coverage for the batched path.

* refactor(infra): unify Semantic Scholar client stack

Implements issue #266 by routing both the shared mixin and the scholar-tracking agent through the same SemanticScholarClient surface, removing the duplicated legacy API client path, and adding focused unification tests.

* feat(infra): harden SSE transport handling

Implements issue #263 by moving heartbeat, timeout, cancellation cleanup, and X-Accel-Buffering handling into the shared SSE wrapper and switching the streaming routes onto the common response helper with regression coverage.

* fix(infra): bound ARQ jobs and reuse event log

Implements issue #268 by adding timeout/max_tries metadata to worker functions, reusing the SqlAlchemyEventLog singleton inside the worker module, and covering both behaviors with focused tests.

* feat(search): add three-tier paper deduplicator (DOI/arxiv_id/rapidfuzz)

Replaces the identity-key-only dedup in PaperSearchService._fuse_with_rrf()
with a dedicated PaperDeduplicator that matches across DOI, arxiv_id (version-
stripped), and fuzzy title similarity via rapidfuzz. Merged papers accumulate
the best metadata (highest citations, longest abstract, union of identities).

Closes #317

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix(search): prevent conflicting identity dedup merges

Refs #317

* fix(embeddings): add CJK character support to hash embedding tokenizer

Extends HashEmbeddingProvider regex to include CJK Unified Ideographs
(U+4E00–U+9FFF) and Extension A (U+3400–U+4DBF), enabling proper
embedding of Chinese text.

Closes #276

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix(embeddings): extend hash tokenizer beyond Han-only CJK

Refs #276

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>

* fix(research): restore feedback track ux on current dev (#341)

* refactor(research): dedupe fetch helpers across Research pages; fix Radix popover import; add @radix-ui/react-popover dep

* fix(web): preserve item order in research tracks selection

Closes #304

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* refactor(web): remove unused components

Closes #305

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix(web): keep active research track visible

Refs #304

* fix(research): align paper feedback toggles with persisted state

Refs #324

* fix(web): add missing @radix-ui/react-popover dependency

Research page crashed with "Module not found: Can't resolve
'@radix-ui/react-popover'" because SearchBox.tsx imports popover.tsx
which depends on this package.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix(research): restore feedback track ux on current dev

---------

Co-authored-by: 林杰 <linjie@v8d1ef6a5.dip.tu-dresden.de>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>

* fix(web): resolve DeepCode Studio page issues

* fix(repro): improve memory and workflow correctness

* feat: add intelligence radar backend

* fix(api): block cross-user intelligence feed access

* fix(intelligence): avoid thread-unsafe refresh races

* refactor: enforce alembic-only store initialization

* fix(db): bootstrap sqlite schemas for lazy stores

* fix(stores): align ports with store interfaces

* fix(stores): align remaining port contracts

* fix(api): harden request boundaries and settings handling

* fix(api): stop trusting spoofable forwarded hosts

* fix(api): gate studio code execution by default

* fix(api): stream request-size checks safely

* feat(export): add Obsidian filesystem exporter

* feat(cli): add Obsidian export command

* feat(obsidian): finish configurable vault export workflow (#348)

* refactor(web): move workflow workbench out of dashboard

* ci(vercel): use --preview flag for dev deploy

* fix(ci): remove invalid Vercel preview flag

* fix(ci): keep Vercel secrets free of inline comments

* refactor: share candidate search boundary and simplify dashboard

* refactor(api): split paperscool search curate and ingest

* refactor(web): flatten dashboard brief snapshot

* refactor(web): reduce dashboard brief visual noise

* refactor(web): feature top dashboard signals

* fix(obsidian): harden export follow-ups

* fix(obsidian): tighten review follow-ups

* feat(web): add Obsidian handoff to Research workspace (#352)

* feat(web): add Obsidian handoff to Research workspace

* fix(web): address obsidian workspace review feedback

* feat: ship track-centric research context stack (#356)

* feat(api): add track context read model service

* feat(api): expose consolidated track context endpoint

* refactor(web): migrate research page to track context endpoint

* refactor(memory): wrap track-scoped memory access

* chore: document track-centric research model and stabilize tests

* docs: refresh readme screenshots

* fix(deps): align package manifests with runtime imports

* docs: refresh email push screenshot

* feat(research): close citation graph and obsidian export gaps (#360)

* feat(obsidian): add bidirectional vault sync

* fix(paper): improve saved papers table layout

* feat(openclaw): add paperbot plugin bridge

* fix(obsidian): address sync review feedback

* feat(web): flatten dashboard action bands

* refactor(web): condense dashboard next-up panel

* refactor(web): simplify dashboard around recommendations

* refactor(web): simplify dashboard surface and workflow copy

* feat(web): add dashboard queue actions

* fix(web): harden dashboard queue links and brief parsing

* fix(paper): unify unsave with feedback API

* fix(paper): scope saved papers per track

* fix: unblock dashboard build and e2e

* ci: add vercel pr preview automation

* fix: repair vercel preview workflow setup

* fix: run vercel build from repo root

* fix: skip preview smoke without bypass secret

* feat(auth): add multi-user authentication foundation. Related to #151

- Add User domain model and SQLAlchemy UserModel with soft-delete support
- Add SqlAlchemyUserStore with email/GitHub user CRUD, password auth, reset tokens
- Add JWT signing/verification (python-jose), bcrypt password hashing
- Add FastAPI auth dependencies: get_user_id (optional fallback) and get_current_user (strict)
- Add /api/auth routes: register, login, github/exchange, me, forgot/reset-password
- Add Alembic migrations for users and password_reset_tokens tables
- Add AUTH_OPTIONAL env var for gradual migration from legacy 'default' user
- Fix account lifecycle bugs: reactivate on OAuth re-login, reject inactive on password login
- Add auth API tests

* fix(auth): address code review feedback on PR #365

* feat(document): add explicit evidence indexing pipeline

* docs(benchmark): define document evidence eval contract

* feat(benchmark): add document evidence eval scaffold

* fix(api): restore py39 auth compatibility after dev rebase

* chore(logging): surface cleanup and FTS5 failures

* fix(paper): refine saved filters and cleanup code

* fix(paper): refine paper context cleanup and year filter

* feat(eval): support dedicated embedding benchmark providers

* feat(settings): add embedding endpoint configuration

* refactor(settings): align embedding endpoint ux with cc-switch

* refactor(settings): simplify embedding endpoint panel

* refactor(settings): tighten embedding layout on wide screens

* feat(studio): refine paper gallery icon animation and context
  workspace layout

* ci: disable native vercel git deploys

* docs: restore master demo gallery assets

* fix: unblock ci for merge-dev-into-master

* fix: address codeql alerts

* fix: harden agent board workspace path validation

---------

Co-authored-by: boyu <oor2020@163.com>
Co-authored-by: WenjingWang <jingnvx@outlook.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: 林杰 <linjie@linjiedeMacBook-Air.local>
Co-authored-by: 林杰 <linjie@v8d1ef64c.dip.tu-dresden.de>
Co-authored-by: Linjie-top <linjie666z@gmail.com>
Co-authored-by: 林杰 <linjie@v8d1ef43e.dip.tu-dresden.de>
Co-authored-by: 林杰 <linjie@v8d1ef6c5.dip.tu-dresden.de>
Co-authored-by: 林杰 <linjie@v8d1ef6a5.dip.tu-dresden.de>
Co-authored-by: 林杰 <linjie@v8d1ef4ea.dip.tu-dresden.de>
Co-authored-by: 林杰 <linjie@v8d1ef41f.dip.tu-dresden.de>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants