Conversation
* feat(push): implement Epic #179 — daily push optimization Multi-channel push infrastructure with structured digest cards: - #187: Integrate HF upvotes into Judge scoring prompt - #180: Add digest_card extraction (highlight/method/finding/tags) with new LLM prompt, email template rendering, markdown output - #181: MinerU Cloud API client for PDF figure extraction - #182: Apprise multi-channel push layer with YAML config - #183-185: Channel formatters (Telegram MarkdownV2, Discord Rich Embed, WeCom markdown+news, Feishu/Lark interactive card) - #186: RSS 2.0 + Atom feed endpoints (/api/feed/daily.xml, .atom) Also creates follow-up issues #190 (Push Channel Settings UI) and #191 (Dashboard Digest Card Enhancement). 56 new tests, all passing. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat: integrate MinerU figure extraction into CLI and ARQ worker daily-paper workflow Refs #179 * fix(daily-push): enable MinerU flags in ARQ cron payload (refs #181) * feat(push): wire channel formatters into apprise digest delivery (refs #182 #183 #184 #185) * feat(feed): add keyword RSS, feed cache, and figure enclosure support (refs #186) * feat(mineru): add extraction result cache for daily push figure pipeline (refs #181) * feat(push): add telegram bot command subscription endpoint (refs #191) * fix(push): add retry/backoff for channel delivery failures (refs #191) * feat(hf): add trending modes, cache, and arxiv identity dedupe (refs #193) * feat(push): add idempotency and error-code mapping for channel delivery (refs #191) * test(push): add mock channel e2e and failure-injection coverage (refs #191) * docs(push): add audit script and ops runbook for acceptance closure (refs #192) * docs: update README for AgentSwarm execution transition * docs: archive stale docs and rename AgentSwarm todo * docs: add AgentSwarm design proposal + web Playwright e2e setup - Add docs/proposals/agentswarm-design.md covering multi-agent orchestration architecture, adapter pattern, session management, skills ecosystem, and OpenCode/OpenClaw/oMo integration (refs #197, #214) - Add Playwright e2e test scaffolding for web dashboard (smoke, papers, research specs) - Update web/.gitignore for Playwright artifacts Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
…LI chat errors (#223) * fix(studio): fix context pack generation, duplicate import UX, and CLI chat errors - 支持内联标题/摘要，以绕过 PaperInputRouter 获取 studio 论文 ID - 使上下文包生成 SSE 流期间的数据库持久化操作不再致命 - 从库中导入重复论文时显示错误消息 - 将左侧文件面板宽度缩小至约 18%（默认宽度）；移除默认模板文件 - 为 Claude CLI 添加 --verbose 标志，以兼容 stream-json 输出 Related to #222 * fix(studio): fix context pack generation, duplicate import UX, and CLI chat errors - 修改Gemini AI review代码问题 * fix(studio): fix context pack generation, duplicate import UX, and CLI chat errors - 修改Gemini AI review代码问题 --------- Co-authored-by: Jerry Zhang <1772030600@qq.com>
* fix(mineru): support v4 async task extraction flow * refactor(mineru): standardize on v4 task API for daily push (#179)
- Add centered hero header with badges (CI, Roadmap, License, Python, Next.js) - Replace verbose 18-row feature table with categorized bullet lists - Move Roadmap Phase 1-6 content to pinned issue #232 - Move module maturity matrix to Roadmap #232 - Consolidate screenshots into collapsible sections - Remove inline API endpoint table (40+ rows) and directory tree - Add Contributing section with Roadmap link - Streamline Getting Started with collapsible config details - Add repo topics for GitHub discoverability References: vLLM, Docling, PaperQA2, R2R README patterns Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Replace ASCII architecture diagram in README with arc.png image - Add TODO comment for future high-res diagram redraw - Update architecture.excalidraw with academic color scheme (blue-gray + gold Domain accent) - Add editable source links for Excalidraw and draw.io Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
|
Caution Review failedPull request was closed or merged during review 📝 WalkthroughWalkthroughThis PR introduces comprehensive enhancements across the PaperBot stack: MinerU-based figure extraction with async task polling and inline image support, hardened path validation for CLI safety, redesigned Studio UI with gallery/workspace modes and per-paper state caching, AgentSwarm design proposal documentation, new E2E and unit test infrastructure (Playwright/Vitest), and README restructuring to emphasize English-centric marketing layout. Multiple interconnected features enable context-pack-driven studio workflows with event-based streaming. Changes
Estimated code review effort🎯 4 (Complex) | ⏱️ ~60 minutes Possibly related issues
Possibly related PRs
Suggested reviewers
Poem
🚥 Pre-merge checks | ✅ 2 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (2 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Summary of ChangesHello, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed! This pull request merges the Highlights
🧠 New Feature in Public Preview: You can now enable Memory to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console. Changelog
Activity
Using Gemini Code AssistThe full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips. Invoking Gemini You can request assistance from Gemini at any point by creating a comment using either
Customization To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a Limitations & Feedback Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here. You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension. Footnotes
|
Resolve README conflict by using dev's version with arc.png image. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
There was a problem hiding this comment.
Pull request overview
This PR merges 13 commits from dev to master, landing three major feature areas: (1) security hardening of path validation in the studio chat and runbook APIs, (2) completion of the Daily Push Epic #179 (MinerU figure extraction, email figure rendering, push channels), and (3) Studio UX improvements (gallery view, per-paper state caching, duplicate import detection, Claude CLI stream-JSON integration).
Changes:
- MinerU Cloud API client reworked to use the v4 async task API (poll-until-done), with SSRF-safe URL validation and inline data URL support for zip-bundled figures
- Studio frontend restructured:
PapersPanelreplaced by aPaperGallerygallery/workspace split, per-paper Zustand state preserved on paper switches, streaming Claude CLI output parsed as structured NDJSON events - Security: path traversal protection hardened with
os.path.realpath-based symlink-safe checks in bothrunbook.pyandstudio_chat.py
Reviewed changes
Copilot reviewed 2 out of 3 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
web/src/lib/store/studio-store.ts |
Adds per-paper state cache and appendToLastAction for streaming text |
web/src/lib/store/studio-store.test.ts |
Unit tests for state cache round-trip and text streaming |
web/src/lib/runbook/deleteProjectFiles.ts |
New helper: lists then deletes files one-by-one via N+1 API calls |
web/src/hooks/useContextPackGeneration.ts |
Passes inline title/abstract to context-pack endpoint |
web/src/components/studio/ReproductionLog.tsx |
Handles structured CLI SSE events (tool_use, tool_result, thinking) |
web/src/components/studio/PapersPanel.tsx |
Delegates deletion to deleteProjectFiles |
web/src/components/studio/PaperIcon.tsx |
New deterministic identicon SVG component |
web/src/components/studio/PaperGallery.tsx |
New gallery view with search, delete confirmation, and duplicated statusConfig |
web/src/components/studio/NewPaperModal.tsx |
Duplicate detection changed from ID-based to title-based |
web/src/app/studio/page.tsx |
Gallery/workspace split, duplicated statusConfig, removes PapersPanel |
web/vitest.config.ts |
New Vitest config |
web/playwright.config.ts |
New Playwright E2E config |
web/package.json / package-lock.json |
Adds Vitest and Playwright devDeps |
web/e2e/*.spec.ts |
Smoke and feature E2E tests |
src/paperbot/infrastructure/extractors/mineru_client.py |
Full v4 async task API implementation with zip extraction, inline data URLs |
src/paperbot/application/workflows/dailypaper.py |
Adds _is_publishable_figure_url guard and passes new MinerU params |
src/paperbot/application/services/email_template.py |
Renders main_figure block (inline data URL or HTTP URL) in email HTML |
src/paperbot/api/routes/studio_chat.py |
Path hardening, context-pack injection, stream-JSON CLI integration |
src/paperbot/api/routes/runbook.py |
Symlink-safe os.path.realpath fix in add_allowed_dir |
src/paperbot/api/routes/repro_context.py |
Inline paper data bypass for input router; persistence errors no longer fatal |
src/paperbot/infrastructure/queue/arq_worker.py |
Forwards MinerU config to job queue |
src/paperbot/presentation/cli/main.py |
Adds --mineru-api-base-url, --mineru-model-version, --mineru-max-wait-seconds flags |
tests/unit/test_mineru_client.py |
New URL validation and markdown/zip parsing tests |
tests/unit/test_dailypaper.py |
New figure filter and inline figure tests |
tests/unit/test_daily_digest_card.py |
New email rendering tests for figure block |
tests/unit/test_arq_daily_papers.py |
Verifies new MinerU env vars are forwarded to job kwargs |
tests/unit/test_studio_chat_path_validation.py |
Tests _resolve_cli_project_dir security logic |
README.md |
Complete rewrite: English, condensed, references arc.png (not in diff) |
env.example |
Adds MinerU and daily-push environment variables |
asset/architecture.excalidraw |
Updated color scheme and layout |
docs/proposals/agentswarm-design.md |
New 779-line AgentSwarm architecture proposal |
web/src/lib/store/project-context.ts |
Removes hardcoded default files |
.gitignore |
Adds .env.* and Playwright report dirs |
web/.gitignore |
Adds Playwright output dirs |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
You can also share your feedback on Copilot code review. Take the survey.
| <details> | ||
| <summary>Data pipeline</summary> | ||
| <!-- TODO: 待重绘高清架构图 --> | ||
|  |
There was a problem hiding this comment.
The README references asset/arc.png as the architecture image (), but this file does not appear in the diff — there is no arc.png added to the asset/ directory. The PR description explicitly states "Verify arc.png architecture image displays" as a checklist item, suggesting this file should have been committed. If arc.png is missing from the repository, the architecture section in the README will show a broken image on GitHub.
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request introduces significant features and improvements, including a major README overhaul, security hardening, and enhancements to the Studio and daily push functionalities, alongside new features like a Paper Gallery, enhanced figure extraction using MinerU, and context pack generation. While the code includes some security hardening measures such as path traversal protections in the Runbook API, it lacks a comprehensive authentication and authorization framework. This results in multiple Insecure Direct Object Reference (IDOR) and Broken Access Control vulnerabilities across the newly introduced API endpoints. Additionally, a weak URL validation logic in the MinerU client facilitates Blind SSRF. Beyond these critical security concerns, there are also areas for improvement concerning code duplication of security-critical logic and potential robustness enhancements in path validation and error handling. Addressing these points, especially implementing robust ownership checks and stricter input validation, will further strengthen the maintainability and security of the codebase.
I am having trouble creating individual review comments. Click here to see my feedback.
src/paperbot/infrastructure/extractors/mineru_client.py (134-144)
The _validate_source_url method only checks if the URL scheme is http or https. It does not prevent requests to private IP ranges (e.g., 127.0.0.1, 169.254.169.254) or internal hostnames. Since these URLs are sent to the MinerU Cloud API for processing, this facilitates a Blind SSRF vulnerability where an attacker can use the MinerU service to probe internal infrastructure.
Remediation: Implement a strict allow-list of trusted domains or use a dedicated library to validate that the provided URL does not resolve to a private or reserved IP address.
src/paperbot/api/routes/studio_chat.py (133-231)
This block of code for path validation (_load_runtime_allowed_dirs, _allowed_workdir_prefixes, _is_under_prefix, _resolve_cli_project_dir) is nearly identical to the logic in src/paperbot/api/routes/runbook.py. Duplicating security-critical code is risky as it can lead to inconsistencies if one file is updated but the other is not.
To improve maintainability and ensure consistent security, this logic should be centralized into a shared utility module (e.g., in src/paperbot/api/utils/). Both studio_chat.py and runbook.py can then import and use these shared functions.
src/paperbot/api/routes/repro_context.py (72-88)
While wrapping the database call in a try...except block improves resilience, catching a generic Exception can mask underlying issues that should be addressed, such as TypeError from incorrect data. This makes debugging more difficult.
Consider catching more specific exceptions related to database operations (e.g., sqlalchemy.exc.OperationalError). This would provide clearer insight into failure modes while still allowing the application to handle expected errors gracefully.
Summary
Commits included
13 commits from
devsince last merge tomaster.Test plan
🤖 Generated with Claude Code
Summary by CodeRabbit
Release Notes
New Features
Documentation
Tests
Chores