feat: deliver daily push epic #179 core pipeline and channel integration#190
Conversation
Multi-channel push infrastructure with structured digest cards: - #187: Integrate HF upvotes into Judge scoring prompt - #180: Add digest_card extraction (highlight/method/finding/tags) with new LLM prompt, email template rendering, markdown output - #181: MinerU Cloud API client for PDF figure extraction - #182: Apprise multi-channel push layer with YAML config - #183-185: Channel formatters (Telegram MarkdownV2, Discord Rich Embed, WeCom markdown+news, Feishu/Lark interactive card) - #186: RSS 2.0 + Atom feed endpoints (/api/feed/daily.xml, .atom) Also creates follow-up issues #190 (Push Channel Settings UI) and #191 (Dashboard Digest Card Enhancement). 56 new tests, all passing. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…y-paper workflow Refs #179
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
|
Caution Review failedThe pull request is closed. ℹ️ Recent review infoConfiguration used: Path: .coderabbit.yaml Review profile: CHILL Plan: Pro 📒 Files selected for processing (34)
📝 WalkthroughWalkthroughThis pull request introduces comprehensive multi-channel push notification infrastructure for DailyPaper digests. It adds RSS/Atom feed generation endpoints, integrates MinerU Cloud API for PDF figure extraction, implements channel-specific formatters (Telegram, Discord, WeCom, Feishu), and enriches reports with LLM-generated daily digest cards via Apprise library. Changes
Sequence Diagram(s)sequenceDiagram
participant Client
participant DailyPushService
participant LLMService
participant MineruClient
participant Formatters
participant AppriseNotifier
participant ExternalAPI as External APIs<br/>(Telegram/Discord/etc)
Client->>DailyPushService: push_dailypaper(report)
loop For each top_item
DailyPushService->>LLMService: extract_daily_digest_card(title, abstract)
LLMService-->>DailyPushService: {highlight, method, finding, tags}
end
alt If figures enabled
DailyPushService->>MineruClient: extract_figures(pdf_url)
MineruClient-->>DailyPushService: [figures + main_figure]
end
DailyPushService->>DailyPushService: channel == "apprise"
DailyPushService->>AppriseNotifier: push_daily_digest(report, markdown, html)
loop For each configured channel URL
AppriseNotifier->>Formatters: format_digest(report, channel_type)
Formatters-->>AppriseNotifier: {formatted_payload}
AppriseNotifier->>ExternalAPI: POST notification
ExternalAPI-->>AppriseNotifier: {status}
end
AppriseNotifier-->>DailyPushService: {results}
DailyPushService-->>Client: {ok, channels_notified}
Estimated code review effort🎯 4 (Complex) | ⏱️ ~60 minutes Poem
✨ Finishing Touches
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Summary of ChangesHello, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed! This pull request delivers the foundational infrastructure for an enhanced daily paper digest system. It significantly improves the content richness of the digests by extracting key insights and figures, expands the delivery capabilities through multi-channel push notifications, and introduces flexible RSS/Atom feed generation. These changes aim to provide users with more comprehensive and accessible research paper summaries across various platforms. Highlights
🧠 New Feature in Public Preview: You can now enable Memory to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console. Changelog
Activity
Using Gemini Code AssistThe full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips. Invoking Gemini You can request assistance from Gemini at any point by creating a comment using either
Customization To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a Limitations & Feedback Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here. You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension. Footnotes
|
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request introduces several new features and improvements, including multi-channel push notifications via Apprise, RSS/Atom feed generation, and MinerU figure extraction. It also includes a daily digest card prompt for LLM-based summarization. Review comments highlight potential security vulnerabilities with an unbounded in-memory cache, suggest using a more reliable deduplication key, and recommend refactoring the push notification logic to preserve rich formatting. Additional feedback focuses on improving error handling, code organization, and cache loading.
| router = APIRouter() | ||
|
|
||
| _REPORTS_DIR = Path("./reports/dailypaper") | ||
| _FEED_CACHE: Dict[str, Dict[str, Any]] = {} |
There was a problem hiding this comment.
The _FEED_CACHE dictionary is an unbounded in-memory cache. Since cache keys are derived from user-supplied path parameters (track_name and keyword), an attacker can send a large number of requests with unique values to exhaust the server's memory, leading to a Denial of Service (DoS).
Recommendation: Use a cache with a maximum size and an eviction policy (e.g., functools.lru_cache or cachetools.LRUCache).
| papers: List[Dict[str, Any]] = [] | ||
| for q in report.get("queries") or []: | ||
| for item in q.get("top_items") or []: | ||
| key = item.get("title") or id(item) |
There was a problem hiding this comment.
Using id(item) as a fallback key for deduplication is not reliable. The id() will be different each time the report is loaded from a file, and it will also be different for two separate dictionary objects even if they represent the same paper. This will cause deduplication to fail. A more stable unique identifier, like the paper's URL, should be used as a primary or secondary key.
| key = item.get("title") or id(item) | |
| key = item.get("url") or item.get("title") |
| def _build_channel_body( | ||
| self, | ||
| url: str, | ||
| *, | ||
| report: Dict[str, Any], | ||
| fallback_html: str, | ||
| fallback_markdown: str, | ||
| subject: str, | ||
| ) -> tuple[str, str]: | ||
| channel_type = self._channel_type_from_url(url) | ||
| payload = self._format_payload(channel_type, report) | ||
|
|
||
| if payload: | ||
| body, body_format = self._payload_to_body(channel_type, payload) | ||
| if body: | ||
| return body, body_format | ||
|
|
||
| if fallback_html: | ||
| return fallback_html, "html" | ||
| if fallback_markdown: | ||
| return fallback_markdown, "markdown" | ||
| return f"New daily digest: {subject}", "text" |
There was a problem hiding this comment.
This implementation degrades the rich, structured payloads generated by the formatters into simple markdown strings before sending. For example, it converts Discord embeds into plain markdown and ignores Telegram's inline_keyboard and photo_url fields. This defeats the purpose of having channel-specific formatters that create native rich messages.
To fix this, push_daily_digest should be refactored to inspect the formatter's payload and pass the rich content to Apprise correctly. Apprise's notify method is flexible and can often handle richer content when the right parameters are used, or you may need to construct the notification body differently for each channel type to preserve the rich formatting.
| # | ||
| # Examples (uncomment and fill in credentials): | ||
|
|
||
| channels: [] |
There was a problem hiding this comment.
The channels: [] syntax can be confusing for users. When they uncomment the example channels, they will also need to remember to remove the [] to make the YAML valid. Using channels: (with a null value) would be more intuitive, as it works correctly for both an empty configuration and when channels are added.
channels:| except Exception: | ||
| continue |
There was a problem hiding this comment.
Silently ignoring all exceptions with except Exception: continue can hide important issues, such as file permission errors or malformed JSON that is not a JSONDecodeError. It would be more robust to catch specific expected exceptions and log a warning for any failures to aid in debugging.
| except Exception: | |
| continue | |
| except (json.JSONDecodeError, IOError) as e: | |
| logger.warning("Failed to load or parse report %s: %s", path, e) | |
| continue |
| try: | ||
| fe.enclosure(url=fig_url, length=0, type="image/jpeg") | ||
| except Exception: | ||
| pass | ||
|
|
||
| # Publication date | ||
| if generated_at: | ||
| try: | ||
| dt = datetime.fromisoformat( | ||
| generated_at.replace("Z", "+00:00") | ||
| ) | ||
| if dt.tzinfo is None: | ||
| dt = dt.replace(tzinfo=timezone.utc) | ||
| fe.pubDate(dt) | ||
| except Exception: | ||
| pass |
There was a problem hiding this comment.
The broad except Exception: pass statements silently ignore potential errors during the generation of the RSS feed, for example when adding an enclosure or parsing a publication date. This can lead to incomplete feed entries without any indication of what went wrong. It's better to log these exceptions as warnings to help with debugging.
| import xml.sax.saxutils as saxutils | ||
|
|
||
| t = saxutils.escape(paper_title) | ||
| d = saxutils.escape(snippet[:500]) | ||
| u = saxutils.escape(url) | ||
| items.append( | ||
| f"<item><title>{t}</title>" | ||
| f"<link>{u}</link>" | ||
| f"<description>{d}</description>" | ||
| f"</item>" | ||
| ) | ||
|
|
||
| import xml.sax.saxutils as saxutils |
There was a problem hiding this comment.
The xml.sax.saxutils module is imported multiple times within the _fallback_rss_xml function. According to PEP 8, imports should be at the top of the file. This improves readability and avoids the small overhead of re-importing within a function that might be called multiple times.
| except Exception: | ||
| return None | ||
| return None |
There was a problem hiding this comment.
While it's reasonable for a cache-loading failure to not crash the application, silently returning None can hide persistent problems with the cache (e.g., permission issues, disk full, corrupted files). Logging a warning here would be beneficial for debugging without changing the graceful failure behavior.
| except Exception: | |
| return None | |
| return None | |
| except Exception as e: | |
| logger.warning("Failed to load figures from cache at %s: %s", cache_path, e) | |
| return None |
There was a problem hiding this comment.
Pull request overview
Implements the core delivery for Epic #179 by enhancing DailyPaper digest content (digest-card fields + HF upvotes in Judge prompt), adding MinerU-based figure extraction, introducing an Apprise-backed multi-channel push layer with per-channel formatters, and exposing RSS/Atom feed endpoints with lightweight caching.
Changes:
- Add
digest_cardLLM extraction (highlight/method/finding/tags) and render it in Markdown + email (HTML/text); include HF upvotes in Judge prompt context. - Integrate MinerU Cloud API figure extraction (with main-figure heuristic) and forward figure flags through CLI + ARQ cron/job.
- Add Apprise notifier + formatter registry (Telegram/Discord/WeCom/Feishu) and add RSS/Atom feed routes (daily/track/keyword) with caching + enclosures.
Reviewed changes
Copilot reviewed 33 out of 34 changed files in this pull request and generated 8 comments.
Show a summary per file
| File | Description |
|---|---|
| tests/unit/test_push_formatters.py | Adds unit coverage for formatter registry and per-channel digest formatting. |
| tests/unit/test_mineru_client.py | Adds tests for MinerU client parsing, caching, and main-figure selection + pipeline hook. |
| tests/unit/test_hf_upvote_judge_integration.py | Verifies HF upvotes are conditionally included in Judge prompt. |
| tests/unit/test_feed_routes.py | Unit-tests feed helpers and RSS/Atom XML generation behavior. |
| tests/unit/test_daily_digest_card.py | Tests digest-card prompt registration, pipeline enrichment, and email/markdown rendering. |
| tests/unit/test_arq_daily_papers.py | Ensures ARQ cron forwards figure flags to the daily job. |
| tests/unit/test_apprise_notifier.py | Tests YAML config parsing and AppriseNotifier digest push path behavior. |
| src/paperbot/workflows/feed.py | Adds export_feed_entries() for converting feed events into feedgen-ready dicts. |
| src/paperbot/presentation/cli/main.py | Adds CLI flags/env support to enable MinerU figure extraction. |
| src/paperbot/infrastructure/queue/arq_worker.py | Adds figure flags to cron/job args and calls extract_figures_for_report() when enabled. |
| src/paperbot/infrastructure/push/formatters/wecom.py | Implements WeCom digest formatting (markdown + news articles). |
| src/paperbot/infrastructure/push/formatters/telegram.py | Implements Telegram MarkdownV2 digest formatting + inline keyboard + optional main figure. |
| src/paperbot/infrastructure/push/formatters/feishu.py | Implements Feishu/Lark interactive card + post digest formatting. |
| src/paperbot/infrastructure/push/formatters/discord.py | Implements Discord embed-style digest formatting. |
| src/paperbot/infrastructure/push/formatters/base.py | Introduces PushFormatter base + shared paper collection/sorting helpers. |
| src/paperbot/infrastructure/push/formatters/init.py | Adds formatter registry and lookup helpers. |
| src/paperbot/infrastructure/push/apprise_notifier.py | Adds Apprise-based notifier with YAML config and per-channel formatting fallback logic. |
| src/paperbot/infrastructure/push/init.py | Exposes AppriseNotifier at package level. |
| src/paperbot/infrastructure/extractors/mineru_client.py | Adds MinerU Cloud API client with cache + main-figure heuristic. |
| src/paperbot/infrastructure/extractors/init.py | Exports MineruClient/Figure symbols. |
| src/paperbot/application/workflows/dailypaper.py | Adds digest_card feature and extract_figures_for_report() pipeline hook. |
| src/paperbot/application/workflows/analysis/judge_prompts.py | Injects HF upvotes into Judge user prompt when present. |
| src/paperbot/application/services/llm_service.py | Adds extract_daily_digest_card() using a new prompt template and safe JSON parsing. |
| src/paperbot/application/services/email_template.py | Renders digest-card fields into email HTML and text output. |
| src/paperbot/application/services/daily_push_service.py | Adds apprise as a push channel and wires AppriseNotifier into digest send flow. |
| src/paperbot/application/prompts/registry.py | Registers the new daily_digest_card prompt template. |
| src/paperbot/application/prompts/paper_analysis.py | Adds system/user prompts for digest-card extraction. |
| src/paperbot/api/routes/feed.py | Adds RSS/Atom endpoints (daily/track/keyword) with helpers + caching + enclosures. |
| src/paperbot/api/main.py | Registers the feed router under /api. |
| requirements.txt | Adds dependencies for Apprise and feedgen. |
| docs/progress/progress.txt | Updates progress log with completed items and validation notes. |
| docs/issues/191-dashboard-digest-card-enhancement.md | Adds follow-up dashboard enhancement issue for digest-card/feed UI. |
| docs/issues/190-settings-push-channel-ui.md | Adds follow-up issue for push channel configuration UI. |
| config/push_channels.yaml | Adds Apprise push channel YAML template/config stub. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| _REPORTS_DIR = Path("./reports/dailypaper") | ||
| _FEED_CACHE: Dict[str, Dict[str, Any]] = {} |
There was a problem hiding this comment.
_FEED_CACHE is an unbounded global dict keyed by user-controlled path params (track_name, keyword). Over time (or via crafted requests) this can grow without limit and increase memory usage. Consider adding an LRU/size cap and/or TTL eviction, and optionally normalizing/limiting the length of cache keys.
| return cls(urls=[]) | ||
|
|
||
| with open(config_path, encoding="utf-8") as fh: | ||
| data = yaml.safe_load(fh) or {} |
There was a problem hiding this comment.
from_yaml() doesn't handle YAML parse errors (e.g., malformed config). yaml.safe_load() can raise and would crash daily push at runtime. Consider wrapping the load in a try/except (yaml.YAMLError/Exception) and returning an empty notifier with a warning log, similar to the missing-file path.
| data = yaml.safe_load(fh) or {} | |
| try: | |
| data = yaml.safe_load(fh) or {} | |
| except yaml.YAMLError as exc: | |
| logger.warning("Failed to parse push config YAML %s: %s", path, exc) | |
| return cls(urls=[]) | |
| except Exception as exc: | |
| logger.warning("Unexpected error loading push config %s: %s", path, exc) | |
| return cls(urls=[]) |
| def _payload_to_body(channel_type: str, payload: Dict[str, Any]) -> tuple[str, str]: | ||
| if channel_type == "telegram": | ||
| return str(payload.get("text") or ""), "markdown" | ||
| if channel_type == "wecom": | ||
| return str((payload.get("markdown") or {}).get("content") or ""), "markdown" | ||
| if channel_type in {"feishu", "lark"}: | ||
| interactive = (payload.get("interactive") or {}).get("card") or {} | ||
| elements = interactive.get("elements") or [] | ||
| lines: List[str] = [] | ||
| for e in elements: | ||
| text_obj = e.get("text") if isinstance(e, dict) else None | ||
| if isinstance(text_obj, dict): | ||
| content = str(text_obj.get("content") or "").strip() | ||
| if content: | ||
| lines.append(content) | ||
| return "\n".join(lines), "markdown" | ||
| if channel_type == "discord": | ||
| embeds = payload.get("embeds") or [] | ||
| if not embeds: | ||
| return "", "text" | ||
| embed = embeds[0] if isinstance(embeds[0], dict) else {} | ||
| chunks: List[str] = [] | ||
| title = str(embed.get("title") or "").strip() | ||
| desc = str(embed.get("description") or "").strip() | ||
| if title: | ||
| chunks.append(f"## {title}") | ||
| if desc: | ||
| chunks.append(desc) | ||
| for field in embed.get("fields") or []: | ||
| if not isinstance(field, dict): | ||
| continue | ||
| name = str(field.get("name") or "").strip() | ||
| value = str(field.get("value") or "").strip() | ||
| if name or value: | ||
| chunks.append(f"- {name}: {value}") | ||
| return "\n".join(chunks), "markdown" | ||
| return "", "text" |
There was a problem hiding this comment.
push_daily_digest() builds rich per-channel payloads (Telegram photo/keyboard, Discord embeds, WeCom news, Feishu interactive cards), but _payload_to_body() flattens them into a plain text/markdown body, so those channel-specific fields are never sent through Apprise. If the goal is rich channel-native messages, this needs a different send path (e.g., plugin-specific kwargs or direct API clients) or the formatters should be simplified to explicitly generate only the body that Apprise will send.
| rows = raw.get("figures") or [] | ||
| figures: List[Figure] = [] | ||
| for row in rows: | ||
| if not isinstance(row, dict): | ||
| continue | ||
| figures.append( | ||
| Figure( | ||
| url=str(row.get("url") or "").strip(), | ||
| caption=str(row.get("caption") or "").strip(), | ||
| page=int(row.get("page") or 0), | ||
| width=int(row.get("width") or 0), | ||
| height=int(row.get("height") or 0), | ||
| index=int(row.get("index") or 0), | ||
| ) | ||
| ) |
There was a problem hiding this comment.
When reading cached figures, entries with an empty url are still converted into Figure objects and may be returned to callers. This can propagate invalid figures into identify_main_figure() and downstream payloads if the cache file is corrupted/partially written. Consider skipping cached rows where url is empty (mirroring _parse_figures() behavior).
| main_figure = item.get("main_figure") or {} | ||
| if isinstance(main_figure, dict): | ||
| fig_url = str(main_figure.get("url") or "").strip() | ||
| if fig_url: | ||
| try: | ||
| fe.enclosure(url=fig_url, length=0, type="image/jpeg") | ||
| except Exception: | ||
| pass |
There was a problem hiding this comment.
RSS enclosure type is hard-coded to image/jpeg, but main_figure.url may frequently be PNG/WebP/etc (and the tests add a .png URL). Consider inferring the MIME type from the URL extension (or storing it in main_figure) to avoid incorrect enclosure metadata for feed readers.
| # | ||
| # Examples (uncomment and fill in credentials): | ||
|
|
||
| channels: [] |
There was a problem hiding this comment.
The template uses channels: [], which makes the YAML invalid if a user simply uncomments any of the example - url: ... entries (flow-style list cannot be extended with block-style items). Consider using channels: (empty block list) instead, or add a clear instruction to remove [] before uncommenting examples.
| channels: [] | |
| channels: |
| for q in report.get("queries") or []: | ||
| for item in q.get("top_items") or []: | ||
| key = item.get("title") or id(item) | ||
| if key not in seen: | ||
| seen.add(key) | ||
| papers.append(item) |
There was a problem hiding this comment.
Deduplicating papers by title alone can drop distinct papers that share the same title (common for arXiv revisions / workshop variants), and it also differs from other dedupe logic in the repo that considers URLs. Consider using a more stable key like url (preferred) or url|title when available, falling back to id(item) only if neither exists.
| papers: List[Dict[str, Any]] = [] | ||
| for q in report.get("queries") or []: | ||
| for item in q.get("top_items") or []: | ||
| key = item.get("title") or id(item) |
There was a problem hiding this comment.
_collect_papers_from_report() deduplicates using title (or id(item)), which can incorrectly drop distinct papers that share a title. Since feed entries use url/external_url elsewhere, consider deduping by url (or url|title) when present to avoid missing items in RSS/Atom output.
| key = item.get("title") or id(item) | |
| url = item.get("url") or item.get("external_url") | |
| title = item.get("title") | |
| if url: | |
| key = ("url", url) | |
| elif title: | |
| key = ("title", title) | |
| else: | |
| key = ("id", id(item)) |
|
Follow-up pushes added:
Issue updates already synced: |
|
Follow-up batch pushed to this PR branch:
Issue progress:
|
* feat(push): implement Epic #179 — daily push optimization Multi-channel push infrastructure with structured digest cards: - #187: Integrate HF upvotes into Judge scoring prompt - #180: Add digest_card extraction (highlight/method/finding/tags) with new LLM prompt, email template rendering, markdown output - #181: MinerU Cloud API client for PDF figure extraction - #182: Apprise multi-channel push layer with YAML config - #183-185: Channel formatters (Telegram MarkdownV2, Discord Rich Embed, WeCom markdown+news, Feishu/Lark interactive card) - #186: RSS 2.0 + Atom feed endpoints (/api/feed/daily.xml, .atom) Also creates follow-up issues #190 (Push Channel Settings UI) and #191 (Dashboard Digest Card Enhancement). 56 new tests, all passing. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat: integrate MinerU figure extraction into CLI and ARQ worker daily-paper workflow Refs #179 * fix(daily-push): enable MinerU flags in ARQ cron payload (refs #181) * feat(push): wire channel formatters into apprise digest delivery (refs #182 #183 #184 #185) * feat(feed): add keyword RSS, feed cache, and figure enclosure support (refs #186) * feat(mineru): add extraction result cache for daily push figure pipeline (refs #181) * feat(push): add telegram bot command subscription endpoint (refs #191) * fix(push): add retry/backoff for channel delivery failures (refs #191) * feat(hf): add trending modes, cache, and arxiv identity dedupe (refs #193) * feat(push): add idempotency and error-code mapping for channel delivery (refs #191) * test(push): add mock channel e2e and failure-injection coverage (refs #191) * docs(push): add audit script and ops runbook for acceptance closure (refs #192) * docs: update README for AgentSwarm execution transition * docs: archive stale docs and rename AgentSwarm todo --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
* feat(push): implement Epic #179 — daily push optimization Multi-channel push infrastructure with structured digest cards: - #187: Integrate HF upvotes into Judge scoring prompt - #180: Add digest_card extraction (highlight/method/finding/tags) with new LLM prompt, email template rendering, markdown output - #181: MinerU Cloud API client for PDF figure extraction - #182: Apprise multi-channel push layer with YAML config - #183-185: Channel formatters (Telegram MarkdownV2, Discord Rich Embed, WeCom markdown+news, Feishu/Lark interactive card) - #186: RSS 2.0 + Atom feed endpoints (/api/feed/daily.xml, .atom) Also creates follow-up issues #190 (Push Channel Settings UI) and #191 (Dashboard Digest Card Enhancement). 56 new tests, all passing. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat: integrate MinerU figure extraction into CLI and ARQ worker daily-paper workflow Refs #179 * fix(daily-push): enable MinerU flags in ARQ cron payload (refs #181) * feat(push): wire channel formatters into apprise digest delivery (refs #182 #183 #184 #185) * feat(feed): add keyword RSS, feed cache, and figure enclosure support (refs #186) * feat(mineru): add extraction result cache for daily push figure pipeline (refs #181) * feat(push): add telegram bot command subscription endpoint (refs #191) * fix(push): add retry/backoff for channel delivery failures (refs #191) * feat(hf): add trending modes, cache, and arxiv identity dedupe (refs #193) * feat(push): add idempotency and error-code mapping for channel delivery (refs #191) * test(push): add mock channel e2e and failure-injection coverage (refs #191) * docs(push): add audit script and ops runbook for acceptance closure (refs #192) * docs: update README for AgentSwarm execution transition * docs: archive stale docs and rename AgentSwarm todo * docs: add AgentSwarm design proposal + web Playwright e2e setup - Add docs/proposals/agentswarm-design.md covering multi-agent orchestration architecture, adapter pattern, session management, skills ecosystem, and OpenCode/OpenClaw/oMo integration (refs #197, #214) - Add Playwright e2e test scaffolding for web dashboard (smoke, papers, research specs) - Update web/.gitignore for Playwright artifacts Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
* Feat/daily push epic 179 (#218) * feat(push): implement Epic #179 — daily push optimization Multi-channel push infrastructure with structured digest cards: - #187: Integrate HF upvotes into Judge scoring prompt - #180: Add digest_card extraction (highlight/method/finding/tags) with new LLM prompt, email template rendering, markdown output - #181: MinerU Cloud API client for PDF figure extraction - #182: Apprise multi-channel push layer with YAML config - #183-185: Channel formatters (Telegram MarkdownV2, Discord Rich Embed, WeCom markdown+news, Feishu/Lark interactive card) - #186: RSS 2.0 + Atom feed endpoints (/api/feed/daily.xml, .atom) Also creates follow-up issues #190 (Push Channel Settings UI) and #191 (Dashboard Digest Card Enhancement). 56 new tests, all passing. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat: integrate MinerU figure extraction into CLI and ARQ worker daily-paper workflow Refs #179 * fix(daily-push): enable MinerU flags in ARQ cron payload (refs #181) * feat(push): wire channel formatters into apprise digest delivery (refs #182 #183 #184 #185) * feat(feed): add keyword RSS, feed cache, and figure enclosure support (refs #186) * feat(mineru): add extraction result cache for daily push figure pipeline (refs #181) * feat(push): add telegram bot command subscription endpoint (refs #191) * fix(push): add retry/backoff for channel delivery failures (refs #191) * feat(hf): add trending modes, cache, and arxiv identity dedupe (refs #193) * feat(push): add idempotency and error-code mapping for channel delivery (refs #191) * test(push): add mock channel e2e and failure-injection coverage (refs #191) * docs(push): add audit script and ops runbook for acceptance closure (refs #192) * docs: update README for AgentSwarm execution transition * docs: archive stale docs and rename AgentSwarm todo * docs: add AgentSwarm design proposal + web Playwright e2e setup - Add docs/proposals/agentswarm-design.md covering multi-agent orchestration architecture, adapter pattern, session management, skills ecosystem, and OpenCode/OpenClaw/oMo integration (refs #197, #214) - Add Playwright e2e test scaffolding for web dashboard (smoke, papers, research specs) - Update web/.gitignore for Playwright artifacts Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> * fix(studio): fix context pack generation, duplicate import UX, and CLI chat errors (#223) * fix(studio): fix context pack generation, duplicate import UX, and CLI chat errors - 支持内联标题/摘要，以绕过 PaperInputRouter 获取 studio 论文 ID - 使上下文包生成 SSE 流期间的数据库持久化操作不再致命 - 从库中导入重复论文时显示错误消息 - 将左侧文件面板宽度缩小至约 18%（默认宽度）；移除默认模板文件 - 为 Claude CLI 添加 --verbose 标志，以兼容 stream-json 输出 Related to #222 * fix(studio): fix context pack generation, duplicate import UX, and CLI chat errors - 修改Gemini AI review代码问题 * fix(studio): fix context pack generation, duplicate import UX, and CLI chat errors - 修改Gemini AI review代码问题 --------- Co-authored-by: Jerry Zhang <1772030600@qq.com> * fix(web): correct package.json scripts JSON syntax * chore(gitignore): block nested .env secrets and keep examples * docs(env): add MinerU key and figure extraction env templates * refactor(mineru): standardize on v4 task API for daily push (#229) * fix(mineru): support v4 async task extraction flow * refactor(mineru): standardize on v4 task API for daily push (#179) * feat(push): render MinerU main figure in email via inline fallback * docs(readme): update email push demo screenshot * docs(readme): refresh Phase 6 daily push descriptions * fix(security): harden runbook allowed-dir path validation * fix(review): resolve security and reliability findings from AI review * fix(codeql): harden studio chat project_dir normalization --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> Co-authored-by: Boyu liu <oor2020@163.com>
* Feat/daily push epic 179 (#218) * feat(push): implement Epic #179 — daily push optimization Multi-channel push infrastructure with structured digest cards: - #187: Integrate HF upvotes into Judge scoring prompt - #180: Add digest_card extraction (highlight/method/finding/tags) with new LLM prompt, email template rendering, markdown output - #181: MinerU Cloud API client for PDF figure extraction - #182: Apprise multi-channel push layer with YAML config - #183-185: Channel formatters (Telegram MarkdownV2, Discord Rich Embed, WeCom markdown+news, Feishu/Lark interactive card) - #186: RSS 2.0 + Atom feed endpoints (/api/feed/daily.xml, .atom) Also creates follow-up issues #190 (Push Channel Settings UI) and #191 (Dashboard Digest Card Enhancement). 56 new tests, all passing. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat: integrate MinerU figure extraction into CLI and ARQ worker daily-paper workflow Refs #179 * fix(daily-push): enable MinerU flags in ARQ cron payload (refs #181) * feat(push): wire channel formatters into apprise digest delivery (refs #182 #183 #184 #185) * feat(feed): add keyword RSS, feed cache, and figure enclosure support (refs #186) * feat(mineru): add extraction result cache for daily push figure pipeline (refs #181) * feat(push): add telegram bot command subscription endpoint (refs #191) * fix(push): add retry/backoff for channel delivery failures (refs #191) * feat(hf): add trending modes, cache, and arxiv identity dedupe (refs #193) * feat(push): add idempotency and error-code mapping for channel delivery (refs #191) * test(push): add mock channel e2e and failure-injection coverage (refs #191) * docs(push): add audit script and ops runbook for acceptance closure (refs #192) * docs: update README for AgentSwarm execution transition * docs: archive stale docs and rename AgentSwarm todo * docs: add AgentSwarm design proposal + web Playwright e2e setup - Add docs/proposals/agentswarm-design.md covering multi-agent orchestration architecture, adapter pattern, session management, skills ecosystem, and OpenCode/OpenClaw/oMo integration (refs #197, #214) - Add Playwright e2e test scaffolding for web dashboard (smoke, papers, research specs) - Update web/.gitignore for Playwright artifacts Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> * fix(studio): fix context pack generation, duplicate import UX, and CLI chat errors (#223) * fix(studio): fix context pack generation, duplicate import UX, and CLI chat errors - 支持内联标题/摘要，以绕过 PaperInputRouter 获取 studio 论文 ID - 使上下文包生成 SSE 流期间的数据库持久化操作不再致命 - 从库中导入重复论文时显示错误消息 - 将左侧文件面板宽度缩小至约 18%（默认宽度）；移除默认模板文件 - 为 Claude CLI 添加 --verbose 标志，以兼容 stream-json 输出 Related to #222 * fix(studio): fix context pack generation, duplicate import UX, and CLI chat errors - 修改Gemini AI review代码问题 * fix(studio): fix context pack generation, duplicate import UX, and CLI chat errors - 修改Gemini AI review代码问题 --------- Co-authored-by: Jerry Zhang <1772030600@qq.com> * fix(web): correct package.json scripts JSON syntax * chore(gitignore): block nested .env secrets and keep examples * docs(env): add MinerU key and figure extraction env templates * refactor(mineru): standardize on v4 task API for daily push (#229) * fix(mineru): support v4 async task extraction flow * refactor(mineru): standardize on v4 task API for daily push (#179) * feat(push): render MinerU main figure in email via inline fallback * docs(readme): update email push demo screenshot * docs(readme): refresh Phase 6 daily push descriptions * fix(security): harden runbook allowed-dir path validation * fix(review): resolve security and reliability findings from AI review * fix(codeql): harden studio chat project_dir normalization * docs(readme): restructure README for professional presentation - Add centered hero header with badges (CI, Roadmap, License, Python, Next.js) - Replace verbose 18-row feature table with categorized bullet lists - Move Roadmap Phase 1-6 content to pinned issue #232 - Move module maturity matrix to Roadmap #232 - Consolidate screenshots into collapsible sections - Remove inline API endpoint table (40+ rows) and directory tree - Add Contributing section with Roadmap link - Streamline Getting Started with collapsible config details - Add repo topics for GitHub discoverability References: vLLM, Docling, PaperQA2, R2R README patterns Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> Co-authored-by: Boyu liu <oor2020@163.com>
What changed
This PR delivers the main implementation for Epic #179 and follow-up fixes on
feat/daily-push-epic-179:digest_cardextraction (highlight/method/finding/tags)daily_papers_job/api/feed/daily.xml/api/feed/daily.atom/api/feed/track/{track}.xml/api/feed/keyword/{keyword}.xml(new)main_figureIssue mapping
Notes on completion scope
This PR is intended as the core delivery for Epic #179. Some issue checklists include product-level acceptance items (for example: manual quality sampling, bot command UX, advanced provider-specific interactions) that may require a follow-up PR for strict checklist closure.
Validation
python -m py_compilepassed for modified Python modules/testspytestremains unstable on this machine (historical segfault); CI should be used as source of truthSummary by CodeRabbit
New Features
Documentation
Tests