Skip to content

fix: handle email-only feeds and HTML responses in feed fetcher#651

Merged
bokelley merged 1 commit into
mainfrom
bokelley/fix-tipsheet-xml-parse
Jan 6, 2026
Merged

fix: handle email-only feeds and HTML responses in feed fetcher#651
bokelley merged 1 commit into
mainfrom
bokelley/fix-tipsheet-xml-parse

Conversation

@bokelley

@bokelley bokelley commented Jan 6, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Add RSS content validation to detect when feeds return HTML instead of XML (e.g., when sites disable their RSS feeds and redirect to homepage)
  • Allow NULL feed_url for email-only feeds in database schema
  • Update getFeedsToFetch() to exclude email-only feeds (feed_url IS NULL)
  • Migrate existing email:// placeholder URLs to NULL

Problem

Tipsheet.ai (and potentially other sites) have disabled their RSS feeds. When the fetcher tried to parse the redirected HTML as XML, it failed with a cryptic error: Invalid character in entity name Line: 43 Column: 1084 Char: &

Additionally, the database schema had a NOT NULL constraint on feed_url, preventing email-only feeds from being created through the UI (which sends feed_url: null).

Solution

  1. Pre-fetch validation: Fetch content before parsing to validate it's actually RSS/XML, not HTML
  2. Clear error messages: Show "Feed URL returned HTML instead of RSS/XML (feed may be disabled)" instead of cryptic XML parsing errors
  3. Schema migration: Allow NULL feed_url with partial unique index for non-null URLs
  4. Query update: Exclude email-only feeds from RSS fetch queries

Test plan

  • Typecheck passes
  • All unit tests pass
  • End-to-end tested: Created email-only feed through UI successfully
  • Verified database shows feed_url = NULL for email-only feeds

🤖 Generated with Claude Code

- Add RSS content validation to detect when feeds return HTML instead of XML
  (e.g., when sites disable their RSS feeds and redirect to homepage)
- Allow NULL feed_url for email-only feeds in database schema
- Update getFeedsToFetch() to exclude email-only feeds (feed_url IS NULL)
- Migrate existing email:// placeholder URLs to NULL

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
@bokelley bokelley force-pushed the bokelley/fix-tipsheet-xml-parse branch from 1a158cb to 930455a Compare January 6, 2026 14:22
@bokelley bokelley merged commit 08394d2 into main Jan 6, 2026
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant