Problem
When converting web pages to markdown, HtmlConverter processes the entire <body>, including navigation bars, sidebars, footers, cookie banners, and other boilerplate. This produces noisy markdown that buries the actual content.
Site-specific converters like WikipediaConverter solve this by targeting known DOM elements (e.g., div#mw-content-text), but there's no generic solution for arbitrary web pages.
Proposed solution
Add a reader mode option that automatically extracts the main article content from a web page before converting to markdown — similar to how Firefox Reader View strips away clutter.
This would be:
- Opt-in: a
reader_mode kwarg on the API and a --reader-mode CLI flag, so existing behavior is unchanged
- Optional dependency: following the existing pattern for pptx, pdf, etc.
Example
md = MarkItDown()
# Without reader mode - full page with nav, sidebar, footer
result = md.convert("https://example.com/blog-post")
# With reader mode - just the article content
result = md.convert("https://example.com/blog-post", reader_mode=True)
Problem
When converting web pages to markdown,
HtmlConverterprocesses the entire<body>, including navigation bars, sidebars, footers, cookie banners, and other boilerplate. This produces noisy markdown that buries the actual content.Site-specific converters like
WikipediaConvertersolve this by targeting known DOM elements (e.g.,div#mw-content-text), but there's no generic solution for arbitrary web pages.Proposed solution
Add a reader mode option that automatically extracts the main article content from a web page before converting to markdown — similar to how Firefox Reader View strips away clutter.
This would be:
reader_modekwarg on the API and a--reader-modeCLI flag, so existing behavior is unchangedExample