GitHub - stilletto/Reddit_Dataset_Parser: Parse Reddit for best posts, comments and anything what can be question-answer pair. For pics I use CLIP to interpret it as text. Links in text checked, so only working links and only final destination of redirects collected. I created it rapidly for collect dataset for my LoRA to Alpaca AI.

Branches Tags

Name		Name	Last commit message	Last commit date
Latest commit History 1 Commit
main.py		main.py

About

Parse Reddit for best posts, comments and anything what can be question-answer pair. For pics I use CLIP to interpret it as text. Links in text checked, so only working links and only final destination of redirects collected. I created it rapidly for collect dataset for my LoRA to Alpaca AI.