🎙️ Shellac — AI Content Repurposer

Self-hosted system that turns YouTube podcasts into multi-platform social media content.

YouTube URL → Transcript → 10 Ideas → Instagram + LinkedIn + Twitter posts + Images → Review → Schedule → Auto-Post

Zero platform dependencies — no N8N, no Make, no monthly subscription fees. Just Python, PostgreSQL, and your own API keys.

---

📋 Table of Contents

• Features
• Quick Start
• Technology Stack
• Architecture
• Configuration
• Cost Optimization
• Development
• Project Status
• Roadmap
• Documentation

---

✨ Features

Current Features (Phase 1)

Feature	Description	Status
YouTube Processing	Extract transcripts from any YouTube video	✅ Working
AI Idea Generation	Generate 10 compelling ideas from transcripts	✅ Working
Multi-Platform Posts	Create Instagram, LinkedIn, Twitter/X posts	✅ Working
Image Generation	AI-generated images for each idea (fal.ai)	✅ Working
Real-Time Progress	Watch processing live via SSE streaming	✅ Working
Content Review	Edit and refine all generated content	✅ Working
Scheduling	Schedule posts for specific dates/times	✅ Working
Prompt Customization	Edit AI prompts via dashboard	✅ Working
Provider Fallback	Auto-switch between LLM providers	✅ Working
Prompt Caching	87% token reduction via caching	✅ Working

Planned Features (Phase 2)

Feature	Description	Priority
Authentication	API key or OAuth login	High
Rate Limiting	Prevent API abuse	High
Token Refresh	Auto-refresh social media tokens	High
Error Alerting	Email/Slack notifications on failures	Medium
Google Sheets Sync	Optional one-way export	Low
Multi-User	User accounts and permissions	Medium
Metrics Tracking	Post engagement feedback loop	Low
Prompt Versioning	Keep history, enable rollback	Low

---

🚀 Quick Start

Prerequisites

• Docker Desktop (running)
• Python 3.11+
• API keys for at least one LLM provider

5-Minute Setup

# 1. Navigate to project
cd /Users/saisumanthbattepati/vibe-coding/shellac

# 2. Start infrastructure (PostgreSQL + Redis)
docker compose up -d postgres redis

# 3. Install Python dependencies
pip install -r requirements.txt

# 4. Initialize database
python -m src.database

# 5. Start application
uvicorn src.app:app --reload

Open: http://localhost:8000

One-Command Start

./start.sh

First Run

1. Click "+ New Content"
2. Paste a YouTube URL
3. Watch real-time progress
4. Review generated content
5. Edit, schedule, or export

---

🏗️ Technology Stack

Layer	Technology	Why
Web Framework	FastAPI	Async-native, type-safe, excellent performance
Frontend	HTMX + Jinja2 + Tailwind CSS	Dynamic UI without JavaScript framework
Database	PostgreSQL + asyncpg	Concurrent writes, JSONB, production-ready
Task Queue	ARQ + Redis	Async-native, job tracking, retries
Scheduler	APScheduler	In-process, dynamic scheduling, PostgreSQL-backed
Real-Time	SSE (Server-Sent Events)	One-way streaming, HTMX-compatible
LLM Provider	Custom abstraction	10+ backends, zero vendor lock-in
Image Gen	fal.ai	Queue-based generation with polling
Social APIs	Direct APIs	Twitter v2, LinkedIn v2, Facebook Graph
Testing	pytest + pytest-asyncio	Async test support

Key Design Principle: Async throughout — one event loop, zero sync/async bridging code.

---

🧠 Architecture

System Flow

┌─────────────────────────────────────────────────────────────────┐
│                    BROWSER (HTMX + SSE)                         │
│  Dashboard • Submit • Review • Edit • Schedule • Prompts        │
└───────────────────────┬─────────────────────────────────────────┘
                        │ HTTP + SSE
┌───────────────────────▼─────────────────────────────────────────┐
│                    FastAPI (Async)                              │
│  Routes: /, /submit, /content, /prompts, /sse/{job_id}        │
└──────┬──────────────────────────────┬───────────────────────────┘
       │                              │
       │ read/write                   │ enqueue
       ▼                              ▼
┌──────────────────┐         ┌──────────────────────┐
│   PostgreSQL     │         │      Redis           │
│   (asyncpg)      │         │   (ARQ broker)       │
│                  │         │                      │
│ • inputs         │         │ • Job queue          │
│ • content        │         │ • Progress pub/sub   │
│ • prompts        │         │                      │
│ • pipeline_jobs  │         └──────────┬───────────┘
└──────────────────┘                    │
                                        ▼
                               ┌──────────────────────┐
                               │    ARQ Worker        │
                               │                      │
                               │ 1. Transcribe        │
                               │ 2. Generate ideas    │
                               │ 3. Generate posts    │
                               │ 4. Generate images   │
                               │ 5. Write to DB       │
                               └──────────────────────┘

Data Model

inputs          — Content sources (YouTube URLs)
content         — Generated posts (one per idea per platform)
prompts         — Editable AI prompts
pipeline_jobs   — Background task progress tracking

---

⚙️ Configuration

LLM Provider Priority

Current order: GLM → Kimi → Gemini → OpenRouter (free tier)

Priority	Provider	Cost/1M Tokens	Model
1st	GLM (Zhipu)	~$0.01	glm-4-plus
2nd	Kimi	~$0.012	moonshot-v1-128k
3rd	Gemini Flash	~$0.075	gemini-2.5-flash
4th	OpenRouter	$0	llama-3.3-70b:free

Switch providers in .env:

# Primary provider
LLM_PROVIDER=glm
GLM_API_KEY=your-key-here

# Automatic fallbacks configured
KIMI_API_KEY=your-kimi-key
GEMINI_API_KEY=your-gemini-key
OPENROUTER_API_KEY=your-openrouter-key

Environment Variables

# Database
POSTGRES_HOST=localhost
POSTGRES_PORT=5432
POSTGRES_DB=shellac
POSTGRES_USER=shellac
POSTGRES_PASSWORD=your-password

# Redis
REDIS_HOST=localhost
REDIS_PORT=6379

# FastAPI
FASTAPI_HOST=0.0.0.0
FASTAPI_PORT=8000
DEBUG=true

# Image Generation (optional)
FAL_API_KEY=your-fal-key

# Social Media (optional)
TWITTER_API_KEY=your-key
TWITTER_ACCESS_TOKEN=your-token
LINKEDIN_ACCESS_TOKEN=your-token
FACEBOOK_PAGE_ACCESS_TOKEN=your-token

---

💰 Cost Optimization

Cost Comparison Per Podcast

Token usage: ~30K input (transcript + prompts) + ~7K output (ideas + posts)

Provider	Cost Per Podcast	Podcasts with $10
GLM	$0.00037	~27,000
Kimi	$0.00044	~22,700
Gemini Flash	$0.0028	~3,500
Llama 3.3 (Free)	$0	Unlimited
Claude Direct	$0.40	~25

Savings: 99% cheaper than using Claude directly

Optimizations Implemented

1. Provider Fallback System — Auto-switch on failure
2. Prompt Caching — Cache transcripts (87% token reduction)
3. Prompt Optimizer — Optimize user inputs automatically

Result: ~$0.04 per year for 100 podcasts (effectively zero cost)

---

🛠️ Development

Project Structure

shellac/
├── src/
│   ├── app.py                  # FastAPI application
│   ├── tasks.py                # ARQ pipeline tasks
│   ├── models.py               # Database models
│   ├── database.py             # DB connection
│   ├── config.py               # Configuration
│   ├── templates/              # HTMX templates
│   ├── llm_provider.py         # LLM abstraction
│   ├── ai_generator.py         # Content generation
│   ├── provider_fallback.py    # Provider fallback
│   ├── prompt_cache.py         # Prompt caching
│   ├── prompt_optimizer.py     # Prompt optimization
│   └── ...
├── tests/                      # Test suite
├── docker-compose.yml          # Infrastructure
├── requirements.txt            # Dependencies
└── start.sh                    # Startup script

Commands

# Start development server
uvicorn src.app:app --reload

# Start ARQ worker
arq src.tasks.WorkerSettings

# Run tests
pytest tests/ -v

# Run specific test
pytest tests/test_llm_provider.py -v

# Test provider fallback
python -m src.provider_fallback

# Check database connection
python -m src.database

Adding New Features

1. New route: Add to src/app.py
2. New task: Add to src/tasks.py and register in WorkerSettings
3. New model: Add to src/models.py and run python -m src.database
4. New template: Add to src/templates/ and extend base.html

---

📊 Project Status

Phase 0: Core Modules ✅ (100% Complete)

Provider-agnostic LLM abstraction, transcript extraction, content generation, image generation, social posting.

Phase 1: Full Stack Build ✅ (~95% Complete)

FastAPI + HTMX dashboard, PostgreSQL models, ARQ pipeline tasks, SSE progress streaming, Docker Compose, test suite.

Missing: Alembic migrations (using SQLModel.metadata.create_all), social posting testing.

Phase 2: Production Hardening ❌ (0% Complete)

Auth, rate limiting, error alerting, token refresh, logging, metrics.

Phase 3: Platform Extension ❌ (0% Complete)

Multi-tenant, billing, white-label dashboard, additional content sources.

---

🗺️ Roadmap

Phase 2: Production Hardening (Next)

High Priority:

• API key authentication
• Rate limiting per user
• Social media token refresh
• Structured error handling
• Logging and monitoring

Medium Priority:

• Error alerting (email/Slack)
• Health check endpoints
• Graceful shutdown
• Multi-user support

Low Priority:

• Google Sheets sync (one-way)
• Engagement feedback loop
• Prompt versioning
• Threads (Meta) platform

Phase 3: Platform Extension (Future)

• Multi-tenant isolation
• Billing and usage tracking
• White-label dashboard
• Additional content sources
• Programmatic API
• Webhook integrations

---

📚 Documentation

File	Description
ARCHITECTURE.md	System design, decision log, data model
DECISIONS.md	Technology choice rationale
COST_OPTIMIZATION.md	Cost breakdown and savings
BUILD_SUMMARY.md	Phase 1 build summary
QUICKSTART.md	5-minute setup guide
ROADMAP.md	Detailed roadmap (TODO)
CLAUDE.md	Claude Code project context

---

🤝 Contributing

This is a personal project, but suggestions are welcome.

1. Fork the repo
2. Create a feature branch
3. Make your changes
4. Submit a pull request

---

📄 License

MIT License — feel free to use this for your own projects.

---

🙏 Acknowledgments

• Claude Code — AI-assisted development
• FastAPI — Excellent async web framework
• HTMX — Dynamic UI without JavaScript fatigue
• ARQ — Async task queue that just works

---

Built with ❤️ using Claude Code