feat(content-studio): Video Generation Pipeline — AI 영상 합성 통합

[thxforall]

Summary

Content Studio의 Short Form Builder가 현재 텍스트 기반 스크립트(voiceover + scene direction)만 생성하고 있는데, 실제 영상 합성(video synthesis) 파이프라인을 추가하여 스크립트 → 영상까지 자동 생성할 수 있도록 확장한다.

현재 상태 (as-is)

Post → ContentPacket → ShortFormPlan (scenes + voiceover 텍스트)
                                      ↑ 여기서 끝남 — 실제 영상 없음

목표 (to-be)

ShortFormPlan (scenes + voiceover)
    ↓
[Video Generation Pipeline]
    ├─ Scene별 이미지/클립 생성 (Image-to-Video, Text-to-Video, Reference-to-Video)
    ├─ 스타일 변환 / 편집 (Video Editing)
    ├─ 클립 연장 (Video Extension)
    └─ 후처리 (합성, 자막, 음악 → 최종 영상)

AI Provider 통합 계획

기존 파이프라인(OpenAI Image, Firecrawl)에 영상 생성 전용 provider를 추가한다.
아래는 후보군이며, 실제 구현 시 벤치마크/비용 비교 후 선택할 수 있다.

Grok Imagine API (xAI) — 1차 후보

모드	설명
Text-to-Video	텍스트 프롬프트 → 영상 생성
Image-to-Video	정지 이미지 → 영상 애니메이션화
Reference-to-Video	참조 이미지(제품/브랜드)를 가이드로 영상 생성
Video Editing	기존 영상에서 객체 추가/제거/교체, 스타일 변환
Video Extension	마지막 프레임에서 이어서 생성

• 해상도: 480p / 720p
• 비율: 9:16 (세로형 숏츠), 16:9, 1:1, 4:3
• 길이: 1~15초 / clip
• 비용: 720p 기준 ~$0.07/초 (10초 숏츠 ≈ $0.70)
• 비동기 처리 (polling 필요)

기존 Provider 연계

Provider	역할
OpenAI GPT Image (gpt-image-1)	Scene별 참조 이미지 생성, 편집
Firecrawl	트렌드 리서치 → 프롬프트 enrichment
OpenAI GPT (gpt-4.1)	Scene 스크립트 refinement, 프롬프트 최적화

구현 범위 (예상)

⚠️ 마케팅 숏츠 파이프라인은 예시이며, 실제 작업 시 변경될 수 있음

Phase 1: Provider 추상화 + Grok 클라이언트

• packages/web/lib/content-studio/video/ 모듈 생성
• Video provider 인터페이스 정의 (provider-agnostic)
• Grok Imagine API 클라이언트 구현 (5개 모드)
• 비동기 job polling 로직
• 생성된 영상 Supabase Storage 업로드

Phase 2: Scene → Video 파이프라인

• ShortFormPlan의 scene 배열 → 개별 클립 생성
• Scene별 최적 생성 모드 자동 선택 (text-to-video vs image-to-video)
• Post 원본 이미지 → Reference-to-Video 활용
• 기존 OpenAI 이미지 → Image-to-Video 변환 연계

Phase 3: 후처리 + 합성

• 클립 합치기 (ffmpeg / server-side 처리)
• 자막 / 텍스트 오버레이
• BGM / 보이스오버 오디오 트랙
• 최종 영상 export (9:16 세로형 기본)

Phase 4: Admin UI + 워크플로우

• ShortFormPanel에 영상 생성 버튼 추가
• 생성 진행 상태 표시 (비동기 polling)
• 미리보기 + 다운로드
• 영상별 Governance check 연동

Phase 5 (선택): API Route + DB

• POST /api/v1/content/assets/videos 엔드포인트
• content_video_jobs 테이블 (job 상태 추적)
• 비용 추적 연동 (cost-tracking 시스템)

참고

• 현재 Content Studio 구조: packages/web/lib/content-studio/ (schemas, LLM, research, assets)
• Short Form 관련: assets/plan.ts, assets/service.ts, ShortFormPanel.tsx
• 관련 PR: feat(content-studio): AI-powered content generation pipeline #498 (AI-powered content generation pipeline)
• Grok Imagine API 공식 문서 기준: 2026-05-13

비용 추정

항목	단가
Grok 영상 생성 480p	$0.05 / 초
Grok 영상 생성 720p	$0.07 / 초
Grok 비디오 입력	$0.01 / 초
Grok 이미지 생성	$0.02 / 장
OpenAI GPT Image	기존 비용 구조 유지

10초 720p 숏츠 1개 생성 시 영상 비용만 약 $0.70 수준.

🤖 Generated with Claude Code

[thxforall]

Status update: Phase 1A/1B job-backed scene clip generation has landed in commit 1318f072 (feat(content-studio): add video job pipeline).

Completed in this slice:

• xAI scene video start/status poll-once/download/storage helpers
• content_studio_video_jobs and content_studio_video_scenes migration + RLS
• job create/get/poll/retry API routes
• xAI pending state keeps scene/job running instead of failing early
• scene-level retry
• ShortFormPanel job creation, polling, status display, preview, and download
• focused tests, typecheck, eslint, and smoke passed for this slice

This does not close the full Issue #508 scope yet. Remaining work has been split into follow-up issues:

• feat(content-studio): add provider-agnostic video provider abstraction #519 provider-agnostic video abstraction and full Grok mode support
• feat(content-studio): select scene video mode and bridge OpenAI images #522 scene-level generation mode selection and OpenAI image-to-video bridge
• feat(content-studio): compose scene clips into final vertical video export #523 ffmpeg composition, subtitles/text overlays, audio tracks, and final 9:16 export
• feat(content-studio): integrate video governance checks and cost tracking #524 video governance checks and cost-tracking integration

Live xAI end-to-end smoke remains intentionally unrun by default because it incurs provider cost; it is covered as an opt-in/manual verification path in the follow-up provider work.

[thxforall]

2026-05-14 update from issue-508-content-studio-video worktree

Implemented one follow-up slice after the persisted xAI video pipeline:

• Added admin-only GET /api/v1/content/assets/video-jobs?packetId=... to list recent persisted video jobs for a packet.
• Added server helper to return each job with the stored ShortFormPlan snapshot, so the UI can reconstruct scenes after refresh.
• Updated Short Form Builder to auto-load the latest saved video job for the current packet, show a Saved video jobs list, resume polling for running jobs, and keep retry/compose updates reflected in the saved-job list.
• Updated docs/agent/api-v1-routes.md for the GET/POST video-jobs contract.

Verification:

• bun run test lib/content-studio/tests/video-jobs.test.ts: 8 passed
• bun run test lib/content-studio/tests: 57 passed
• bun run typecheck: passed
• Local dev smoke on http://localhost:3108/admin/content-studio compiled and redirected unauthenticated admin access to /admin/login; GET video-jobs returned 401 unauthenticated as expected.

Still open for full Issue #508 scope:

• Background worker or queue-driven polling instead of admin UI polling.
• Cost tracking and governance integration.
• Native voiceover/BGM generation; current final composition accepts audio URLs only.
• Real xAI API key end-to-end video generation validation.
• Branch still needs reconciliation with origin/dev before PR because this worktree remains ahead/behind.

[thxforall]

2026-05-14 follow-up update

Continued from the persisted video-job resume slice:

• Rebasing is complete: feature/508-content-studio-video-pipeline is now based on latest origin/dev (39a9951) and is ahead by 9 commits.
• Resolved the rebase conflict in Content Studio tests by preserving both the upstream chore(dx): dev-reset PRD seed + pre-push lint fixes #525 test updates and the video pipeline generated-image flow.
• Added a governance integration slice: ShortFormPanel now passes the current Content Studio governance result into video-job creation, shortFormVideoRequestSchema accepts it, and createShortFormVideoJob persists the snapshot into the stored job request metadata.
• Hardened saved-job resume handling so malformed/legacy GET video-jobs responses cannot set savedVideoJobs to undefined.

Verification after rebase + governance slice:

• bun run test lib/content-studio/tests app/admin/content-studio/tests app/api/v1/content/assets/video-jobs/tests: 10 files passed, 74 tests passed
• bun run typecheck: passed

Still intentionally open in #508:

• Background worker or queue-driven polling.
• Native voiceover/BGM generation; final composition still accepts audio URLs.
• Cost tracking beyond governance/request metadata snapshot.
• Real xAI API-key E2E generation validation before production rollout.