feat! Improve chunk generation speed, watermark rendering, and visual grading (#20)

* feat: Enhance audio management and dashboard UI

- Introduced functions to format audio durations and manage audio file caching, improving audio file handling.
- Updated the dashboard to display audio files with their durations, allowing for better user interaction.
- Added filtering and sorting options for audio files based on duration, enhancing usability.
- Improved the layout of the stats page and dashboard for a more cohesive user experience.

* feat: Enhance chunk metadata handling and dashboard improvements

- Introduced caching for chunk metadata to optimize performance and reduce redundant file reads.
- Added functions to normalize source video entries and parse chunk metadata from .meta.json files.
- Updated the dashboard to improve the display of source videos and model information.
- Enhanced the video generation script to support a new video wall layout for multi-clip displays.
- Improved error handling and user feedback in the chunk generation process.
- Added a configuration option for video wall layout in the Docker setup.

* feat: Enhance audio management and dashboard UI

- Added timezone configuration to Docker and environment files for consistent time handling.
- Improved audio stats tracking by normalizing audio entries and formatting display times.
- Updated the stats page to reflect changes in audio play counts and durations, enhancing user experience.
- Introduced caching mechanisms for video durations and model labels to optimize performance during chunk generation.

Author: vineethvijay

.env.example

@@ -25,6 +25,9 @@ PORT=8081
 # HLS streaming port (for VLC, Safari, or external players)
 HLS_PORT=8082
 
+# Container timezone (used by cron run history, chunk metadata timestamps, UI time display)
+# TZ=Europe/Amsterdam
+
 
 # ── 3. Chunk Generation Tuning ───────────────────────────────────
 # Hardware Acceleration: "nvidia" for GPU acceleration or "none" for CPU encoding
@@ -53,3 +56,6 @@ CLIP_MAX=6
 # "Model - <url>" from descriptions. Requires both URL and token.
 # TUBEARCHIVIST_URL=https://ta.example.com
 # TUBEARCHIVIST_TOKEN=your_api_token
+# If TubeArchivist returns no model, use this as the watermark text (optional).
+# Test mode (/generate_chunk.sh test) uses "Sample watermark" when both are empty.
+# WATERMARK_FALLBACK=My channel

.gitignore

@@ -14,6 +14,7 @@ build/
 
 .cursor/
 .cursorrules
+*.mdc
 
 # Configuration
 .env
@@ -45,3 +46,4 @@ chunks/
 stats/
 
 deploy/
+

Dockerfile

@@ -4,6 +4,7 @@ FROM python:3.11-slim
 RUN apt-get update && apt-get install -y \
     ffmpeg \
     curl \
+    tzdata \
     && rm -rf /var/lib/apt/lists/*
 
 # Set working directory
@@ -25,6 +26,7 @@ EXPOSE 8080
 
 # Set Python to unbuffered mode for logs
 ENV PYTHONUNBUFFERED=1
+ENV TZ=Europe/Amsterdam
 
 # Switch to non-root user
 USER appuser

Dockerfile.generator

@@ -1,8 +1,11 @@
 FROM linuxserver/ffmpeg:version-6.1.1-cli
 
-# Install required packages for generation script + segment tracker (Python) + font for drawtext
-RUN apt-get update && apt-get install -y bash bc python3 fonts-dejavu-core fontconfig && \
+# Install required packages for generation script + segment tracker (Python) + fonts.
+# Install distro ffmpeg so drawtext is available in the runtime binary.
+RUN apt-get update && apt-get install -y bash bc python3 ffmpeg fonts-dejavu-core fontconfig tzdata && \
     apt-get clean && rm -rf /var/lib/apt/lists/*
+ENV PATH="/usr/bin:${PATH}"
+ENV TZ=Europe/Amsterdam
 
 # Suppress "Cannot load default config file" from libass/subtitles filter
 RUN mkdir -p /etc/fonts

clip_pusher.py

@@ -41,6 +41,7 @@ services:
       - ${STATS_DIR:-./stats}:/app/stats
       - trigger_volume:/app/trigger
     environment:
+      - TZ=${TZ:-Europe/Amsterdam}
       - CHUNK_DURATION=${CHUNK_DURATION:-300}
       - CLIP_MIN=${CLIP_MIN:-5}
       - CLIP_MAX=${CLIP_MAX:-10}
@@ -87,6 +88,7 @@ services:
       - ./templates:/app/templates:ro
       - /var/run/docker.sock:/var/run/docker.sock
     environment:
+      - TZ=${TZ:-Europe/Amsterdam}
       - CHUNK_FOLDER=/chunks
       - PORT=8080
       - EXTERNAL_PORT=${PORT:-8081}

docker-compose.yml

@@ -106,6 +106,59 @@ def pick_start(
     return random.randint(fallback_min, fallback_max)
 
 
+def pick_and_record_start(
+    json_path: str,
+    video_path: str,
+    duration_sec: float,
+    clip_len_sec: int,
+    zone_min: Optional[float] = None,
+    zone_max: Optional[float] = None,
+) -> int:
+    """Pick a start and immediately record [start, start+clip_len] in one file read/write cycle."""
+    data = load_used(json_path)
+    used = data.get("videos", {}).get(video_path, [])
+    duration_sec = max(0, duration_sec)
+    clip_len_sec = max(1, clip_len_sec)
+    max_start = int(duration_sec - clip_len_sec)
+    if max_start <= 0:
+        start = 0
+    else:
+        free = free_intervals(duration_sec, used)
+        long_enough = [(a, b) for a, b in free if (b - a) >= clip_len_sec]
+
+        if zone_min is not None and zone_max is not None:
+            z_min, z_max = float(zone_min), float(zone_max)
+            long_enough = [
+                (max(a, z_min), min(b - clip_len_sec, z_max))
+                for a, b in long_enough
+                if a <= z_max and b >= z_min + clip_len_sec
+            ]
+            long_enough = [(a, b) for a, b in long_enough if b >= a]
+            fallback_min = max(0, int(z_min))
+            fallback_max = min(max_start, int(z_max))
+            if fallback_max < fallback_min:
+                fallback_max = fallback_min
+        else:
+            fallback_min, fallback_max = 0, max_start
+
+        if long_enough:
+            a, b = random.choice(long_enough)
+            start = int(a + random.random() * max(0, b - a))
+        else:
+            start = random.randint(fallback_min, fallback_max)
+
+    videos = data.setdefault("videos", {})
+    intervals = videos.setdefault(video_path, [])
+    end = float(start + clip_len_sec)
+    intervals.append([float(start), end])
+    merged = merge_intervals(intervals)
+    if len(merged) > MAX_INTERVALS_PER_VIDEO:
+        merged = merged[-MAX_INTERVALS_PER_VIDEO:]
+    videos[video_path] = merged
+    save_used(json_path, data)
+    return int(start)
+
+
 def record_used(json_path: str, video_path: str, start_sec: float, end_sec: float) -> None:
     data = load_used(json_path)
     videos = data.setdefault("videos", {})
@@ -133,6 +186,15 @@ def main() -> None:
         zone_max = float(sys.argv[7]) if len(sys.argv) >= 8 else None
         start = pick_start(json_path, video_path, float(dur_s), int(float(clip_s)), zone_min, zone_max)
         print(start)
+    elif cmd == "pick_record":
+        if len(sys.argv) not in (6, 8):
+            print(f"Usage: {FILENAME} pick_record <json_path> <video_path> <duration_sec> <clip_len_sec> [zone_min] [zone_max]", file=sys.stderr)
+            sys.exit(2)
+        json_path, video_path, dur_s, clip_s = sys.argv[2:6]
+        zone_min = float(sys.argv[6]) if len(sys.argv) >= 8 else None
+        zone_max = float(sys.argv[7]) if len(sys.argv) >= 8 else None
+        start = pick_and_record_start(json_path, video_path, float(dur_s), int(float(clip_s)), zone_min, zone_max)
+        print(start)
     elif cmd == "record":
         if len(sys.argv) != 6:
             print(f"Usage: {FILENAME} record <json_path> <video_path> <start_sec> <end_sec>", file=sys.stderr)

generate_chunk.sh

@@ -192,21 +192,24 @@ <h3 class="text-sm font-medium text-gray-400 uppercase tracking-wider">
             <!-- Top audio by play count -->
             <div class="bg-gray-800 rounded-lg shadow border border-gray-700 overflow-hidden" x-data="{ collapsed: true }">
                 <div @click="collapsed = !collapsed" class="px-6 py-4 cursor-pointer hover:bg-gray-700/50 transition-colors flex items-center justify-between">
-                    <h3 class="text-sm font-medium text-gray-400 uppercase tracking-wider">Music (most played)</h3>
+                    <h3 class="text-sm font-medium text-gray-400 uppercase tracking-wider">Music (most streamed)</h3>
                     <span class="text-gray-500" x-text="collapsed ? '▶' : '▼'"></span>
                 </div>
                 <div x-show="!collapsed" x-cloak class="px-6 pb-6">
                 {% if play_counts.audio %}
                 <ul class="grid grid-cols-2 gap-3 text-sm text-gray-300">
-                    {% for name, count in play_counts.audio %}
+                    {% for row in play_counts.audio %}
                     <li class="flex justify-between items-center gap-2 border-b border-gray-700 pb-3 odd:border-r odd:border-gray-700 odd:pr-3">
-                        <span class="truncate min-w-0" title="{{ name }}">{{ name }}</span>
-                        <span class="font-mono text-gray-400 shrink-0">{{ count }}</span>
+                        <span class="truncate min-w-0" title="{{ row.name }}">{{ row.name }}</span>
+                        <span class="text-right shrink-0">
+                            <span class="font-mono text-gray-300">{{ row.time_display }}</span>
+                            <span class="block text-[10px] text-gray-500 font-mono">{{ row.chunks }} chunks · {{ row.seconds }}s</span>
+                        </span>
                     </li>
                     {% endfor %}
                 </ul>
                 {% else %}
-                <p class="text-gray-500 text-sm">No audio play data yet. Counts increment per chunk played while each track is active.</p>
+                <p class="text-gray-500 text-sm">No audio stats yet. Time and chunk counts accrue while each track plays on stream.</p>
                 {% endif %}
                 </div>
             </div>