feat(videos): reference conditioning over /v1/videos, image and clip and voice

[localai-bot]

Row SERVE-VIDEOS-REFS, stacked on SERVE-VIDEOS-OAI (#71, merged). That row made an OpenAI client's body parse; this one makes its references do something. Before it, no reference modality was reachable over HTTP at all: an image-to-video request generated silently from the prompt alone.

This is the second half of the #71 split (the combined change was 1241 non-exempt lines, over the 900 check-pr-size.py cap). 552 non-exempt lines here.

input_reference maps to fl2va, not ref2va

OpenAI documents it as the image the generated video starts from, which is exactly what fl2va expresses: MiniMaxH3EncodeKeyframeCondRows pins frame 0 of the output to the supplied image. MiniMaxH3EncodeReferenceImages means something else entirely, whole reference images prepended as their own blocks (subject or style guidance that never becomes a frame), so mapping input_reference there would have silently changed what the API promises.

The source is a filesystem path or an RFC 2397 data: URL, decoded by the same DecodeDataUri the chat multimodal parts use rather than a second, subtly different decoder. An http(s) URL is refused by name rather than stat-ed as a path and failing later with a confusing message.

The two modalities OpenAI has no slot for ride metadata

H3 supports three (image, silent video, audio) and the Sora schema carries one, so the other two enter through the standard free-form string map every strict client already tolerates, rather than invented top-level fields that would fail a client's schema validation. The whole map is kept verbatim; two keys are lifted into typed fields.

• metadata.input_reference_video is a directory of frame_%06d.ppm, the exact layout minimax-h3-gen and this server write, so one run's frames chain straight into the next request. It is not a container: no demuxer is vendored, and a data: URL cannot name a directory.
• metadata.input_reference_audio is a 16-bit PCM WAV path or data: URL.

A video reference is silent, by construction and stated. MiniMaxH3EncodeReferenceVideo emits ref_audio_t == 0. An audio reference supplied alongside attaches to that same kVideoAudio block, the layout packed_sequence.py builds; alone it is its own block.

Legality is enforced in the parser

The rule is the pipeline's own (minimax_h3_pipeline.cpp:251: fl2va keyframes and ref2va blocks are exclusive), so an illegal pair is a 400 naming it rather than a failed job, and never a silently dropped reference, which is the failure that looks like it worked.

Two refusals up front rather than deep in the denoise: a reference image must be a binary PPM (P6), since no PNG/JPEG codec is vendored (the same named residual the chat multimodal path carries), and it must already be at the resolved output geometry, since no resampler is vendored. Both VAE encoder halves load lazily and once each under one mutex.

Gate (CPU, foreground)

• test_video_api 14/14, 167 assertions
• test_openai_api_server 41/41, 525 assertions
• server builds clean
• Each modality is gated as arriving at the runner (a reference that parsed and then never reached the pipeline is the failure that looks like it worked), and an illegal pair is a 400 that generates nothing (calls == 0).

FOLLOWING_AGENTS_PROTOCOL