feat: ali video support#573
Merged
Merged
Conversation
There was a problem hiding this comment.
Pull request overview
This PR adds end-to-end “Ali video support” by introducing OpenAI-compatible /v1/videos relay endpoints, extending async-usage polling to capture video-specific usage context, and updating the web UI/types to configure video/image constraints and video duration limits.
Changes:
- Add relay routing + adaptor support for OpenAI-style Videos API (
/v1/videos, content, delete, remix) and hook it into async usage polling. - Introduce
model.UsageContext(service tier + media price conditions likesize/quality) to drive conditional pricing/amount calculation and persist it in logs/async usage records. - Update web schemas/UI for video configuration (
video_sizes,max_video_generation_seconds) and price condition fields (size,quality).
Reviewed changes
Copilot reviewed 87 out of 88 changed files in this pull request and generated 8 comments.
Show a summary per file
| File | Description |
|---|---|
| web/src/validation/model.ts | Extends validation schemas for size/quality conditions and video-related config fields. |
| web/src/types/model.ts | Updates frontend model types for video/image sizes, video duration limits, and model type options. |
| web/src/types/group.ts | Extends group model override types to include max video generation seconds and removes legacy image price maps. |
| web/src/feature/model/components/ModelForm.tsx | Adds UI controls for image_sizes, video_sizes, and max_video_generation_seconds based on model type. |
| web/src/feature/group/components/GroupModelConfigsTab.tsx | Adds group-level override UI/state for max video generation seconds and removes legacy image price summary handling. |
| web/src/components/price/PriceFormFields.tsx | Adds size/quality fields to conditional pricing UI. |
| web/src/components/price/PriceDisplay.tsx | Displays size/quality as part of conditional price labels. |
| web/public/locales/zh/translation.json | Adds i18n strings for new video/image size fields and max video seconds UI. |
| web/public/locales/en/translation.json | Adds i18n strings for new video/image size fields and max video seconds UI. |
| core/task/task.go | Updates async usage processing to propagate UsageContext through fetch/complete/log update paths. |
| core/task/async_usage_test.go | Expands async usage tests to cover stored price conditions and failure log updates. |
| core/router/relay.go | Registers new relay routes for /v1/videos endpoints. |
| core/relay/utils/testreq.go | Removes dependency on legacy ImagePrices when building image test requests. |
| core/relay/plugin/timeout/timeout.go | Applies timeouts to new Videos modes. |
| core/relay/model/video.go | Adds OpenAI-style video request/response structs and helpers for pricing size normalization. |
| core/relay/model/errors.go | Maps new Videos modes to OpenAI video error wrapper. |
| core/relay/model/constant.go | Adds video object constant. |
| core/relay/mode/define.go | Introduces new relay modes for Videos endpoints and string representations. |
| core/relay/mode/define_test.go | Adds a stability test ensuring persisted mode numeric IDs remain unchanged. |
| core/relay/meta/meta.go | Adds request usage context + video ID to relay meta. |
| core/relay/controller/video.go | Implements request parsing, validation, usage + pricing for video generation requests. |
| core/relay/controller/video_test.go | Adds controller tests for video usage parsing, size filtering, and duration limits. |
| core/relay/controller/usage.go | Introduces RequestUsage wrapper (usage + context) for request accounting. |
| core/relay/controller/tts.go | Migrates TTS request usage to return RequestUsage. |
| core/relay/controller/stt.go | Migrates STT request usage to return RequestUsage. |
| core/relay/controller/responses.go | Migrates Responses request usage to return RequestUsage. |
| core/relay/controller/rerank.go | Migrates Rerank request usage to return RequestUsage. |
| core/relay/controller/pdf.go | Migrates PDF request usage to return RequestUsage. |
| core/relay/controller/image.go | Adds image size allowlist validation and records image size/quality into usage context. |
| core/relay/controller/image_test.go | Adds tests for conditional price units, unmatched condition handling, size allowlist, and usage context. |
| core/relay/controller/handle.go | Plumbs UsageContext through relay handling results. |
| core/relay/controller/gemini.go | Migrates Gemini request usage to return RequestUsage. |
| core/relay/controller/embed.go | Migrates Embeddings request usage to return RequestUsage. |
| core/relay/controller/edits.go | Adds size allowlist validation and records size/quality in edits usage context. |
| core/relay/controller/edits_test.go | Adds tests for edits size allowlist and updates usage assertions for RequestUsage. |
| core/relay/controller/completions.go | Migrates Completions request usage to return RequestUsage. |
| core/relay/controller/chat.go | Migrates Chat request usage to return RequestUsage. |
| core/relay/controller/anthropic.go | Migrates Anthropic request usage to return RequestUsage. |
| core/relay/adaptor/zhipu/constants.go | Migrates image pricing from legacy image price maps to conditional prices. |
| core/relay/adaptor/utils.go | Extends success-status logic for Videos modes. |
| core/relay/adaptor/stepfun/constants.go | Migrates image pricing to conditional prices. |
| core/relay/adaptor/siliconflow/constants.go | Removes legacy image price maps from model constants. |
| core/relay/adaptor/openai/video.go | Adds multipart support for Videos requests + handlers for Videos object/content/delete endpoints and store pinning. |
| core/relay/adaptor/openai/response_test.go | Adds tests ensuring /videos responses store video IDs and mark async usage. |
| core/relay/adaptor/openai/integration_test.go | Ensures Videos mode models route to /videos. |
| core/relay/adaptor/openai/image_test.go | Adds test for multipart video conversion model rewrite. |
| core/relay/adaptor/openai/async_usage.go | Extends async usage fetching to return UsageContext and support Videos endpoints. |
| core/relay/adaptor/openai/async_usage_test.go | Adds test for video size normalization when calculating usage context. |
| core/relay/adaptor/openai/adaptor.go | Adds Videos modes support + URL building + request/response conversion wiring. |
| core/relay/adaptor/interface.go | Extends adaptor interfaces to include UsageContext and adds AsyncUsageRequest. |
| core/relay/adaptor/baidu/constants.go | Migrates image pricing to conditional prices. |
| core/relay/adaptor/azure/main.go | Adds Azure routing support for Videos endpoints (preview api-version). |
| core/relay/adaptor/azure/main_test.go | Adds Azure URL tests for Videos endpoint variants. |
| core/relay/adaptor/azure/async_usage.go | Extends Azure async usage fetcher to return UsageContext and support Videos modes. |
| core/relay/adaptor/azure/async_usage_test.go | Adds test for video size normalization helper behavior. |
| core/relay/adaptor/ali/constants.go | Adds Ali video model entries using VideoGenerationsJobs mode. |
| core/relay/adaptor/ali/async_usage.go | Adds Ali async usage fetching for video tasks and sets size-based usage context when possible. |
| core/relay/adaptor/ali/adaptor.go | Enables Ali support for video modes and wires Ali video handlers/conversion. |
| core/model/yaml_integration.go | Adds YAML type-name mappings for new Videos modes. |
| core/model/usage.go | Adds size/quality conditions, UsageContext, and updates conditional price selection signature/logic. |
| core/model/usage_test.go | Updates tests for new selection signature and adds media-conditions coverage. |
| core/model/modelconfig.go | Removes legacy image price maps and adds MaxVideoGenerationSeconds + group override wiring. |
| core/model/modelconfig_test.go | Updates tests to reflect new pricing model and adds tests for video seconds override clearing/override behavior. |
| core/model/log.go | Persists UsageContext in logs and threads it through consumption recording. |
| core/model/log_test.go | Updates log tests for new RecordConsumeLog signature. |
| core/model/local_cache.go | Removes cloning of legacy ImagePrices from group model config cache. |
| core/model/groupmodel.go | Removes legacy group image price maps and adds max video seconds override fields. |
| core/model/configkey.go | Adds video_sizes config key constant. |
| core/model/batch.go | Threads UsageContext through batch log recording. |
| core/model/async_usage.go | Persists UsageContext on async usage info and updates log update helpers for failure/usage context. |
| core/model/async_usage_postgres_integration_test.go | Updates integration tests for new async completion signature. |
| core/middleware/distributor.go | Adds Videos mode mapping/dispatch, video ID storage, and model extraction for multipart/videos flows. |
| core/middleware/distributor_video_test.go | Adds tests for extracting model names from video multipart/JSON requests. |
| core/middleware/ctxkey.go | Adds context key for video ID. |
| core/docs/swagger.yaml | Documents new fields (usage_context, max_video_generation_seconds, size/quality) and adds Videos endpoints/models. |
| core/docs/swagger.json | Generated swagger JSON update reflecting Videos endpoints and new fields. |
| core/docs/docs.go | Generated docs template update reflecting Videos endpoints and new fields. |
| core/controller/relay.go | Adds controller handlers for Videos endpoints (create/get/content/delete/remix) and swagger annotations. |
| core/controller/relay-controller.go | Updates request usage plumbing to include UsageContext in balance checks, logging, and async usage persistence. |
| core/controller/relay-channel.go | Pins channels for stored video read/delete endpoints. |
| core/controller/group.go | Updates group model config API DTO to remove legacy image prices and add max video seconds override. |
| core/controller/dashboard.go | Updates dashboard model DTO to remove legacy image price maps and expose max video seconds. |
| core/common/consume/record.go | Records consumption using UsageContext.ServiceTier and persists usage context. |
| core/common/consume/consume.go | Updates amount calculation/selection to use UsageContext and adds test hook for record/skip logic. |
| core/common/consume/consume_test.go | Updates tests for new amount calculation signature. |
| core/common/consume/consume_record_test.go | Adds tests for record/skip behavior on Videos stored-read modes. |
Files not reviewed (1)
- core/docs/docs.go: Language not supported
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
+497
to
+507
| if condition.Size != "" && | ||
| normalizeConditionValue(condition.Size) != normalizeConditionValue(c.PriceCondition.Size) { | ||
| return false | ||
| } | ||
|
|
||
| if condition.Quality != "" && | ||
| normalizeConditionValue( | ||
| condition.Quality, | ||
| ) != normalizeConditionValue( | ||
| c.PriceCondition.Quality, | ||
| ) { |
Comment on lines
+123
to
+137
| func validateSupportedImageSize(size string, mc model.ModelConfig) error { | ||
| if size == "" { | ||
| return nil | ||
| } | ||
|
|
||
| sizes, ok := model.GetModelConfigStringSlice(mc.Config, model.ModelConfigImageSizes) | ||
| if !ok || len(sizes) == 0 { | ||
| return nil | ||
| } | ||
|
|
||
| if slices.Contains(sizes, size) { | ||
| return nil | ||
| } | ||
|
|
||
| return NewBadRequestParamError(fmt.Sprintf("unsupported image size `%s`", size)) |
Comment on lines
+51
to
+66
| func validateSupportedVideoSize(size string, mc model.ModelConfig) error { | ||
| if size == "" { | ||
| return nil | ||
| } | ||
|
|
||
| sizes, ok := model.GetModelConfigStringSlice(mc.Config, model.ModelConfigVideoSizes) | ||
| if !ok || len(sizes) == 0 { | ||
| return nil | ||
| } | ||
|
|
||
| if slices.Contains(sizes, size) { | ||
| return nil | ||
| } | ||
|
|
||
| return NewBadRequestParamError(fmt.Sprintf("unsupported video size `%s`", size)) | ||
| } |
Comment on lines
+116
to
+123
| func getVideoGenerationJobRequest(c *gin.Context) (*relaymodel.VideoGenerationJobRequest, error) { | ||
| contentType := c.Request.Header.Get("Content-Type") | ||
| if strings.HasPrefix(contentType, "multipart/form-data") { | ||
| if err := common.ParseMultipartFormWithLimit(c.Request); err != nil { | ||
| return nil, err | ||
| } | ||
|
|
||
| return getMultipartVideoGenerationJobRequest(c) |
Comment on lines
+146
to
+155
| func validateParsedVideoGenerationJobRequest(request *relaymodel.VideoGenerationJobRequest) error { | ||
| if request.NSeconds < 0 { | ||
| return errors.New("invalid n_seconds: must be non-negative") | ||
| } | ||
|
|
||
| if request.NVariants < 0 { | ||
| return errors.New("invalid n_variants: must be non-negative") | ||
| } | ||
|
|
||
| return nil |
Comment on lines
+169
to
+190
| if valueNode.TypeSafe() == ast.V_STRING { | ||
| value, err := valueNode.String() | ||
| if err != nil { | ||
| return 0, true, fmt.Errorf("invalid %s: %w", name, err) | ||
| } | ||
|
|
||
| parsed, err := parseOptionalPositiveInt(value, name) | ||
| if err != nil { | ||
| return 0, true, err | ||
| } | ||
|
|
||
| return parsed, true, nil | ||
| } | ||
|
|
||
| value, err := valueNode.Int64() | ||
| if err != nil { | ||
| return 0, true, fmt.Errorf("invalid %s: %w", name, err) | ||
| } | ||
|
|
||
| if value < 0 { | ||
| return 0, true, fmt.Errorf("invalid %s: must be non-negative", name) | ||
| } |
Comment on lines
+232
to
+245
| func parseOptionalPositiveInt(value, name string) (int, error) { | ||
| value = strings.TrimSpace(value) | ||
| if value == "" { | ||
| return 0, nil | ||
| } | ||
|
|
||
| parsed, err := strconv.Atoi(value) | ||
| if err != nil { | ||
| return 0, fmt.Errorf("invalid %s: %w", name, err) | ||
| } | ||
|
|
||
| if parsed < 0 { | ||
| return 0, fmt.Errorf("invalid %s: must be non-negative", name) | ||
| } |
Comment on lines
45
to
+53
| func ValidateImagesEditsRequest(c *gin.Context, mc model.ModelConfig) error { | ||
| if err := parseImagesEditsForm(c); err != nil { | ||
| return err | ||
| } | ||
|
|
||
| if err := validateSupportedImageSize(c.PostForm("size"), mc); err != nil { | ||
| return err | ||
| } | ||
|
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.