Add vLLM OCI artifact format support with media type application/vnd.docker.ai.vllm.v1#7
Draft
Copilot wants to merge 3 commits into
Draft
Add vLLM OCI artifact format support with media type application/vnd.docker.ai.vllm.v1#7Copilot wants to merge 3 commits into
Copilot wants to merge 3 commits into
Conversation
…docker.ai.vllm.v1 Co-authored-by: ericcurtin <1694275+ericcurtin@users.noreply.github.com>
Co-authored-by: ericcurtin <1694275+ericcurtin@users.noreply.github.com>
Copilot
AI
changed the title
[WIP] There is a application/vnd.docker.ai.gguf.v3 OCI artifact format that can be used with llama.cpp in this repo, create and add another OCI artifact format for vLLM.
Add vLLM OCI artifact format support with media type application/vnd.docker.ai.vllm.v1
Sep 11, 2025
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR adds support for vLLM models as a new OCI artifact format alongside the existing GGUF support. The implementation introduces the media type
application/vnd.docker.ai.vllm.v1for vLLM models and enables seamless packaging and distribution of both GGUF and vLLM formats.Changes Made
New OCI Media Type
MediaTypeVLLMconstant with valueapplication/vnd.docker.ai.vllm.v1FormatVLLMconstant for vLLM format identificationvLLM Support Infrastructure
internal/vllmpackage with complete model creation and handling logicFromVLLM()function to the builder package for creating vLLM model artifactsDockerfile.vllmfor containerizing vLLM modelsEnhanced CLI Tool
The CLI tool now automatically detects model format based on file extensions:
.gguffiles → GGUF format (existing behavior).safetensorsand.binfiles → vLLM format (new)Updated Interfaces
Extended existing interfaces to support both formats:
VLLMPaths()method toModelinterfaceVLLMPath()method toModelBundleinterfaceExamples
Packaging a vLLM model:
Packaging a GGUF model (unchanged):
The tool will automatically detect the format and use the appropriate media type in the OCI manifest.
Testing
Backward Compatibility
This is a purely additive change that maintains full backward compatibility. Existing GGUF workflows continue to work exactly as before, while new vLLM workflows are now supported alongside them.
✨ Let Copilot coding agent set things up for you — coding agent works faster and does higher quality work when set up for your repo.