Skip to content

Multimodal response generation support #206

Description

@inspire-boy

Some models support generate image in context.

Google doc

Documentation for Gemini: https://cloud.google.com/vertex-ai/generative-ai/docs/multimodal-response-generation

Model

  • gemini-2.0-flash-exp
  • gemini-2.0-flash-exp-image-generation

Request raw content sample

{
  "contents": [
    {
      "parts": [
        {
          "text": "Create a image of a red cat."
        }
      ],
      "role": "user"
    }
  ],
  "generationConfig": {
    "responseMimeType": "text/plain",
    "responseModalities": [
      "Text",
      "Image"
    ]
  }
}

Response raw content sample

{
      "content": {
        "parts": [
          {
            "inlineData": {
              "mimeType": "image/png",
              "data": "[------IMAGE BASE64------]"
            }
          }
        ],
        "role": "model"
      },
      "index": 0
    },
}

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type
No fields configured for issues without a type.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions