[BUG] The result is inconsistent after serializing and deserializing ChatRequestUserMessage

[fairjm]

Describe the bug
After serializing ChatRequestUserMessage and then deserializing it, the structure of the result returned is inconsistent with the original structure.
It seems that BinaryData directly converted the List into a string structure.

Exception or Stack Trace

To Reproduce
run the following code:

Code Snippet

        final ChatRequestUserMessage user = new ChatRequestUserMessage(List.of(
                new ChatMessageTextContentItem("text"),
                new ChatMessageImageContentItem(new ChatMessageImageUrl("testImage"))
        ));
        // log.info("user:{}", user);
        final String content = BinaryData.fromObject(user).toString();
        log.info("json content:{}", content);
        final ChatRequestUserMessage converted = BinaryData.fromString(content).toObject(ChatRequestUserMessage.class);
        // log.info("converted:{}", converted);

Expected behavior
converted should have the same structure as user.

Screenshots
the user:

the converted:

Setup (please complete the following information):

• OS: windows 11
• IDE: IntelliJ
• Library/Libraries: azure-ai-openai 1.0.0-beta.8
• Java version: 17

[fairjm]

I found a relatively simple temporary solution - perform an additional conversion on the content of ChatRequestUserMessage, but this method would also bring some strange issues if the text content itself is an array.
Not good but it works.

final BinaryData b = converted.getContent();
try {
    final List<ChatMessageContentItem> items = b.toObject(new TypeReference<>() {
    });
    converted = new ChatRequestUserMessage(items);
} catch (Exception ignored) {}

and for ChatCompetionsOptions:

private void fixMessage(ChatCompletionsOptions options) {
    boolean changed = false;
    List<ChatRequestMessage> messages = new ArrayList<>();
    for (ChatRequestMessage message : options.getMessages()) {
        if (message instanceof ChatRequestUserMessage u) {
            final BinaryData content = u.getContent();
            try {
                final List<ChatMessageContentItem> items = content.toObject(new TypeReference<>() {
                });
                messages.add(new ChatRequestUserMessage(items));
                changed = true;
                continue;
            } catch (Exception ignored) {
            }
        }
        messages.add(message);
    }
    if (changed) {
        try {
            Field messagesField = ChatCompletionsOptions.class.getDeclaredField("messages");
            messagesField.setAccessible(true);
            messagesField.set(options, messages);
        } catch (Exception e) {
            log.error("fixMessage - set message error", e);
        }
    }
}

[mssfang]

@fairjm Thank you for reporting this bug. I have a fix in linked PR #40687. It should fix the issue.

[fairjm]

sorry, but I found another issue.
I update to beta.10.
The deserialized object is :

You can see stringContent and chatMessageContentItems are both null.
And the toJson method in ChatRequestUserMessage:

    @Override
    public JsonWriter toJson(JsonWriter jsonWriter) throws IOException {
        jsonWriter.writeStartObject();
        if (stringContent != null) {
            jsonWriter.writeStringField("content", stringContent);
        } else if (chatMessageContentItems != null) {
            jsonWriter.writeArrayField("content", chatMessageContentItems, JsonWriter::writeJson);
        }
        jsonWriter.writeStringField("role", this.role == null ? null : this.role.toString());
        jsonWriter.writeStringField("name", this.name);
        return jsonWriter.writeEndObject();
    }

it means it will return no content if toJson:

Send request to openAI will fail in OpenAIClient:

    @Generated
    @ServiceMethod(returns = ReturnType.SINGLE)
    public ChatCompletions getChatCompletions(String deploymentOrModelName,
        ChatCompletionsOptions chatCompletionsOptions) {
        // Generated convenience method for getChatCompletionsWithResponse
        RequestOptions requestOptions = new RequestOptions();
        return getChatCompletionsWithResponse(deploymentOrModelName, BinaryData.fromObject(chatCompletionsOptions),
            requestOptions).getValue().toObject(ChatCompletions.class);
    }

I think you'd better check it out. @mssfang