Python: .NET: FunctionResultContent behaves differently between Python and C#

[markwallace-microsoft]

The result property in FunctionResultContent behaves differently between Python and C# when running the mcp case.

In python, the result of FunctionResultContent is list of TextContent

In C#, the result of FunctionResultContent is JsonElement of chatmessage

[stephentoub]

For .NET, the FunctionInvokingChatClient is the primary creator of FunctionResultContent, and it just takes whatever the AIFunction returns and uses that as the result directly, so it's not that FunctionResultContent is formatted a certain way, it's based entirely on what AIFunction.InvokeAsync returns.

The AIFunction instances created by AIFunctionFactory.Create by default serialize to JSON whatever the function returns. That's done primarily for practical reasons, in that it's the AIFunction itself that knows what kind of data is being produced and thus has the best chance to correctly serialize it. This is especially the case with trimming and Native AOT, where to be serialized correctly the JsonSerializerOptions needs to have information about the type, and the AIFunction carries with it the JsonSerializerOptions to use. If it were instead the job of the FunctionInvokingChatClient or the leaf IChatClient to serialize the result of the function, it may not have the necessary information to do so. Thus, by default, AIFunctionFactory.Create serializes the object the function returns and then returns out of InvokeAsync a JsonElement. That's stored into the FunctionResultContent by the FunctionInvokingChatClient, and then 99% of the time the leaf IChatClient that sends the result back to the LLM does so as JSON, so that JsonElement can just be forwarded along as-is.

It is customizable, however. AIFunctionFactory.Create accepts an options instance that has a MarshalResult delegate on it. If someone doesn't want the serialization behavior, they can specify a delegate to transform the result object however is desired, including doing nothing to it, in which case the raw object will be stored into the FunctionResultContent.

[SergeyMenshykh]

@markwallace-microsoft, what is the expected outcome here? Do we need the result types to be aligned? What scenarios are we targeting with this?

[SergeyMenshykh]

@eavanvalkenburg is the reason provided above regarding AIFunction being responsible for result serialization (as it's in the best position to have all relevant information about result type and settings for its serialization) applicable to the Python version of AIFunction as well? If so, how feasible is it?

[eavanvalkenburg]

In the general case, we do the same, a AIFunction returns whatever it wants to return and only when it get's into the function invocation decorator code is it turned into a serializable object.

This case, a local MCP server, is the exception, because it returns a set of types from the MCP spec, and we turn those into our types because then serialization is easier downstream, then if we would have left the MCP types (classes from the MCP package), so in all other cases, we indeed leave the results as is. @SergeyMenshykh

[SergeyMenshykh]

In the general case, we do the same, a AIFunction returns whatever it wants to return and only when it get's into the function invocation decorator code is it turned into a serializable object.

That's not the case with MEAI AIFunction, as far as I understand. It serializes the result to a JsonElement in all cases except when the result is of the AIContent type or a collection of them, which is returned as is: https://github.com/dotnet/extensions/blob/bbd5b59f54ebd3850b05dba2e95c240c1d1a7655/src/Libraries/Microsoft.Extensions.AI.Abstractions/Functions/AIFunctionFactory.cs#L979

@markwallace-microsoft, do you have access to the code you took the screenshots from?