.NET: [Bug]: Per-request ChatClient middleware in ChatClientFactory only logs once per run despite multiple LLM exchanges

[helloxubo]

Description

When a per-request chat-client middleware is registered via ChatClientAgentRunOptions.ChatClientFactory (e.g. .Use(PerRequestChatClientMiddleware, null)), the middleware appears to be executed only once per agent run (one Pre/Post pair), even when the underlying chat client performs multiple exchanges with the LLM within that run (e.g. model requests a function, function executes, then model returns final response). This makes per-request middleware behave like an outer wrapper rather than triggering on each actual chat exchange.

Reproduction steps

1. In the sample samples/02-agents/Agents/Agent_Step11_Middleware/Program.cs, register a per-request chat-client factory that uses PerRequestChatClientMiddleware:
   • ChatClientFactory = chatClient => chatClient.AsBuilder().Use(PerRequestChatClientMiddleware, null).Build()
2. Run an agent prompt that causes the model to call a function (so the run includes two underlying chat exchanges: one before the function call and one after).
3. Observe console logs.

Observed behavior (example log)

• Pii Middleware - Filtered Messages Pre-Run
• Guardrail Middleware - Filtered messages Pre-Run
• Per-Request Chat Client Middleware - Pre-Chat <-- appears once
• Chat Client Middleware A - Pre-Chat
• Chat Client Middleware B - Pre-Chat
• Chat Client Middleware B - Post-Chat
• Chat Client Middleware A - Post-Chat
• Function Name: GetDateTime - Middleware B Pre-Invoke
• Function Name: GetDateTime - Middleware A Pre-Invoke
• Function Name: GetDateTime - Middleware A Post-Invoke
• Function Name: GetDateTime - Middleware B Post-Invoke
• Chat Client Middleware A - Pre-Chat
• Chat Client Middleware B - Pre-Chat
• Chat Client Middleware B - Post-Chat
• Chat Client Middleware A - Post-Chat
• Per-Request Chat Client Middleware - Post-Chat <-- appears once
• Guardrail Middleware - Filtered messages Post-Run
• Pii Middleware - Filtered Messages Post-Run

Expected behavior

• The per-request chat-client middleware registered via ChatClientFactory should observe (Pre/Post) each underlying IChatClient exchange with the LLM that occurs during that run. In the example above, it is expected to log Pre/Post twice (once per exchange).

Code Sample

csharp
chatClient = chatClient.AsBuilder()
    .Use(getResponseFunc: ChatClientMiddlewareA, getStreamingResponseFunc: null)
    .Use(getResponseFunc: ChatClientMiddlewareB, getStreamingResponseFunc: null)
    .ConfigureOptions(options =>
    {
        options.RawRepresentationFactory = (response) =>
        {
            var opt = new OpenAI.Chat.ChatCompletionOptions();
#pragma warning disable SCME0001 // 类型仅用于评估，在将来的更新中可能会被更改或删除。取消此诊断以继续。
            opt.Patch.Set("$thinking.type"u8, "disabled");
#pragma warning restore SCME0001 // 类型仅用于评估，在将来的更新中可能会被更改或删除。取消此诊断以继续。
            return opt;
        };
    })
    .Build();

var originalAgent = chatClient
    .AsBuilder()
    .BuildAIAgent(
        instructions: "You are an AI assistant that helps people find information.",
        tools: [AIFunctionFactory.Create(GetDateTime, name: nameof(GetDateTime))]);

var middlewareAgent = originalAgent
    .AsBuilder()
    .Use(FunctionCallMiddlewareA)
    .Use(FunctionCallMiddlewareB)
    .Use(PIIMiddleware,null)
    .Use(GuardrailMiddleware,null)
    .Build();

var optionsWithApproval = new ChatClientAgentRunOptions()
{
    ChatClientFactory = (chatClient) => chatClient
        .AsBuilder()
        .Use(PerRequestChatClientMiddleware, null) // Using the non-streaming for handling streaming as well
        .Build()
};

var response = await middlewareAgent.RunAsync("Hello, what time is it?",options: optionsWithApproval, cancellationToken: default);


async Task<ChatResponse> ChatClientMiddlewareA(IEnumerable<ChatMessage> message, ChatOptions? options, IChatClient innerChatClient, CancellationToken cancellationToken)
{
    Console.WriteLine("Chat Client Middleware A - Pre-Chat");
    var response = await innerChatClient.GetResponseAsync(message, options, cancellationToken);
    Console.WriteLine("Chat Client Middleware A - Post-Chat");

    return response;
}

async Task<ChatResponse> ChatClientMiddlewareB(IEnumerable<ChatMessage> message, ChatOptions? options, IChatClient innerChatClient, CancellationToken cancellationToken)
{
    Console.WriteLine("Chat Client Middleware B - Pre-Chat");
    var response = await innerChatClient.GetResponseAsync(message, options, cancellationToken);
    Console.WriteLine("Chat Client Middleware B - Post-Chat");

    return response;
}

async ValueTask<object?> FunctionCallMiddlewareA(AIAgent agent, FunctionInvocationContext context, Func<FunctionInvocationContext, CancellationToken, ValueTask<object?>> next, CancellationToken cancellationToken)
{
    Console.WriteLine($"Function Name: {context!.Function.Name} - Middleware A Pre-Invoke");
    var result = await next(context, cancellationToken);
    Console.WriteLine($"Function Name: {context!.Function.Name} - Middleware A Post-Invoke");

    return result;
}

async ValueTask<object?> FunctionCallMiddlewareB(AIAgent agent, FunctionInvocationContext context, Func<FunctionInvocationContext, CancellationToken, ValueTask<object?>> next, CancellationToken cancellationToken)
{
    Console.WriteLine($"Function Name: {context!.Function.Name} - Middleware B Pre-Invoke");
    var result = await next(context, cancellationToken);
    Console.WriteLine($"Function Name: {context!.Function.Name} - Middleware B Post-Invoke");

    return result;
}

async Task<AgentResponse> PIIMiddleware(IEnumerable<ChatMessage> messages, AgentSession? session, AgentRunOptions? options, AIAgent innerAgent, CancellationToken cancellationToken)
{
    // Redact PII information from input messages
    var filteredMessages = FilterMessages(messages);
    Console.WriteLine("Pii Middleware - Filtered Messages Pre-Run");

    var response = await innerAgent.RunAsync(filteredMessages, session, options, cancellationToken).ConfigureAwait(false);

    // Redact PII information from output messages
    response.Messages = FilterMessages(response.Messages);

    Console.WriteLine("Pii Middleware - Filtered Messages Post-Run");

    return response;

    static IList<ChatMessage> FilterMessages(IEnumerable<ChatMessage> messages)
    {
        return messages.Select(m => new ChatMessage(m.Role, FilterPii(m.Text))).ToList();
    }

    static string FilterPii(string content)
    {
        return content;
    }
}

async Task<AgentResponse> GuardrailMiddleware(IEnumerable<ChatMessage> messages, AgentSession? session, AgentRunOptions? options, AIAgent innerAgent, CancellationToken cancellationToken)
{
    // Redact keywords from input messages
    var filteredMessages = FilterMessages(messages);

    Console.WriteLine("Guardrail Middleware - Filtered messages Pre-Run");

    // Proceed with the agent run
    var response = await innerAgent.RunAsync(filteredMessages, session, options, cancellationToken);

    // Redact keywords from output messages
    response.Messages = FilterMessages(response.Messages);

    Console.WriteLine("Guardrail Middleware - Filtered messages Post-Run");

    return response;

    List<ChatMessage> FilterMessages(IEnumerable<ChatMessage> messages)
    {
        return messages.Select(m => new ChatMessage(m.Role, FilterContent(m.Text))).ToList();
    }

    static string FilterContent(string content)
    {
        foreach (var keyword in new[] { "harmful", "illegal", "violence" })
        {
            if (content.Contains(keyword, StringComparison.OrdinalIgnoreCase))
            {
                return "[REDACTED: Forbidden content]";
            }
        }

        return content;
    }
}

async Task<ChatResponse> PerRequestChatClientMiddleware(IEnumerable<ChatMessage> message, ChatOptions? options, IChatClient innerChatClient, CancellationToken cancellationToken)
{
    Console.WriteLine("Per-Request Chat Client Middleware - Pre-Chat");
    var response = await innerChatClient.GetResponseAsync(message, options, cancellationToken);
    Console.WriteLine("Per-Request Chat Client Middleware - Post-Chat");

    return response;
}

[Description("The current datetime offset.")]
static string GetDateTime()
    => DateTimeOffset.Now.ToString();

Error Messages / Stack Traces

Package Versions

1.8.0

.NET Version

.net 10

Additional Context

No response

[github-actions]

🤖 Automated triage reproduction notes (agent-authored — trust but verify)

Agent analysis

Repro: ChatClientAgent.RunCoreAsync() in dotnet/src/Microsoft.Agents.AI/ChatClient/ChatClientAgent.cs applies ChatClientFactory (line 222) AFTER retrieving this.ChatClient which already includes FunctionInvokingChatClient; this places per-request middleware outside FIC so it only intercepts the single top-level GetResponseAsync. Trigger: register any middleware via ChatClientAgentRunOptions.ChatClientFactory and invoke a prompt that causes a function call (two LLM exchanges). Minimal repro: create ChatClientAgent with a tool, pass ChatClientAgentRunOptions with a ChatClientFactory adding a counting middleware, send a message that triggers function calling, assert middleware fires twice.

• Failing test: dotnet/tests/Microsoft.Agents.AI.UnitTests/ChatClient/ChatClientAgent_PerRequestMiddlewareTests.cs
• Files examined: dotnet/src/Microsoft.Agents.AI/ChatClient/ChatClientAgent.cs, dotnet/src/Microsoft.Agents.AI/ChatClient/ChatClientAgentRunOptions.cs, dotnet/src/Microsoft.Agents.AI/ChatClient/ChatClientExtensions.cs, dotnet/tests/Microsoft.Agents.AI.UnitTests/ChatClient/ChatClientAgentRunOptionsTests.cs, dotnet/tests/Microsoft.Agents.AI.UnitTests/ChatClient/ChatClientAgentTestHelper.cs, dotnet/tests/Microsoft.Agents.AI.UnitTests/ChatClient/ChatClientAgent_ChatHistoryManagementTests.cs
• Tests run: dotnet/tests/Microsoft.Agents.AI.UnitTests/ChatClient/ChatClientAgent_PerRequestMiddlewareTests.cs
• Reported version: 1.8.0
• Current version: 1.8.0