.NET: Support for intermediate state from Tools for AG-UI

[anktsrkr]

Hi,
I am trying to build Deep Research Agent using AGUI and CopilotKit. I would like to do below steps -

1. Break the objective into 3-4 short steps for the UI to display

This is working using useCoAgentStateRender with DelegatingAIAgent as above.

2. For Each Steps It will call perform_research tool. <-- This steps also works fine. Here is my tool definition -

[Description("Perform a research task and track progress.")]
static async IAsyncEnumerable<AgentRunResponseUpdate> PerformResearch([Description("The research query")] string query)
{
    var agentState = new AgentStateSnapshot { Searches = [] };
    agentState.Searches.Add(new SearchInfo { Query = query, Status= "InProgress"});

    byte[] stateBytes = JsonSerializer.SerializeToUtf8Bytes(
                agentState);

    // Emit state snapshot as DataContent
    yield return new AgentRunResponseUpdate
    {
        Contents = [new DataContent(stateBytes, "application/json")]
    };
    // Simulate initial research phase
    var research_content = $" Research Overview: Actual content......";
  

    agentState.Searches.Add(new SearchInfo { Query = query, Status= "Done" });
    stateBytes = JsonSerializer.SerializeToUtf8Bytes(
              agentState);

    yield return new AgentRunResponseUpdate
    {
        Contents = [new DataContent(stateBytes, "application/json")]
    };

   yield return new AgentRunResponseUpdate
    {
        Contents = [new TextContent(research_content)]
    };
}

Above snippet has few problem

• Unable to access existing state
• all three outputs are returning together at once it seems
• all three outputs are returning as TEXT_MESSAGE_CONTENT so not invoking useCoAgentStateRender and it is getting treated as Tool's actual output.

My question -

• How do I get existing Agent State in Tool?
• How do I return intermediate state seperatly.
• How do I stream these intermediate state as STATE_SNAPSHOT

[anktsrkr]

Instead of above, I can think of CustomFunctionCallingMiddleware but seems we cant yield the AgentRunResponseUpdate from it.

Any suggestion on this @javiercn ?

[javiercn]

@anktsrkr thanks for contacting us.

In general, you need to decorate the agent to do these things. We will have more demos coming soon, but I'll try to give you some pointers.

• How do I get existing Agent State in Tool?

Since the state is sent on every turn, there are a couple of ways I can think of doing this. You can get the state on RunStreamingAsync from the chat options, create new chat options and add a tool instance to the options that contains the state.

• How do I return intermediate state separately.
• How do I stream these intermediate state as STATE_SNAPSHOT

Typically, you can't do this from tool calls. the pattern in AG-UI is that you define additional tool definitions that map tool invocations to state updates, and then you convert those tool calls into state snapshots/deltas before sending them to the client. We will have a sample for this pattern in the following days. I'll update this issue when that's available.

https://github.com/microsoft/agent-framework/blob/main/dotnet/samples/AGUIClientServer/AGUIDojoServer/SharedStateAgent.cs shows how to access the state from AG-UI and how to emit it.

Let me know if this helps or you need more assistance.

[markwallace-microsoft]

@anktsrkr, friendly reminder — this issue is waiting on your response. Please share any updates when you get a chance. (This is an automated message.)