Python: Application Insights logs a single model span when using AzureOpenAIResponsesClient Agent regardless of tool calls

[jomalsan]

I am working with the AzureOpenAIResponsesClient and using setup_observability to log to application insights. I noticed this in my application, but confirmed that it occurs on the agent_observability sample as well. My slight modification to use AzureOpenAIResponsesClient is attached as a repro.

When I look at the telemetry in Application Insights, there is a single model span that aligns with the span for the full agent execution. The desired state is for each call to the model (in this case 7, 6 serial tool calls and a final assistant message) to be it's own span with token usage information. I also suspect that the token usage is not being summarized correctly. While this example looks correct, I have seen many examples of the model generating 100's of tokens in tool call arguments and output, but the final span has 20's of tokens

---

# Modified form of: https://github.com/microsoft/agent-framework/blob/main/python/samples/getting_started/observability/agent_observability.py
import asyncio
import os
from random import randint
from typing import Annotated

from agent_framework import ChatAgent
from agent_framework.azure import AzureOpenAIResponsesClient
from agent_framework.observability import get_tracer, setup_observability
from azure.identity import DefaultAzureCredential
from opentelemetry.trace import SpanKind
from opentelemetry.trace.span import format_trace_id
from pydantic import Field

"""
This sample shows how you can observe an agent in Agent Framework by using the
same observability setup function with Azure OpenAI Responses Client.
"""


async def get_weather(
    location: Annotated[str, Field(description="The location to get the weather for.")],
) -> str:
    """Get the weather for a given location."""
    await asyncio.sleep(randint(0, 10) / 10.0)  # Simulate a network call
    conditions = ["sunny", "cloudy", "rainy", "stormy"]
    return f"The weather in {location} is {conditions[randint(0, 3)]} with a high of {randint(10, 30)}°C."


async def main():
    # This will enable tracing and create the necessary tracing, logging and metrics providers
    # based on environment variables. See the .env.example file for the available configuration options.
    connection_string = os.getenv("APPLICATION_INSIGHTS_CONNECTION_STRING")
    setup_observability(
        enable_sensitive_data=True,
        applicationinsights_connection_string=connection_string,
    )

    questions = [
        "What's the weather in Amsterdam, Paris, London, New York City, Las Vegas and Seattle?",
    ]

    with get_tracer().start_as_current_span(
        "Scenario: Agent Chat", kind=SpanKind.CLIENT
    ) as current_span:
        print(f"Trace ID: {format_trace_id(current_span.get_span_context().trace_id)}")

        # Create Azure OpenAI Responses client
        azure_endpoint = os.getenv("AZURE_OPENAI_API_BASE")
        if not azure_endpoint:
            raise ValueError("AZURE_OPENAI_API_BASE environment variable is required")

        agent = ChatAgent(
            chat_client=AzureOpenAIResponsesClient(
                endpoint=azure_endpoint,
                credential=DefaultAzureCredential(),
                deployment_name="o4-mini",
            ),
            tools=get_weather,
            name="WeatherAgent",
            instructions="You are a weather assistant.",
        )
        thread = agent.get_new_thread()
        for question in questions:
            print(f"User: {question}")
            print(f"{agent.display_name}: ", end="")
            async for update in agent.run_stream(
                question,
                thread=thread,
            ):
                if update.text:
                    print(update.text, end="")


if __name__ == "__main__":
    asyncio.run(main())

[eavanvalkenburg]

Thanks for reporting this @jomalsan I found a bug in our setup (specifically, the order of decorators for the Azure OpenAI Responses Client, others were all correct, just that one), fixed it now in #1255, we'll let you know when it will be released.