Skip to content

Demo Toolcall App #1

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
keith-decker wants to merge 3 commits into
base: toolcall-handler-and-spans
Choose a base branch
from
Draft

Demo Toolcall App #1

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
86cccfb
Demo Toolcall App
keith-decker Mar 24, 2026
6675451
update example to use langchain
keith-decker Mar 24, 2026
f573876
update readme
keith-decker Mar 24, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
74 changes: 74 additions & 0 deletions ...instrumentation-langchain/src/opentelemetry/instrumentation/langchain/callback_handler.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,7 @@
MessagePart,
OutputMessage,
Text,
ToolCall,
)


Expand Down Expand Up @@ -273,3 +274,76 @@ def on_llm_error(
)
if llm_invocation.span and not llm_invocation.span.is_recording():
self._invocation_manager.delete_invocation_state(run_id=run_id)

def on_tool_start(
self,
serialized: dict[str, Any],
input_str: str,
*,
run_id: UUID,
parent_run_id: Optional[UUID] = None,
tags: Optional[list[str]] = None,
metadata: Optional[dict[str, Any]] = None,
inputs: Optional[dict[str, Any]] = None,
**kwargs: Any,
) -> None:
"""Start a tool call span when LangChain invokes a tool."""
tool_name = serialized.get("name", "unknown")
tool_description = serialized.get("description")

# Parse arguments - prefer structured inputs over input_str
arguments: Any = inputs if inputs is not None else input_str

tool_call = ToolCall(
name=tool_name,
arguments=arguments,
id=str(run_id),
tool_type="function",
tool_description=tool_description,
)
tool_call = self._telemetry_handler.start_tool_call(tool_call)
self._invocation_manager.add_invocation_state(
run_id=run_id,
parent_run_id=parent_run_id,
invocation=tool_call,
)

def on_tool_end(
self,
output: Any,
*,
run_id: UUID,
parent_run_id: Optional[UUID] = None,
**kwargs: Any,
) -> None:
"""End a tool call span successfully."""
tool_call = self._invocation_manager.get_invocation(run_id=run_id)
if tool_call is None or not isinstance(tool_call, ToolCall):
return

# Store the tool result
tool_call.tool_result = output

tool_call = self._telemetry_handler.stop_tool_call(tool_call)
if tool_call.span and not tool_call.span.is_recording():
self._invocation_manager.delete_invocation_state(run_id=run_id)

def on_tool_error(
self,
error: BaseException,
*,
run_id: UUID,
parent_run_id: Optional[UUID] = None,
**kwargs: Any,
) -> None:
"""End a tool call span with error."""
tool_call = self._invocation_manager.get_invocation(run_id=run_id)
if tool_call is None or not isinstance(tool_call, ToolCall):
return

error_otel = Error(message=str(error), type=type(error))
tool_call = self._telemetry_handler.fail_tool_call(
tool_call, error_otel
)
if tool_call.span and not tool_call.span.is_recording():
self._invocation_manager.delete_invocation_state(run_id=run_id)
9 changes: 9 additions & 0 deletions util/opentelemetry-util-genai/examples/toolcall/.env.example
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
# OTLP endpoint for trace/metric/log export
OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4317

# Enable experimental semantic conventions for GenAI
OTEL_SEMCONV_STABILITY_OPT_IN=gen_ai_latest_experimental

# Capture tool arguments and results in spans (optional)
# Options: SPAN_ONLY, EVENT_ONLY, SPAN_AND_EVENT, OFF
OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT=SPAN_ONLY
127 changes: 127 additions & 0 deletions util/opentelemetry-util-genai/examples/toolcall/README.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,127 @@
OpenTelemetry LangChain Tool Call Example
=========================================

This example demonstrates tool call instrumentation with LangChain.
The ``LangChainInstrumentor`` automatically creates spans for:

- LLM calls (chat completions)
- Tool executions (via ``on_tool_start``/``on_tool_end`` callbacks)

The demo wraps both in an ``agent_workflow`` parent span to maintain trace continuity.

When ``main.py`` runs, it exports traces to an OTLP-compatible endpoint showing:

- Agent workflow span as the root
- Chat span and tool call span as children of the workflow
- Proper semantic convention attributes on all spans

Sample Trace Output
-------------------

::

Span: agent_workflow
├── Kind: Internal
├── Attributes:
│ └── gen_ai.operation.name: invoke_agent
│
├── Span: chat gpt-4o-mini
│ ├── Kind: Client
│ └── Attributes:
│ ├── gen_ai.operation.name: chat
│ ├── gen_ai.request.model: gpt-4o-mini
│ └── gen_ai.provider.name: openai
│
└── Span: execute_tool get_weather
├── Kind: Internal
└── Attributes:
├── gen_ai.operation.name: execute_tool
├── gen_ai.tool.name: get_weather
├── gen_ai.tool.call.id: <uuid>
└── gen_ai.tool.type: function

Setup
-----

1. Copy ``.env.example`` to ``.env`` and add your OpenAI API key:

::

cp .env.example .env
# Edit .env and set OPENAI_API_KEY=sk-...

2. Start an OTLP-compatible collector (e.g., Jaeger):

::

docker run -d --name jaeger \
-p 16686:16686 \
-p 4317:4317 \
jaegertracing/jaeger:2.6

3. Set up a virtual environment and install dependencies:

::

python3 -m venv .venv
source .venv/bin/activate
pip install "python-dotenv[cli]"
pip install -r requirements.txt

# Install local packages (from repo root)
pip install -e ../../ # util-genai
pip install -e ../../../../instrumentation-genai/opentelemetry-instrumentation-langchain

Run
---

::

dotenv run -- python main.py

You should see console output like:

::

OpenTelemetry LangChain Tool Call Demo
========================================

Sending query: 'What is the weather in Paris?'
LLM Response:

Tool calls requested: 1
- get_weather({'location': 'Paris'})
Result: Weather in Paris: 18°C, cloudy

========================================
Demo complete! Check your OTLP endpoint for traces.

Traces will appear in Jaeger UI at http://localhost:16686.

Content Capturing
-----------------

To capture tool arguments and results in span attributes, set the environment
variables (already in ``.env.example``):

::

export OTEL_SEMCONV_STABILITY_OPT_IN=gen_ai_latest_experimental
export OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT=SPAN_ONLY

This adds ``gen_ai.tool.call.arguments`` and ``gen_ai.tool.call.result``
attributes to tool call spans.

How It Works
------------

The demo wraps the LLM call and tool execution in an ``agent_workflow`` span
to ensure they share the same trace. The ``LangChainInstrumentor`` uses callback
handlers to intercept LangChain operations:

1. ``on_chat_model_start`` - Creates a chat span when the LLM is invoked
2. ``on_llm_end`` - Ends the chat span with token usage and response data
3. ``on_tool_start`` - Creates an ``execute_tool`` span when a tool runs
4. ``on_tool_end`` - Ends the tool span with the result

All spans follow the OpenTelemetry GenAI semantic conventions.
Loading
Loading