Tracing non-Gemini models

[anirudh-cogent]

Hi,
What is the current recommended approach for tracing non-Gemini models. Is there a plan to support integrations with to W&B, MLFlow or Braintrust?

[BenMawnMahlauNBTC]

likely would be more robust otel support

#320
#569
#474

I dont think those PRs account for litellm instrumentation, but getting the foundation for more robust tracing and observability is definitely the first step.

[Maddog-max]

Just wanted to add some observations that I've encountered which relates to this, specifically when combining Google ADK with LiteLLM and MLflow's autologging feature (specifically, mlflow.litellm.autolog()).

My understanding is that the Google ADK Base Agent initializes an OpenTelemetry (OTEL) Tracing context. When MLflow's autologging for LiteLLM is then enabled, it also attempts to leverage an OTEL context. However, it seems MLflow detects the already-established context from ADK and tries to use it. This re-use of a context created by ADK appears to lead to an incompatibility that breaks MLflow's tracing, preventing it from accurately capturing the LLM interactions.

This prevents proper observability via MLflow when ADK is part of the stack. I'm curious if this is a known issue or if there are best practices for managing OTEL contexts in such a combined environment to avoid this interference!

Hope this helps.

[surajksharma07]

Hi @anirudh-cogent and thanks to everyone who participated in this thread.

We're currently doing some housekeeping on our issue tracker. Given the rapid pace of development and the number of updates since this was last active, there's a good chance this issue has been resolved in a newer version. To help us keep our backlog focused on current issues, we are closing this as stale.

If you're still facing this problem, could you please update to the very latest version of the library and confirm the issue still exists? If it does, please comment here, and we will be happy to reopen this issue and investigate further.

Thanks for your contribution and understanding!

[pitta-bread]

I have tested this on latest version (1.16.0) and can confirm it is still an issue. The google-adk OTEL tracing is breaking the ability of mlflow to pick up and report litellm traces via the mlflow.litellm.autolog() feature.

Please reopen :)

[salvatore-cipolla]

I confirm it is an issue. Is it possible to reopen it? Basically this page of the doc seems to work only for google models.