Proxied file uploads bypass the LiteLLM Proxy and are sent to api.openai.com

[tranminhquang4421]

Describe the bug

When a model is served through a LiteLLM Proxy, ADK uploads file content (PDFs
and the other types in _SUPPORTED_FILE_CONTENT_MIME_TYPES) to an endpoint
resolved independently of the completion call. The completion goes to the
proxy. The upload does not.

For azure this fails locally with a missing credentials error. For openai
it does not fail at all. get_openai_credentials defaults api_base to
https://api.openai.com/v1, so if an OPENAI_API_KEY is present in the
environment the upload succeeds against the public OpenAI API. File content
meant for a self hosted or vendored proxy leaves it, and the developer's own
OpenAI key is what sends it.

That is the case I care about here. A proxy is usually deployed precisely
because the data is not supposed to reach the provider directly.

To Reproduce

.env, which is a plausible developer setup: proxy credentials plus a personal
OpenAI key left over from other work.

LITELLM_PROXY_API_BASE=http://127.0.0.1:8877
LITELLM_PROXY_API_KEY=sk-proxy-secret
OPENAI_API_KEY=sk-personal-developer-key

repro.py. The local server stands in for the proxy so we can see what it
receives, and the httpx patch records anything that leaves localhost instead of
sending it.

import asyncio, json, threading
from http.server import BaseHTTPRequestHandler, HTTPServer

from dotenv import load_dotenv
load_dotenv()

MODEL = "litellm_proxy/openai/gpt-4o"

PROXY_RECEIVED = []


class _Proxy(BaseHTTPRequestHandler):

  def do_POST(self):
    n = int(self.headers.get("Content-Length") or 0)
    self.rfile.read(n)
    PROXY_RECEIVED.append(self.path)
    if "files" in self.path:
      payload = {"id": "file-via-proxy", "object": "file", "bytes": n,
                 "created_at": 0, "filename": "f.pdf",
                 "purpose": "assistants", "status": "processed"}
    else:
      payload = {"id": "chatcmpl-1", "object": "chat.completion", "created": 0,
                 "model": "m", "choices": [{"index": 0, "finish_reason": "stop",
                 "message": {"role": "assistant", "content": "ok"}}],
                 "usage": {"prompt_tokens": 1, "completion_tokens": 1,
                           "total_tokens": 2}}
    body = json.dumps(payload).encode()
    self.send_response(200)
    self.send_header("Content-Type", "application/json")
    self.send_header("Content-Length", str(len(body)))
    self.end_headers()
    self.wfile.write(body)

  def log_message(self, *a):
    pass


HTTPServer.allow_reuse_address = True
threading.Thread(
    target=HTTPServer(("127.0.0.1", 8877), _Proxy).serve_forever,
    daemon=True).start()

import httpx
LEFT_THE_PROXY = []
_orig_send = httpx.AsyncClient.send


async def _spy(self, request, *args, **kwargs):
  if request.url.host not in ("127.0.0.1", "localhost"):
    LEFT_THE_PROXY.append(str(request.url))
    raise RuntimeError(f"blocked egress to {request.url}")
  return await _orig_send(self, request, *args, **kwargs)


httpx.AsyncClient.send = _spy

from google.adk.models.lite_llm import LiteLlm
from google.adk.models.llm_request import LlmRequest
from google.genai import types

llm = LiteLlm(model=MODEL)
request = LlmRequest(model=MODEL, contents=[types.Content(role="user", parts=[
    types.Part(text="summarize this"),
    types.Part.from_bytes(data=b"%PDF-1.4 confidential",
                          mime_type="application/pdf"),
])])


async def main():
  try:
    async for _ in llm.generate_content_async(request, stream=False):
      pass
  except Exception as e:
    print(f"raised         : {type(e).__name__}: {str(e)[:100]}")
  print(f"proxy received : {PROXY_RECEIVED or []}")
  print(f"left the proxy : {LEFT_THE_PROXY or []}")


asyncio.run(main())

Output on main at 370027a:

raised         : APIConnectionError: Connection error.
proxy received : []
left the proxy : ['https://api.openai.com/v1/files', ...]

The proxy receives nothing. The upload goes to OpenAI. It only surfaces as a
connection error because the script blocks the request. Without the block it
succeeds, using OPENAI_API_KEY.

Expected behavior

The upload goes wherever the completion goes. If the proxy endpoint cannot be
determined, the upload fails instead of falling back to a provider default.

Three configurations reach this

All three were run with the script above, changing only .env and MODEL.

#	Configuration	Model	proxy received	left the proxy
1	LITELLM_PROXY_API_BASE + LITELLM_PROXY_API_KEY	litellm_proxy/openai/gpt-4o	[]	api.openai.com/v1/files
2	USE_LITELLM_PROXY=true	openai/gpt-4o	[]	api.openai.com/v1/files
3	neither, only OPENAI_API_KEY	litellm_proxy/openai/gpt-4o	[]	api.openai.com/v1/files

Case 2 is worth spelling out. LiteLLM supports routing unprefixed models
through the proxy with USE_LITELLM_PROXY, and its own docstring says that
flag exists for Google ADK (BerriAI/litellm#10559). A model string of
openai/gpt-4o gives no hint that it is proxied, so anything matching on the
litellm_proxy/ prefix alone will miss it.

For reference, LiteLLM resolves the proxy endpoint for the completion call in
litellm/llms/litellm_proxy/chat/transformation.py:

api_base = api_base or get_secret_str("LITELLM_PROXY_API_BASE")
dynamic_api_key = api_key or get_secret_str("LITELLM_PROXY_API_KEY")

litellm.acreate_file does not consult those, so unless ADK forwards an
endpoint the two halves of the request diverge.

Relation to #6538 and #6578

#6578 (merged as 602e58d) fixed the payload shape for proxied models. Nested
litellm_proxy/azure/... identifiers now resolve to the underlying provider,
so the upload path runs instead of emitting an inline file_data block. That
part is correct and nothing here contests it.

The same commit also set custom_llm_provider="openai" for proxied uploads.
That is the line that turns cases 1 and 3 from a local failure into a request
to api.openai.com, because openai is the one provider whose credential
resolution defaults to the public API:

# litellm/llms/openai/common_utils.py
resolved_api_base = (
    api_base or litellm.api_base or os.getenv("OPENAI_BASE_URL")
    or os.getenv("OPENAI_API_BASE") or "https://api.openai.com/v1"
)

On main, _get_content takes only parts, provider and model. There is
no parameter through which a proxy endpoint could arrive, and ADK never sets
litellm.api_base. So there is no configuration in which that override reaches
a proxy. It resolves to the public endpoint every time.

Environment

• ADK main at 370027a. The behavior starts at 602e58d.
• litellm 1.85.7, and also reproduced on 1.84.0, the floor of the
  litellm>=1.84 constraint in pyproject.toml. In both,
  OPENAI_COMPATIBLE_BATCH_AND_FILES_PROVIDERS is {openai, hosted_vllm} and
  create_file has no litellm_proxy branch, so routing the upload as
  litellm_proxy is not an option either.
• Python 3.11, macOS.

Additional context

I have a fix and will open a PR referencing this issue. One part of it is a
behavior change I would rather have decided explicitly than slipped into a
diff: when no proxy endpoint can be determined, the upload raises instead of
quietly using the provider default. For anyone who has OPENAI_API_KEY set
that converts a currently "working" upload into an error. I think an explicit
error is better than a silent misroute, but it is a trade off and it is your
call.

[surajksharma07]

@tranminhquang4421 The hardcoded openai fallback in the upload path is what's sending it to api.openai.com.
#6723 lines up with the root cause and passed our own before/after check locally so it looks like the right fix.

Since you flagged the Azure-backed proxy case as unverified against a live endpoint could you try that PR against your actual setup and confirm it holds before we look at merging?
Would rather have your real-world config confirm it than assume from our synthetic repro alone.

[tranminhquang4421]

Thanks for checking it locally, glad the before/after lines up on your side too.

On the live Azure verification: I don't have an Azure backed proxy deployment.
That's what I meant by flagging it as unverified in #6723, and the same in
#6578 earlier. It's a gap in what I can test rather than a test I skipped, so
it isn't something I can close myself.

Two things that might help you decide how much that blocks merging.

The reported leak doesn't depend on Azure. Two of the three paths in #6722
resolve to openai and never enter Azure code at all:

#	Model	Provider ADK resolves
1	litellm_proxy/openai/gpt-4o	openai
2	openai/gpt-4o with USE_LITELLM_PROXY=true	openai
3	litellm_proxy/openai/gpt-4o, no proxy endpoint set	openai

The root cause is get_openai_credentials defaulting api_base to
https://api.openai.com/v1. Nothing in that chain is Azure specific, which is
why your before/after run reproduced it without an Azure deployment.

On main, an Azure backed proxy model doesn't reach Azure either. The
custom_llm_provider="openai" override replaces the provider before the
upload, so the azure branch in litellm/files/main.py never runs:

model                 : litellm_proxy/azure/my-deployment
provider ADK resolves : azure
custom_llm_provider   : openai      <- what acreate_file actually receives

Same repro, same .env, only the model swapped:

proxy received : []
left the proxy : ['https://api.openai.com/v1/files', ...]

So on current main the Azure case fails in the same place and for the same
reason as the OpenAI one, and a live Azure endpoint would mostly confirm that
the upload goes to api.openai.com instead of to Azure.

That said, I don't want to argue the Azure test is pointless, because after
#6723 it stops being the same path. With the fix the provider is no longer
overridden, so the azure branch runs for the first time:

custom_llm_provider : azure
api_base            : http://127.0.0.1:8877

That branch goes through get_azure_credentials and puts an api-version on
the URL (/openai/files?api-version=...), which the openai branch never
does. My stub accepts that happily. A real proxy in front of Azure is where
you'd find out whether the forwarded api_base and the resolved api-version
actually line up with what the deployment expects, and whether the returned
file_id is then accepted by the completion. That is a genuine gap and it is
exactly the part I can't cover from here.

So the request is fair, I just can't be the one to satisfy it. Two ways to get
it if you want it before merging:

• @ross-p reported [Bug] LiteLlm sends invalid PDF content for Azure-backed models through litellm_proxy #6538 and is set up for this. I asked on fix(models): resolve nested provider behind litellm_proxy prefix #6578 whether they
  had AZURE_API_BASE/AZURE_API_KEY set alongside the proxy config and
  didn't hear back, but a run of this branch on their deployment would answer
  both questions above.
• If there's an internal Azure backed proxy on your side, you've already got
  the before/after harness running, so pointing it at that would be a shorter
  path than anything I can do.

Happy to dig into whatever either turns up, including the api-version
handling if it doesn't match a real deployment.

One housekeeping note since you tagged this account: I opened this issue from
@tranminhquang4421 and #6723 from @vietnamesekid. Same person, two
accounts, and @vietnamesekid is the one that signed the CLA on #6578. Worth
knowing in case the CLA check on #6723 looks inconsistent with this thread.

[MilkyWay008]

That's a nasty one...... completion goes to the proxy but the upload resolves its own endpoint, the kind of thing that slips through testing. If you need a stopgap before the PR lands: get_openai_credentials checks OPENAI_BASE_URL then OPENAI_API_BASE before defaulting to api.openai.com, so pointing either at your proxy routes the upload through it too. And if you'd rather it never touch that key, unset OPENAI_API_KEY and it fails loudly instead of silently exfiltrating. Nice repro btw, the egress-blocking spy makes it obvious.