Skip to content

API proxy token normalizer does not map OpenAI cached_tokens to cache_read_tokens #5203

Description

@lpcox

Problem

The AWF API proxy / token usage parser does not correctly map OpenAI-style cached prompt tokens into the cache read tokens field. When OpenAI models report cached token usage via usage.prompt_tokens_details.cached_tokens, AWF fails to recognize these as cache reads.

Impact

  • Token usage reports show 0 cache reads even when the underlying model heavily uses caching (e.g., 3.3M cached tokens over 33 turns)
  • AI credits guard is affected: the max-ai-credits limit is calculated without accounting for cached reads, causing the reported AIC usage to be much higher than actual consumption. Users must set artificially high limits to avoid being rate-limited.

Expected Behavior

When usage.prompt_tokens_details.cached_tokens is present and explicit cache_read_tokens is absent, the normalizer should map cached_tokenscache_read_tokens.

Root Cause

The token usage normalization path (in the API proxy token tracker / parse_token_usage.cjs) does not handle the OpenAI-specific prompt_tokens_details.cached_tokens field.

Reference

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions