Skip to content

Commit 648c8f3

Browse files
jeremyphilemonjeremyphilemon
authored
fix(xai): make usage nullable in responses schema for streaming compatibility (#12004)
<!-- Welcome to contributing to AI SDK! We're excited to see your changes. We suggest you read the following contributing guide we've created before submitting: https://github.com/vercel/ai/blob/main/CONTRIBUTING.md --> ## Background xAI returns `usage: null` in early streaming response events (`response.created`, `response.in_progress`) because token counts aren't available until the stream completes. The actual usage is sent in the final `response.completed` event. This causes `AI_TypeValidationError` during streaming because the schema expects `usage` to be an object, not null. ## Summary Make `usage` field nullish in `xaiResponsesResponseSchema` to accept null values during streaming ## Manual Verification 1. Run a streaming request to xAI via the gateway: ```ts import { streamText } from 'ai'; import { createGateway } from '@ai-sdk/gateway'; const gateway = createGateway({ baseURL: '...', apiKey: '...' }); const result = streamText({ model: gateway('xai/grok-3-fast'), prompt: 'Say hello', }); for await (const part of result.fullStream) { console.log(part.type); } console.log('Usage:', await result.usage); ``` 2. Verify no AI_TypeValidationError is logged for response.created / response.in_progress events 3. Verify final usage is correctly captured ## Checklist <!-- Do not edit this list. Leave items unchecked that don't apply. If you need to track subtasks, create a new "## Tasks" section Please check if the PR fulfills the following requirements: --> - [x] Tests have been added / updated (for bug fixes / features) - [ ] Documentation has been added / updated (for bug fixes / features) - [ ] A _patch_ changeset for relevant packages has been added (for bug fixes / features - run `pnpm changeset` in the project root) - [x] I have reviewed this pull request (self-review) ## Future Work <!-- Feel free to mention things not covered by this PR that can be done in future PRs. Remove the section if it's not needed. --> ## Related Issues <!-- List related issues here, e.g. "Fixes #1234". Remove the section if it's not needed. -->
1 parent 4bbb1f6 commit 648c8f3

File tree

4 files changed

+192
-2
lines changed

4 files changed

+192
-2
lines changed

‎.changeset/four-days-relate.md‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,7 @@
1+
---
2+
'@ai-sdk/xai': patch
3+
---
4+
5+
fix(xai): make usage nullable in responses schema for streaming compatibility
6+
7+
xAI sends `usage: null` in early streaming events (`response.created`, `response.in_progress`) because token counts aren't available until the stream completes. This change makes the `usage` field nullish in `xaiResponsesResponseSchema` to accept these values without validation errors.

‎packages/xai/src/responses/xai-responses-api.ts‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -222,7 +222,7 @@ export const xaiResponsesResponseSchema = z.object({
222222
model: z.string().nullish(),
223223
object: z.literal('response'),
224224
output: z.array(outputItemSchema),
225-
usage: xaiResponsesUsageSchema,
225+
usage: xaiResponsesUsageSchema.nullish(),
226226
status: z.string(),
227227
});
228228

‎packages/xai/src/responses/xai-responses-language-model.test.ts‎

Lines changed: 178 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1800,4 +1800,182 @@ describe('XaiResponsesLanguageModel', () => {
18001800
});
18011801
});
18021802
});
1803+
1804+
describe('schema validation', () => {
1805+
it('should accept response.created with usage: null', async () => {
1806+
prepareStreamChunks([
1807+
JSON.stringify({
1808+
type: 'response.created',
1809+
response: {
1810+
id: 'resp_123',
1811+
object: 'response',
1812+
model: 'grok-4-fast',
1813+
created_at: 1700000000,
1814+
status: 'in_progress',
1815+
output: [],
1816+
usage: null,
1817+
},
1818+
}),
1819+
JSON.stringify({
1820+
type: 'response.output_item.added',
1821+
item: {
1822+
id: 'msg_001',
1823+
type: 'message',
1824+
role: 'assistant',
1825+
content: [],
1826+
status: 'in_progress',
1827+
},
1828+
output_index: 0,
1829+
}),
1830+
JSON.stringify({
1831+
type: 'response.content_part.added',
1832+
item_id: 'msg_001',
1833+
output_index: 0,
1834+
content_index: 0,
1835+
part: { type: 'output_text', text: '' },
1836+
}),
1837+
JSON.stringify({
1838+
type: 'response.output_text.delta',
1839+
item_id: 'msg_001',
1840+
output_index: 0,
1841+
content_index: 0,
1842+
delta: 'Hello',
1843+
}),
1844+
JSON.stringify({
1845+
type: 'response.completed',
1846+
response: {
1847+
id: 'resp_123',
1848+
object: 'response',
1849+
model: 'grok-4-fast',
1850+
created_at: 1700000000,
1851+
status: 'completed',
1852+
output: [
1853+
{
1854+
id: 'msg_001',
1855+
type: 'message',
1856+
role: 'assistant',
1857+
content: [{ type: 'output_text', text: 'Hello' }],
1858+
status: 'completed',
1859+
},
1860+
],
1861+
usage: {
1862+
input_tokens: 10,
1863+
output_tokens: 5,
1864+
total_tokens: 15,
1865+
},
1866+
},
1867+
}),
1868+
]);
1869+
1870+
const { stream } = await createModel().doStream({
1871+
prompt: TEST_PROMPT,
1872+
});
1873+
1874+
const parts = await convertReadableStreamToArray(stream);
1875+
1876+
expect(parts).toContainEqual(
1877+
expect.objectContaining({
1878+
type: 'text-delta',
1879+
delta: 'Hello',
1880+
}),
1881+
);
1882+
1883+
expect(parts).toContainEqual(
1884+
expect.objectContaining({
1885+
type: 'finish',
1886+
}),
1887+
);
1888+
});
1889+
1890+
it('should accept response.in_progress with usage: null', async () => {
1891+
prepareStreamChunks([
1892+
JSON.stringify({
1893+
type: 'response.created',
1894+
response: {
1895+
id: 'resp_123',
1896+
object: 'response',
1897+
model: 'grok-4-fast',
1898+
created_at: 1700000000,
1899+
status: 'in_progress',
1900+
output: [],
1901+
usage: null,
1902+
},
1903+
}),
1904+
JSON.stringify({
1905+
type: 'response.in_progress',
1906+
response: {
1907+
id: 'resp_123',
1908+
object: 'response',
1909+
model: 'grok-4-fast',
1910+
created_at: 1700000000,
1911+
status: 'in_progress',
1912+
output: [],
1913+
usage: null,
1914+
},
1915+
}),
1916+
JSON.stringify({
1917+
type: 'response.output_item.added',
1918+
item: {
1919+
id: 'msg_001',
1920+
type: 'message',
1921+
role: 'assistant',
1922+
content: [],
1923+
status: 'in_progress',
1924+
},
1925+
output_index: 0,
1926+
}),
1927+
JSON.stringify({
1928+
type: 'response.content_part.added',
1929+
item_id: 'msg_001',
1930+
output_index: 0,
1931+
content_index: 0,
1932+
part: { type: 'output_text', text: '' },
1933+
}),
1934+
JSON.stringify({
1935+
type: 'response.output_text.delta',
1936+
item_id: 'msg_001',
1937+
output_index: 0,
1938+
content_index: 0,
1939+
delta: 'Hi',
1940+
}),
1941+
JSON.stringify({
1942+
type: 'response.completed',
1943+
response: {
1944+
id: 'resp_123',
1945+
object: 'response',
1946+
model: 'grok-4-fast',
1947+
created_at: 1700000000,
1948+
status: 'completed',
1949+
output: [
1950+
{
1951+
id: 'msg_001',
1952+
type: 'message',
1953+
role: 'assistant',
1954+
content: [{ type: 'output_text', text: 'Hi' }],
1955+
status: 'completed',
1956+
},
1957+
],
1958+
usage: {
1959+
input_tokens: 5,
1960+
output_tokens: 1,
1961+
total_tokens: 6,
1962+
},
1963+
},
1964+
}),
1965+
]);
1966+
1967+
const { stream } = await createModel().doStream({
1968+
prompt: TEST_PROMPT,
1969+
});
1970+
1971+
const parts = await convertReadableStreamToArray(stream);
1972+
1973+
expect(parts).toContainEqual(
1974+
expect.objectContaining({
1975+
type: 'text-delta',
1976+
delta: 'Hi',
1977+
}),
1978+
);
1979+
});
1980+
});
18031981
});

‎packages/xai/src/responses/xai-responses-language-model.ts‎

Lines changed: 6 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -343,7 +343,12 @@ export class XaiResponsesLanguageModel implements LanguageModelV3 {
343343
unified: mapXaiResponsesFinishReason(response.status),
344344
raw: response.status ?? undefined,
345345
},
346-
usage: convertXaiResponsesUsage(response.usage),
346+
usage: response.usage
347+
? convertXaiResponsesUsage(response.usage)
348+
: {
349+
inputTokens: { total: 0, noCache: 0, cacheRead: 0, cacheWrite: 0 },
350+
outputTokens: { total: 0, text: 0, reasoning: 0 },
351+
},
347352
request: { body },
348353
response: {
349354
...getResponseMetadata(response),

0 commit comments

Comments
 (0)