Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions .changeset/v2-fault-injection.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
"@moonshot-ai/kimi-code": patch
---

v2 engine: expose the prompt scheduler over /api/v2 for native clients, and add an experimental fault-injection service (KIMI_CODE_EXPERIMENTAL_FAULT_INJECTION) that arms a one-shot provider failure so the media-degraded / media-stripped recovery resends can be exercised end-to-end.
5 changes: 5 additions & 0 deletions .changeset/v2-image-format-gate.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
"@moonshot-ai/kimi-code": patch
---

v2 engine: block unsupported image formats (AVIF, HEIC, BMP, TIFF, ICO) at every ingestion point so they can no longer poison session history, and auto-recover provider image-format rejections with a media-stripped resend.
5 changes: 5 additions & 0 deletions .changeset/v2-media-degraded-413.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
"@moonshot-ai/kimi-code": patch
---

v2 engine: recover image-heavy sessions from provider request-size rejections (HTTP 413) by resending with older media degraded to text markers, re-encode oversized WebP images instead of passing them through, and keep downscaled PNGs readable by switching to JPEG below 1000px.
1 change: 1 addition & 0 deletions packages/agent-core-v2/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -58,6 +58,7 @@
"@antfu/utils": "^9.3.0",
"@anthropic-ai/sdk": "^0.95.2",
"@google/genai": "^1.49.0",
"@jsquash/webp": "^1.5.0",
"@modelcontextprotocol/sdk": "^1.29.0",
"@moonshot-ai/kimi-code-oauth": "workspace:^",
"@moonshot-ai/minidb": "workspace:^",
Expand Down
1 change: 1 addition & 0 deletions packages/agent-core-v2/scripts/check-domain-layers.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -162,6 +162,7 @@ const DOMAIN_LAYER = new Map([
// the domain to L4 beside the other agent-behaviour tools.
['edit', 4],
['llmRequester', 4],
['faultInjection', 4],
['profile', 4],
['prompt', 4],
// `shellCommand` orchestrates user `!` commands through `toolRegistry` (L3),
Expand Down
36 changes: 36 additions & 0 deletions packages/agent-core-v2/scripts/generate-webp-dec-wasm.mjs
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
/**
* Regenerate `src/agent/media/webp-dec-wasm.ts` from the installed
* `@jsquash/webp` package.
*
* The WebP decoder wasm is committed as a base64 string module because the
* published CLI bundles every dependency into a single file with no runtime
* node_modules — a file-path lookup for the .wasm would break there, while a
* string constant survives every packaging (vitest on sources, tsdown
* bundling, nix builds) unchanged. Run this after bumping @jsquash/webp:
*
* node scripts/generate-webp-dec-wasm.mjs
*/
import { createRequire } from 'node:module';
import { readFileSync, writeFileSync } from 'node:fs';
import { resolve } from 'node:path';

const packageRoot = resolve(import.meta.dirname, '..');
const require = createRequire(resolve(packageRoot, 'package.json'));

const wasmPath = require.resolve('@jsquash/webp/codec/dec/webp_dec.wasm');
const version = require('@jsquash/webp/package.json').version;
const wasm = readFileSync(wasmPath);

const target = resolve(packageRoot, 'src/agent/media/webp-dec-wasm.ts');
writeFileSync(
target,
`// GENERATED FILE — do not edit by hand.
// WebP decoder wasm from @jsquash/webp@${version} (codec/dec/webp_dec.wasm),
// base64-encoded so the bundled CLI needs no on-disk wasm asset.
// Regenerate with: node scripts/generate-webp-dec-wasm.mjs

export const WEBP_DECODER_WASM_BASE64 =
'${wasm.toString('base64')}';
`,
);
console.log(`Wrote ${target} (${wasm.length} bytes of wasm)`);
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,8 @@ export interface IAgentContextProjectorService {

project(messages: readonly ContextMessage[]): readonly Message[];
projectStrict(messages: readonly ContextMessage[]): readonly Message[];
projectMediaDegraded(messages: readonly ContextMessage[]): readonly Message[];
projectMediaStripped(messages: readonly ContextMessage[]): readonly Message[];
}

export const IAgentContextProjectorService = createDecorator<IAgentContextProjectorService>(
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,14 @@
* dropped) are reported through an optional sink and surfaced once here as a
* single deduped warning plus a `context_projection_repaired` telemetry event,
* so a silently-mangled history always leaves a trace.
*
* `projectMediaDegraded` / `projectMediaStripped` are the fallback
* projections for the two deterministic provider rejections: media-degraded
* (all but the most recent media replaced by text markers) resends after an
* HTTP 413 body-size rejection; media-stripped (every media part replaced)
* resends after an image-format rejection, since that error never says WHICH
* image is poison and only a full strip guarantees a clean request. Both are
* read-side only — the history keeps its media.
*/

import { InstantiationType } from '#/_base/di/extensions';
Expand Down Expand Up @@ -44,6 +52,17 @@ export class AgentContextProjectorService implements IAgentContextProjectorServi
return this.projectWithTrace(messages, projectStrict);
}

projectMediaDegraded(messages: readonly ContextMessage[]): readonly Message[] {
return degradeOlderMediaParts(
this.projectWithTrace(messages, project),
MEDIA_DEGRADE_KEEP_RECENT,
);
}

projectMediaStripped(messages: readonly ContextMessage[]): readonly Message[] {
return degradeOlderMediaParts(this.projectWithTrace(messages, project), 0, MEDIA_STRIPPED_PLACEHOLDERS);
}

private projectWithTrace(
messages: readonly ContextMessage[],
fn: (history: readonly ContextMessage[], onAnomaly?: (anomaly: ProjectionAnomaly) => void) => Message[],
Expand Down Expand Up @@ -149,6 +168,81 @@ type ProjectionAnomaly =

type OnAnomaly = (anomaly: ProjectionAnomaly) => void;

/**
* How many of the most recent media parts survive the media-degraded
* projection. The tail images are what the model is actively working from
* (the screenshot it just took); everything older is replaced by a marker.
*/
export const MEDIA_DEGRADE_KEEP_RECENT = 2;

const MEDIA_DEGRADED_PLACEHOLDERS = {
image_url:
'[image omitted: dropped to fit the provider request size limit; re-read the file to view it]',
audio_url:
'[audio omitted: dropped to fit the provider request size limit; re-read the file to hear it]',
video_url:
'[video omitted: dropped to fit the provider request size limit; re-read the file to view it]',
} as const;

/**
* Markers for the media-stripped resend after the provider rejected an
* image's FORMAT (not its size): the image marker points the model at
* re-reading the file, whose refusal carries per-OS conversion instructions;
* audio/video are collateral of the full strip and say so.
*/
export const MEDIA_STRIPPED_PLACEHOLDERS = {
image_url:
'[image omitted: the provider rejected this image; re-read the file for conversion instructions]',
audio_url:
'[audio omitted: dropped along with a rejected image; re-read the file to hear it]',
video_url:
'[video omitted: dropped along with a rejected image; re-read the file to view it]',
} as const;

type MediaPlaceholderSet = typeof MEDIA_DEGRADED_PLACEHOLDERS | typeof MEDIA_STRIPPED_PLACEHOLDERS;

function isDegradableMediaPart(
part: ContentPart,
): part is ContentPart & { type: keyof MediaPlaceholderSet } {
return part.type in MEDIA_DEGRADED_PLACEHOLDERS;
}

/**
* Replace all but the `keepRecent` most recent media parts with deterministic
* text markers. This is the media-degraded projection used to resend a request
* the provider rejected as too large (HTTP 413 on accumulated base64 media)
* and — with `keepRecent = 0` and `MEDIA_STRIPPED_PLACEHOLDERS` — the resend
* after an image-format rejection, where the poisoned image could be anywhere
* and only a full strip guarantees a clean request. A purely read-side
* transform — the underlying history is left untouched — that trades pixels
* for deliverability while the surrounding text (including ReadMediaFile's
* `<image path="...">` wrapper) survives, so the model can re-read any file
* it still needs. Untouched messages are returned by reference, and when
* nothing needs degrading the input array itself is returned.
*/
export function degradeOlderMediaParts(
messages: readonly Message[],
keepRecent: number,
placeholders: MediaPlaceholderSet = MEDIA_DEGRADED_PLACEHOLDERS,
): readonly Message[] {
const mediaCount = messages.reduce(
(count, message) => count + message.content.filter(isDegradableMediaPart).length,
0,
);
let toDegrade = Math.max(0, mediaCount - keepRecent);
if (toDegrade === 0) return messages;

return messages.map((message) => {
if (toDegrade === 0 || !message.content.some(isDegradableMediaPart)) return message;
const content = message.content.map((part): ContentPart => {
if (toDegrade === 0 || !isDegradableMediaPart(part)) return part;
toDegrade -= 1;
return { type: 'text', text: placeholders[part.type] };
});
return { ...message, content };
});
}

function projectStrict(history: readonly ContextMessage[], onAnomaly?: OnAnomaly): Message[] {
const projected = project(history, onAnomaly);
return dropLeadingNonUserMessages(
Expand Down
56 changes: 56 additions & 0 deletions packages/agent-core-v2/src/agent/faultInjection/faultInjection.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,56 @@
/**
* `faultInjection` domain (L4) — deterministic provider-failure simulation
* for testing the requester's recovery projections over a live channel.
*
* The turn-loop recovery resends (media-degraded after an HTTP 413 body-size
* rejection, media-stripped after an image-format rejection) are
* deterministic given a provider error, but a real provider cannot be asked
* to produce one on demand. Arming a one-shot fault makes the next LLM
* request attempt raise the chosen error BEFORE the provider is contacted,
* so the recovery path — projection rebuild, per-turn stickiness, wire
* records — runs end-to-end while the (successful) resend still goes to the
* real provider.
*
* `arm` is refused unless the `fault-injection` experimental flag is enabled
* (see ./flag); `take` is the requester's consumption point and stays inert
* otherwise.
*/

import { createDecorator, type ServiceIdentifier } from '#/_base/di/instantiation';

/** The deterministic failures the requester can be armed to raise. */
export type FaultKind = 'request-too-large' | 'image-format';

export interface FaultInjectionStatus {
/** The armed one-shot fault, if any (consumed by the next request attempt). */
readonly armed: FaultKind | undefined;
/** Faults that actually fired, in fire order. */
readonly fired: readonly FaultKind[];
}

export interface IFaultInjectionService {
readonly _serviceBrand: undefined;

/**
* Arm a one-shot fault: the next LLM request attempt raises it before
* hitting the provider. Refused unless the `fault-injection` experimental
* flag is enabled.
*/
arm(kind: FaultKind): void;

/** Current arming and fire history. */
status(): FaultInjectionStatus;

/** Clear the armed fault and the fire history. */
clear(): void;

/**
* Consume the armed one-shot fault — the requester's consumption point,
* called once per request attempt. Returns undefined when nothing is
* armed; a consumed fault is recorded in {@link FaultInjectionStatus.fired}.
*/
take(): FaultKind | undefined;
}

export const IFaultInjectionService: ServiceIdentifier<IFaultInjectionService> =
createDecorator<IFaultInjectionService>('faultInjectionService');
Original file line number Diff line number Diff line change
@@ -0,0 +1,65 @@
/**
* `faultInjection` domain (L4) — `IFaultInjectionService` implementation.
*
* Agent-scope one-shot latch: `arm` (flag-gated) stores the next fault,
* `take` (the llmRequester's per-attempt consumption point) consumes and
* records it. Bound at Agent scope.
*/

import { InstantiationType } from '#/_base/di/extensions';
import { LifecycleScope, registerScopedService } from '#/_base/di/scope';
import { IFlagService } from '#/app/flag/flag';
import { ErrorCodes, Error2 } from '#/errors';

import { FAULT_INJECTION_FLAG_ID } from './flag';
import {
IFaultInjectionService,
type FaultInjectionStatus,
type FaultKind,
} from './faultInjection';

export class FaultInjectionService implements IFaultInjectionService {
declare readonly _serviceBrand: undefined;

private armed: FaultKind | undefined;
private readonly fired: FaultKind[] = [];

constructor(@IFlagService private readonly flags: IFlagService) {}

arm(kind: FaultKind): void {
if (!this.flags.enabled(FAULT_INJECTION_FLAG_ID)) {
throw new Error2(
ErrorCodes.REQUEST_INVALID,
'Fault injection is disabled; enable the fault-injection experimental flag ' +
'(KIMI_CODE_EXPERIMENTAL_FAULT_INJECTION=1, the master flag, or the ' +
'[experimental] config section).',
);
}
this.armed = kind;
}

status(): FaultInjectionStatus {
return { armed: this.armed, fired: [...this.fired] };
}

clear(): void {
this.armed = undefined;
this.fired.length = 0;
}

take(): FaultKind | undefined {
const kind = this.armed;
if (kind === undefined) return undefined;
this.armed = undefined;
this.fired.push(kind);
return kind;
}
}

registerScopedService(
LifecycleScope.Agent,
IFaultInjectionService,
FaultInjectionService,
InstantiationType.Delayed,
'faultInjection',
);
29 changes: 29 additions & 0 deletions packages/agent-core-v2/src/agent/faultInjection/flag.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
/**
* `faultInjection` domain (L4) — registers the `fault-injection` experimental
* flag into `flag`.
*
* Gates the fault-injection Service's `arm`: deterministic provider-failure
* simulation for exercising the requester's recovery projections over a live
* channel. Off by default; enable via
* `KIMI_CODE_EXPERIMENTAL_FAULT_INJECTION`, the master
* `KIMI_CODE_EXPERIMENTAL_FLAG`, or the `[experimental]` config section.
* Imported for its side effect (registers the definition) from the package
* barrel.
*/

import { type FlagDefinitionInput, registerFlagDefinition } from '#/app/flag/flagRegistry';

export const FAULT_INJECTION_FLAG_ID = 'fault-injection';
export const FAULT_INJECTION_FLAG_ENV = 'KIMI_CODE_EXPERIMENTAL_FAULT_INJECTION';

export const faultInjectionFlag: FlagDefinitionInput = {
id: FAULT_INJECTION_FLAG_ID,
title: 'Fault injection (LLM request failures)',
description:
'Allow arming a one-shot deterministic provider failure (HTTP 413 body-size or image-format rejection) on the next LLM request, for testing the media-degraded / media-stripped recovery projections over a live channel.',
env: FAULT_INJECTION_FLAG_ENV,
default: false,
surface: 'core',
};

registerFlagDefinition(faultInjectionFlag);
Original file line number Diff line number Diff line change
Expand Up @@ -70,7 +70,10 @@ export const llmRequest = LlmRequestTraceModel.defineOp('llm.request', {
messageCount: z.number(),
turnStep: z.string().optional(),
attempt: z.string().optional(),
projection: z.literal('strict').optional(),
/** Set when this request is a recovery resend (strict rebuild after a
* structural rejection, media-degraded rebuild after an HTTP 413 body-size
* rejection, media-stripped rebuild after an image-format rejection). */
projection: z.enum(['strict', 'media-degraded', 'media-stripped']).optional(),
droppedCount: z.number().optional(),
}),
apply: (s) => s,
Expand Down
Loading
Loading