Skip to content

Commit f7947bd

Browse files
author
DavidQ
committed
Add expected-vs-actual tool-load diagnostics classification - PR 10.6I
1 parent b58f893 commit f7947bd

6 files changed

Lines changed: 635 additions & 229 deletions

‎docs/dev/codex_commands.md‎

Lines changed: 9 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -1,8 +1,12 @@
1-
# Codex Commands - BUILD_PR_LEVEL_10_6H_TOOL_LOAD_EXPECTED_DIAGNOSTICS
1+
# Codex Commands — BUILD_PR_LEVEL_10_6I_TOOL_LOAD_EXPECTED_VS_ACTUAL_CLASSIFICATION_V3
22

3-
Model: GPT-5.4
4-
Reasoning: high
3+
## Model
4+
GPT-5.3-codex
55

6-
```powershell
7-
codex --model gpt-5.4 --reasoning high "Run BUILD_PR_LEVEL_10_6H_TOOL_LOAD_EXPECTED_DIAGNOSTICS. Read docs/pr/BUILD_PR_LEVEL_10_6H_TOOL_LOAD_EXPECTED_DIAGNOSTICS.md. Extend the existing tool-load diagnostics to include explicit expected contract details at request, fetch, loaded, warning, and error boundaries. Keep the change diagnostic-only, no fallback data, no hardcoded paths, no schema rewrites. Run npm run test:launch-smoke:games and npm run test:sample-standalone:data-flow. Write docs/dev/reports/level_10_6H_tool_load_expected_diagnostics_report.md. Package the changed repo-relative files into tmp/BUILD_PR_LEVEL_10_6H_TOOL_LOAD_EXPECTED_DIAGNOSTICS.zip."
6+
## Reasoning
7+
high
8+
9+
## Command
10+
```text
11+
Implement BUILD_PR_LEVEL_10_6I_TOOL_LOAD_EXPECTED_VS_ACTUAL_CLASSIFICATION_V3 from docs/pr/BUILD_PR_LEVEL_10_6I_TOOL_LOAD_EXPECTED_VS_ACTUAL_CLASSIFICATION_V3.md. Keep the change diagnostics-only. Add expected-vs-actual dependency classification to existing tool-load diagnostics. Do not normalize palette files in this PR. Do not add fallback data or hardcoded paths. Run npm run test:launch-smoke:games and npm run test:sample-standalone:data-flow. Put results in docs/dev/reports/level_10_6i_tool_load_expected_vs_actual_classification_report.md. Return a ZIP artifact at tmp/BUILD_PR_LEVEL_10_6I_TOOL_LOAD_EXPECTED_VS_ACTUAL_CLASSIFICATION_V3.zip.
812
```

‎docs/dev/commit_comment.txt‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1 +1 @@
1-
Add expected contract fields to tool-load diagnostics - PR 10.6H
1+
Add expected-vs-actual tool-load diagnostics classification - PR 10.6I

‎docs/dev/reports/launch_smoke_report.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
# Launch Smoke Report
22

3-
Generated: 2026-04-27T18:15:42.698Z
3+
Generated: 2026-04-27T18:30:35.400Z
44

55
Filters: games=true, samples=false, tools=false, sampleRange=all
66

‎docs/dev/reports/level_10_6i_tool_load_expected_vs_actual_classification_report.md‎

Lines changed: 115 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,115 @@
1+
# Level 10.6I Tool Load Expected vs Actual Classification Report
2+
3+
## Files changed
4+
5+
- `tools/shared/toolLoadDiagnostics.js`
6+
- `docs/dev/reports/level_10_6i_tool_load_expected_vs_actual_classification_report.md`
7+
- `docs/dev/reports/launch_smoke_report.md` (test-generated output refresh)
8+
9+
## Diagnostic events added or updated
10+
11+
- Existing boundaries preserved with compact `expected` + `actual` blocks attached:
12+
- `[tool-load:request]`
13+
- `[tool-load:fetch]`
14+
- `[tool-load:loaded]`
15+
- `[tool-load:warning]`
16+
- New boundary added:
17+
- `[tool-load:classification]`
18+
19+
Classification values emitted exactly from shared diagnostics:
20+
21+
- `missing`
22+
- `wrong-path`
23+
- `wrong-shape`
24+
- `success`
25+
26+
Expected block fields emitted per dependency:
27+
28+
- `dependencyId`
29+
- `expectedPathKey`
30+
- `expectedPath`
31+
- `expectedSchema`
32+
- `expectedTopLevelShape`
33+
- `requiredFields`
34+
- `requiredArrayFields`
35+
36+
Actual block fields emitted per dependency:
37+
38+
- `requestedPath`
39+
- `fetchUrl`
40+
- `httpStatus`
41+
- `loadedSchema`
42+
- `topLevelKeys`
43+
- `fieldPresence`
44+
- `arrayCounts`
45+
46+
## Palette dependency classification behavior
47+
48+
Palette expectation is now classified explicitly even when palette was never requested/fetched/loaded.
49+
50+
Observed classification event for sample `0219` (`sprite-editor`):
51+
52+
```js
53+
{
54+
classification: "missing",
55+
expected: {
56+
dependencyId: "palette",
57+
expectedPathKey: "palettePath",
58+
expectedSchema: "html-js-gaming.palette",
59+
expectedTopLevelShape: ["schema", "version", "name", "swatches"]
60+
},
61+
actual: {
62+
requestedPath: "",
63+
fetchUrl: "",
64+
loadedSchema: "",
65+
topLevelKeys: []
66+
},
67+
note: "Expected palette dependency was not requested, not fetched, and not loaded."
68+
}
69+
```
70+
71+
## Test command output summary
72+
73+
1. `npm run test:launch-smoke:games`
74+
- Result: PASS
75+
- Summary: `PASS=12`, `FAIL=0`, `TOTAL=12`
76+
77+
2. `npm run test:sample-standalone:data-flow`
78+
- Result: PASS
79+
- Summary arrays all empty:
80+
- `schemaFailures: []`
81+
- `contractFailures: []`
82+
- `roundtripPathFailures: []`
83+
- `genericFailures: []`
84+
85+
## Manual browser-console validation notes
86+
87+
Manual launch performed for:
88+
89+
- `tools/Sprite Editor/index.html?sampleId=0219&samplePresetPath=/samples/phase-02/0219/sample.0219.sprite-editor.json`
90+
91+
Observed console prefixes:
92+
93+
- `[tool-load:request]`
94+
- `[tool-load:fetch]`
95+
- `[tool-load:loaded]`
96+
- `[tool-load:warning]`
97+
- `[tool-load:error]`
98+
- `[tool-load:classification]`
99+
100+
Observed classification outcomes:
101+
102+
1. Palette dependency classified as `missing` with note: expected palette was not requested/fetched/loaded.
103+
2. Sprite dependency classified as `wrong-shape` with:
104+
- `expected.dependencyId = "sprite-project"`
105+
- `expected.expectedPath = "/samples/phase-02/0219/sample.0219.sprite-editor.json"`
106+
- `actual.topLevelKeys = ["$schema", "tool", "version", "config"]`
107+
- `expected.contractMatch = false`
108+
109+
Scope guardrails held for this BUILD:
110+
111+
- diagnostics-only change
112+
- no palette normalization
113+
- no fallback data
114+
- no hardcoded asset paths
115+
- no start_of_day changes

‎docs/pr/BUILD_PR_LEVEL_10_6I_TOOL_LOAD_EXPECTED_VS_ACTUAL_CLASSIFICATION_V3.md‎

Lines changed: 77 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,77 @@
1+
# BUILD_PR_LEVEL_10_6I_TOOL_LOAD_EXPECTED_VS_ACTUAL_CLASSIFICATION_V3
2+
3+
## Purpose
4+
Add expected-vs-actual classification to tool-load diagnostics so missing palette requests, wrong paths, wrong payload shapes, and successful loads are visible in browser console output.
5+
6+
## Scope
7+
- Diagnostics only.
8+
- No production contract reshaping.
9+
- No silent fallback data.
10+
- No hardcoded asset paths.
11+
- No start_of_day changes.
12+
- Do not normalize palette files in this PR.
13+
14+
## Required Behavior
15+
Codex must update the existing tool-load diagnostics so each emitted diagnostic event includes a compact `expected` block and a compact `actual` block.
16+
17+
Required event categories:
18+
- `[tool-load:request]`
19+
- `[tool-load:fetch]`
20+
- `[tool-load:loaded]`
21+
- `[tool-load:warning]`
22+
- `[tool-load:classification]`
23+
24+
## Classification Values
25+
Each tool data dependency must classify as exactly one of:
26+
- `missing`
27+
- `wrong-path`
28+
- `wrong-shape`
29+
- `success`
30+
31+
## Expected Block
32+
Each dependency log should include:
33+
- `dependencyId`
34+
- `expectedPathKey`
35+
- `expectedPath`
36+
- `expectedSchema`
37+
- `expectedTopLevelShape`
38+
- `requiredFields`
39+
- `requiredArrayFields`
40+
41+
For palette diagnostics, the expected block must make the palette requirement visible even when no palette fetch occurs.
42+
43+
## Actual Block
44+
Each dependency log should include:
45+
- `requestedPath`
46+
- `fetchUrl`
47+
- `httpStatus`
48+
- `loadedSchema`
49+
- `topLevelKeys`
50+
- `fieldPresence`
51+
- `arrayCounts`
52+
53+
## Palette-Specific Requirement
54+
If a tool expects palette data and no palette path is present in launch query, manifest-derived requested data paths, or normalized input, log:
55+
56+
```text
57+
[tool-load:classification] classification: missing dependencyId: palette
58+
```
59+
60+
The log must state that the expected palette dependency was not requested, not fetched, and not loaded.
61+
62+
## Acceptance
63+
- Running the Sprite Editor sample that currently only fetches `sample.0219.sprite-editor.json` must clearly show whether palette was expected.
64+
- If palette was expected but absent, console output must classify palette as `missing`.
65+
- If preset loads but does not include the expected sprite project shape, classify that dependency as `wrong-shape`.
66+
- Existing smoke/data-flow tests must still run.
67+
- No duplicate palette normalization is performed in this PR.
68+
69+
## Tests
70+
Run:
71+
72+
```powershell
73+
npm run test:launch-smoke:games
74+
npm run test:sample-standalone:data-flow
75+
```
76+
77+
Then manually launch the failing tool sample and inspect browser console diagnostics.

0 commit comments

Comments
 (0)