Skip to content

Commit 6520e6f

Browse files
ToolboxAidToolboxAid
committed
build(validation-suite): add platform validation suite for full pipeline coverage
1 parent a83fd36 commit 6520e6f

12 files changed

Lines changed: 573 additions & 31 deletions

‎docs/dev/CODEX_COMMANDS.md‎

Lines changed: 19 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -2,20 +2,26 @@ MODEL: GPT-5.4
22
REASONING: high
33

44
COMMAND:
5-
Execute this Level 14 expansion suite in order, treating each PLAN as the governing architecture contract for its corresponding BUILD/APPLY.
5+
Execute this atomic Level 15 package in order.
66

7-
Order:
8-
1. PLAN/BUILD/APPLY_PR_EDITOR_EXPERIENCE_LAYER
9-
2. PLAN/BUILD/APPLY_PR_RUNTIME_STREAMING_SYSTEM
10-
3. PLAN/BUILD/APPLY_PR_PLUGIN_ARCHITECTURE
11-
4. PLAN/BUILD/APPLY_PR_PROJECT_VERSIONING_SYSTEM
7+
Step 1:
8+
Treat docs/pr/PLAN_PR_PLATFORM_VALIDATION_SUITE.md as the governing architecture contract.
129

13-
Global requirements:
14-
- Preserve Level 13 accepted platform boundaries
10+
Step 2:
11+
Create BUILD_PR_PLATFORM_VALIDATION_SUITE.
12+
13+
BUILD requirements:
14+
- Implement a repeatable platform validation suite for accepted system flows
15+
- Cover golden-path, failure-path, remediation, packaging, runtime, streaming, plugin, and versioning scenarios
16+
- Produce stable, readable validation reports
17+
- Preserve accepted platform boundaries
1518
- Do not modify engine core APIs
16-
- Keep docs/dev command and commit files authoritative
17-
- Package each completed slice to the repo tmp directory using the PR/package naming conventions already established
18-
- Maintain deterministic behavior and readable validation/reporting wherever applicable
1919

20-
Master package:
21-
HTML-JavaScript-Gaming/tmp/LEVEL_14_EXPANSION_SUITE_ATOMIC_PACKAGE_delta.zip
20+
Step 3:
21+
Validate BUILD against docs/pr/BUILD_PR_PLATFORM_VALIDATION_SUITE.md.
22+
23+
Step 4:
24+
Treat docs/pr/APPLY_PR_PLATFORM_VALIDATION_SUITE.md as the acceptance boundary and package results.
25+
26+
Package:
27+
HTML-JavaScript-Gaming/tmp/LEVEL_15_PLATFORM_VALIDATION_SUITE_delta.zip

‎docs/dev/COMMIT_COMMENT.txt‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1 +1 @@
1-
build(level-14): execute expansion suite over accepted platform baseline
1+
build(validation-suite): add platform validation suite for full pipeline coverage

‎docs/dev/change_summary.txt‎

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1 +1,3 @@
1-
Combined Level 14 suite containing PLAN/BUILD/APPLY docs for UX, streaming, plugins, and versioning.
1+
Single atomic Level 15 package containing PLAN, BUILD, and APPLY documentation for a
2+
platform validation suite. This bundle is docs-first and Codex-ready, with implementation
3+
ownership delegated to Codex.

‎docs/dev/file_tree.txt‎

Lines changed: 3 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -1,15 +1,6 @@
1-
docs/pr/PLAN_PR_EDITOR_EXPERIENCE_LAYER.md
2-
docs/pr/BUILD_PR_EDITOR_EXPERIENCE_LAYER.md
3-
docs/pr/APPLY_PR_EDITOR_EXPERIENCE_LAYER.md
4-
docs/pr/PLAN_PR_RUNTIME_STREAMING_SYSTEM.md
5-
docs/pr/BUILD_PR_RUNTIME_STREAMING_SYSTEM.md
6-
docs/pr/APPLY_PR_RUNTIME_STREAMING_SYSTEM.md
7-
docs/pr/PLAN_PR_PLUGIN_ARCHITECTURE.md
8-
docs/pr/BUILD_PR_PLUGIN_ARCHITECTURE.md
9-
docs/pr/APPLY_PR_PLUGIN_ARCHITECTURE.md
10-
docs/pr/PLAN_PR_PROJECT_VERSIONING_SYSTEM.md
11-
docs/pr/BUILD_PR_PROJECT_VERSIONING_SYSTEM.md
12-
docs/pr/APPLY_PR_PROJECT_VERSIONING_SYSTEM.md
1+
docs/pr/PLAN_PR_PLATFORM_VALIDATION_SUITE.md
2+
docs/pr/BUILD_PR_PLATFORM_VALIDATION_SUITE.md
3+
docs/pr/APPLY_PR_PLATFORM_VALIDATION_SUITE.md
134
docs/dev/codex_commands.md
145
docs/dev/commit_comment.txt
156
docs/dev/change_summary.txt

‎docs/dev/reports/platform_validation_suite_validation.txt‎

Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,20 @@
1+
BUILD_PR_PLATFORM_VALIDATION_SUITE validation
2+
3+
Files:
4+
- tools/shared/platformValidationSuite.js
5+
- tests/tools/PlatformValidationSuite.test.mjs
6+
- tests/run-tests.mjs
7+
8+
Checks passed:
9+
- node --check tools/shared/platformValidationSuite.js
10+
- node --check tests/tools/PlatformValidationSuite.test.mjs
11+
- node ./scripts/run-node-tests.mjs
12+
13+
Behavior confirmed:
14+
- golden-path validation, packaging, and runtime flow passes
15+
- invalid references fail at enforced boundaries
16+
- remediation demonstrates recoverability
17+
- packaging and runtime ordering remain deterministic
18+
- runtime fail-fast, streaming, plugin, and versioning scenarios are covered explicitly
19+
- suite report text is stable and readable
20+
- no engine core API changes

‎docs/dev/validation_checklist.txt‎

Lines changed: 5 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,6 @@
1-
- All four Level 14 capability groups are present
2-
- Each capability has PLAN, BUILD, and APPLY docs
3-
- docs/dev contains codex_commands.md and commit_comment.txt
4-
- Existing Level 13 baseline remains the prerequisite authority
1+
- PLAN, BUILD, and APPLY docs are all present
2+
- codex_commands.md and commit_comment.txt are under docs/dev
3+
- Full-flow platform validation coverage is explicit
4+
- Plugin, versioning, and streaming are included in scope
55
- No engine core API changes are authorized
6+
- APPLY remains docs-only acceptance boundary

‎docs/pr/APPLY_PR_PLATFORM_VALIDATION_SUITE.md‎

Lines changed: 31 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,31 @@
1+
# APPLY_PR_PLATFORM_VALIDATION_SUITE
2+
3+
## Purpose
4+
Apply the completed platform validation suite as a docs-only acceptance boundary.
5+
6+
## Apply Scope
7+
- Confirm PLAN and BUILD are complete
8+
- Confirm validation suite behavior remains within approved scope
9+
- Confirm APPLY introduces no implementation expansion
10+
- Lock platform validation suite as accepted project architecture baseline
11+
12+
## Verification Summary
13+
- BUILD artifact exists and matches scoped validation suite implementation
14+
- APPLY remains docs-only
15+
- Full-flow validation coverage is now part of the accepted platform baseline
16+
- No engine core API changes are required by APPLY
17+
18+
## Accepted Baseline
19+
- Project owns a repeatable platform validation suite
20+
- Full-flow regression coverage protects the accepted platform baseline
21+
- Reports provide stable validation evidence for future work
22+
23+
## Manual Validation Checklist
24+
1. Confirm BUILD artifact exists in repo tmp path.
25+
2. Confirm APPLY bundle only contains docs/pr and docs/dev files.
26+
3. Confirm accepted scope does not exceed BUILD scope.
27+
4. Confirm no new implementation files are introduced by APPLY.
28+
5. Confirm future work can rely on validation suite coverage.
29+
30+
## Approved Commit Comment
31+
build(validation-suite): add platform validation suite for full pipeline coverage

‎docs/pr/BUILD_PR_PLATFORM_VALIDATION_SUITE.md‎

Lines changed: 50 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,50 @@
1+
# BUILD_PR_PLATFORM_VALIDATION_SUITE
2+
3+
## Goal
4+
Implement the platform-level validation suite defined in `PLAN_PR_PLATFORM_VALIDATION_SUITE` without changing engine core APIs.
5+
6+
## Implemented Scope
7+
- Added shared repeatable platform validation suite runner in `tools/shared/platformValidationSuite.js`
8+
- validates full-flow golden-path authoring, validation, packaging, and runtime behavior
9+
- validates invalid reference enforcement boundaries
10+
- validates remediation recoverability
11+
- validates packaging/runtime determinism
12+
- validates runtime fail-fast behavior
13+
- validates streaming correctness
14+
- validates plugin participation constraints
15+
- validates versioning compatibility and migration-needed reporting
16+
- emits stable readable suite report text for future automation
17+
- Added automated coverage in `tests/tools/PlatformValidationSuite.test.mjs`
18+
- suite passes with all eight deterministic scenarios
19+
- repeated suite execution produces stable identical results
20+
- Registered suite coverage in `tests/run-tests.mjs`
21+
22+
## Manual Validation Checklist
23+
1. Baseline valid scenario passes. `PASS`
24+
2. Invalid reference scenario fails at enforced boundary. `PASS`
25+
3. Remediation scenario demonstrates fix path. `PASS`
26+
4. Packaging determinism scenario is repeatable. `PASS`
27+
5. Runtime ready/fail-fast scenarios behave correctly. `PASS`
28+
6. Streaming scenario preserves correctness. `PASS`
29+
7. Plugin integration scenario works within constraints. `PASS`
30+
8. Versioning scenario verifies compatibility behavior. `PASS`
31+
9. Engine core APIs remain unchanged. `PASS`
32+
33+
## Validation Summary
34+
- Syntax checks passed:
35+
- `node --check tools/shared/platformValidationSuite.js`
36+
- `node --check tests/tools/PlatformValidationSuite.test.mjs`
37+
- Full Node test suite passed:
38+
- `node ./scripts/run-node-tests.mjs`
39+
40+
## Scope Guard
41+
- The suite composes accepted shared platform services instead of replacing them.
42+
- Enforced boundaries remain owned by validation, packaging, runtime, streaming, plugin, and versioning modules.
43+
- Reports remain deterministic and readable.
44+
- No engine core API files were modified.
45+
46+
## Approved Commit Comment
47+
build(validation-suite): add platform validation suite for full pipeline coverage
48+
49+
## Next Command
50+
APPLY_PR_PLATFORM_VALIDATION_SUITE

‎docs/pr/PLAN_PR_PLATFORM_VALIDATION_SUITE.md‎

Lines changed: 81 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,81 @@
1+
# PLAN_PR_PLATFORM_VALIDATION_SUITE
2+
3+
## Goal
4+
Define a platform-level validation suite that verifies the full authoring, validation, remediation,
5+
packaging, runtime, streaming, plugin, and versioning flows across the accepted platform baseline.
6+
7+
## Context
8+
The project baseline now includes:
9+
- registry-aware editors
10+
- dependency graph
11+
- enforced validation
12+
- assistive remediation
13+
- strict packaging
14+
- strict runtime loader
15+
- editor experience layer
16+
- runtime streaming
17+
- plugin architecture
18+
- project versioning
19+
20+
The next architectural layer is a repeatable validation suite that protects the platform against regressions
21+
and gives future PRs a stable target for acceptance.
22+
23+
## Scope
24+
- Define end-to-end platform scenarios
25+
- Define golden-path and failure-path validation cases
26+
- Define deterministic packaging/runtime verification cases
27+
- Define plugin/versioning/streaming validation coverage
28+
- Define reporting expectations for suite output
29+
- Define acceptance thresholds for future changes
30+
31+
## Non-Goals
32+
- No engine core API changes
33+
- No replacement of existing validation engine contracts
34+
- No destructive migration requirements
35+
- No ad hoc one-off manual-only validation strategy
36+
37+
## Core Suite Categories
38+
- baseline valid project flow
39+
- invalid reference enforcement flow
40+
- remediation confirmation flow
41+
- deterministic packaging comparison flow
42+
- runtime ready/fail-fast flow
43+
- streaming flow validation
44+
- plugin integration flow
45+
- project version compatibility flow
46+
47+
## Proposed Suite Model
48+
```json
49+
{
50+
"platformValidationSuite": {
51+
"version": 1,
52+
"scenarios": [],
53+
"reports": [],
54+
"status": "pass | fail"
55+
}
56+
}
57+
```
58+
59+
## Core Contracts
60+
1. The suite validates full flows, not isolated helpers only.
61+
2. Golden-path valid projects must pass end-to-end.
62+
3. Known-invalid projects must fail at the correct enforced boundaries.
63+
4. Remediation scenarios must demonstrate recoverability.
64+
5. Packaging and runtime determinism must be testable repeatedly.
65+
6. Plugin, versioning, and streaming support must be covered explicitly.
66+
7. Reports must be stable and readable for future automation.
67+
8. Engine core APIs remain unchanged.
68+
69+
## Manual Validation Checklist
70+
1. Valid baseline project passes full flow.
71+
2. Invalid project fails at expected boundary.
72+
3. Remediation can return project to valid state.
73+
4. Packaging results remain deterministic.
74+
5. Runtime ready/fail-fast behavior is validated.
75+
6. Streaming behavior preserves correctness.
76+
7. Plugin integration is validated safely.
77+
8. Versioned project compatibility is validated.
78+
9. Reports are readable and stable.
79+
80+
## Next Command
81+
BUILD_PR_PLATFORM_VALIDATION_SUITE

‎tests/run-tests.mjs‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -92,6 +92,7 @@ import { run as runEditorExperienceLayer } from './tools/EditorExperienceLayer.t
9292
import { run as runRuntimeStreamingSystem } from './tools/RuntimeStreamingSystem.test.mjs';
9393
import { run as runPluginArchitectureSystem } from './tools/PluginArchitectureSystem.test.mjs';
9494
import { run as runProjectVersioningSystem } from './tools/ProjectVersioningSystem.test.mjs';
95+
import { run as runPlatformValidationSuite } from './tools/PlatformValidationSuite.test.mjs';
9596
import './render/Renderer.test.mjs';
9697
import { run as runReplaySystem } from './replay/ReplaySystem.test.mjs';
9798
import { run as runOrbitLabModel } from './samples/OrbitLabModel.test.mjs';
@@ -178,6 +179,7 @@ const tests = [
178179
['RuntimeStreamingSystem', runRuntimeStreamingSystem],
179180
['PluginArchitectureSystem', runPluginArchitectureSystem],
180181
['ProjectVersioningSystem', runProjectVersioningSystem],
182+
['PlatformValidationSuite', runPlatformValidationSuite],
181183
['ReplaySystem', runReplaySystem],
182184
['OrbitLabModel', runOrbitLabModel],
183185
['OrbitLabScene', runOrbitLabScene],

0 commit comments

Comments
 (0)