-
Notifications
You must be signed in to change notification settings - Fork 0
refactor: rename assert: to assertions: in EVAL.yaml schema #603
Copy link
Copy link
Closed
Description
Objective
Rename the assert: key to assertions: across the EVAL.yaml schema, parser, validator, types, docs, and all examples.
Motivation
assertions:reads naturally as a list key —assert:sounds imperative and collides with theassertstatement keyword in JavaScript and Python, which is confusing in code-judge and test contexts.- Renaming pre-v1 is cheap; post-v1 with external eval files in the wild it becomes a breaking migration.
Scope
All occurrences of the YAML key assert: used as an evaluator list in eval files must become assertions:. This includes:
- Schema definition and validation (
eval-file.schema.tsor equivalent) - YAML parser / loader
- TypeScript types (
EvalTest,EvalConfig, etc.) - Builtin evaluator registry and evaluator parser
- All
*.yaml/*.ymleval files underexamples/ - All test fixtures and unit tests that reference the key
- Documentation, README, and skill docs that show EVAL.yaml snippets
- The
agentv validatecommand output if it references the key by name
Design Latitude
- Simple find-and-replace across the repo is acceptable — no API redesign needed.
- Add a deprecation warning (parse
assert:asassertions:with a console warning) if backwards compatibility during transition is desired; otherwise a hard rename is fine. - Update
agentv validateerror messages to useassertions:in examples.
Acceptance Signals
assert:no longer appears as a schema key in any eval file, type, or validatorassertions:is accepted byagentv validatewithout errors- All existing tests pass with the renamed key
- Examples under
examples/all useassertions: agentv validateerror output referencesassertions:notassert:
Non-Goals
- Renaming
assertin TypeScript code (variable names, test assertions, etc.) — only the YAML key - Changes to evaluator internals or evaluation logic
Related
- feat: create EVAL.yaml to evals.json transpiler #598 — EVAL.yaml → evals.json transpiler (update field mapping table:
assert→assertions) - feat: input_files shorthand in EVAL.yaml test cases #602 —
input_files:shorthand
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
No labels