Skip to content

refactor: rename assert: to assertions: in EVAL.yaml schema #603

@christso

Description

@christso

Objective

Rename the assert: key to assertions: across the EVAL.yaml schema, parser, validator, types, docs, and all examples.

Motivation

  • assertions: reads naturally as a list key — assert: sounds imperative and collides with the assert statement keyword in JavaScript and Python, which is confusing in code-judge and test contexts.
  • Renaming pre-v1 is cheap; post-v1 with external eval files in the wild it becomes a breaking migration.

Scope

All occurrences of the YAML key assert: used as an evaluator list in eval files must become assertions:. This includes:

  • Schema definition and validation (eval-file.schema.ts or equivalent)
  • YAML parser / loader
  • TypeScript types (EvalTest, EvalConfig, etc.)
  • Builtin evaluator registry and evaluator parser
  • All *.yaml / *.yml eval files under examples/
  • All test fixtures and unit tests that reference the key
  • Documentation, README, and skill docs that show EVAL.yaml snippets
  • The agentv validate command output if it references the key by name

Design Latitude

  • Simple find-and-replace across the repo is acceptable — no API redesign needed.
  • Add a deprecation warning (parse assert: as assertions: with a console warning) if backwards compatibility during transition is desired; otherwise a hard rename is fine.
  • Update agentv validate error messages to use assertions: in examples.

Acceptance Signals

  • assert: no longer appears as a schema key in any eval file, type, or validator
  • assertions: is accepted by agentv validate without errors
  • All existing tests pass with the renamed key
  • Examples under examples/ all use assertions:
  • agentv validate error output references assertions: not assert:

Non-Goals

  • Renaming assert in TypeScript code (variable names, test assertions, etc.) — only the YAML key
  • Changes to evaluator internals or evaluation logic

Related

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions