-
Notifications
You must be signed in to change notification settings - Fork 0
Support env interpolation consistently across all config string fields #506
Copy link
Copy link
Closed
Description
Problem
Environment variable interpolation appears inconsistent across eval YAML fields.
In practice, values like ${VAR} / ${{ VAR }} are resolved in some contexts (for example, certain command/provider fields), but are treated literally in others such as:
workspace.repos[*].source.path
This blocks portable eval configs that need machine-specific local paths.
Repro
Given an eval file with:
workspace:
repos:
- path: ./RepoA
source:
type: local
path: ${{ REPO_A_PATH }}and environment:
REPO_A_PATH=/abs/path/to/RepoArunning agentv eval ... attempts:
git clone --no-checkout ${{ REPO_A_PATH }} ...instead of cloning /abs/path/to/RepoA.
Expected behavior
Interpolation should be supported uniformly across all string fields in eval/targets/workspace config (or at least all fields where string values are accepted), not only selected subsets.
At minimum, these should work consistently in all relevant config files:
${VAR}${{ VAR }}
Proposal
- Add a single, centralized interpolation pass for loaded config objects.
- Apply it recursively to all string-valued fields before validation/execution.
- Keep behavior backward compatible: strings without interpolation syntax remain unchanged.
- If a variable is missing, provide a clear error with key and field path.
Why this matters
- Portable eval definitions across dev machines and CI
- No hardcoded absolute paths in committed YAML
- Predictable behavior for users (same syntax everywhere)
Acceptance criteria
workspace.repos[*].source.pathresolves env variables before clone.- Interpolation works in nested objects/arrays of string fields.
- Both
${VAR}and${{ VAR }}are supported (or one canonical form is documented + enforced with explicit error for the other). - Missing variables produce actionable errors including field path.
- Existing configs without interpolation continue to work unchanged.
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
No labels