-
Notifications
You must be signed in to change notification settings - Fork 0
chore(examples): add claude-cli and claude-sdk targets to agent-skills-evals example #877
Copy link
Copy link
Open
Labels
in-progressClaimed by an agent — do not duplicate workClaimed by an agent — do not duplicate work
Description
Problem
The examples/features/agent-skills-evals/ example demonstrates multi-provider skill-trigger evals, but the targets file (.agentv/targets.yaml) only defines an echo provider. Users who want to test with claude-cli or claude-sdk must create their own targets, which adds friction.
The parent examples/features/.agentv/targets.yaml has a claude target (provider: claude), but:
- No explicit
claude-clitarget - No
claude-sdktarget - The README shows
--target claude --targets ../.agentv/targets.yamlbut doesn't document the setup
Proposed Fix
Add claude-cli and claude-sdk targets to examples/features/.agentv/targets.yaml:
- name: claude-cli
provider: claude-cli
grader_target: gemini-flash
log_format: json
- name: claude-sdk
provider: claude-sdk
grader_target: gemini-flash
log_format: jsonUpdate the multi-provider example README to document running with both providers.
Blocked By
- fix(claude-sdk): workspace skills not discovered by Claude Agent SDK provider #876 (claude-sdk workspace skill discovery) — claude-sdk target will fail for skill-trigger evals until that issue is resolved
Acceptance Criteria
-
examples/features/.agentv/targets.yamlincludesclaude-cliandclaude-sdktargets - README documents how to run with each provider
- Running
agentv eval multi-provider-skill-trigger.EVAL.yaml --target claude-cliworks out of the box
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
in-progressClaimed by an agent — do not duplicate workClaimed by an agent — do not duplicate work
Type
Projects
Status
In progress