diff --git a/docs/evaluator/agent-eval/evaluate-deployed-agent.mdx b/docs/evaluator/agent-eval/evaluate-deployed-agent.mdx new file mode 100644 index 0000000000..e974fca3eb --- /dev/null +++ b/docs/evaluator/agent-eval/evaluate-deployed-agent.mdx @@ -0,0 +1,210 @@ +--- +title: "Evaluate a Deployed Agent over HTTP" +description: "Point NeMo Evaluator at an agent reachable over HTTP by describing it as a GenericAgent target — the request to send and where the answer is in the response — then score it exactly like the quickstart." +--- + +The [quickstart](/documentation/evaluate-models/agent-eval/quickstart) evaluated an in-process agent. +This guide evaluates an agent that runs **behind an HTTP endpoint** — a service you (or someone else) +deployed. Everything else is the same: the same tasks, the same metrics, the same `run()`. Only the +**target** changes. + +You describe an HTTP agent as a `GenericAgent`: its URL, the JSON request to send, and where the +answer lives in the response. The evaluator then calls that endpoint over real HTTP for each task. + + + +This guide assumes you've done the [quickstart](/documentation/evaluate-models/agent-eval/quickstart) +(install, tasks, metrics, `run()`). It reuses that quickstart's `KeywordMatchMetric` and tasks. + + + +## 1. Describe the agent + +A `GenericAgent` has three parts you configure: + +- **`url`** — where the agent is reachable. +- **`body`** — the JSON request to POST, as a Jinja template rendered against the task's `inputs`. + Reference them directly: the task's instruction is `{{ instruction }}`. The payload is entirely + yours — the evaluator sends exactly what `body` produces. +- **`response_path`** — a JSONPath into the response that selects the agent's answer. + +```python +from nemo_evaluator_sdk.enums import AgentFormat +from nemo_evaluator_sdk.values import GenericAgent + +agent = GenericAgent( + name="my-http-agent", + url="http://127.0.0.1:8000/agent", + format=AgentFormat.GENERIC, + body={"message": "{{ instruction }}"}, # -> {"message": "What is the capital of France?"} + response_path="$.answer", # reads {"answer": "..."} from the response + # api_key_secret="MY_AGENT_TOKEN", # optional: env var (local run) or platform secret (job), sent as a bearer token +) +``` + +Set `body` and `response_path` to match **your** agent's request and response shape. Uncomment +`api_key_secret` for an agent that needs a token — for a local `run()` it names an environment +variable in your process, and for a submitted job it names a platform secret in the workspace; either +way its value is sent as a bearer token on each request. The local stand-in below needs no auth, so +this walkthrough leaves it off. + +## 2. Run it against a local stand-in + +To try this end to end without deploying anything, stand up a tiny local HTTP server that plays the +role of the agent. In practice you'd skip this and point `url` at your real endpoint. + +```python +import json +import threading +from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer + + +class _AgentHandler(BaseHTTPRequestHandler): + def do_POST(self) -> None: + payload = json.loads(self.rfile.read(int(self.headers.get("Content-Length", "0"))) or b"{}") + text = str(payload.get("message", "")).lower() + answer = "Paris" if "france" in text else "Tokyo" if "japan" in text else "unsure" + body = json.dumps({"answer": answer}).encode() + self.send_response(200) + self.send_header("Content-Type", "application/json") + self.end_headers() + self.wfile.write(body) + + def log_message(self, *args: object) -> None: + return # keep the demo output quiet +``` + +## 3. Run the evaluation + +With an agent target and no inference function supplied, `run()` generates answers by making real HTTP +calls to the agent's `url`. + +```python +from nemo_evaluator_sdk.agent_eval.evaluator import AgentEvaluator +from nemo_evaluator_sdk.agent_eval.tasks import AgentEvalRunConfig + +result = await AgentEvaluator().run( + tasks=make_tasks(), # from the quickstart + target=agent, + config=AgentEvalRunConfig(parallelism=2), +) + +for aggregate in result.summary.scores.scores: + print(f"{aggregate.name}: {aggregate.mean}") +``` + +## Point at your real agent + +To evaluate your own deployed agent, change three things and drop the local server: + +1. **`url`** → your agent's endpoint. +2. **`body`** → the request your agent expects (reference the task's inputs, e.g. `{{ instruction }}`). +3. **`response_path`** → the JSONPath to the answer in your agent's response. + +Add `api_key_secret="MY_AGENT_TOKEN"` if it needs auth (an env var for a local run, a platform secret for a job). If your agent is a **NeMo Agent Toolkit** +workflow, use `NemoAgentToolkitAgent` instead of `GenericAgent` — it targets a NAT endpoint's fixed +protocol, so you don't hand-write a `body` template. + +## Full script + +```python +import asyncio +import json +import threading +from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer + +from nemo_evaluator_sdk.agent_eval.evaluator import AgentEvaluator +from nemo_evaluator_sdk.agent_eval.tasks import AgentEvalRunConfig, AgentEvalTask +from nemo_evaluator_sdk.enums import AgentFormat +from nemo_evaluator_sdk.metrics.protocol import MetricInput, MetricOutput, MetricOutputSpec, MetricResult +from nemo_evaluator_sdk.values import GenericAgent + + +class KeywordMatchMetric: + @property + def type(self) -> str: + return "keyword_match" + + def output_spec(self) -> list[MetricOutputSpec]: + return [MetricOutputSpec.continuous_score("score")] + + async def compute_scores(self, input: MetricInput) -> MetricResult: + expected = str(input.row.data.get("reference", {}).get("expected", "")).lower() + answer = (input.candidate.output_text or "").lower() + return MetricResult(outputs=[MetricOutput(name="score", value=1.0 if expected and expected in answer else 0.0)]) + + +def make_tasks() -> list[AgentEvalTask]: + return [ + AgentEvalTask(id="capital-france", intent="Answer the geography question.", + inputs={"instruction": "What is the capital of France?"}, + reference={"expected": "Paris"}, metrics=[KeywordMatchMetric()]), + AgentEvalTask(id="capital-japan", intent="Answer the geography question.", + inputs={"instruction": "What is the capital of Japan?"}, + reference={"expected": "Tokyo"}, metrics=[KeywordMatchMetric()]), + ] + + +# A stand-in "deployed agent": a tiny local HTTP server. Replace with your real agent's URL. +class _AgentHandler(BaseHTTPRequestHandler): + def do_POST(self) -> None: + payload = json.loads(self.rfile.read(int(self.headers.get("Content-Length", "0"))) or b"{}") + text = str(payload.get("message", "")).lower() + answer = "Paris" if "france" in text else "Tokyo" if "japan" in text else "unsure" + body = json.dumps({"answer": answer}).encode() + self.send_response(200) + self.send_header("Content-Type", "application/json") + self.end_headers() + self.wfile.write(body) + + def log_message(self, *args: object) -> None: + return + + +async def main() -> None: + httpd = ThreadingHTTPServer(("127.0.0.1", 0), _AgentHandler) + threading.Thread(target=httpd.serve_forever, daemon=True).start() + host, port = httpd.server_address + + agent = GenericAgent( + name="my-http-agent", + url=f"http://{host}:{port}/agent", + format=AgentFormat.GENERIC, + body={"message": "{{ instruction }}"}, + response_path="$.answer", + ) + try: + result = await AgentEvaluator().run( + tasks=make_tasks(), + target=agent, + config=AgentEvalRunConfig(parallelism=2), + ) + finally: + httpd.shutdown() + + for aggregate in result.summary.scores.scores: + print(f"{aggregate.name}: {aggregate.mean}") + for trial in result.trials: + print(f" {trial.task_id}: {trial.output.output_text!r}") + + +asyncio.run(main()) +``` + +Expected output — the answers came back over HTTP and both contain the expected keyword: + +``` +keyword_match.score: 1.0 + capital-france: 'Paris' + capital-japan: 'Tokyo' +``` + +## Next steps + + + + + + +- Score the agent's **trajectory** (its tool use), not just the final answer. +- Give a task **multiple metrics** and combine them into a named **view**. diff --git a/docs/evaluator/agent-eval/harbor-runner.mdx b/docs/evaluator/agent-eval/harbor-runner.mdx new file mode 100644 index 0000000000..cff1a1814b --- /dev/null +++ b/docs/evaluator/agent-eval/harbor-runner.mdx @@ -0,0 +1,167 @@ +--- +title: "Evaluate a Harbor Task Suite" +description: "Run an existing Harbor task dataset through NeMo Evaluator's Harbor runner — Harbor executes each task in a Docker sandbox and its verifier emits a reward, which the SDK scores and reports like any other agent-eval run." +--- + +[Harbor](https://www.harborframework.com) is a container-based harness for agentic tasks: it runs each +task in a Docker sandbox, lets an agent work in it, then runs a **verifier** that emits a **reward** +(see Harbor's [Core Concepts](https://www.harborframework.com/docs/core-concepts) for its task, trial, +and job model). If you already have Harbor task datasets, the **Harbor runner** runs them and scores +the verifier reward through agent-eval — the same +[`AgentEvaluator`](/documentation/evaluate-models/agent-eval) and the same result and bundle as the +[quickstart](/documentation/evaluate-models/agent-eval/quickstart). Only the runner changes. + + + +Unlike the quickstart, this runner is **not** zero-dependency — it shells out to Harbor and Docker: + +- **Python ≥ 3.12** +- **Docker** installed and running +- **Harbor**, installed separately: `uv pip install "harbor>=0.16.1"`. Harbor is intentionally **not** + a dependency of `nemo-platform[nemo-evaluator-sdk]`, so the rest of the SDK stays lightweight. + +The runner raises a clear error pointing at this install step if `harbor` is missing. + + + +## The dataset + +A Harbor dataset is a directory of **task folders**. For discovery, the runner needs two files per +task — the rest of the task format (environment, verifier, and solution config) is Harbor's own: + +``` +my-suite/ + hello-world/ + task.toml # [task] name = "harbor/hello-world" + instruction.md # Create a file called hello.txt with "Hello, world!" as the content. + ... +``` + +`discover_harbor_tasks` reads each folder into an `AgentEvalTask`: the `[task] name` becomes the task +`id` and its human-readable `intent`, and `instruction.md` becomes `inputs["instruction"]` — the +instruction the agent is prompted with. See +[Harbor's task documentation](https://www.harborframework.com/docs/tasks) for the full task format. + + + +The repo ships a one-task +[example dataset](https://github.com/NVIDIA-NeMo/nemo-platform/tree/main/packages/nemo_evaluator_sdk/examples/harbor/hello_world_dataset) +you can clone and point at (it is not shipped in the installed wheel). Or point the dataset path at +your own Harbor suite. + + + +## Run it + +A Harbor run is a normal agent-eval run: `AgentEvaluator().run(tasks=..., target=runner)`, exactly like +the [quickstart](/documentation/evaluate-models/agent-eval/quickstart) (a callable) and the +[deployed-agent](/documentation/evaluate-models/agent-eval/evaluate-deployed-agent) guide (an HTTP +target). Here the target is a `HarborAgentTaskRunner`. + +```python +import asyncio +from pathlib import Path + +from nemo_evaluator_sdk.agent_eval.evaluator import AgentEvaluator +from nemo_evaluator_sdk.agent_eval.runtimes.harbor_runtime import ( + HarborAgentTaskRunner, + HarborRuntimeConfig, + discover_harbor_tasks, +) + + +async def main() -> None: + # INPUT: your Harbor task suite (a directory of task folders). + tasks = discover_harbor_tasks("path/to/my-suite") + + runner = HarborAgentTaskRunner( + config=HarborRuntimeConfig( + # OUTPUT: where Harbor writes its / results tree (not the dataset). + jobs_dir=Path("./harbor-jobs"), + agent_name="oracle", # a built-in Harbor agent — see "Choosing an agent" + ) + ) + + # Same AgentEvaluator as every agent-eval run; only the target changes. + result = await AgentEvaluator().run(tasks=tasks, target=runner) + + for aggregate in result.summary.scores.scores: + print(f"{aggregate.name}: {aggregate.mean}") + + +asyncio.run(main()) +``` + +The three pieces map onto the model: `discover_harbor_tasks` turns the suite into tasks, +`HarborAgentTaskRunner` runs the Harbor job and returns one trial per Harbor trial, and `AgentEvaluator` +scores each trial's reward with `HarborRewardMetric`. + + + +**`dataset_path` is input; `jobs_dir` is output.** The dataset is your read-only task suite. `jobs_dir` +is a directory the runner writes into — Harbor's per-trial results land under `jobs_dir//`, +and that directory doubles as a re-run cache (see [below](#attempts-concurrency-and-caching)). + + + +The `oracle` agent is Harbor's reference agent — it produces a passing trial on a well-formed task — +so for the one-task example above the verifier reward is `1.0`: + +``` +harbor_reward.reward: 1.0 +``` + +Swap `agent_name` (or `agent_import_path`) for your own agent to get a real score. + +## Choosing an agent + +`HarborRuntimeConfig` decides what runs inside each sandbox (see +[Harbor's agent documentation](https://www.harborframework.com/docs/agents)): + +- **`agent_name`** — a built-in Harbor agent (for example `"oracle"`, Harbor's reference agent, handy + as a smoke test that the harness and dataset are wired up). +- **`agent_import_path`** — your own Harbor agent, e.g. `"my_agent_module:MyAgent"`. Set `agent_dir` + as well when it's a loose file rather than an installed package. Overrides `agent_name`. +- **`agent_model_name`** — the model slug handed to the agent. + +## How scoring works + +1. Harbor runs each task in its sandbox and writes a `__/result.json` per trial, including + the reward from the task's [verifier](https://www.harborframework.com/docs/tasks). +2. The runner reads that reward onto each trial's metadata, and `HarborRewardMetric` scores it — the + `reward` value, or `0.0` for a trial whose verifier emitted none. +3. `result.summary` aggregates the reward across tasks (`harbor_reward.reward`), `result.trials` holds + each trial's status and evidence, and — if you pass a `config` with an `output_dir` to `run()` — the + run **bundle** (including `report.html`) is written like any other agent-eval run. + +## Attempts, concurrency, and caching + +- **`n_attempts`** — trials Harbor runs per task; **`n_concurrent_trials`** — how many run at once. +- The job directory doubles as a **cache**. Pin a stable `job_name` and a completed job whose results + already cover every requested task is re-scored instead of re-run. The default timestamped `job_name` + always runs fresh; set `force_rerun=True` to delete an existing job dir first. + +## Shortcut: `run_harbor_eval` + +When a run is exactly "one Harbor suite, scored by its reward," `run_harbor_eval` collapses the three +steps above — discover, run, score — into a single call: + +```python +from nemo_evaluator_sdk.agent_eval.runtimes.harbor_runtime import HarborRuntimeConfig, run_harbor_eval + +result = await run_harbor_eval( + HarborRuntimeConfig(jobs_dir="./harbor-jobs", agent_name="oracle"), + dataset_path="path/to/my-suite", +) +``` + +It uses the same `AgentEvaluator` and `HarborRewardMetric` under the hood. Prefer the explicit form +above when you want to mix Harbor tasks with other tasks or metrics, or share one evaluator across +targets. + +## Next steps + + + + + diff --git a/docs/evaluator/agent-eval/index.mdx b/docs/evaluator/agent-eval/index.mdx new file mode 100644 index 0000000000..e561c8796c --- /dev/null +++ b/docs/evaluator/agent-eval/index.mdx @@ -0,0 +1,119 @@ +--- +title: "Agent Evaluation" +description: "The task-driven evaluation model — define tasks, run an agent to produce trials, and score each trial (output and trajectory) with per-task metrics." +--- + + + +Agent evaluation is the **task-driven** shape of NeMo Evaluator (see +[Dataset-Driven vs Task-Driven Evaluation](/documentation/evaluate-models/dataset-driven-vs-task-driven-evaluation) +for how it compares to metrics). You define **tasks**, an **agent** performs each one, +and you score the resulting **trial** — not just whether the final answer is right, but *how the agent +got there*. Each task carries its own metrics, so a single suite can grade heterogeneous work. + +## The model + +One run flows through five pieces: + +**Task → (runner) → Trial → (metrics) → Scores → Result** + +- **Task** (`AgentEvalTask`) — the unit of work. Its fields: + - `intent` — a human-readable description of the goal; metadata for the suite's authors, **not** shown to the agent. + - `inputs` — the instruction and anything the agent starts from (what the agent actually sees). + - `reference` (optional) — grader-only, held-out ground truth the agent never sees. + - `metrics` — the scorers for **this** task. + - `views` (optional) — combine several of the task's metric outputs into one named, reported score (see *Score by component* under [Key properties](#key-properties)). +- **Runner** (`AgentTaskRunner`) — performs each task and returns trials through one small interface, + `run_tasks(tasks) -> [AgentEvalTrial]`. The SDK ships runners for a plain async callable, a deployed + agent over HTTP, and container/harness backends; you can supply your own. Scoring never depends on + which runner produced a trial. +- **Trial** (`AgentEvalTrial`) — the durable record of one attempt: the agent's final `output`, the + `evidence` it produced (an [Agent Trajectory Interchange Format (ATIF)](https://github.com/harbor-framework/harbor/blob/main/rfcs/0001-trajectory-format.md) trace of steps and tool calls, final filesystem state, logs), a + `status` (`completed` / `partial` / `failed`), and arbitrary metadata. The trial — not the runner — is what + gets scored, and it can be re-scored offline later. +- **Metrics** (`Metric`) — each task's metrics score its trials through + `compute_scores(input) -> MetricResult`. A metric can read the final output **and** the evidence, so + it can grade the outcome (did it answer correctly?) or the trajectory (did it use the expected tool?). +- **Result** (`AgentEvaluator` → `AgentEvalResult`) — the evaluator orchestrates the run and returns + the trials, per-trial scores, and an aggregated `summary`. Point it at an output directory and it + also writes a run **bundle**: `run.json`, `trials.jsonl`, `scores.jsonl`, `summary.json`, and a + browsable `report.html`. + +## A minimal example + + + +Conceptual — for a runnable, end-to-end version see the quickstart (coming in this section). + + + +```python +from nemo_evaluator_sdk.agent_eval.evaluator import AgentEvaluator +from nemo_evaluator_sdk.agent_eval.tasks import AgentEvalTask + +task = AgentEvalTask( + id="capital-france", + intent="Name the capital of France.", + inputs={"instruction": "What is the capital of France?"}, + reference={"expected": "Paris"}, # grader-only; never shown to the agent + metrics=[OutcomeMetric()], # this task's own scorer(s) +) + +# target is a Model, a deployed Agent (HTTP), or any AgentTaskRunner. +result = await AgentEvaluator().run(tasks=[task], target=my_agent) +print(result.summary) +``` + +## Key properties + +- **Runner-agnostic scoring.** The scorer only ever sees an `AgentEvalTrial`, so the same tasks and + metrics score any runner's output — or previously stored trials re-scored offline. +- **Per-task metrics.** Metrics are attached to each task, not once for the whole run, so a suite can + grade heterogeneous work (docs, Q&A, tests, net-new code) each with the checks that fit it. +- **Score by component.** A single run can score at several levels: + - the **task outcome** — the final answer; + - the **trajectory** — how the agent worked (its tool use and steps); + - **views** — named roll-ups you define on a task that combine two or more of its metric outputs into one reported score (for example, averaging an accuracy metric and a tool-use metric into a single `quality` score); + - the **run-level aggregate** — results also roll up across the whole run. +- **Runs locally.** A full run — including the `report.html` dashboard — is produced on your machine + with no platform services required; the same run can also be submitted as a platform job. +- **Measurement, not decisions.** The evaluator produces scores, aggregates, and provenance — it + doesn't decide pass/fail, gate a release, or compare runs. Those decisions belong to whatever + consumes the results. + +## Targets + +An evaluation target is what performs the tasks: + +- a **`Model`** — a chat/completions endpoint; +- a deployed **`Agent`** reachable over HTTP — a `GenericAgent` (any JSON endpoint) or a + `NemoAgentToolkitAgent`; +- a custom **runner** (`AgentTaskRunner`) — anything that can turn tasks into trials. + +See [Evaluate a Deployed Agent over HTTP](/documentation/evaluate-models/agent-eval/evaluate-deployed-agent) +for how to describe an HTTP agent target. + +## In this section + +- [Quickstart](/documentation/evaluate-models/agent-eval/quickstart) — evaluate an agent end to end in + a local process, no services required. +- [Evaluate a Deployed Agent over HTTP](/documentation/evaluate-models/agent-eval/evaluate-deployed-agent) + — point a run at an agent behind an HTTP endpoint. +- [Evaluate a Harbor Task Suite](/documentation/evaluate-models/agent-eval/harbor-runner) — run an + existing Harbor dataset in Docker and score its verifier reward. +- [Score by Component](/documentation/evaluate-models/agent-eval/score-by-component) — score the + trajectory (not just the answer) and roll metrics into a named view. +- [Targets and Runners](/documentation/evaluate-models/agent-eval/targets-and-runners) — reference for + what a run can point at. +- [Writing Metrics](/documentation/evaluate-models/agent-eval/writing-metrics) — reference for the + `Metric` protocol. +- [Reading Results](/documentation/evaluate-models/agent-eval/reading-results) — what a run returns and + the on-disk bundle it writes. + +## Related + + + + + + diff --git a/docs/evaluator/agent-eval/quickstart.mdx b/docs/evaluator/agent-eval/quickstart.mdx new file mode 100644 index 0000000000..1e95bcae9d --- /dev/null +++ b/docs/evaluator/agent-eval/quickstart.mdx @@ -0,0 +1,244 @@ +--- +title: "Agent Evaluation Quickstart" +description: "Evaluate a simple agent end to end with the NeMo Evaluator SDK — define tasks and a metric, run locally, and read the scores and HTML report. No platform services or API keys required." +--- + +In this quickstart you'll evaluate an agent on two tasks, score its answers, and produce a run +bundle with a browsable HTML report — all on your machine, with **no platform services and no API +keys**. It takes about five minutes. + +For the concepts behind tasks, trials, runners, and metrics, see +[Agent Evaluation](/documentation/evaluate-models/agent-eval). This page is the hands-on version. + +## Prerequisites + +- Python 3.11–3.14 +- The SDK: + +```bash +pip install nemo-platform[nemo-evaluator-sdk] +``` + + + +Everything here runs in a single local Python process. You'll swap the stand-in agent for a real +model or deployed agent once you've seen the flow. + + + +## 1. Define a metric + +A metric scores one trial. It implements three things: a `type` (its name), an `output_spec()` (the +values it emits), and `compute_scores()` (the scoring logic). Here's a tiny one that scores `1.0` +when the agent's answer contains an expected keyword and `0.0` otherwise. It reads the agent's answer +from `input.candidate.output_text` and the task's grader-only `reference` from `input.row.data`. + +```python +from nemo_evaluator_sdk.metrics.protocol import MetricInput, MetricOutput, MetricOutputSpec, MetricResult + + +class KeywordMatchMetric: + """Score 1.0 when the agent's answer contains the expected keyword, else 0.0.""" + + @property + def type(self) -> str: + return "keyword_match" + + def output_spec(self) -> list[MetricOutputSpec]: + return [MetricOutputSpec.continuous_score("score")] + + async def compute_scores(self, input: MetricInput) -> MetricResult: + expected = str(input.row.data.get("reference", {}).get("expected", "")).lower() + answer = (input.candidate.output_text or "").lower() + return MetricResult(outputs=[MetricOutput(name="score", value=1.0 if expected and expected in answer else 0.0)]) +``` + +## 2. Define your tasks + +Each `AgentEvalTask` is one unit of work: an `intent`, the `inputs` the agent acts on (here, an +`instruction`), a grader-only `reference` (held-out truth the agent never sees), and the `metrics` +that score **this** task. The `intent` is a human-readable note about the task's goal — metadata for +whoever reads or maintains the suite. It is **not** passed to the agent, which only ever sees +`inputs`. Because metrics live on the task, different tasks can use different scorers. + +```python +from nemo_evaluator_sdk.agent_eval.tasks import AgentEvalTask + + +def make_tasks() -> list[AgentEvalTask]: + return [ + AgentEvalTask( + id="capital-france", + intent="Answer the geography question.", + inputs={"instruction": "What is the capital of France?"}, + reference={"expected": "Paris"}, + metrics=[KeywordMatchMetric()], + ), + AgentEvalTask( + id="capital-japan", + intent="Answer the geography question.", + inputs={"instruction": "What is the capital of Japan?"}, + reference={"expected": "Tokyo"}, + metrics=[KeywordMatchMetric()], + ), + ] +``` + +## 3. Provide an agent + +An agent is anything that turns a task into an answer. The simplest option is an async function; the +SDK provides a runner — `CallableAgentTaskRunner` — that wraps your function so it plugs into a run. +The runner hands your function the whole task and expects an answer back (here, a canned response +keyed by task `id`). We'll use a stand-in so the quickstart runs with no external services; later +you'll point the evaluation at a real model or a +[deployed agent](/documentation/evaluate-models/agent-eval/evaluate-deployed-agent) instead. + +```python +async def my_agent(task: AgentEvalTask) -> str: + # Stand-in agent — replace with a real model or deployed agent later. + answers = { + "capital-france": "The capital of France is Paris.", + "capital-japan": "The capital of Japan is Tokyo.", + } + return answers[task.id] +``` + +## 4. Run the evaluation + +`AgentEvaluator.run()` sends the tasks to the runner, collects the trials, scores them, and returns +an `AgentEvalResult`. Setting `output_dir` also writes a run bundle to disk. (`run()` is async, so it +lives inside an `async` function — see the full script below.) + +```python +from nemo_evaluator_sdk.agent_eval.evaluator import AgentEvaluator +from nemo_evaluator_sdk.agent_eval.runtimes.callable_runtime import CallableAgentTaskRunner +from nemo_evaluator_sdk.agent_eval.tasks import AgentEvalRunConfig + +result = await AgentEvaluator().run( + tasks=make_tasks(), + target=CallableAgentTaskRunner(my_agent), + config=AgentEvalRunConfig(output_dir="./agent-eval-run", parallelism=2), +) + +for aggregate in result.summary.scores.scores: + print(f"{aggregate.name}: {aggregate.mean}") +``` + +## 5. Read the results + +`run()` returns an `AgentEvalResult` you can work with directly in Python: + +- `result.summary` — aggregated scores per metric output (the mean you just printed), plus coverage + counts (how many trials were scored, failed, or missing an output). +- `result.scores` — one entry per (task, trial, metric): the metric outputs and their status. +- `result.trials` — each trial: the agent's answer, the evidence it produced (trajectory, final + state, logs), and its status (`completed` / `partial` / `failed`). +- `result.run_id` — a stable identifier for this run. + +Because you set `output_dir`, the same data was also written to `./agent-eval-run/` as a run bundle: + +| File | Contents | +|---|---| +| `summary.json` | aggregated scores per metric output (mean / min / max / std-dev / counts) and coverage | +| `scores.jsonl` | one row per (task, trial, metric) — the metric outputs, status, and any diagnostics | +| `trials.jsonl` | one row per trial — the agent's output, its evidence, and status | +| `tasks.jsonl` | the tasks that were evaluated | +| `run.json` | the run manifest — the run id and a map of the artifact files | +| `benchmark.json` | benchmark-grouping metadata for the run | +| `report.html` | a browsable dashboard of the run — open it in a browser | + +The in-memory result and the on-disk bundle hold the same information: use the result object for +programmatic follow-up, and the bundle (especially `report.html`) to inspect or share a run. + +## Full script + +```python +import asyncio + +from nemo_evaluator_sdk.agent_eval.evaluator import AgentEvaluator +from nemo_evaluator_sdk.agent_eval.runtimes.callable_runtime import CallableAgentTaskRunner +from nemo_evaluator_sdk.agent_eval.tasks import AgentEvalRunConfig, AgentEvalTask +from nemo_evaluator_sdk.metrics.protocol import MetricInput, MetricOutput, MetricOutputSpec, MetricResult + + +class KeywordMatchMetric: + """Score 1.0 when the agent's answer contains the expected keyword, else 0.0.""" + + @property + def type(self) -> str: + return "keyword_match" + + def output_spec(self) -> list[MetricOutputSpec]: + return [MetricOutputSpec.continuous_score("score")] + + async def compute_scores(self, input: MetricInput) -> MetricResult: + expected = str(input.row.data.get("reference", {}).get("expected", "")).lower() + answer = (input.candidate.output_text or "").lower() + return MetricResult(outputs=[MetricOutput(name="score", value=1.0 if expected and expected in answer else 0.0)]) + + +def make_tasks() -> list[AgentEvalTask]: + return [ + AgentEvalTask( + id="capital-france", + intent="Answer the geography question.", + inputs={"instruction": "What is the capital of France?"}, + reference={"expected": "Paris"}, + metrics=[KeywordMatchMetric()], + ), + AgentEvalTask( + id="capital-japan", + intent="Answer the geography question.", + inputs={"instruction": "What is the capital of Japan?"}, + reference={"expected": "Tokyo"}, + metrics=[KeywordMatchMetric()], + ), + ] + + +async def my_agent(task: AgentEvalTask) -> str: + # Stand-in agent — replace with a real model or deployed agent later. + answers = { + "capital-france": "The capital of France is Paris.", + "capital-japan": "The capital of Japan is Tokyo.", + } + return answers[task.id] + + +async def main() -> None: + result = await AgentEvaluator().run( + tasks=make_tasks(), + target=CallableAgentTaskRunner(my_agent), + config=AgentEvalRunConfig(output_dir="./agent-eval-run", parallelism=2), + ) + for aggregate in result.summary.scores.scores: + print(f"{aggregate.name}: {aggregate.mean}") + + +asyncio.run(main()) +``` + +Run it: + +```bash +python quickstart.py +``` + +Expected output (both answers contain the expected keyword, so the mean is `1.0`): + +``` +keyword_match.score: 1.0 +``` + +Open `./agent-eval-run/report.html` to see the run in a browser. + +## Next steps + + + + + + +- Swap the stand-in agent for a **real model or a deployed agent over HTTP**. +- Add a **trajectory** metric that scores *how* the agent worked (its tool use), not just the answer. +- Give a task **more than one metric**, or combine metric outputs into a named **view**. diff --git a/docs/evaluator/agent-eval/reading-results.mdx b/docs/evaluator/agent-eval/reading-results.mdx new file mode 100644 index 0000000000..1df54b1fdf --- /dev/null +++ b/docs/evaluator/agent-eval/reading-results.mdx @@ -0,0 +1,94 @@ +--- +title: "Reading Results" +description: "Reference for what a run returns — the in-memory AgentEvalResult (summary, per-metric scores, trials, run id) — and the on-disk run bundle it writes when you set output_dir, including the browsable HTML report." +--- + +`AgentEvaluator().run(...)` returns an `AgentEvalResult`. Set an `output_dir` and it *also* writes a +**run bundle** to disk. The object and the bundle hold the same data — use the object for programmatic +follow-up, and the bundle (especially `report.html`) to inspect or share a run. + +## The result object + +```python +result = await AgentEvaluator().run(tasks=..., target=...) +``` + +| Attribute | What it holds | +|---|---| +| `result.run_id` | stable identifier for this run (e.g. `agent-eval-20260715…`) | +| `result.summary` | aggregated scores and coverage — see [below](#the-summary) | +| `result.scores` | one entry per **(task, trial, metric)** | +| `result.trials` | one entry per **trial** | +| `result.tasks` | the tasks that were evaluated | +| `result.output_dir` / `result.dashboard_path` | where the bundle and `report.html` were written (when `output_dir` was set) | + +### The summary + +`result.summary` (an `AgentEvalSummary`): + +- **`summary.scores.scores`** — the aggregates. Each is named `.` (and + `view.` for a [view](/documentation/evaluate-models/agent-eval/score-by-component)), with + `mean`, `min`, `max`, and `std_dev`. This is what the guides print: + + ```python + for aggregate in result.summary.scores.scores: + print(f"{aggregate.name}: {aggregate.mean}") + ``` + +- **`summary.metric_coverage`** — per metric output, how many trials were `total` / `scored` / failed / + missing, so you can tell a low mean from low coverage. +- **`summary.task_count`**, **`summary.trial_count`**, **`summary.score_count`**. + +### Per-metric scores + +Each entry in `result.scores` carries: `id`, `run_id`, `task_id`, `trial_id`, `metric_type`, `status` +(e.g. `completed` / `failed`), `outputs` (the metric's named outputs), `diagnostics`, and `metadata`. +Use these to drill from an aggregate down to the individual (task, metric) that produced it. + +### Trials + +Each entry in `result.trials` carries: `id`, `task_id`, `status` (`completed` / `partial` / `failed`), +`output` (the agent's final answer), `evidence` (trajectory, final state, logs), and `metadata`. Trials +are the durable, scorer-agnostic record — they can be re-scored offline later. + +## The run bundle + +Set `output_dir` and `run()` writes these files (the same data, on disk): + +| File | Contents | +|---|---| +| `run.json` | the run manifest — run id and a map of the artifact files | +| `summary.json` | the aggregated summary (means / min / max / std-dev and coverage) | +| `scores.jsonl` | one row per (task, trial, metric) | +| `trials.jsonl` | one row per trial — output, evidence, status | +| `tasks.jsonl` | the tasks that were evaluated | +| `benchmark.json` | benchmark-grouping metadata for the run | +| `report.html` | a browsable dashboard — open it in a browser | + +```python +from nemo_evaluator_sdk.agent_eval.tasks import AgentEvalRunConfig + +result = await AgentEvaluator().run( + tasks=..., target=..., config=AgentEvalRunConfig(output_dir="./agent-eval-run"), +) +# -> ./agent-eval-run/report.html, summary.json, scores.jsonl, trials.jsonl, ... +``` + +`report.html` is the fastest way to eyeball a run or hand it to someone else; the `.jsonl` files are +convenient for loading scores and trials into your own tooling. + + + +`report.html` is written only when `AgentEvalRunConfig.write_dashboard` is `True` (the default). Set +`write_dashboard=False` to emit just the JSON/JSONL artifacts and skip the HTML; the `.json` and +`.jsonl` files are always written whenever `output_dir` is set. + + + +## Related + + + + + + diff --git a/docs/evaluator/agent-eval/score-by-component.mdx b/docs/evaluator/agent-eval/score-by-component.mdx new file mode 100644 index 0000000000..5f70c62646 --- /dev/null +++ b/docs/evaluator/agent-eval/score-by-component.mdx @@ -0,0 +1,326 @@ +--- +title: "Score by Component" +description: "Go beyond the final answer: add a metric that scores the agent's trajectory (its tool use), attach it alongside an outcome metric, and roll both into one named view — all runnable locally with no services." +--- + +The [quickstart](/documentation/evaluate-models/agent-eval/quickstart) scored one thing: whether the +final answer contained the right keyword. But an agent can reach the right answer the *wrong way* — +guessing instead of looking something up, or looping on a tool. Agent evaluation lets you score **how** +the agent worked, not just the outcome, and combine several signals into one reported **view**. + +This guide extends the quickstart. You'll add a **trajectory metric** that reads the agent's tool +calls, keep the quickstart's **outcome metric**, and combine them into a `quality` **view**. It stays +zero-dependency — one local process, no API keys. + + + +This builds on the [quickstart](/documentation/evaluate-models/agent-eval/quickstart)'s +`KeywordMatchMetric` and tasks. The full runnable script is at the end. + + + +## 1. The outcome metric + +Reuse the quickstart's `KeywordMatchMetric` — it scores the final answer, reading it from +`input.candidate.output_text` and the grader-only truth from `input.row.data`: + +```python +from nemo_evaluator_sdk.metrics.protocol import MetricInput, MetricOutput, MetricOutputSpec, MetricResult + + +class KeywordMatchMetric: + @property + def type(self) -> str: + return "keyword_match" + + def output_spec(self) -> list[MetricOutputSpec]: + return [MetricOutputSpec.continuous_score("score")] + + async def compute_scores(self, input: MetricInput) -> MetricResult: + expected = str(input.row.data.get("reference", {}).get("expected", "")).lower() + answer = (input.candidate.output_text or "").lower() + return MetricResult(outputs=[MetricOutput(name="score", value=1.0 if expected and expected in answer else 0.0)]) +``` + +## 2. A trajectory metric + +A metric can read more than the final output. `input.candidate.evidence` exposes the trial's +**evidence** — for the trajectory, an [Agent Trajectory Interchange Format (ATIF)](https://github.com/harbor-framework/harbor/blob/main/rfcs/0001-trajectory-format.md) trace of the +agent's steps. `evidence.trace(...)` gives a handle whose `tool_calls()` returns the tool calls in +order; each `ToolCall` has a `function_name` and `arguments`. This metric scores `1.0` when the agent +used the tool you expected and `0.0` otherwise: + +```python +from nemo_evaluator_sdk.metrics.protocol import MetricInput, MetricOutput, MetricOutputSpec, MetricResult +from nemo_evaluator_sdk.values.evidence import EVIDENCE_TRACE + + +class UsedExpectedToolMetric: + """Score whether the agent's trajectory used a given tool.""" + + def __init__(self, expected_tool: str) -> None: + self._expected = expected_tool + + @property + def type(self) -> str: + return "used_expected_tool" + + def output_spec(self) -> list[MetricOutputSpec]: + return [MetricOutputSpec.continuous_score("tool_use")] + + async def compute_scores(self, input: MetricInput) -> MetricResult: + used = False + evidence = input.candidate.evidence + if evidence is not None and evidence.get(EVIDENCE_TRACE) is not None: + calls = await (await evidence.trace(EVIDENCE_TRACE)).tool_calls() + used = any(call.function_name == self._expected for call in calls) + return MetricResult(outputs=[MetricOutput(name="tool_use", value=1.0 if used else 0.0)]) +``` + +The metric checks for evidence first — a trial without a trace simply scores `0.0` rather than +erroring. + +## 3. Produce a trajectory + +Runners that execute the agent capture this evidence for you — a +[deployed HTTP agent](/documentation/evaluate-models/agent-eval/evaluate-deployed-agent) returns a +trajectory when you set `trajectory_path`, and container/harness backends record one as the agent runs. +(A [Harbor](/documentation/evaluate-models/agent-eval/harbor-runner) run scores a verifier reward +instead.) Here the agent is a local callable, so it returns a `TrialDraft` — its final output plus, +when it used a tool, a small ATIF trace built from the SDK's trajectory models. One task's agent uses the `search` tool; the +other skips it and just guesses, returning no trace at all: + +```python +from nemo_evaluator_sdk.agent_eval.runtimes.callable_runtime import TrialDraft +from nemo_evaluator_sdk.agent_eval.tasks import AgentEvalTask +from nemo_evaluator_sdk.agent_eval.trials import AgentOutput +from nemo_evaluator_sdk.values.atif import Step, ToolCall, Trajectory +from nemo_evaluator_sdk.values.evidence import ( + EVIDENCE_FORMAT_ATIF, + EVIDENCE_TRACE, + CandidateEvidence, + EvidenceDescriptor, +) + + +def _trace(tool_name: str, query: str) -> CandidateEvidence: + trajectory = Trajectory( + schema_version="ATIF-v1.7", + steps=[Step(source="agent", tool_calls=[ToolCall(function_name=tool_name, arguments={"query": query})])], + ) + return CandidateEvidence( + descriptors={ + EVIDENCE_TRACE: EvidenceDescriptor( + kind="trace", format=EVIDENCE_FORMAT_ATIF, data=trajectory.model_dump(mode="json") + ) + } + ) + + +async def my_agent(task: AgentEvalTask) -> TrialDraft: + answers = { + "capital-france": ("The capital of France is Paris.", "search"), # used the expected tool + "capital-japan": ("The capital of Japan is Tokyo.", None), # right answer, but skipped the tool + } + text, tool = answers[task.id] + evidence = _trace(tool, task.inputs["instruction"]) if tool else None + return TrialDraft(output=AgentOutput(output_text=text), evidence=evidence) +``` + +## 4. Combine the signals into a view + +Attach both metrics to each task, then define a **view** — a named roll-up of this task's metric +outputs. `SemanticView` reduces its `signals` (each a `metric.output`) into one score; `MEAN` averages +them. The view is reported per task and aggregated across the run as `view.`. + +```python +from nemo_evaluator_sdk.agent_eval.tasks import AgentEvalTask, SemanticReducer, SemanticView, ViewSignal + + +def _quality_view() -> SemanticView: + return SemanticView( + reducer=SemanticReducer.MEAN, + signals=[ + ViewSignal(metric="keyword_match", output="score"), + ViewSignal(metric="used_expected_tool", output="tool_use"), + ], + ) + + +def make_tasks() -> list[AgentEvalTask]: + metrics = [KeywordMatchMetric(), UsedExpectedToolMetric("search")] + return [ + AgentEvalTask( + id="capital-france", intent="Answer the geography question.", + inputs={"instruction": "What is the capital of France?"}, reference={"expected": "Paris"}, + metrics=metrics, views={"quality": _quality_view()}, + ), + AgentEvalTask( + id="capital-japan", intent="Answer the geography question.", + inputs={"instruction": "What is the capital of Japan?"}, reference={"expected": "Tokyo"}, + metrics=metrics, views={"quality": _quality_view()}, + ), + ] +``` + +## 5. Run it and read the components + +```python +from nemo_evaluator_sdk.agent_eval.evaluator import AgentEvaluator +from nemo_evaluator_sdk.agent_eval.runtimes.callable_runtime import CallableAgentTaskRunner +from nemo_evaluator_sdk.agent_eval.tasks import AgentEvalRunConfig + +result = await AgentEvaluator().run( + tasks=make_tasks(), + target=CallableAgentTaskRunner(my_agent), + config=AgentEvalRunConfig(parallelism=2), +) + +for aggregate in result.summary.scores.scores: + print(f"{aggregate.name}: {aggregate.mean}") +``` + +Output: + +``` +keyword_match.score: 1.0 +used_expected_tool.tool_use: 0.5 +view.quality: 0.75 +``` + +This is the whole point of scoring by component. The **outcome** looks perfect — every answer was +correct (`1.0`). But the **trajectory** metric shows only half the agents actually used the expected +tool (`0.5`); the other got the right answer by guessing. The **view** combines the two into a single +`quality` score (`0.75`) you can track over time. Scoring only the answer would have hidden the +shortcut entirely. + +## Full script + +```python +import asyncio + +from nemo_evaluator_sdk.agent_eval.evaluator import AgentEvaluator +from nemo_evaluator_sdk.agent_eval.runtimes.callable_runtime import CallableAgentTaskRunner, TrialDraft +from nemo_evaluator_sdk.agent_eval.tasks import ( + AgentEvalRunConfig, + AgentEvalTask, + SemanticReducer, + SemanticView, + ViewSignal, +) +from nemo_evaluator_sdk.agent_eval.trials import AgentOutput +from nemo_evaluator_sdk.metrics.protocol import MetricInput, MetricOutput, MetricOutputSpec, MetricResult +from nemo_evaluator_sdk.values.atif import Step, ToolCall, Trajectory +from nemo_evaluator_sdk.values.evidence import ( + EVIDENCE_FORMAT_ATIF, + EVIDENCE_TRACE, + CandidateEvidence, + EvidenceDescriptor, +) + + +class KeywordMatchMetric: + @property + def type(self) -> str: + return "keyword_match" + + def output_spec(self) -> list[MetricOutputSpec]: + return [MetricOutputSpec.continuous_score("score")] + + async def compute_scores(self, input: MetricInput) -> MetricResult: + expected = str(input.row.data.get("reference", {}).get("expected", "")).lower() + answer = (input.candidate.output_text or "").lower() + return MetricResult(outputs=[MetricOutput(name="score", value=1.0 if expected and expected in answer else 0.0)]) + + +class UsedExpectedToolMetric: + """Score whether the agent's trajectory used a given tool.""" + + def __init__(self, expected_tool: str) -> None: + self._expected = expected_tool + + @property + def type(self) -> str: + return "used_expected_tool" + + def output_spec(self) -> list[MetricOutputSpec]: + return [MetricOutputSpec.continuous_score("tool_use")] + + async def compute_scores(self, input: MetricInput) -> MetricResult: + used = False + evidence = input.candidate.evidence + if evidence is not None and evidence.get(EVIDENCE_TRACE) is not None: + calls = await (await evidence.trace(EVIDENCE_TRACE)).tool_calls() + used = any(call.function_name == self._expected for call in calls) + return MetricResult(outputs=[MetricOutput(name="tool_use", value=1.0 if used else 0.0)]) + + +def _trace(tool_name: str, query: str) -> CandidateEvidence: + trajectory = Trajectory( + schema_version="ATIF-v1.7", + steps=[Step(source="agent", tool_calls=[ToolCall(function_name=tool_name, arguments={"query": query})])], + ) + return CandidateEvidence( + descriptors={ + EVIDENCE_TRACE: EvidenceDescriptor( + kind="trace", format=EVIDENCE_FORMAT_ATIF, data=trajectory.model_dump(mode="json") + ) + } + ) + + +def _quality_view() -> SemanticView: + return SemanticView( + reducer=SemanticReducer.MEAN, + signals=[ + ViewSignal(metric="keyword_match", output="score"), + ViewSignal(metric="used_expected_tool", output="tool_use"), + ], + ) + + +def make_tasks() -> list[AgentEvalTask]: + metrics = [KeywordMatchMetric(), UsedExpectedToolMetric("search")] + return [ + AgentEvalTask( + id="capital-france", intent="Answer the geography question.", + inputs={"instruction": "What is the capital of France?"}, reference={"expected": "Paris"}, + metrics=metrics, views={"quality": _quality_view()}, + ), + AgentEvalTask( + id="capital-japan", intent="Answer the geography question.", + inputs={"instruction": "What is the capital of Japan?"}, reference={"expected": "Tokyo"}, + metrics=metrics, views={"quality": _quality_view()}, + ), + ] + + +async def my_agent(task: AgentEvalTask) -> TrialDraft: + answers = { + "capital-france": ("The capital of France is Paris.", "search"), # used the expected tool + "capital-japan": ("The capital of Japan is Tokyo.", None), # right answer, but skipped the tool + } + text, tool = answers[task.id] + evidence = _trace(tool, task.inputs["instruction"]) if tool else None + return TrialDraft(output=AgentOutput(output_text=text), evidence=evidence) + + +async def main() -> None: + result = await AgentEvaluator().run( + tasks=make_tasks(), + target=CallableAgentTaskRunner(my_agent), + config=AgentEvalRunConfig(parallelism=2), + ) + for aggregate in result.summary.scores.scores: + print(f"{aggregate.name}: {aggregate.mean}") + + +asyncio.run(main()) +``` + +## Next steps + + + + + diff --git a/docs/evaluator/agent-eval/targets-and-runners.mdx b/docs/evaluator/agent-eval/targets-and-runners.mdx new file mode 100644 index 0000000000..7cdb4858b1 --- /dev/null +++ b/docs/evaluator/agent-eval/targets-and-runners.mdx @@ -0,0 +1,146 @@ +--- +title: "Targets and Runners" +description: "Reference for what an agent-eval run can point at — a Model, a deployed Agent over HTTP, or an AgentTaskRunner (a callable, Harbor, or your own) — with each target's key fields and when to use it." +--- + +`AgentEvaluator().run(target=...)` accepts one of three kinds of target. Whatever you pick, it produces +**trials**, and trials are scored the same way — so the same tasks and metrics work against any target +(see [Agent Evaluation](/documentation/evaluate-models/agent-eval) for the model). + +## At a glance + +| Target | What it is | Extra dependencies | How-to | +|---|---|---|---| +| `Model` | a chat/completions LLM endpoint | an inference endpoint + key | — | +| `GenericAgent` | any HTTP JSON endpoint | none | [Evaluate a Deployed Agent](/documentation/evaluate-models/agent-eval/evaluate-deployed-agent) | +| `NemoAgentToolkitAgent` | a NeMo Agent Toolkit endpoint | a running NAT workflow | [Evaluate a Deployed Agent](/documentation/evaluate-models/agent-eval/evaluate-deployed-agent) | +| `CallableAgentTaskRunner` | an in-process async function | none | [Quickstart](/documentation/evaluate-models/agent-eval/quickstart) | +| `HarborAgentTaskRunner` | a Harbor task suite | `harbor` + Docker | [Harbor Task Suite](/documentation/evaluate-models/agent-eval/harbor-runner) | +| *your* `AgentTaskRunner` | anything that turns tasks into trials | up to you | *(this page)* | + +The union is `AgentEvalTarget = Model | Agent | AgentTaskRunner`, where `Agent = GenericAgent | +NemoAgentToolkitAgent`. + +## `Model` + +A chat/completions endpoint evaluated directly on your tasks — a useful **baseline** (how well does a +bare model do before you wrap it in an agent?). The evaluator prompts it with each task's +`instruction`. + +| Field | Required | Notes | +|---|---|---| +| `url` | yes | endpoint URL (e.g. `.../v1/chat/completions` or `.../v1/completions`) | +| `name` | yes | model identifier, stamped on trials | +| `format` | no | `ModelFormat.NVIDIA_NIM` (default), `ModelFormat.OPEN_AI`, or `ModelFormat.LLAMA_STACK` — serialized as `nim` / `openai` / `llama_stack` | +| `api_key_secret` | no | credential reference — `workspace/secret_name` or `secret_name` | + +```python +from nemo_evaluator_sdk.enums import ModelFormat +from nemo_evaluator_sdk.values import Model + +target = Model(url="https://integrate.api.nvidia.com/v1/chat/completions", name="meta/llama-3.1-8b-instruct", + format=ModelFormat.OPEN_AI, api_key_secret="NVIDIA_API_KEY") +``` + +For a local `run()`, `api_key_secret` names an **environment variable** in your process; for a submitted +job it names a **platform secret** in the workspace. + +## `Agent` (HTTP) + +A deployed agent reachable over HTTP. Two variants, both authenticated with **`api_key_secret`** — the +same credential reference `Model` uses: for a local `run()` it names an environment variable, for a +submitted job a platform secret. Its value is sent as a bearer token on each request. + +### `GenericAgent` + +Any JSON endpoint. You define the request with a Jinja `body` (rendered against the task inputs) and +pull the answer out with JSONPath. Full walkthrough: +[Evaluate a Deployed Agent over HTTP](/documentation/evaluate-models/agent-eval/evaluate-deployed-agent). + +| Field | Required | Notes | +|---|---|---| +| `url` | yes | endpoint the evaluator POSTs to | +| `name` | yes | agent identifier, stamped on trials | +| `format` | no | `AgentFormat.GENERIC` (the default and only value) | +| `body` | yes | Jinja template for the request payload, rendered against task inputs (e.g. `{{ instruction }}`) | +| `response_path` | yes | JSONPath selecting the answer from the response | +| `trajectory_path` | no | JSONPath selecting a trajectory to score | +| `api_key_secret` | no | credential reference (env var locally, platform secret for a job); its value is sent as a bearer token | +| `stream` | no | read JSON SSE `data:` frames instead of a single JSON body (default `false`) | + +### `NemoAgentToolkitAgent` + +A [NeMo Agent Toolkit](https://docs.nvidia.com/nemo/agent-toolkit/latest/index.html) endpoint. It +speaks NAT's fixed request/response protocol, so you don't hand-write a `body` — point it at the +workflow's URL. + +| Field | Required | Notes | +|---|---|---| +| `url` | yes | the NAT workflow endpoint | +| `name` | yes | agent identifier, stamped on trials | +| `format` | no | `AgentFormat.NEMO_AGENT_TOOLKIT` (the default and only value) | +| `nat` | no | `NatAgentConfig` — endpoint / query-param / response-path overrides; defaults preserve `/generate/full` | +| `api_key_secret` | no | credential reference (env var locally, platform secret for a job); its value is sent as a bearer token | + +## `AgentTaskRunner` (callable, Harbor, or your own) + +The most general target: anything implementing the one-method protocol. + +```python +from collections.abc import Sequence + +from nemo_evaluator_sdk.agent_eval.tasks import AgentEvalRunConfig, AgentEvalTask +from nemo_evaluator_sdk.agent_eval.trials import AgentEvalTrial + + +class AgentTaskRunner: + async def run_tasks( + self, tasks: Sequence[AgentEvalTask], config: AgentEvalRunConfig | None = None + ) -> Sequence[AgentEvalTrial]: ... +``` + +The SDK ships two runners you'll usually reach for first: + +- **`CallableAgentTaskRunner`** wraps an `async def agent(task) -> str | AgentOutput | TrialDraft`. The + smallest possible target — no Docker, no HTTP. See the + [Quickstart](/documentation/evaluate-models/agent-eval/quickstart). Return a `TrialDraft` to attach + a trajectory or other evidence (see [Score by Component](/documentation/evaluate-models/agent-eval/score-by-component)). +- **`HarborAgentTaskRunner`** runs a [Harbor](https://www.harborframework.com) task suite in Docker and + scores its verifier reward. See [Harbor Task Suite](/documentation/evaluate-models/agent-eval/harbor-runner). + +Write your own when your agent doesn't fit those — a bespoke harness, a queue, a replay of stored runs. +Return one `AgentEvalTrial` per task; the evaluator scores them exactly like any other target: + +```python +from nemo_evaluator_sdk.agent_eval.trials import AgentEvalTrial, AgentEvalTrialStatus, AgentOutput + + +class EchoRunner: + async def run_tasks(self, tasks, config=None): + return [ + AgentEvalTrial( + id=f"{task.id}:trial", + task_id=task.id, + status=AgentEvalTrialStatus.COMPLETED, + output=AgentOutput(output_text=task.inputs["instruction"]), + ) + for task in tasks + ] +``` + +## Choosing a target + +- Just trying the flow, or you already have the agent in Python → **`CallableAgentTaskRunner`**. +- The agent is deployed behind HTTP → **`GenericAgent`** (any endpoint) or **`NemoAgentToolkitAgent`** + (a NAT workflow). +- You want a model baseline, no agent → **`Model`**. +- You have Harbor task datasets → **`HarborAgentTaskRunner`**. +- None of the above fits → implement **`AgentTaskRunner`**. + +## Related + + + + + + diff --git a/docs/evaluator/agent-eval/writing-metrics.mdx b/docs/evaluator/agent-eval/writing-metrics.mdx new file mode 100644 index 0000000000..a3aa382c0e --- /dev/null +++ b/docs/evaluator/agent-eval/writing-metrics.mdx @@ -0,0 +1,142 @@ +--- +title: "Writing Metrics" +description: "Reference for the Metric protocol — the three members every metric implements, what compute_scores receives (the answer, the trajectory/evidence, the grader-only truth), the output value types, and how results are validated and reported." +--- + +A **metric** scores one trial. Metrics are attached to each task (not once per run), so a suite can +grade heterogeneous work — a Q&A task and a coding task can carry different scorers. Every metric, +however simple or elaborate, implements the same small protocol. + +## The protocol + +```python +from nemo_evaluator_sdk.metrics.protocol import MetricInput, MetricOutputSpec, MetricResult + + +class MyMetric: + @property + def type(self) -> str: + """A unique name for this metric within a task (also the summary key prefix).""" + + def output_spec(self) -> list[MetricOutputSpec]: + """The named values this metric emits, and their types.""" + + async def compute_scores(self, input: MetricInput) -> MetricResult: + """Score one trial and return its outputs.""" +``` + +No base class — a metric is any object with these three members (a structural `Metric` protocol). + +## What `compute_scores` receives + +`input.candidate` is the trial under evaluation: + +| Field | Type | What it holds | +|---|---|---| +| `candidate.output_text` | `str \| None` | the agent's final answer | +| `candidate.evidence` | `CandidateEvidence \| None` | trajectory, final filesystem state, logs — see [Reading evidence](#reading-evidence) | +| `candidate.metadata` | `dict` | trial metadata (e.g. a reward a runner stamped on) | + +`input.row.data` is a dict describing the task and trial: + +| Key | What it holds | +|---|---| +| `input.row.data["reference"]` | grader-only ground truth (the task's `reference`), never shown to the agent | +| `input.row.data["inputs"]` | the task `inputs` (`instruction`, …) | +| `input.row.data["task"]` | `{id, intent, metadata}` | +| `input.row.data["trial"]` | `{id, task_id, status, metadata}` | + +So an **outcome** metric reads `candidate.output_text` and `row.data["reference"]`; a **trajectory** +metric reads `candidate.evidence`. + +## Declaring outputs + +A metric declares its outputs up front; the runtime validates that `compute_scores` returns **exactly** +those names, each coercible to the declared type (a missing or undeclared output raises). Build specs +with the `MetricOutputSpec` factories: + +| Factory | Value type | Use for | +|---|---|---| +| `MetricOutputSpec.continuous_score(name)` | `float` | a numeric score (0–1 or unbounded) | +| `MetricOutputSpec.discrete_score(name)` | `int` | counts or ordinal levels | +| `MetricOutputSpec.boolean(name)` | `bool` | a pass/fail check | +| `MetricOutputSpec.label(name)` | `str` | a category label | +| `MetricOutputSpec.model(name, value_schema)` | your `BaseModel` | structured/custom values | + +A metric may emit **several** outputs — for example an efficiency metric returning both a boolean and a +count: + +```python +def output_spec(self) -> list[MetricOutputSpec]: + return [ + MetricOutputSpec.boolean("efficient_tool_use"), + MetricOutputSpec.discrete_score("max_repeated_tool_calls"), + ] +``` + + + +Prefer `continuous_score` when you want a numeric **mean** in the run summary. A `boolean` output +reports per-trial pass/fail but does not aggregate to a numeric mean on its own (though it still +contributes as 0/1 to a [view](/documentation/evaluate-models/agent-eval/score-by-component)). + + + +## Returning a result + +Return a `MetricResult` whose `outputs` match `output_spec` by name: + +```python +from nemo_evaluator_sdk.metrics.protocol import MetricInput, MetricOutput, MetricOutputSpec, MetricResult + + +class KeywordMatchMetric: + @property + def type(self) -> str: + return "keyword_match" + + def output_spec(self) -> list[MetricOutputSpec]: + return [MetricOutputSpec.continuous_score("score")] + + async def compute_scores(self, input: MetricInput) -> MetricResult: + expected = str(input.row.data.get("reference", {}).get("expected", "")).lower() + answer = (input.candidate.output_text or "").lower() + return MetricResult(outputs=[MetricOutput(name="score", value=1.0 if expected and expected in answer else 0.0)]) +``` + +## Reading evidence + +`candidate.evidence` (a `CandidateEvidence`) holds named descriptors, each exposed through a typed +handle. Always guard first — a trial may not carry a given descriptor: + +```python +from nemo_evaluator_sdk.values.evidence import EVIDENCE_TRACE + +evidence = input.candidate.evidence +if evidence is not None and evidence.get(EVIDENCE_TRACE) is not None: + calls = await (await evidence.trace(EVIDENCE_TRACE)).tool_calls() +``` + +| Evidence | Handle | Reads | +|---|---|---| +| **trace** (`EVIDENCE_TRACE`) | `await evidence.trace(name)` | `.tool_calls()`, `.steps()`, `.token_usage()` — the [Agent Trajectory Interchange Format (ATIF)](https://github.com/harbor-framework/harbor/blob/main/rfcs/0001-trajectory-format.md) trajectory | +| **filesystem** (`EVIDENCE_FINAL_STATE`, `EVIDENCE_INITIAL_STATE`) | `await evidence.filesystem(name)` | `.run_verifier(command)`, `.diff(other)` — run a check or diff two snapshots | +| **logs** | `await evidence.logs(name)` | `.read_text(file)`, `.tail(file)` | + +The [Score by Component](/documentation/evaluate-models/agent-eval/score-by-component) guide has a +complete, runnable trajectory metric; the SDK's `example_metrics.py` (under +`examples/run_agent_eval/`) shows filesystem and trace metrics. + +## How outputs are reported + +Each output aggregates in `result.summary` under the key `.` (mean / min / max / +std-dev). To roll several outputs into one reported score, define a **view** on the task — see +[Score by Component](/documentation/evaluate-models/agent-eval/score-by-component). + +## Related + + + + + + diff --git a/docs/evaluator/evaluation-approaches.mdx b/docs/evaluator/evaluation-approaches.mdx new file mode 100644 index 0000000000..44fd208330 --- /dev/null +++ b/docs/evaluator/evaluation-approaches.mdx @@ -0,0 +1,113 @@ +--- +title: "Dataset-Driven vs Task-Driven Evaluation" +description: "Two complementary shapes of evaluation in NeMo Evaluator — scoring model outputs over a dataset, versus scoring an agent that performs tasks — and how to choose between them." +--- + + + +NeMo Evaluator supports two complementary shapes of evaluation. They share the same scoring +foundation but differ in **what produces the thing you score** and **what you're allowed to look at** +when you score it. + +- **Dataset-driven evaluation** — you have a dataset of homogeneously structured rows, a target + generates an output for each row (or you supply outputs), and metrics score each output against a + reference. This is the path + behind [metrics](/documentation/evaluate-models/metrics). +- **Task-driven evaluation (agent evaluation)** — you have a set of *tasks*, an agent performs each + task, and you score the resulting *trial* — the agent's final output **and** how it got there + (its trajectory, tool calls, and other evidence). + + + +Both shapes score with the **same `Metric` interface**. A deterministic/code scorer or an +LLM-as-a-judge scorer works in either one — the difference is the shape of the input being scored, +not the scorer. + + + +## Dataset-driven evaluation + +You start from a **dataset** — rows of inputs, usually with a reference/expected value. A **model** +(or pipeline) produces an output per row, either generated online at evaluation time or supplied +offline, and metrics score each output against its reference. The suite is uniform — the **same +metric set applies to every row**. + +**Use it when** the thing under test is a model or pipeline you can run over a fixed dataset and you +care about output quality: model quality checks, RAG pipelines, and regression testing against a +labeled set. + +Entry point: [Evaluation Metrics](/documentation/evaluate-models/metrics). + +## Task-driven evaluation (agent evaluation) + +You start from **tasks**. Each task gives an agent an instruction and any inputs it needs; the agent +performs the task through a **runner**, producing a **trial** — the agent's final output plus +evidence such as its trajectory (an [Agent Trajectory Interchange Format (ATIF)](https://github.com/harbor-framework/harbor/blob/main/rfcs/0001-trajectory-format.md) trace of steps and tool calls), final filesystem state, and +logs. Container-based runners also provide an environment (and its final state) the agent works in. +Metrics then score the trial. Because the trajectory is part of the trial, you can score not +just *whether the answer is right* but *how the agent worked* — for example, whether it used the +expected tool. + +Metrics are attached **per task**, not once for the whole run — each task carries its own scorers. +That lets a single evaluation grade **heterogeneous work**: a coding agent whose tasks span +documentation, Q&A, unit tests, and net-new code can score each task type with the checks that fit +it — an LLM-as-a-judge for documentation clarity, a tests-pass check for code, an exact-match check +for a factual answer — all in one suite. Dataset-driven runs, by contrast, apply the same metric set +to every row. + +**Use it when** the thing under test is an agent that *acts*: it calls tools, takes multiple steps, +or edits state — and you care about the process as well as the outcome, or you're comparing skills or +models on agentic tasks. + +Entry point: [Agent Evaluation](/documentation/evaluate-models/agent-eval). + +## At a glance + +| | Dataset-driven | Task-driven (agent evaluation) | +|---|---|---| +| **Unit of evaluation** | a dataset row | a task, and the trial it produces | +| **What produces the candidate** | a model/pipeline generates an output per row | an agent performs the task (via a runner) | +| **What you score** | the output, against a reference | the trial: final output **and** trajectory/evidence | +| **Evidence available to the scorer** | the output (and row fields) | ATIF trace, tool calls, final state, logs | +| **Metric assignment** | one metric set applied to every row | each task defines its own metrics (heterogeneous suites) | +| **Scoring granularity** | per-row metric + aggregate | task · trajectory · component (views) · session | +| **Target** | a `Model` or `Agent` (plus the dataset) | a `Model`, a deployed `Agent` (HTTP), or a custom runner | +| **Entry point** | metrics (`RunConfig`, dataset rows) | `AgentEvaluator` + `AgentEvalTask` | +| **Result** | metric results + aggregates | an `AgentEvalResult` bundle (trials incl. ATIF trajectory & file evidence, scores, summary, report) | +| **Best for** | model quality, RAG, regression on labeled data | agents, tool use, multi-step behavior, skills, model/skill A/B | + +## Which should I use? + + + +Start from *what produces the answer*. If you can generate outputs by running a model over a fixed +dataset, use **dataset-driven**. If an agent has to *do something* to produce the answer — and how it +does it matters — use **task-driven**. + + + +- **Dataset-driven** — you have (or can produce) a labeled dataset and want to score outputs against + references: LLM quality, RAG, or a regression suite. +- **Task-driven** — the system under test is an agent that uses tools or takes multiple steps, you + want to score the trajectory as well as the outcome, your suite is **heterogeneous** so different + tasks need different metrics (e.g. a coding agent producing docs, Q&A, tests, and net-new code), or + you're running a skills / model A/B on agentic tasks. +- **Both** — the two compose. It's common to track base model quality with dataset-driven metrics + *and* agentic behavior with task-driven evaluation. + +## What both share + +- **One scoring interface.** Every scorer implements the same `Metric` protocol, so the same + deterministic and LLM-as-a-judge scorers apply to either shape. +- **Measurement, not decisions.** The evaluator *measures* — it produces scores, aggregates, and + provenance. It does not decide pass/fail, gate a release, or compare runs against each other — + those decisions belong to whatever consumes the results. Both evaluation shapes emit measurements; + decisions live above them. + +## Related + + + + + + diff --git a/docs/evaluator/index.mdx b/docs/evaluator/index.mdx index 376fc17a26..b4de6fd84f 100644 --- a/docs/evaluator/index.mdx +++ b/docs/evaluator/index.mdx @@ -5,7 +5,7 @@ description: "" -Evaluation is powered by NeMo Platform, a cloud-native platform for evaluating large language models (LLMs), RAG pipelines, and AI agents at enterprise scale. The evaluation API provides automated workflows for over 100 industry benchmarks, LLM-as-a-judge scoring, and specialized metrics for RAG and agent systems. +Evaluation is powered by NeMo Platform, a cloud-native platform for evaluating large language models (LLMs), RAG pipelines, and AI agents at enterprise scale. The evaluation API provides LLM-as-a-judge scoring, deterministic and similarity metrics, and specialized metrics for RAG and agent systems. NeMo Platform enables real-time evaluations of your LLM application through APIs, guiding you in refining and optimizing LLMs for enhanced performance and real-world applicability. The NeMo Evaluator APIs can be seamlessly automated within development pipelines, enabling faster iterations without the need for live data. It is cost-effective and suitable for pre-deployment checks and regression testing. @@ -96,9 +96,10 @@ When using Evaluator as a NeMo Platform plugin : ## Evaluation Concepts -NeMo Platform supports two core evaluation primitives: +NeMo Platform supports two shapes of evaluation (see [Dataset-Driven vs Task-Driven Evaluation](/documentation/evaluate-models/dataset-driven-vs-task-driven-evaluation)): -- **Metrics**: Scoring logic that evaluates model outputs. Use metrics when you need flexible, reusable scoring for your own datasets and task-specific criteria. +- **Metrics (dataset-driven)**: Scoring logic that evaluates model outputs over a dataset. Use metrics when you need flexible, reusable scoring for your own datasets and task-specific criteria. +- **Agent evaluation (task-driven)**: Score an agent that performs tasks — its final output and how it got there. See [Agent Evaluation](/documentation/evaluate-models/agent-eval). There are two execution modes and two evaluation patterns: @@ -114,7 +115,7 @@ For deeper details, see [Evaluation Metrics](/documentation/evaluate-models/metr ## Tutorials -After [setting up a local instance of the platform](/documentation/get-started), use the following tutorials to learn how to accomplish common evaluation tasks. These step-by-step guides help you evaluate models using different benchmarks and metrics. +After [setting up a local instance of the platform](/documentation/get-started), use the following tutorials to learn how to accomplish common evaluation tasks. These step-by-step guides help you evaluate models using different metrics. @@ -140,7 +141,7 @@ Learn how to write a domain-specific Python metric, test it locally, and run it ## Recommended Evaluation Journey -Most teams get the best results by starting metric-first, then moving to benchmarks: +Most teams get the best results by starting metric-first: 1. **Develop and validate your metrics first** - Start with [Metrics](/documentation/evaluate-models/metrics) to define how quality should be scored for your use case. diff --git a/docs/fern/versions/latest.yml b/docs/fern/versions/latest.yml index 61fe54847f..7b96f02c75 100644 --- a/docs/fern/versions/latest.yml +++ b/docs/fern/versions/latest.yml @@ -308,9 +308,38 @@ navigation: path: ../../guardrails/terminology.mdx - page: Observability path: ../../guardrails/observability.mdx - - section: Evaluate Models + - section: Evaluate Models & Agents + slug: evaluate-models path: ../../evaluator/index.mdx contents: + - page: Dataset-Driven vs Task-Driven Evaluation + slug: dataset-driven-vs-task-driven-evaluation + path: ../../evaluator/evaluation-approaches.mdx + - section: Agent Evaluation + slug: agent-eval + path: ../../evaluator/agent-eval/index.mdx + contents: + - page: Quickstart + slug: quickstart + path: ../../evaluator/agent-eval/quickstart.mdx + - page: Evaluate a Deployed Agent over HTTP + slug: evaluate-deployed-agent + path: ../../evaluator/agent-eval/evaluate-deployed-agent.mdx + - page: Evaluate a Harbor Task Suite + slug: harbor-runner + path: ../../evaluator/agent-eval/harbor-runner.mdx + - page: Score by Component + slug: score-by-component + path: ../../evaluator/agent-eval/score-by-component.mdx + - page: Targets and Runners + slug: targets-and-runners + path: ../../evaluator/agent-eval/targets-and-runners.mdx + - page: Writing Metrics + slug: writing-metrics + path: ../../evaluator/agent-eval/writing-metrics.mdx + - page: Reading Results + slug: reading-results + path: ../../evaluator/agent-eval/reading-results.mdx - section: Tutorials path: ../../evaluator/tutorials/index.mdx contents: