feat(agents): nest analyst, eval author, and experimentalist under nemo agents - #1026
Merged
Merged
Conversation
Contributor
Contributor
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: CHILL Plan: Enterprise Run ID: 📒 Files selected for processing (12)
🚧 Files skipped from review as they are similar to previous changes (12)
📝 WalkthroughWalkthroughThe PR registers Analyst, Experimentalist, and Eval Author under ChangesAgent CLI migration
Possibly related PRs
Suggested reviewers: 🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✨ Finishing Touches 💡 1📝 Generate docstrings 💡
🧪 Generate unit tests (beta)
Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@plugins/nemo-experimentalist/pyproject.toml`:
- Around line 24-26: The Experimentalist command path must consistently use
top-level nemo experimentalist rather than nested nemo agents experimentalist.
In plugins/nemo-experimentalist/pyproject.toml lines 24-26, remove the nested
entry-point registration; update the command references in
plugins/nemo-experimentalist/src/nemo_experimentalist_plugin/cli.py lines 4-8,
58, 82, 120, and 235, the consumer command in
plugins/nemo-experimentalist/src/nemo_experimentalist_plugin/profile.py lines
7-9, and the invocation in docs/get-started/example-agent.mdx line 105 to the
top-level path.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Enterprise
Run ID: fff68b37-3e5a-494f-8a52-09d79a17fa8e
📒 Files selected for processing (11)
docs/get-started/example-agent.mdxplugins/nemo-eval-author/pyproject.tomlplugins/nemo-eval-author/src/nemo_eval_author_plugin/cli.pyplugins/nemo-eval-author/tests/test_cli.pyplugins/nemo-experimentalist/pyproject.tomlplugins/nemo-experimentalist/src/nemo_experimentalist_plugin/cli.pyplugins/nemo-experimentalist/src/nemo_experimentalist_plugin/profile.pyplugins/nemo-experimentalist/tests/test_cli_profile.pyplugins/nemo-insights/pyproject.tomlplugins/nemo-insights/src/nemo_insights_plugin/analyst/cli.pyplugins/nemo-insights/src/nemo_insights_plugin/cli.py
Contributor
|
…emo agents` The three platform agents each had their own top-level command group, so the CLI gave no hint that they are agents rather than unrelated tools. Register them under the `nemo.cli.agents` entry-point group instead, which `AgentsCLI` already discovers and mounts, making `nemo agents <agent> <verb>` the canonical path. Experimentalist and Eval Author dual-register the same class, so their verbs are unchanged. The analyst had no command group of its own -- it lived as `nemo insights analyze` -- so its `analyze` and `doctor` callbacks move to module scope and a new `AnalystCLI` mounts them as `run` and `doctor`. Both trees share the same function objects rather than duplicating an implementation. Periodic-analysis management and the analyze job stay on `nemo insights`: they drive scheduled runs, not the analyst. Every existing top-level command keeps working; the `nemo.cli` entries are untouched. Eval Author's placeholder message now reports the invoked path via `ctx.command_path` so it names whichever mount the caller used. Signed-off-by: Nico Tonozzi <ntonozzi@nvidia.com>
Step 2 sourced `config/local.env` and passed `--config config/local.yaml` to `nemo services start`. Neither is needed: the runner falls back to its bundled config, Studio resolves `web/packages/studio/dist` from the source checkout on its own, and the tau3 `.env` copied in step 3 already exports `NMP_BASE_URL`. `nemo setup` also registers the inference provider and picks a default model, which the manual block left to the reader. ClickHouse stays a separate step. Nothing in platform startup provisions it, and Intake returns 503 on trace ingest until it is reachable. Also point the analyst and experimentalist steps at the nested `nemo agents` paths. Signed-off-by: Nico Tonozzi <ntonozzi@nvidia.com>
…S.md The 2026-07-24 rename log asserted that the command group is top-level "rather than the eventual `nemo agents experimentalist`," because the platform's `nemo.cli` entry-point group was flat at the time. That constraint is gone -- `nemo.cli.agents` exists and `AgentsCLI` mounts it -- but the paragraph reads as standing guidance, so review bots and coding agents cite it to reject the nested path. Add a dated entry for the move and correct the stale paragraph in place. Signed-off-by: Nico Tonozzi <ntonozzi@nvidia.com>
nicot
force-pushed
the
nest-agent-clis-under-nemo-agents/ntonozzi
branch
from
July 31, 2026 22:26
ad9e0d6 to
fe949e2
Compare
jeremysilva1098
approved these changes
Jul 31, 2026
5 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
The analyst, Eval Author, and experimentalist each had their own top-level command group, so nothing in the CLI signalled that they are agents rather than unrelated tools. They now register under the
nemo.cli.agentsentry-point group, whichAgentsCLIalready discovers and mounts, makingnemo agents <agent> <verb>the canonical path.Nothing breaks. The
nemo.clientries are untouched, so every existing top-level command still resolves.Command inventory
nemo agents analyst runnemo insights analyzenemo agents analyst doctornemo insights doctornemo agents eval-author {discover,audit,propose,run,doctor}nemo eval-author <verb>nemo agents experimentalist {run,doctor}nemo experimentalist <verb>Periodic-analysis management (
nemo insights analysis {enable,disable,status}) and the analyze job (nemo agents analyze {run,submit,explain}) are unchanged. They drive scheduled runs rather than the analyst itself, so they stay where they are.Implementation notes
nemo insights analyze. Itsanalyzeanddoctorcallbacks move to module scope and a newAnalystCLImounts them asrunanddoctor. Both trees register the same function objects, so the implementations cannot drift.ctx.command_pathinstead of a hardcoded string, so it names whichever mount the caller actually used.Tutorial cleanup
docs/get-started/example-agent.mdxnow starts the platform withuv run nemo setupinstead of sourcingconfig/local.envand passing--config config/local.yaml. Verified that neither is needed: the runner falls back to its bundled config,StudioService._get_static_files_path()resolvesweb/packages/studio/distfrom the source checkout on its own, and the tau3.envcopied in step 3 already exportsNMP_BASE_URL.nemo setupadditionally registers the inference provider and picks a default model, which the manual block left to the reader.ClickHouse remains a separate step — nothing in platform startup provisions it, and Intake returns 503 on trace ingest until it is reachable.
Test plan
pytest plugins/nemo-insights/tests plugins/nemo-eval-author/tests plugins/nemo-experimentalist/tests plugins/nemo-agents/tests/unit/test_cli_extensions.py— 1220 passed, 3 skippedruff check/ruff format --checkclean across the three pluginsnemoCLInemo agents analyst runexecuted end to end against a live platform and wrote insightsmake docs-checkpasses; generated CLI reference is unchanged (plugin CLIs are excluded from generation)NMP_CONFIG_FILE_PATHunsetSummary by CodeRabbit
New Features
nemo agents analyst runandnemo agents analyst doctor.nemo agents experimentalistcommand support.nemo agents eval-authorwith discover, audit, propose, run, and doctor subcommands.Documentation
nemo setup.