feat: allow runExperiment to accept a VoltOpsClient directly#970
Conversation
🦋 Changeset detectedLatest commit: 61cee96 The changes in this PR will be included in the next version bump. This PR includes changesets to release 1 package
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
📝 WalkthroughWalkthroughThis change extends the Changes
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~25 minutes Poem
🚥 Pre-merge checks | ✅ 2 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (2 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing touches
Comment |
This comment has been minimized.
This comment has been minimized.
![cubic-dev-ai[bot]](https://avatars.githubusercontent.com/in/1082092?s=60&v=4){kind=small}
Deploying voltagent with
|
| Latest commit: |
61cee96
|
| Status: | ✅ Deploy successful! |
| Preview URL: | https://87ab11a2.voltagent.pages.dev |
| Branch Preview URL: | https://feat-update-voltops-client-e.voltagent.pages.dev |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 2
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
website/evaluation-docs/offline-evaluations.md (1)
253-272: Align VoltOpsClient import with evaluation documentation standard.Line 264 imports from
@voltagent/sdk, butevaluation-docs/overview.mduses@voltagent/core. While both work (@voltagent/sdkre-exports the core version), import from@voltagent/corefor consistency across evaluation docs.
🤖 Fix all issues with AI agents
In @.changeset/clear-signs-cry.md:
- Around line 7-18: Replace the import source for VoltOpsClient to match the
codebase pattern: change the module specifier used in the import that declares
VoltOpsClient (the same symbol referenced when creating voltOpsClient passed
into runExperiment with experiment) from "@voltagent/sdk" to "@voltagent/core"
so the file imports VoltOpsClient from the canonical package used elsewhere.
In `@website/evaluation-docs/overview.md`:
- Around line 102-110: The import of VoltOpsClient in this doc uses the wrong
package: change the import source for VoltOpsClient (and any sibling imports in
this snippet such as runExperiment if applicable) from "@voltagent/core" to
"@voltagent/sdk" so it matches the other evaluation-docs (live-evaluations.md,
offline-evaluations.md); update the import line that references VoltOpsClient to
use the `@voltagent/sdk` package and verify the example still resolves.
🧹 Nitpick comments (2)
examples/with-offline-evals/src/index.ts (1)
1-13: ClarifyVoltOpsClientinitialization in the example.Line 12 constructs
new VoltOpsClient({}); if the client doesn’t default to environment credentials, this example won’t connect. Consider passingVOLTAGENT_PUBLIC_KEY/SECRET_KEYor add a note about env defaults. Also confirm the correct package (@voltagent/corevs@voltagent/sdk) and keep the example aligned with docs.packages/evals/src/experiment/run-experiment.ts (1)
612-691: Throw a clear error whenvoltOpsClientis provided but fails to coerce with VoltOps dataset references.When a client is passed to
attachVoltOpsDatasetResolverbut doesn't match the expected shape for dataset resolution, the error is silent—coerceVoltOpsDatasetClientreturnsundefinedand the descriptor is returned unchanged. If the descriptor has VoltOps identifiers (id,name, orversionId), this likely indicates the user intended VoltOps resolution, making the silent failure confusing.The suggested guard improves error visibility: when both
voltOpsClientand a VoltOps dataset reference are present but coercion fails, throw a clear error instead of silently returning the unresolved descriptor.♻️ Optional guard to surface a clearer error
const datasetClient = coerceVoltOpsDatasetClient(voltOpsClient); if (!datasetClient) { + if (voltOpsClient && (descriptor.id || descriptor.name || descriptor.versionId)) { + throw new Error( + "voltOpsClient does not support dataset resolution; pass a VoltOpsRestClient or VoltOpsDatasetClient.", + ); + } return descriptor; }
| ```ts | ||
| import { VoltOpsClient } from "@voltagent/sdk"; | ||
| import { runExperiment } from "@voltagent/evals"; | ||
| import experiment from "./experiments/support-nightly.experiment"; | ||
|
|
||
| const voltOpsClient = new VoltOpsClient({ | ||
| publicKey: process.env.VOLTAGENT_PUBLIC_KEY, | ||
| secretKey: process.env.VOLTAGENT_SECRET_KEY, | ||
| }); | ||
|
|
||
| const result = await runExperiment(experiment, { | ||
| voltOpsClient, |
There was a problem hiding this comment.
🧩 Analysis chain
🏁 Script executed:
# Search for VoltOpsClient imports across the codebase
rg "VoltOpsClient" --type ts --type js -A 1 -B 1Repository: VoltAgent/voltagent
Length of output: 50375
🏁 Script executed:
# Check the changeset file content
cat .changeset/clear-signs-cry.mdRepository: VoltAgent/voltagent
Length of output: 621
🏁 Script executed:
# Look for package.json files to understand exports
fd "package.json" --type f | head -20Repository: VoltAgent/voltagent
Length of output: 906
🏁 Script executed:
# Search for `@voltagent/core` and `@voltagent/sdk` imports to see patterns
rg "from ['\"]@voltagent/(sdk|core)" --type ts --type js -A 0 -B 0Repository: VoltAgent/voltagent
Length of output: 38161
Update the VoltOpsClient import to use @voltagent/core.
The changeset uses @voltagent/sdk, but the codebase pattern—particularly the with-offline-evals example which is most comparable to this evals feature—imports VoltOpsClient from @voltagent/core. For consistency, change:
import { VoltOpsClient } from "@voltagent/sdk";to:
import { VoltOpsClient } from "@voltagent/core";Both packages export the same class, but @voltagent/core is the canonical source and the consistent pattern across the codebase.
🤖 Prompt for AI Agents
In @.changeset/clear-signs-cry.md around lines 7 - 18, Replace the import source
for VoltOpsClient to match the codebase pattern: change the module specifier
used in the import that declares VoltOpsClient (the same symbol referenced when
creating voltOpsClient passed into runExperiment with experiment) from
"@voltagent/sdk" to "@voltagent/core" so the file imports VoltOpsClient from the
canonical package used elsewhere.
| ```ts title="Node script" | ||
| import { VoltOpsRestClient } from "@voltagent/sdk"; | ||
| import { VoltOpsClient } from "@voltagent/core"; | ||
| import { runExperiment } from "@voltagent/evals"; | ||
| import experiment from "./experiments/support-nightly.experiment"; | ||
|
|
||
| const voltOpsClient = new VoltOpsRestClient({ | ||
| const voltOpsClient = new VoltOpsClient({ | ||
| publicKey: process.env.VOLTAGENT_PUBLIC_KEY, | ||
| secretKey: process.env.VOLTAGENT_SECRET_KEY, | ||
| }); |
There was a problem hiding this comment.
🧩 Analysis chain
🏁 Script executed:
# First, let's check the file in question
cd website && cat -n evaluation-docs/overview.md | sed -n '100,115p'Repository: VoltAgent/voltagent
Length of output: 694
🏁 Script executed:
# Search for all VoltOpsClient imports/usages across the entire repo
rg "VoltOpsClient" -A 1 -B 1Repository: VoltAgent/voltagent
Length of output: 50378
🏁 Script executed:
# Search for package definitions or exports related to VoltOpsClient
rg "export.*VoltOpsClient" -g "*.ts" -g "*.js" -g "*.d.ts"Repository: VoltAgent/voltagent
Length of output: 1313
🏁 Script executed:
# Check for package.json files that might define exports
find . -name "package.json" -type f | head -20Repository: VoltAgent/voltagent
Length of output: 946
Align VoltOpsClient import with sibling evaluation-docs files.
Line 103 imports from @voltagent/core, but other files in evaluation-docs/ (live-evaluations.md, offline-evaluations.md) import from @voltagent/sdk. Choose one pattern consistently across the evaluation-docs folder to avoid confusion.
🤖 Prompt for AI Agents
In `@website/evaluation-docs/overview.md` around lines 102 - 110, The import of
VoltOpsClient in this doc uses the wrong package: change the import source for
VoltOpsClient (and any sibling imports in this snippet such as runExperiment if
applicable) from "@voltagent/core" to "@voltagent/sdk" so it matches the other
evaluation-docs (live-evaluations.md, offline-evaluations.md); update the import
line that references VoltOpsClient to use the `@voltagent/sdk` package and verify
the example still resolves.
1 participant
PR Checklist
Please check if your PR fulfills the following requirements:
Bugs / Features
What is the current behavior?
What is the new behavior?
fixes (issue)
Notes for reviewers
Summary by cubic
Allow runExperiment to accept a VoltOpsClient directly for cloud dataset resolution and telemetry, so you don’t need to initialize a separate REST client. Docs and examples updated to show the new usage.
New Features
Refactors
Written for commit 61cee96. Summary will update on new commits.
Summary by CodeRabbit
New Features
runExperimentnow accepts an optionalvoltOpsClientparameter in the configuration object, enabling direct dataset resolution.Documentation
voltOpsClientconfiguration option for offline evaluations.✏️ Tip: You can customize this high-level summary in your review settings.