fix(plugin): layer discovered config for dynamic hosts#418
Conversation
Signed-off-by: Bryan Bednarski <bbednarski@nvidia.com>
WalkthroughPlugin activation now layers discovered ChangesPlugin discovery activation
Go scope test refactor
Estimated code review effort: 3 (Moderate) | ~20 minutes Sequence Diagram(s)sequenceDiagram
participant Caller
participant PluginHostActivation
participant resolve_plugin_config
participant initialize_plugins_exact
Caller->>PluginHostActivation: activate_with_discovered_config(config, dynamic_plugins)
PluginHostActivation->>resolve_plugin_config: layer discovered plugins.toml with config
resolve_plugin_config-->>PluginHostActivation: resolved PluginConfig
PluginHostActivation->>PluginHostActivation: validate dynamic plugin specs
PluginHostActivation->>initialize_plugins_exact: activate validated plugins
Possibly related PRs
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Comment |
There was a problem hiding this comment.
Actionable comments posted: 2
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
crates/core/src/plugin.rs (1)
1-1: 🚀 Performance & Scalability | 🔵 Trivial | ⚡ Quick winSynchronous file I/O runs directly on the async task in both call sites of
resolve_plugin_config.resolve_plugin_configreads and merges discoveredplugins.tomlfiles synchronously (viaresolve_default_file_plugin_config→load_plugin_config_files) with nospawn_blocking, and it is now invoked from two async entrypoints, one of which (activate_with_discovered_config) is explicitly documented as the entrypoint for language/FFI bindings — a context where blocking the calling async task on disk I/O is more likely to matter for runtime responsiveness. As per coding guidelines, "bindings should preserve callback and future lifetimes rather than blocking or hiding async work unexpectedly."
crates/core/src/plugin.rs#L1462-1466: wrap the discovery/merge work (or at minimum theresolve_default_file_plugin_configcall) intokio::task::spawn_blockingsoresolve_plugin_configdoesn't block its calling executor thread.crates/core/src/plugin.rs#L1453-1456: no code change needed here beyond benefiting from the fix above onceresolve_plugin_configis non-blocking.crates/core/src/plugin/dynamic/host.rs#L91-102: same —activate_with_discovered_configwill stop blocking its caller onceresolve_plugin_configis fixed at the root.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@crates/core/src/plugin.rs` at line 1, Update resolve_plugin_config to execute the synchronous discovery and merge work, including resolve_default_file_plugin_config and its load_plugin_config_files path, inside tokio::task::spawn_blocking. Preserve the existing resolved configuration and error behavior while ensuring both async callers, including activate_with_discovered_config, do not perform blocking file I/O on their executor threads.Source: Coding guidelines
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@crates/core/tests/integration/native_plugin_tests.rs`:
- Around line 1247-1262: Extend the post-clear assertions in the
discovered-config activation test after activation.clear() to verify
lookup_plugin("fixture_native") returns None, while retaining the existing
static deregistration checks and confirming the static base plugin remains
registered as established by the sibling test.
- Around line 46-77: After activation.clear() in the affected native plugin
discovery test, add an assertion that lookup_plugin("fixture_native") returns
None, matching the sibling test. Keep the existing static-base deregistration
assertion and ensure both plugin kinds are verified as deregistered.
---
Outside diff comments:
In `@crates/core/src/plugin.rs`:
- Line 1: Update resolve_plugin_config to execute the synchronous discovery and
merge work, including resolve_default_file_plugin_config and its
load_plugin_config_files path, inside tokio::task::spawn_blocking. Preserve the
existing resolved configuration and error behavior while ensuring both async
callers, including activate_with_discovered_config, do not perform blocking file
I/O on their executor threads.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Enterprise
Run ID: d13414ad-5b5d-4ad9-93bd-fba8fe74335c
📒 Files selected for processing (3)
crates/core/src/plugin.rscrates/core/src/plugin/dynamic/host.rscrates/core/tests/integration/native_plugin_tests.rs
📜 Review details
⏰ Context from checks skipped due to timeout. (2)
- GitHub Check: Check / Run
- GitHub Check: Preview docs
🧰 Additional context used
📓 Path-based instructions (18)
**/*.rs
📄 CodeRabbit inference engine (.agents/skills/prepare-pr/SKILL.md)
**/*.rs: Any Rust change must runjust test-rust
Any Rust change must runcargo fmt --all
Any Rust change must runcargo clippy --workspace --all-targets -- -D warnings
**/*.rs: Runcargo fmt --allfor all FFI work since it is Rust work
Runjust test-rustto validate FFI changes
Runcargo clippy --workspace --all-targets -- -D warningsto enforce strict linting on FFI workWhen Rust files changed as part of Go work, also run
cargo fmt --all,just test-rust, andcargo clippy --workspace --all-targets -- -D warnings
**/*.rs: Runcargo fmt --allwhen Rust files are changed as part of Node work
Runcargo clippy --workspace --all-targets -- -D warningswhen Rust files are changed as part of Node work
Runjust test-rustwhen Rust files are changed as part of Node workWhen changing the core Rust runtime or Rust-facing API surface, format Rust code with
cargo fmt(rustfmt defaults), keepcargo clippy -- -D warningsclean, and satisfycargo deny checkperdeny.toml.
**/*.rs: If any Rust code changed, always runjust test-rust.
If any Rust code changed, also runcargo fmt --all.
If any Rust code changed, also runcargo clippy --workspace --all-targets -- -D warnings.
For Rust changes headed for review, runcargo fmt --allandcargo clippy --workspace --all-targets -- -D warningseven if relying on pre-commit.
Files:
crates/core/src/plugin/dynamic/host.rscrates/core/tests/integration/native_plugin_tests.rscrates/core/src/plugin.rs
{crates/core,crates/adaptive}/**/*
📄 CodeRabbit inference engine (.agents/skills/prepare-pr/SKILL.md)
Changes to
crates/coreorcrates/adaptivemust run the full language matrix
Files:
crates/core/src/plugin/dynamic/host.rscrates/core/tests/integration/native_plugin_tests.rscrates/core/src/plugin.rs
crates/core/**/*.rs
📄 CodeRabbit inference engine (.agents/skills/test-go-binding/SKILL.md)
If the change touched
crates/coreor shared runtime semantics, also usevalidate-changefor broader validation
Files:
crates/core/src/plugin/dynamic/host.rscrates/core/tests/integration/native_plugin_tests.rscrates/core/src/plugin.rs
**/*.{rs,py}
📄 CodeRabbit inference engine (AGENTS.md)
Follow binding naming conventions in Rust and Python: use
snake_case.
Files:
crates/core/src/plugin/dynamic/host.rscrates/core/tests/integration/native_plugin_tests.rscrates/core/src/plugin.rs
**/*.{rs,py,js,mjs,cjs,ts,tsx}
📄 CodeRabbit inference engine (AGENTS.md)
**/*.{rs,py,js,mjs,cjs,ts,tsx}: UseJson = serde_json::Valuein Rust-facing runtime APIs where the existing code expects JSON payloads.
UseResult<T>withFlowErrorin core runtime paths, and keep errors explicit and binding-appropriate at the wrapper layer.
Keep async behavior on the existing tokio-based model; bindings should preserve callback and future lifetimes rather than blocking or hiding async work unexpectedly.
Files:
crates/core/src/plugin/dynamic/host.rscrates/core/tests/integration/native_plugin_tests.rscrates/core/src/plugin.rs
**/*.{rs,py,go,js,ts,c,h}
📄 CodeRabbit inference engine (CONTRIBUTING.md)
Use language-appropriate naming conventions: Rust
snake_case, C FFI exports prefixednemo_relay_, GoPascalCase, Node.jscamelCase, and Pythonsnake_case.
Files:
crates/core/src/plugin/dynamic/host.rscrates/core/tests/integration/native_plugin_tests.rscrates/core/src/plugin.rs
**/*.{rs,go,js,ts}
📄 CodeRabbit inference engine (CONTRIBUTING.md)
Add the SPDX license header to all Rust, Go, JavaScript, and TypeScript source files using the corresponding
//comment form.
Files:
crates/core/src/plugin/dynamic/host.rscrates/core/tests/integration/native_plugin_tests.rscrates/core/src/plugin.rs
{crates/core/src/plugin/dynamic/**,crates/plugin/**,crates/worker/**,crates/worker-proto/**,crates/types/**,python/plugin/**,examples/rust-native-plugin/**,examples/python-grpc-worker-plugin/**,docs/build-plugins/**}
📄 CodeRabbit inference engine (.agents/skills/maintain-dynamic-plugins/SKILL.md)
Keep the stable boundary explicit: native plugins cross a C ABI, and worker plugins cross
grpc-v1.
Files:
crates/core/src/plugin/dynamic/host.rs
{crates/core/src/plugin/dynamic/**/*.rs,examples/rust-native-plugin/**/*.rs}
📄 CodeRabbit inference engine (.agents/skills/maintain-dynamic-plugins/SKILL.md)
Do not pass Rust runtime types, trait objects, futures, or allocator-owned strings across the native dynamic-library boundary.
Files:
crates/core/src/plugin/dynamic/host.rs
{crates/**/src/**/*.rs,python/**/*.py}
📄 CodeRabbit inference engine (.agents/skills/maintain-dynamic-plugins/SKILL.md)
Do not add tests under
src; Rust tests belong in cratetests/trees, and Python SDK tests belong underpython/tests.
Files:
crates/core/src/plugin/dynamic/host.rscrates/core/src/plugin.rs
{crates/core/src/plugin/dynamic/**,examples/rust-native-plugin/**,examples/python-grpc-worker-plugin/**,docs/build-plugins/**}
📄 CodeRabbit inference engine (.agents/skills/maintain-dynamic-plugins/SKILL.md)
Native and worker plugins are trusted extensions; document that native plugins are in-process and unsandboxed, and worker plugins provide process isolation but not a security sandbox.
Files:
crates/core/src/plugin/dynamic/host.rs
{crates/core/src/plugin/dynamic/**/*.rs,crates/plugin/**/*.rs,crates/worker/**/*.rs,crates/worker-proto/**/*.rs,python/plugin/**/*.py}
📄 CodeRabbit inference engine (.agents/skills/maintain-dynamic-plugins/SKILL.md)
Manifest validation must cover kind, compatibility, load contract, integrity, capability mismatch, and disabled-plugin behavior.
Files:
crates/core/src/plugin/dynamic/host.rs
crates/core/src/plugin/dynamic/**/*.rs
📄 CodeRabbit inference engine (.agents/skills/maintain-dynamic-plugins/SKILL.md)
The native loader must keep libraries alive until registered callbacks are cleared and must deregister plugin kinds before unload.
Files:
crates/core/src/plugin/dynamic/host.rs
**/*
📄 CodeRabbit inference engine (.agents/skills/validate-change/SKILL.md)
**/*: Format changed files with the language-native formatter before the final lint/test pass.
If dynamic plugin behavior changed, usemaintain-dynamic-pluginsand include the native SDK, worker protocol, Python SDK, docs, packaging, and Codecov surfaces in the validation plan.
If code changes alter APIs, bindings, commands, paths, packaging behavior, observability/adaptive semantics, or documented best practices, update any dependent maintainer or consumer skills in the same branch.
During iteration, preferuv run pre-commit run --files <changed files...>.
Before review or handoff, runuv run pre-commit run --all-files.
Files:
crates/core/src/plugin/dynamic/host.rscrates/core/tests/integration/native_plugin_tests.rscrates/core/src/plugin.rs
crates/{core,adaptive}/**/*
📄 CodeRabbit inference engine (.agents/skills/validate-change/SKILL.md)
If
crates/coreorcrates/adaptivechanged, run the full validation matrix across Rust, Python, Go, and Node.js.
Files:
crates/core/src/plugin/dynamic/host.rscrates/core/tests/integration/native_plugin_tests.rscrates/core/src/plugin.rs
**/*.{rs,py,go,js,ts}
📄 CodeRabbit inference engine (.agents/skills/validate-change/SKILL.md)
If a language surface changed, always run that language's test target even when Rust core did not change.
Files:
crates/core/src/plugin/dynamic/host.rscrates/core/tests/integration/native_plugin_tests.rscrates/core/src/plugin.rs
crates/{core,adaptive}/**/*.rs
⚙️ CodeRabbit configuration file
crates/{core,adaptive}/**/*.rs: Review the Rust runtime for async correctness, scope isolation, middleware ordering, and event lifecycle regressions.
Pay close attention to task-local/thread-local scope propagation, callback lifetimes, stream finalization, and root_uuid isolation.
Public API changes should preserve existing behavior unless tests and docs show the intended migration path.
Files:
crates/core/src/plugin/dynamic/host.rscrates/core/tests/integration/native_plugin_tests.rscrates/core/src/plugin.rs
{crates/**/tests/**,python/tests/**,go/nemo_relay/**/*_test.go}
⚙️ CodeRabbit configuration file
{crates/**/tests/**,python/tests/**,go/nemo_relay/**/*_test.go}: Tests should cover the behavior promised by the changed API surface, including error paths and cross-request isolation where relevant.
Prefer assertions on lifecycle events, scope stacks, middleware ordering, and binding parity over shallow smoke tests.
Files:
crates/core/tests/integration/native_plugin_tests.rs
🔇 Additional comments (2)
crates/core/src/plugin.rs (1)
1453-1466: 🎯 Functional CorrectnessClean extraction; blocking file I/O concern noted separately.
The
resolve_plugin_configextraction correctly preserves prior layering behavior and letshost.rsreuse the same one-time resolution. The synchronous file-discovery I/O invoked here (and fromactivate_with_discovered_config) is addressed in the consolidated comment below.crates/core/src/plugin/dynamic/host.rs (1)
65-102: LGTM! Theactivate/activate_with_discovered_config/activate_validatedsplit cleanly preservesactivate's "exact" semantics while adding the discovery path, matching the PR's stated goal of keepingactivateimplicit-discovery-free for hosts like the Relay CLI.
Signed-off-by: Bryan Bednarski <bbednarski@nvidia.com>
|
Caution Failed to replace (edit) comment. This is likely due to insufficient permissions or the comment being deleted. Error details |
Signed-off-by: Bryan Bednarski <bbednarski@nvidia.com>
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@go/nemo_relay/scope/error_coverage_test.go`:
- Around line 15-18: Update the call to runWithTestScopeStack so it receives a
no-argument closure that invokes testWithScopeCleanupNoopsWhenPushFails with the
existing t handle. Keep testWithScopeCleanupNoopsWhenPushFails unchanged and
preserve the current test execution flow.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Enterprise
Run ID: cd440358-f955-416b-91cd-875e78609369
📒 Files selected for processing (2)
crates/core/tests/integration/native_plugin_tests.rsgo/nemo_relay/scope/error_coverage_test.go
📜 Review details
⏰ Context from checks skipped due to timeout. (2)
- GitHub Check: Check / Run
- GitHub Check: Preview docs
🧰 Additional context used
📓 Path-based instructions (19)
go/nemo_relay/**/*.go
📄 CodeRabbit inference engine (.agents/skills/test-go-binding/SKILL.md)
go/nemo_relay/**/*.go: Format changed Go packages withcd go/nemo_relay && go fmt ./...
Run Go tests withjust test-goto build and test the NeMo Relay Go binding
Usejust build-gowhen you want an explicit build-only pass or need the artifact for other work
Usejust ci=true test-gowhen you need the CI-style coverage and JUnit path
On macOS, setDYLD_LIBRARY_PATHto the../../target/releasedirectory before running the rawgo testcommand directlyUse
PascalCasefor public Go APIs.
Files:
go/nemo_relay/scope/error_coverage_test.go
**/*.go
📄 CodeRabbit inference engine (CONTRIBUTING.md)
When changing the experimental Go binding, format Go code with
gofmtand keepgo vet ./...passing.
Files:
go/nemo_relay/scope/error_coverage_test.go
**/*.{rs,py,go,js,ts,c,h}
📄 CodeRabbit inference engine (CONTRIBUTING.md)
Use language-appropriate naming conventions: Rust
snake_case, C FFI exports prefixednemo_relay_, GoPascalCase, Node.jscamelCase, and Pythonsnake_case.
Files:
go/nemo_relay/scope/error_coverage_test.gocrates/core/tests/integration/native_plugin_tests.rs
**/*.{rs,go,js,ts}
📄 CodeRabbit inference engine (CONTRIBUTING.md)
Add the SPDX license header to all Rust, Go, JavaScript, and TypeScript source files using the corresponding
//comment form.
Files:
go/nemo_relay/scope/error_coverage_test.gocrates/core/tests/integration/native_plugin_tests.rs
{crates/python/src/py_api/mod.rs,python/nemo_relay/**/*.py,python/nemo_relay/**/*.pyi,go/nemo_relay/**/*.go,crates/node/src/api/**/*.rs}
📄 CodeRabbit inference engine (.agents/skills/add-binding-feature/SKILL.md)
Update the language-native bindings for every exposed surface in Python, Go, and Node.js.
Files:
go/nemo_relay/scope/error_coverage_test.go
{python/nemo_relay/**/*.py,python/nemo_relay/**/*.pyi,go/nemo_relay/**/*.go}
📄 CodeRabbit inference engine (.agents/skills/add-binding-feature/SKILL.md)
Update language wrapper helpers such as Python wrapper modules, Python type stubs, and Go shorthand packages when the new behavior belongs in those helper layers.
Files:
go/nemo_relay/scope/error_coverage_test.go
**/*.{py,go,js,ts}
📄 CodeRabbit inference engine (.agents/skills/maintain-observability/SKILL.md)
Keep Python, Go, and Node.js config objects and subscriber/exporter methods aligned so all bindings expose the same logical knobs and semantics.
Files:
go/nemo_relay/scope/error_coverage_test.go
go/nemo_relay/**
📄 CodeRabbit inference engine (.agents/skills/maintain-optimizer/SKILL.md)
Keep shared plugin helpers in
go/nemo_relayaligned with plugin registration, composition, and lifecycle behavior.
Files:
go/nemo_relay/scope/error_coverage_test.go
**/*
📄 CodeRabbit inference engine (.agents/skills/validate-change/SKILL.md)
**/*: Format changed files with the language-native formatter before the final lint/test pass.
If dynamic plugin behavior changed, usemaintain-dynamic-pluginsand include the native SDK, worker protocol, Python SDK, docs, packaging, and Codecov surfaces in the validation plan.
If code changes alter APIs, bindings, commands, paths, packaging behavior, observability/adaptive semantics, or documented best practices, update any dependent maintainer or consumer skills in the same branch.
During iteration, preferuv run pre-commit run --files <changed files...>.
Before review or handoff, runuv run pre-commit run --all-files.
Files:
go/nemo_relay/scope/error_coverage_test.gocrates/core/tests/integration/native_plugin_tests.rs
**/*.{rs,py,go,js,ts}
📄 CodeRabbit inference engine (.agents/skills/validate-change/SKILL.md)
If a language surface changed, always run that language's test target even when Rust core did not change.
Files:
go/nemo_relay/scope/error_coverage_test.gocrates/core/tests/integration/native_plugin_tests.rs
go/nemo_relay/**/*
⚙️ CodeRabbit configuration file
go/nemo_relay/**/*: Review Go binding changes for cgo memory ownership, race safety, callback cleanup, idiomatic exported APIs, and parity with Rust/FFI behavior.
Any API change should include focused Go tests and consider race-test behavior.
Files:
go/nemo_relay/scope/error_coverage_test.go
{crates/**/tests/**,python/tests/**,go/nemo_relay/**/*_test.go}
⚙️ CodeRabbit configuration file
{crates/**/tests/**,python/tests/**,go/nemo_relay/**/*_test.go}: Tests should cover the behavior promised by the changed API surface, including error paths and cross-request isolation where relevant.
Prefer assertions on lifecycle events, scope stacks, middleware ordering, and binding parity over shallow smoke tests.
Files:
go/nemo_relay/scope/error_coverage_test.gocrates/core/tests/integration/native_plugin_tests.rs
**/*.rs
📄 CodeRabbit inference engine (.agents/skills/prepare-pr/SKILL.md)
**/*.rs: Any Rust change must runjust test-rust
Any Rust change must runcargo fmt --all
Any Rust change must runcargo clippy --workspace --all-targets -- -D warnings
**/*.rs: Runcargo fmt --allfor all FFI work since it is Rust work
Runjust test-rustto validate FFI changes
Runcargo clippy --workspace --all-targets -- -D warningsto enforce strict linting on FFI workWhen Rust files changed as part of Go work, also run
cargo fmt --all,just test-rust, andcargo clippy --workspace --all-targets -- -D warnings
**/*.rs: Runcargo fmt --allwhen Rust files are changed as part of Node work
Runcargo clippy --workspace --all-targets -- -D warningswhen Rust files are changed as part of Node work
Runjust test-rustwhen Rust files are changed as part of Node workWhen changing the core Rust runtime or Rust-facing API surface, format Rust code with
cargo fmt(rustfmt defaults), keepcargo clippy -- -D warningsclean, and satisfycargo deny checkperdeny.toml.
**/*.rs: If any Rust code changed, always runjust test-rust.
If any Rust code changed, also runcargo fmt --all.
If any Rust code changed, also runcargo clippy --workspace --all-targets -- -D warnings.
For Rust changes headed for review, runcargo fmt --allandcargo clippy --workspace --all-targets -- -D warningseven if relying on pre-commit.
Files:
crates/core/tests/integration/native_plugin_tests.rs
{crates/core,crates/adaptive}/**/*
📄 CodeRabbit inference engine (.agents/skills/prepare-pr/SKILL.md)
Changes to
crates/coreorcrates/adaptivemust run the full language matrix
Files:
crates/core/tests/integration/native_plugin_tests.rs
crates/core/**/*.rs
📄 CodeRabbit inference engine (.agents/skills/test-go-binding/SKILL.md)
If the change touched
crates/coreor shared runtime semantics, also usevalidate-changefor broader validation
Files:
crates/core/tests/integration/native_plugin_tests.rs
**/*.{rs,py}
📄 CodeRabbit inference engine (AGENTS.md)
Follow binding naming conventions in Rust and Python: use
snake_case.
Files:
crates/core/tests/integration/native_plugin_tests.rs
**/*.{rs,py,js,mjs,cjs,ts,tsx}
📄 CodeRabbit inference engine (AGENTS.md)
**/*.{rs,py,js,mjs,cjs,ts,tsx}: UseJson = serde_json::Valuein Rust-facing runtime APIs where the existing code expects JSON payloads.
UseResult<T>withFlowErrorin core runtime paths, and keep errors explicit and binding-appropriate at the wrapper layer.
Keep async behavior on the existing tokio-based model; bindings should preserve callback and future lifetimes rather than blocking or hiding async work unexpectedly.
Files:
crates/core/tests/integration/native_plugin_tests.rs
crates/{core,adaptive}/**/*
📄 CodeRabbit inference engine (.agents/skills/validate-change/SKILL.md)
If
crates/coreorcrates/adaptivechanged, run the full validation matrix across Rust, Python, Go, and Node.js.
Files:
crates/core/tests/integration/native_plugin_tests.rs
crates/{core,adaptive}/**/*.rs
⚙️ CodeRabbit configuration file
crates/{core,adaptive}/**/*.rs: Review the Rust runtime for async correctness, scope isolation, middleware ordering, and event lifecycle regressions.
Pay close attention to task-local/thread-local scope propagation, callback lifetimes, stream finalization, and root_uuid isolation.
Public API changes should preserve existing behavior unless tests and docs show the intended migration path.
Files:
crates/core/tests/integration/native_plugin_tests.rs
🔇 Additional comments (2)
crates/core/tests/integration/native_plugin_tests.rs (2)
6-6: LGTM!Also applies to: 46-78, 1221-1264
1221-1264: 📐 Maintainability & Code QualityConfirm the complete required validation matrix was run.
Because this change is under
crates/core, confirmjust test-rust,cargo fmt --all, strict Clippy, the full Rust/Python/Go/Node matrix, and final all-files pre-commit validation.Source: Coding guidelines
Signed-off-by: Bryan Bednarski <bbednarski@nvidia.com>
There was a problem hiding this comment.
♻️ Duplicate comments (1)
crates/core/tests/integration/native_plugin_tests.rs (1)
1246-1249: 🎯 Functional Correctness | 🟡 Minor | ⚡ Quick winStill missing the post-clear "implementation remains registered" assertion.
A prior review on this same test requested two assertions after
activation.clear():lookup_plugin(STATIC_BASE_PLUGIN_KIND).is_some()andlookup_plugin("fixture_native").is_none(). Only the second was added here. SinceSTATIC_BASE_PLUGIN_KINDwas registered viaregister_pluginoutside the activation transaction (line 1230),clear()should only fire the runtime dereg callback (hence the counter check) without removing the plugin implementation from the registry — solookup_plugin(STATIC_BASE_PLUGIN_KIND)should still resolve until the explicitderegister_plugincall on line 1249. Without this assertion, a regression whereclear()incorrectly also drops the implementation from the registry would go undetected.As per path instructions, tests matching
{crates/**/tests/**,...}should "cover the behavior promised by the changed API surface" and "prefer assertions on lifecycle events, scope stacks, middleware ordering, and binding parity over shallow smoke tests."✅ Proposed added assertion
activation.clear().expect("discovered host should clear"); assert_eq!(STATIC_BASE_DEREGISTRATIONS.load(Ordering::SeqCst), 1); + assert!(lookup_plugin(STATIC_BASE_PLUGIN_KIND).is_some()); assert!(lookup_plugin("fixture_native").is_none()); assert!(deregister_plugin(STATIC_BASE_PLUGIN_KIND));🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@crates/core/tests/integration/native_plugin_tests.rs` around lines 1246 - 1249, Extend the test after activation.clear() to assert lookup_plugin(STATIC_BASE_PLUGIN_KIND).is_some() before the existing fixture_native absence assertion. Keep the registration-count check and final deregister_plugin(STATIC_BASE_PLUGIN_KIND) unchanged so the test verifies the implementation remains registered until explicit deregistration.Source: Path instructions
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Duplicate comments:
In `@crates/core/tests/integration/native_plugin_tests.rs`:
- Around line 1246-1249: Extend the test after activation.clear() to assert
lookup_plugin(STATIC_BASE_PLUGIN_KIND).is_some() before the existing
fixture_native absence assertion. Keep the registration-count check and final
deregister_plugin(STATIC_BASE_PLUGIN_KIND) unchanged so the test verifies the
implementation remains registered until explicit deregistration.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Enterprise
Run ID: 91284c39-ee4f-4e3e-937d-2754814b8299
📒 Files selected for processing (1)
crates/core/tests/integration/native_plugin_tests.rs
📜 Review details
🧰 Additional context used
📓 Path-based instructions (12)
**/*.rs
📄 CodeRabbit inference engine (.agents/skills/prepare-pr/SKILL.md)
**/*.rs: Any Rust change must runjust test-rust
Any Rust change must runcargo fmt --all
Any Rust change must runcargo clippy --workspace --all-targets -- -D warnings
**/*.rs: Runcargo fmt --allfor all FFI work since it is Rust work
Runjust test-rustto validate FFI changes
Runcargo clippy --workspace --all-targets -- -D warningsto enforce strict linting on FFI workWhen Rust files changed as part of Go work, also run
cargo fmt --all,just test-rust, andcargo clippy --workspace --all-targets -- -D warnings
**/*.rs: Runcargo fmt --allwhen Rust files are changed as part of Node work
Runcargo clippy --workspace --all-targets -- -D warningswhen Rust files are changed as part of Node work
Runjust test-rustwhen Rust files are changed as part of Node workWhen changing the core Rust runtime or Rust-facing API surface, format Rust code with
cargo fmt(rustfmt defaults), keepcargo clippy -- -D warningsclean, and satisfycargo deny checkperdeny.toml.
**/*.rs: If any Rust code changed, always runjust test-rust.
If any Rust code changed, also runcargo fmt --all.
If any Rust code changed, also runcargo clippy --workspace --all-targets -- -D warnings.
For Rust changes headed for review, runcargo fmt --allandcargo clippy --workspace --all-targets -- -D warningseven if relying on pre-commit.
Files:
crates/core/tests/integration/native_plugin_tests.rs
{crates/core,crates/adaptive}/**/*
📄 CodeRabbit inference engine (.agents/skills/prepare-pr/SKILL.md)
Changes to
crates/coreorcrates/adaptivemust run the full language matrix
Files:
crates/core/tests/integration/native_plugin_tests.rs
crates/core/**/*.rs
📄 CodeRabbit inference engine (.agents/skills/test-go-binding/SKILL.md)
If the change touched
crates/coreor shared runtime semantics, also usevalidate-changefor broader validation
Files:
crates/core/tests/integration/native_plugin_tests.rs
**/*.{rs,py}
📄 CodeRabbit inference engine (AGENTS.md)
Follow binding naming conventions in Rust and Python: use
snake_case.
Files:
crates/core/tests/integration/native_plugin_tests.rs
**/*.{rs,py,js,mjs,cjs,ts,tsx}
📄 CodeRabbit inference engine (AGENTS.md)
**/*.{rs,py,js,mjs,cjs,ts,tsx}: UseJson = serde_json::Valuein Rust-facing runtime APIs where the existing code expects JSON payloads.
UseResult<T>withFlowErrorin core runtime paths, and keep errors explicit and binding-appropriate at the wrapper layer.
Keep async behavior on the existing tokio-based model; bindings should preserve callback and future lifetimes rather than blocking or hiding async work unexpectedly.
Files:
crates/core/tests/integration/native_plugin_tests.rs
**/*.{rs,py,go,js,ts,c,h}
📄 CodeRabbit inference engine (CONTRIBUTING.md)
Use language-appropriate naming conventions: Rust
snake_case, C FFI exports prefixednemo_relay_, GoPascalCase, Node.jscamelCase, and Pythonsnake_case.
Files:
crates/core/tests/integration/native_plugin_tests.rs
**/*.{rs,go,js,ts}
📄 CodeRabbit inference engine (CONTRIBUTING.md)
Add the SPDX license header to all Rust, Go, JavaScript, and TypeScript source files using the corresponding
//comment form.
Files:
crates/core/tests/integration/native_plugin_tests.rs
**/*
📄 CodeRabbit inference engine (.agents/skills/validate-change/SKILL.md)
**/*: Format changed files with the language-native formatter before the final lint/test pass.
If dynamic plugin behavior changed, usemaintain-dynamic-pluginsand include the native SDK, worker protocol, Python SDK, docs, packaging, and Codecov surfaces in the validation plan.
If code changes alter APIs, bindings, commands, paths, packaging behavior, observability/adaptive semantics, or documented best practices, update any dependent maintainer or consumer skills in the same branch.
During iteration, preferuv run pre-commit run --files <changed files...>.
Before review or handoff, runuv run pre-commit run --all-files.
Files:
crates/core/tests/integration/native_plugin_tests.rs
crates/{core,adaptive}/**/*
📄 CodeRabbit inference engine (.agents/skills/validate-change/SKILL.md)
If
crates/coreorcrates/adaptivechanged, run the full validation matrix across Rust, Python, Go, and Node.js.
Files:
crates/core/tests/integration/native_plugin_tests.rs
**/*.{rs,py,go,js,ts}
📄 CodeRabbit inference engine (.agents/skills/validate-change/SKILL.md)
If a language surface changed, always run that language's test target even when Rust core did not change.
Files:
crates/core/tests/integration/native_plugin_tests.rs
crates/{core,adaptive}/**/*.rs
⚙️ CodeRabbit configuration file
crates/{core,adaptive}/**/*.rs: Review the Rust runtime for async correctness, scope isolation, middleware ordering, and event lifecycle regressions.
Pay close attention to task-local/thread-local scope propagation, callback lifetimes, stream finalization, and root_uuid isolation.
Public API changes should preserve existing behavior unless tests and docs show the intended migration path.
Files:
crates/core/tests/integration/native_plugin_tests.rs
{crates/**/tests/**,python/tests/**,go/nemo_relay/**/*_test.go}
⚙️ CodeRabbit configuration file
{crates/**/tests/**,python/tests/**,go/nemo_relay/**/*_test.go}: Tests should cover the behavior promised by the changed API surface, including error paths and cross-request isolation where relevant.
Prefer assertions on lifecycle events, scope stacks, middleware ordering, and binding parity over shallow smoke tests.
Files:
crates/core/tests/integration/native_plugin_tests.rs
🔇 Additional comments (1)
crates/core/tests/integration/native_plugin_tests.rs (1)
44-44: LGTM! Subprocess isolation for env-sensitive discovery is well-structured.The re-exec-via-
current_exe()pattern cleanly isolatesXDG_CONFIG_HOME/current_dirmutation to a child process, avoiding races with other tests sharingNATIVE_PLUGIN_TEST_LOCKand process-wide env state, and the TOML fixture matches the documentedplugins.tomlschema (version,[[components]],kind,enabled).Also applies to: 1189-1245
|
/merge |
#### Overview This adds an owned Python binding for initializing Relay configuration with explicit native and worker dynamic plugins. It builds on the shared process-wide host lifecycle merged in #364 and the discovered-configuration layering merged in #418. `initialize_with_dynamic_plugins(...)` is the owned initialization path when at least one dynamic specification is present. It resolves discovered `plugins.toml` configuration once, layers the explicit base configuration over it, then initializes statically registered components before appending dynamic-plugin components in specification order. Static-only harness-native applications continue to use `plugin.initialize()` and `plugin.clear()`. - [x] I confirm this contribution is my own work, or I have the right to submit it under this project license. - [x] I searched existing issues and open pull requests, and this does not duplicate existing work. #### Details - Add `DynamicPluginActivationSpec` and async `initialize_with_dynamic_plugins(...)` to `nemo_relay.plugin`. - Return an owned `PluginHostActivation` with the validation report and activation-handle state. - Support async `close()`, `async with`, and best-effort finalization without blocking the Python finalizer or holding the GIL during teardown. - Share one cancellation-resistant close operation across repeated or concurrent callers; every caller observes the same cached result. - Document that an inactive handle has begun teardown, but does not guarantee another process-wide activation can start if teardown retained ownership for safety. - Preserve raw plugin-document omission semantics: structural `None` fields are omitted, while component-local `config` maps preserve nested `None` as JSON `null`. - Preserve the existing static `plugin.initialize()` and `plugin.clear()` APIs. Empty dynamic initialization is rejected without claiming process ownership, so static initialization remains available. - Use one Python exception-classification path for initialization and teardown failures. - Add neutral native and worker fixture coverage for callbacks, discovered/static-plus-dynamic configuration, conflicts, partial-load rollback, explicit close, context cleanup, concurrent close, cancellation, finalization, and platform-safe manifest paths. #### Where should the reviewer start? Start with `python/nemo_relay/plugin.py` for the public `initialize_with_dynamic_plugins(...)` API and `crates/python/src/py_plugin.rs` for ownership, cancellation-resistant cleanup, and exception translation. End-to-end fixture cases are in `python/tests/test_dynamic_plugin_host.py`; core discovered-configuration layering is covered by #418. #### Validation - `python/tests/test_dynamic_plugin_host.py` — 14 passed - `cargo test -p nemo-relay-python` — 63 unit tests and 1 integration test passed - `cargo clippy -p nemo-relay-python --all-targets -- -D warnings` - `cargo fmt --check` - Ruff check and format passed for changed Python files - `git diff --check` #### Related Issues Builds on #364 and #418. Authors: - Bryan Bednarski (https://github.com/bbednarski9) Approvers: - Zhongxuan (Daniel) Wang (https://github.com/zhongxuanwang-nv) - Will Killian (https://github.com/willkill07) URL: #365
#### Overview This adds an owned Node binding for explicit native and worker dynamic-plugin activation. The shared process-wide activation lifecycle landed in #364, and discovery-aware configuration layering landed in #418. This PR now contains only the Node binding and its tests. Static-only harness-native applications keep using `plugin.initialize()` / `plugin.clear()`. `initializeWithDynamicPlugins(...)` requires at least one dynamic specification. For harness-native activation, the supplied config is layered over discovered `plugins.toml` configuration; static components from that effective config initialize before components appended by the dynamic plugins. - [x] I confirm this contribution is my own work, or I have the right to submit it under this project's license. - [x] I searched existing issues and open pull requests, and this does not duplicate existing work. #### Details - Add `initializeWithDynamicPlugins(config, specs)` with a dedicated JavaScript/TypeScript specification shape for native and worker plugins. - Return an owned activation exposing `report`, `active`, async `close()`, and `[Symbol.asyncDispose]()` for deterministic `await using` cleanup. - Define `active` as handle state: it becomes false when teardown begins and does not by itself guarantee that process-wide ownership has been released after a teardown failure. - Share one detached close operation across repeated and concurrent callers; every caller observes the same teardown result. - Run cleanup off the JavaScript thread and retain defensive finalization when callers omit explicit close. - Preserve existing static `plugin.initialize()` and `plugin.clear()` behavior. Empty dynamic activation is rejected without claiming process ownership, so static initialization remains available. - Keep complete core validation, manifest, load, ownership, and teardown diagnostics in rejected promises. - Add neutral native and worker fixture coverage for discovered and explicit static configuration, managed LLM/tool callbacks, callback absence after close, partial-load rollback, conflicts, repeated/concurrent close, structured disposal, cancellation-safe cleanup, and garbage collection. #### Where should the reviewer start? Start with `crates/node/src/api/mod.rs` for N-API ownership and cleanup, then `crates/node/plugin.d.ts` and `crates/node/plugin.js` for the `initializeWithDynamicPlugins` public contract and deliberate absence of an old-name alias. The lifecycle cases are in `crates/node/tests/dynamic_plugin_tests.mjs`. #### Validation - `npm run build-debug` - Dynamic-plugin Node tests — 9 passed - Full Node suite — 263 passed - `npm run check:docstrings --workspace crates/node` - `cargo fmt --all -- --check` - `cargo test -p nemo-relay-node` - `cargo clippy -p nemo-relay-node --all-targets -- -D warnings` - Generated native/wrapper export consistency and stale-name checks passed - Changed-file Prettier checks passed - `git diff --check` #### Related Issues: (use one of the action keywords Closes / Fixes / Resolves / Relates to) Relates to #364. Relates to #418. ## Summary by CodeRabbit - **New Features** - Added a Node.js API to activate and manage dynamically resolved plugins (`initializeWithDynamicPlugins`). - Supports native and worker dynamic plugins with per-plugin identifiers, optional environment targeting, and per-plugin JSON config. - Returns an owned activation handle exposing `active` status and a configuration validation `report`. - Provides deterministic, idempotent lifecycle teardown via `close()` and async disposal (`Symbol.asyncDispose`). - **Bug Fixes** - Improved dynamic plugin cleanup for idempotent and concurrent shutdown, including recovery after rejected activations. - **Tests** - Added comprehensive dynamic plugin host tests covering lifecycle, disposal, concurrency, and finalization behavior (native + worker). Authors: - Bryan Bednarski (https://github.com/bbednarski9) Approvers: - Zhongxuan (Daniel) Wang (https://github.com/zhongxuanwang-nv) - Will Killian (https://github.com/willkill07) URL: #366
## Summary This exposes the shared owned dynamic-plugin host through the C `nemo_relay_initialize_with_dynamic_plugins` and Go `InitializeWithDynamicPlugins` entry points for evaluation. Static-only harness-native applications continue using `nemo_relay_initialize_plugins` or Go `InitializePlugins`. Dynamic activation requires at least one specification. At startup, the binding discovers the standard `plugins.toml` files once, layers the explicit configuration over them, and then appends components loaded from the dynamic-plugin specifications. File-configured and explicitly configured static components initialize before dynamic components. Configuration files are not watched or reloaded. - expose the shared dynamic-plugin host through `nemo_relay_initialize_with_dynamic_plugins`, returning an opaque C ABI activation handle - return the validation report as caller-released JSON while keeping all library and worker ownership in Rust - layer explicit configuration over one-time `plugins.toml` discovery with explicit values taking precedence - require explicit resolved manifest paths for dynamic plugins; installation-state and manifest discovery remain outside the binding - provide sequentially idempotent clear/free functions with null and pointer-to-pointer safety - serialize concurrent C clear calls, while requiring callers to synchronize free against every use of the raw activation handle - report a C clear failure only on the call that performs teardown because the activation is consumed regardless of outcome - provide a Go `PluginActivation` wrapper with synchronous explicit `Close` and asynchronous defensive finalization - make copied Go activation values share one cleanup state and the same cached close result - reject nil or empty dynamic specifications before crossing CGo; the C API returns `InvalidArg` with actionable guidance - preserve the existing public Go config structs and JSON behavior while using a private activation wire shape so disabled static components remain disabled - preserve thread-local FFI diagnostics across Go/CGo calls - mark the dynamic activation surface as experimental in Rust docs, the generated C header, Go docs, and binding READMEs ## Reviewer guidance - The entry points are named as initialization APIs because they perform one-shot discovered, static, and dynamic component initialization. - This revision is naming-only: lifecycle behavior, validation, layering, cleanup, and ownership are unchanged. - The old C and Go names are intentionally not retained as aliases because this experimental API has not shipped. ## Validation - native and worker dynamic plugins load through the C ABI and Go/CGo wrapper - a static component discovered from a project `plugins.toml` and an explicitly supplied dynamic plugin activate together and both callbacks execute - explicit configuration overrides discovered values, static components initialize before dynamic components, and discovery occurs only once at startup - callbacks are unavailable after close - valid-first/invalid-second activation rolls back without leaking kinds or callbacks - asynchronous defensive finalization releases process-wide ownership without blocking Go's finalizer goroutine - empty specifications, invalid JSON, missing manifests, repeated sequential clear/free, copied or concurrent Go close, cached teardown errors, null cleanup, and incomplete FFI outputs are covered - public Go config JSON remains backward-compatible while activation serializes disabled static components explicitly - `cargo test -p nemo-relay-ffi --all-features` — 79 unit and 74 integration tests passed - `cargo clippy --workspace --all-targets -- -D warnings` - full `go test -v ./...` suite passed across all Go binding packages - focused finalizer and parser tests passed 20 repetitions under the Go race detector - `go vet ./...`, formatting, generated-header synchronization, and pre-commit checks passed ## Production gate Before removing the **DO NOT MERGE** label, identify a real consumer and validate at minimum: host startup/shutdown integration, activation-handle ownership, in-flight callback teardown, failure diagnostics, packaging/linking, and repeated process lifecycle behavior. ## Stack - Depends on #418 for one-time harness-native `plugins.toml` discovery and layering - Builds on the shared activation host merged through #364 ## Summary by CodeRabbit * **New Features** * Added experimental dynamic plugin activation in the FFI and Go bindings, including an opaque activation handle and JSON activation reports. * Introduced new lifecycle APIs to clear and free activation handles. * Enabled layering discovered `plugins.toml` configuration with explicitly provided activation specs. * **Bug Fixes** * Improved robustness of activation teardown (idempotent clear/free and safer cleanup after failures). * Added stricter validation for empty/invalid specs and malformed inputs. * **Documentation** * Added prominent “not production-ready” warnings for the experimental activation lifecycle. * **Tests** * Added extensive Rust and Go unit/integration tests covering activation, layering, rollback, cleanup, and finalization behavior. Authors: - Bryan Bednarski (https://github.com/bbednarski9) Approvers: - Will Killian (https://github.com/willkill07) URL: #368
Overview
Adds a shared harness-native activation path that layers explicit plugin configuration over the default discovered plugins.toml files before loading dynamic plugins. The existing exact activation path remains available for hosts such as the Relay CLI that already resolved their configuration.
Details
Validation
Related Issues
Relates to #365, #366, and #368.
Summary by CodeRabbit
plugins.tomlconfiguration with dynamically provided plugin components.