Skip to content

feat(provider): allow scoped LLM provider injection (#243)#249

Merged
rng1995 merged 1 commit into
NVIDIA:mainfrom
rodboev:pr/provider-injection-243
Jul 14, 2026
Merged

feat(provider): allow scoped LLM provider injection (#243)#249
rng1995 merged 1 commit into
NVIDIA:mainfrom
rodboev:pr/provider-injection-243

Conversation

@rodboev

@rodboev rodboev commented Jul 4, 2026

Copy link
Copy Markdown
Contributor

Summary

Embedding applications can now bind a governed in-process LLM provider for the current scan context. That lets hosts reuse their existing completion API without exporting raw keys, invoking an agent CLI, or monkeypatching SkillSpector internals.

Closes #243

Root cause

Provider selection lived entirely behind SKILLSPECTOR_PROVIDER and the built-in fallback chain. Every public helper routed through _select_active_provider(), but that selector had no scoped override for a provider object that the embedding application already owns.

The first implementation also left three provider-context edges open: bound HTTP providers could be treated as available even when their native chat-model construction failed, bound CLI providers could be sent through the HTTP provider path, and model context length was cached only by label even though the same label can resolve differently under different providers.

Diff Notes

  • Add a ContextVar-backed provider binding API in skillspector.providers.
  • Keep the existing env/default provider dispatch when no provider is bound.
  • Route LLM availability through the active provider path, including CLI-capable providers, bound HTTP providers with native chat-model probes, and the existing unbound fallback.
  • Keep MCP scan accounting honest by using the same availability result that the graph will use for provider selection.
  • Resolve model context length from the current provider context at call time rather than caching only by model label.
  • Cover metadata, active-provider, credential, chat-model, reset, nested-token, availability, MCP accounting, CLI binding, unavailable bound providers, and provider-specific context length behavior in focused tests.

Scope

This does not add a new provider family, credential store, CLI transport, or graph-level policy. The hook stays in the provider adapter layer, and MCP scan accounting only observes whether a provider is bound for the current context.

Attribution

The upstream issue proposed the ContextVar binding shape; this PR implements that design in the existing provider selector.

Verification

  • pytest tests/unit/test_llm_utils.py tests/unit/test_mcp_server.py tests/unit/test_model_info.py tests/unit/test_providers.py - pass
  • ruff check src/ tests/
  • ruff format --check src/ tests/
  • Invariant enumeration gate - pass, 24 rows, state domains checked
  • Local adversarial review - no findings
  • CI Lint & Test (Python 3.12), Lint & Test (Python 3.13), and DCO Check - pending maintainer approval and rerun after push

@rng1995 rng1995 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[Automated SkillSpector Review]

Requesting changes. Context-local provider dispatch is the right shape, but unusable bindings are reported as available, injected CLI providers take the wrong construction path, and model-limit caching can leak values across provider contexts. These make scoped injection unreliable and can produce dishonest LLM accounting.

Comment thread src/skillspector/mcp_server.py Outdated
Comment thread src/skillspector/llm_utils.py
Comment thread src/skillspector/providers/__init__.py
Signed-off-by: Rod Boev <rod.boev@gmail.com>
@rodboev
rodboev force-pushed the pr/provider-injection-243 branch from 295fdb8 to f0f2f3e Compare July 9, 2026 18:10

@rng1995 rng1995 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[Automated SkillSpector Review]

Re-review of head f0f2f3e. This PR adds a ContextVar-backed scoped provider binding (use_provider/reset_provider/has_provider_binding) so embedding applications can inject a governed LLM provider without exporting keys or monkeypatching. All three previously raised blockers are resolved; no new blockers found.

Prior-issue resolution checklist

  1. Unusable bindings reported as available (mcp_server.py) — Resolved. run_scan now derives llm_available from the shared is_llm_available() path instead of resolve_provider_credentials(). For bound HTTP providers, is_llm_available() probes native chat-model construction; providers.create_chat_model raises the no-LLM-key ValueError when a bound provider returns None instead of silently falling back to OpenAI (fallback preserved for the unbound path). Covered by test_run_scan_disables_llm_for_unavailable_bound_provider, test_run_scan_uses_bound_provider_without_credentials, and test_injected_provider_without_native_model_does_not_fall_back_to_openai (which sets OPENAI_API_KEY to prove no fallback leak).
  2. Injected CLI providers sent through create_chat_model (llm_utils.py) — Resolved. is_llm_available() now checks has_cli_capability(get_active_provider()) before the binding probe, and get_chat_model() routes CLI-capable bound providers to AgentCLIChatModel (duck-typed has_cli_capability covers externally injected providers). Regression test_bound_cli_provider_uses_cli_availability asserts is_available() is used and create_chat_model is never called.
  3. Process-global context-length cache keyed only by model label (model_info.py) — Resolved. @functools.cache removed from _resolve_context_length; resolution now consults the context-local get_metadata_provider() at call time. The remaining registry._load cache is keyed by YAML path, so no cross-provider leakage. Regression test_token_limits_follow_current_bound_provider_for_same_model_label verifies the same label yields different limits under different bindings.

Verification

Applied the head diff onto current main locally: it applies cleanly, the full unit/node test suite passes (1267 passed, 12 skipped, 6 xfailed), and ruff check is clean.

Non-blocking notes

  • run_scan now calls is_llm_available() on every scan, including use_llm=False; for CLI providers this delegates to is_available(), which may spawn a subprocess (binary/auth probe) per scan. Consider whether static-only scans should short-circuit if MCP latency matters.
  • With the functools.cache removed, the "No token-limit info for model ... using default" warning in _resolve_context_length now logs on every call for unknown labels rather than once per process. Consider a logger.warning-once guard if log volume becomes noisy.
  • SKILLSPECTOR_MODEL_{SLOT} env overrides still take precedence over an injected provider in build_model_config(). This matches the documented precedence, but embedders should be aware host env vars can override their bound provider's model choices; a docstring note on use_provider would help.

raise ValueError(f"output_format must be one of {VALID_FORMATS}, got {output_format!r}")

llm_available = resolve_provider_credentials() is not None
llm_available, _ = is_llm_available()

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Non-blocking: this now runs on every scan, including use_llm=False. For CLI providers is_llm_available() delegates to is_available(), which can spawn a subprocess (binary/auth probe) per request. If MCP scan latency matters, consider computing availability lazily or only probing when use_llm is requested (the payload's llm_available field would need a cheaper source in that case).

@rng1995
rng1995 merged commit 36cb67d into NVIDIA:main Jul 14, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Feature] Let an embedding application inject an LLM provider

2 participants