feat: route orcarouter/ models through the OrcaRouter gateway - #395
Open
XiaoHuo888-hue wants to merge 1 commit into
Open
feat: route orcarouter/ models through the OrcaRouter gateway#395XiaoHuo888-hue wants to merge 1 commit into
XiaoHuo888-hue wants to merge 1 commit into
Conversation
Adds OrcaRouter as a named OpenAI-compatible provider to the local-mode LLM routing layer. 'orcarouter/...' models (e.g. 'orcarouter/deepseek/deepseek-v4-pro') now use the OpenAI SDK against https://api.orcarouter.ai/v1 with ORCAROUTER_API_KEY, mirroring how the 'openai/' prefix is handled; the routing prefix is stripped before the request, except for OrcaRouter's own 'orcarouter/auto' alias. Updates the CLI key gates (run_pageindex.py, tree_optimize.py) to require ORCAROUTER_API_KEY for orcarouter models, keeps retrieve_model normalization passthrough in sync, and documents the provider in README.md and config.yaml. Verified: pytest suite (76 passed; the one failure is a pre-existing Windows-only path test), sync + async live calls through llm_completion/llm_acompletion against the real endpoint (deepseek model and orcarouter/auto), and 401 rejection for an invalid key.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds OrcaRouter as a model provider. OrcaRouter is an OpenAI-compatible model routing gateway that exposes 150+ models from OpenAI, Anthropic, Google, DeepSeek, Qwen, MiniMax, xAI and others behind a single endpoint and API key. It also provides gateway-level security controls for AI agents.
I'm an engineer on the OrcaRouter team.
Concretely,
orcarouter/...models now route through the gateway in local mode:pageindex/utils.py—orcarouter/models use the OpenAI SDK againsthttps://api.orcarouter.ai/v1withORCAROUTER_API_KEY, mirroring how the existingopenai/prefix is handled. The routing prefix is stripped before the request, except for OrcaRouter's ownorcarouter/autoalias. Both sync and async completion paths are covered, and a missingORCAROUTER_API_KEYfails immediately instead of being retried.run_pageindex.py,pageindex/tree_optimize.py— the CLI key gates now requireORCAROUTER_API_KEYfororcarouter/models (previously these only checkedOPENAI_API_KEY).pageindex/client.py—retrieve_modelnormalization keepsorcarouter/...as passthrough (nolitellm/wrapping).README.md,pageindex/config.yaml— documented the provider with usage examples.tests/test_client.py— tests for the routing helpers, model-id resolution, missing-key failure, and the completion path.Testing
python -m pytest -q→ 76 passed, 1 failed. The failure (test_list_documents_skips_unsafe_directory_names) is a pre-existing Windows-only path test (it creates a directory namedbad/name, whichos.mkdircannot create on Windows) — it also fails on cleanmain.llm_completion("orcarouter/deepseek/deepseek-v4-pro", ...)andllm_acompletion(...)both return the expected reply;orcarouter/orcarouter/autoalso works; an invalid key is rejected with a 401AuthenticationError.litellm.token_counter(used by the standard indexing path) handlesorcarouter/...model strings without error.Checklist
main)