feat: Add multi-provider LLM support via LiteLLM integration - #110
Closed
joeVenner wants to merge 3 commits into
Closed
feat: Add multi-provider LLM support via LiteLLM integration#110joeVenner wants to merge 3 commits into
joeVenner wants to merge 3 commits into
Conversation
|
HOW CAN I UNSUBSCRIBE TO THIS!?!
…On Sat, 14 Feb 2026 at 14:06, JoeVenner ***@***.***> wrote:
🌟 Add Multi-Provider LLM Support - Use Any LLM with PageIndex! Summary
This PR transforms PageIndex from an OpenAI-only tool into a *universal
LLM-powered document indexing solution*. By integrating LiteLLM
<https://docs.litellm.ai/>, users can now choose from *100+ LLM providers*
including:
Provider Models Setup
🟢 *OpenAI* GPT-4o, GPT-4 Turbo OPENAI_API_KEY
🟣 *Anthropic* Claude 3 Opus, Sonnet, Haiku ANTHROPIC_API_KEY
🔵 *Google* Gemini Pro, Gemini 1.5 GEMINI_API_KEY
🟦 *Azure* Azure OpenAI deployments AZURE_API_KEY + base
🟠 *AWS Bedrock* Claude, Llama on Bedrock AWS credentials
⚡ *Groq* Llama 3.1, Mixtral GROQ_API_KEY
🦙 *Ollama (Local)* Llama3, Mistral, Qwen No API key! 🚀 Key Features
- *Zero Breaking Changes*: Fully backward compatible with existing
setups
- *Local Model Support*: Run completely offline with Ollama
- *Flexible Token Counting*: Automatic tokenizer selection per provider
- *Simple Migration*: Just change the model string!
📝 Usage Examples
# OpenAI (default - no changes needed)
python3 run_pageindex.py --pdf_path doc.pdf
# Claude 3 Opusexport ANTHROPIC_API_KEY=your_key
python3 run_pageindex.py --pdf_path doc.pdf --model claude-3-opus-20240229
# Local Ollama (no API key!)
ollama serve
python3 run_pageindex.py --pdf_path doc.pdf --model ollama/llama3
------------------------------
You can view, comment on, or merge this pull request online at:
#110
Commit Summary
- 6a2bc47
<6a2bc47>
feat: Add multi-provider LLM support via LiteLLM integration
File Changes
(4 files <https://github.com/VectifyAI/PageIndex/pull/110/files>)
- *M* README.md
<https://github.com/VectifyAI/PageIndex/pull/110/files#diff-b335630551682c19a781afebcf4d07bf978fb1f8ac04c6bf87428ed5106870f5>
(49)
- *M* pageindex/config.yaml
<https://github.com/VectifyAI/PageIndex/pull/110/files#diff-9606b16dba62797da3dfb022659759f9f88ae63a5e0c33f40166c5fb37677faf>
(35)
- *M* pageindex/utils.py
<https://github.com/VectifyAI/PageIndex/pull/110/files#diff-1702cf55b6cc21929c7df118b0d61833878e4ebb72fea4c4b991ea6fbfcf3388>
(149)
- *M* requirements.txt
<https://github.com/VectifyAI/PageIndex/pull/110/files#diff-4d7c51b1efe9043e44439a949dfd92e5827321b34082903477fd04876edb7552>
(3)
Patch Links:
- https://github.com/VectifyAI/PageIndex/pull/110.patch
- https://github.com/VectifyAI/PageIndex/pull/110.diff
—
Reply to this email directly, view it on GitHub
<#110>, or unsubscribe
<https://github.com/notifications/unsubscribe-auth/BSGAVO6I3LGS6OBLSXTE2J34L4TVFAVCNFSM6AAAAACVEF3EFWVHI2DSMVQWIX3LMV43ASLTON2WKOZTHE2DCNJRGYYDKOI>
.
You are receiving this because you are subscribed to this thread.Message
ID: ***@***.***>
|
Author
|
@zmtomorrow Can you check please ? |
|
I really need this pull request. |
Author
|
@rejojer can you check ? |
ClimenteA
reviewed
Mar 22, 2026
| else: | ||
| # Use LiteLLM's token counter for other providers | ||
| try: | ||
| return litellm.token_counter(model=model, text=text) |
There was a problem hiding this comment.
Why not use directly litellm.token_counter? Why is this needed: if is_openai_model:
ClimenteA
suggested changes
Mar 22, 2026
Replace OpenAI-only implementation with LiteLLM to support 100+ LLM providers including Anthropic Claude, Google Gemini, Azure OpenAI, AWS Bedrock, Groq, and local Ollama models. Changes: - Add litellm>=1.0.0 dependency - Refactor ChatGPT_API functions to use litellm.completion() - Enhance count_tokens() for multi-provider token counting - Update config.yaml with provider-specific model examples - Update README.md with multi-provider setup instructions Backward compatible: Existing OPENAI_API_KEY and CHATGPT_API_KEY still work. Default model remains gpt-4o-2024-11-20.
…removing reliance on direct tiktoken and hardcoded models, while cleaning up tokenization logic to prevent crashes
Accept upstream's LiteLLM implementation (llm_completion/llm_acompletion, pinned litellm==1.82.0, ConfigLoader, drop_params) which supersedes our parallel implementation. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
joeVenner
force-pushed
the
feat/Add_multi_provider_LLM_support_including_local_ollama
branch
from
March 22, 2026 13:24
acc22f7 to
7d63209
Compare
Author
|
@ClimenteA Can you please check this PR when you have time ? |
Collaborator
|
Thanks @joeVenner! This is exactly the direction we took — multi-provider support via LiteLLM landed in #168 (merged), so |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
5 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
🌟 Add Multi-Provider LLM Support - Use Any LLM with PageIndex!
Summary
This PR transforms PageIndex from an OpenAI-only tool into a universal LLM-powered document indexing solution. By integrating LiteLLM, users can now choose from 100+ LLM providers including:
OPENAI_API_KEYANTHROPIC_API_KEYGEMINI_API_KEYAZURE_API_KEY+ baseGROQ_API_KEY🚀 Key Features
📝 Usage Examples