Enhancement request: Allow using an API endpoint for model usage

[dprothero]

For enterprise environments where downloading models from huggingface is blocked by policy, it would be nice to be able to configure the tool to use an OpenAI API compatible endpoint and custom model.

[stephantul]

Hey @dprothero!

If avoiding hugging face is all you need: we can easily work around this by making the path to the model configurable. This would allow you to store it somewhere locally/put it in some docker image. This is something we're planning to do anyway.

If you want to embed using a different kind of model: that would require some more details from your side in terms of what you'd like to load. It would certainly be a lot slower.