Orchestration

[hammad93]

This issue tracks orchestration of multiple containers. There are 2 services: one is serving the large language model (LLM) by itself while the other is the front end that provides a secure access point to it. The ideal design is just one container but the LLM requires significantly more computing resources and optimizations than standard.

[hammad93]

For the front end, Open WebUI is utilized that is regularly updated with cutting edge features. The software in the container is updated through another container, Watchtower. However, there are security concerns.

The automatic updating feature will be customized with security scans through advanced configuration of Watchtower at a later time. Until then, manual updates will occur.

Alternatives include chatbot-api, something I created in 2016 for Markov Chain and RNN based chatbots utilizing cobe. However, the current chat history data is compatible with the former approach and there are a few features missing.

[hammad93]

The orchestration using docker compose is done. See #4 for details. This issue can be closed, granted that the changes make it to the main branch.