This repository provides inference sever to host any open-source LLM, providing the user OpenAI API colpatabile inference server, and AutoGen agents with function calling.

Ansible role to provision and configure a host server

Socket Programming is done using python

Python Flask server for hosting and serving multiple ML models locally or over the internet using ngrok tunneling.