Swap GPT for any LLM by changing a single line of code. Xinference lets you run open-source, speech, and multimodal models on cloud, on-prem, or your laptop — all through one unified, production-ready inference API.

This custom_node for ComfyUI adds one-click "Virtual VRAM" for any UNet and CLIP loader as well MultiGPU integration in WanVideoWrapper, managing the offload/Block Swap of layers to DRAM *or* VRAM to maximize the latent space of your card. Also includes nodes for directly loading entire components (UNet, CLIP, VAE) onto the device you choose

Self-evaluating interview for AI coders

Run inference on MPT-30B using CPU

Chat with your data privately using MPT-30b

Run inference on replit-3B code instruct model using CPU

🪶 Lightweight OpenAI drop-in replacement for Kubernetes

stable-diffusion.cpp bindings for python

This repository contains a web application designed to execute relatively compact, locally-operated Large Language Models (LLMs).

Finetuning instruct-LLaMA on german datasets.

PyGPTPrompt: A CLI tool that manages context windows for AI models, facilitating user interaction and data ingestion for optimized long-term memory and task automation.

✨ Your Custom Offline Role Play with LLM and Stable Diffusion on Mac and Linux (for now) 🧙‍♂️

Some random tools for working with the GGUF file format

convert a saved pytorch model to gguf and generate as much corresponding ggml c code as possible

Development and deployment on AWS of a question-answer LLM model using Llama2 with 7B parameters and RAG with LangChain

✨ LLM CMD is a toolbox allowing you to use LLM in daily developer commands 💻

A simple chat app with embeddings and vector database, exclusively for local execution on mac/apple silicon.

finLLM using llama-2-7b-chat-ggml, langchain, chainlit, and faiss vector db on encyclopedia of finance, sfsu

A fairly comprehensive Python library allowing for exploration, self-education and categorizaton of AI models

An AI_Podcast developed with running Xinference platform using GGML and Pytorch Model