Skip to content

Mattbusel/llm-batch

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

10 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

llm-batch

Batch LLM processing with thread


Part of the llm-cpp toolkit -- 26 single-header C++ libraries for LLM infrastructure. Drop in what you need.

About

Zero-dependency single-header C++ library for batching LLM API requests. Reduce latency and cost with intelligent request grouping.

Topics

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

 
 
 

Contributors