onde-inference

https://github.com/ondeinference/onde

Ruby bindings for the Onde inference engine. Run LLMs and speech-to-text locally with automatic HuggingFace model management, cache inspection, and GPU acceleration.