Local Models

27B-class models running fast on one consumer GPU: quants, llama.cpp flags, MTP speculative decoding and context tuning.

Ongoing

What it does

Qwen3.8 27B at about 108 tokens per second on a single RTX 5080, using llama.cpp with MTP speculative decoding.
One big model on the card at a time, with builds picked to fit its 16 GB.
One launcher per model, with Open WebUI and LM Studio as the chat front ends.
Voice in and out without the cloud: Whisper for speech to text and Piper for text to speech.