Windows(11)-tested fork of llama-cpp-turboquant TurboQuant KV cache experiments and benchmarking.
Production-grade KV-cache and weight quantization for llama.cpp, with cross-backend kernel support for Apple Silicon, NVIDIA CUDA, AMD ROCm, and Vulkan.
Some results have been hidden because they may be inaccessible to you
Show inaccessible results